2024-09-12T03:00:37.387 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-12T03:00:37.413 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T03:00:37.506 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901668 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901668' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request - SLOW_OPS - slow request sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.dir_split install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi006.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBgcUbNX3P5/V1ktraIAY+2BVfX5jwJBrJjYeMqcYmvDXNP+U52DHnRqiKW1bJyt3V/6hDkXQRJ+V3DaWoFUZcw= smithi118.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIp03T3rsd0fZDVc3HW2YTsdqkRWGAsbF919SLxdj6xaLxSNQkOyQILibHB1ujcQrw0tHkoIM4n8C9Y360LoIiE= smithi161.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPuZ0S33YorEVKkJ9tzynohOJwNaKmdCMMcyv6z/YMOyZ5CpdJhGvv1UKEECFLUneIjDVjr4H9h84/pCgCt2Acs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - quiescer: null - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-12T03:00:37.507 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-12T03:00:37.507 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-12T03:00:37.507 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-12T03:00:37.509 INFO:teuthology.task.internal:Checking packages... 2024-09-12T03:00:37.532 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-12T03:00:37.532 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-12T03:00:37.532 INFO:teuthology.packaging:ref: None 2024-09-12T03:00:37.532 INFO:teuthology.packaging:tag: None 2024-09-12T03:00:37.532 INFO:teuthology.packaging:branch: squid 2024-09-12T03:00:37.532 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:00:37.532 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-12T03:00:37.744 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-12T03:00:37.746 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-12T03:00:37.752 INFO:teuthology.task.internal:no buildpackages task found 2024-09-12T03:00:37.753 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-12T03:00:37.830 INFO:teuthology.task.internal:Saving configuration 2024-09-12T03:00:37.851 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-12T03:00:37.910 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-12T03:00:37.936 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi006.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901668', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 02:55:03.275881', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBgcUbNX3P5/V1ktraIAY+2BVfX5jwJBrJjYeMqcYmvDXNP+U52DHnRqiKW1bJyt3V/6hDkXQRJ+V3DaWoFUZcw='} 2024-09-12T03:00:37.959 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi118.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901668', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 02:55:03.274386', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIp03T3rsd0fZDVc3HW2YTsdqkRWGAsbF919SLxdj6xaLxSNQkOyQILibHB1ujcQrw0tHkoIM4n8C9Y360LoIiE='} 2024-09-12T03:00:37.984 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901668', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 02:55:03.277201', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPuZ0S33YorEVKkJ9tzynohOJwNaKmdCMMcyv6z/YMOyZ5CpdJhGvv1UKEECFLUneIjDVjr4H9h84/pCgCt2Acs='} 2024-09-12T03:00:37.985 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-12T03:00:37.991 INFO:teuthology.task.internal:roles: ubuntu@smithi006.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-12T03:00:37.991 INFO:teuthology.task.internal:roles: ubuntu@smithi118.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-12T03:00:37.991 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-12T03:00:37.991 INFO:teuthology.run_tasks:Running task console_log... 2024-09-12T03:00:38.229 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdc8ab27be0>, signals=[15]) 2024-09-12T03:00:38.229 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-12T03:00:38.312 INFO:teuthology.task.internal:Opening connections... 2024-09-12T03:00:38.313 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi006.front.sepia.ceph.com 2024-09-12T03:00:38.314 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:00:38.390 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi118.front.sepia.ceph.com 2024-09-12T03:00:38.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:00:38.465 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2024-09-12T03:00:38.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:00:38.541 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-12T03:00:38.547 DEBUG:teuthology.orchestra.run.smithi006:> uname -m 2024-09-12T03:00:38.566 INFO:teuthology.orchestra.run.smithi006.stdout:x86_64 2024-09-12T03:00:38.566 DEBUG:teuthology.orchestra.run.smithi006:> cat /etc/os-release 2024-09-12T03:00:38.623 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="CentOS Stream" 2024-09-12T03:00:38.624 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="9" 2024-09-12T03:00:38.624 INFO:teuthology.orchestra.run.smithi006.stdout:ID="centos" 2024-09-12T03:00:38.624 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE="rhel fedora" 2024-09-12T03:00:38.624 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="9" 2024-09-12T03:00:38.624 INFO:teuthology.orchestra.run.smithi006.stdout:PLATFORM_ID="platform:el9" 2024-09-12T03:00:38.624 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T03:00:38.624 INFO:teuthology.orchestra.run.smithi006.stdout:ANSI_COLOR="0;31" 2024-09-12T03:00:38.624 INFO:teuthology.orchestra.run.smithi006.stdout:LOGO="fedora-logo-icon" 2024-09-12T03:00:38.624 INFO:teuthology.orchestra.run.smithi006.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T03:00:38.624 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://centos.org/" 2024-09-12T03:00:38.624 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T03:00:38.625 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T03:00:38.625 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T03:00:38.625 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2024-09-12T03:00:38.650 DEBUG:teuthology.orchestra.run.smithi118:> uname -m 2024-09-12T03:00:38.669 INFO:teuthology.orchestra.run.smithi118.stdout:x86_64 2024-09-12T03:00:38.669 DEBUG:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2024-09-12T03:00:38.726 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="CentOS Stream" 2024-09-12T03:00:38.726 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="9" 2024-09-12T03:00:38.726 INFO:teuthology.orchestra.run.smithi118.stdout:ID="centos" 2024-09-12T03:00:38.726 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE="rhel fedora" 2024-09-12T03:00:38.726 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="9" 2024-09-12T03:00:38.727 INFO:teuthology.orchestra.run.smithi118.stdout:PLATFORM_ID="platform:el9" 2024-09-12T03:00:38.727 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T03:00:38.727 INFO:teuthology.orchestra.run.smithi118.stdout:ANSI_COLOR="0;31" 2024-09-12T03:00:38.727 INFO:teuthology.orchestra.run.smithi118.stdout:LOGO="fedora-logo-icon" 2024-09-12T03:00:38.727 INFO:teuthology.orchestra.run.smithi118.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T03:00:38.727 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://centos.org/" 2024-09-12T03:00:38.727 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T03:00:38.727 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T03:00:38.727 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T03:00:38.728 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2024-09-12T03:00:38.742 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2024-09-12T03:00:38.760 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2024-09-12T03:00:38.761 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2024-09-12T03:00:38.819 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Stream" 2024-09-12T03:00:38.819 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="9" 2024-09-12T03:00:38.819 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos" 2024-09-12T03:00:38.819 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora" 2024-09-12T03:00:38.819 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="9" 2024-09-12T03:00:38.819 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el9" 2024-09-12T03:00:38.819 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T03:00:38.819 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2024-09-12T03:00:38.819 INFO:teuthology.orchestra.run.smithi161.stdout:LOGO="fedora-logo-icon" 2024-09-12T03:00:38.819 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T03:00:38.820 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://centos.org/" 2024-09-12T03:00:38.820 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T03:00:38.820 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T03:00:38.820 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T03:00:38.820 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2024-09-12T03:00:38.841 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-12T03:00:38.852 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-12T03:00:38.879 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-12T03:00:38.880 DEBUG:teuthology.orchestra.run.smithi006:> test '!' -e /home/ubuntu/cephtest 2024-09-12T03:00:38.882 DEBUG:teuthology.orchestra.run.smithi118:> test '!' -e /home/ubuntu/cephtest 2024-09-12T03:00:38.884 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2024-09-12T03:00:38.901 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-12T03:00:38.935 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-12T03:00:38.935 DEBUG:teuthology.orchestra.run.smithi006:> test -z $(ls -A /var/lib/ceph) 2024-09-12T03:00:38.939 DEBUG:teuthology.orchestra.run.smithi118:> test -z $(ls -A /var/lib/ceph) 2024-09-12T03:00:38.942 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2024-09-12T03:00:39.066 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-12T03:00:39.139 INFO:teuthology.run_tasks:Running task kernel... 2024-09-12T03:00:39.153 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-12T03:00:39.153 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-12T03:00:39.154 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-12T03:00:39.154 DEBUG:teuthology.orchestra.run.smithi006:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T03:00:39.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:00:39.171 DEBUG:teuthology.orchestra.run.smithi006:> uname -r 2024-09-12T03:00:39.228 INFO:teuthology.orchestra.run.smithi006.stdout:5.14.0-503.el9.x86_64 2024-09-12T03:00:39.229 INFO:teuthology.task.kernel:Running kernel on smithi006: 5.14.0-503.el9.x86_64 2024-09-12T03:00:39.229 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum install -y kernel 2024-09-12T03:00:42.459 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - BaseOS 5.7 MB/s | 8.2 MB 00:01 2024-09-12T03:00:47.335 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - AppStream 5.5 MB/s | 20 MB 00:03 2024-09-12T03:00:51.986 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.5 MB 00:00 2024-09-12T03:00:53.537 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 19 kB 00:00 2024-09-12T03:00:54.341 INFO:teuthology.orchestra.run.smithi006.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-12T03:00:59.989 INFO:teuthology.orchestra.run.smithi006.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-12T03:01:02.145 INFO:teuthology.orchestra.run.smithi006.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-12T03:01:02.145 INFO:teuthology.orchestra.run.smithi006.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-12T03:01:02.228 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-09-12T03:01:02.233 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-09-12T03:01:02.233 INFO:teuthology.orchestra.run.smithi006.stdout: Package Architecture Version Repository Size 2024-09-12T03:01:02.233 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-09-12T03:01:02.234 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-09-12T03:01:02.234 INFO:teuthology.orchestra.run.smithi006.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-12T03:01:02.234 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-09-12T03:01:02.234 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-12T03:01:02.234 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-12T03:01:02.234 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-12T03:01:02.234 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:01:02.234 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-09-12T03:01:02.234 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-09-12T03:01:02.234 INFO:teuthology.orchestra.run.smithi006.stdout:Install 4 Packages 2024-09-12T03:01:02.234 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:01:02.235 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 79 M 2024-09-12T03:01:02.235 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 126 M 2024-09-12T03:01:02.235 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-09-12T03:01:02.507 INFO:teuthology.orchestra.run.smithi006.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 162 kB/s | 23 kB 00:00 2024-09-12T03:01:02.915 INFO:teuthology.orchestra.run.smithi006.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-09-12T03:01:03.332 INFO:teuthology.orchestra.run.smithi006.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 35 MB/s | 29 MB 00:00 2024-09-12T03:01:03.633 INFO:teuthology.orchestra.run.smithi006.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 27 MB/s | 35 MB 00:01 2024-09-12T03:01:03.633 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:01:03.633 INFO:teuthology.orchestra.run.smithi006.stdout:Total 56 MB/s | 79 MB 00:01 2024-09-12T03:01:04.343 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-09-12T03:01:04.747 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-09-12T03:01:04.748 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-09-12T03:01:05.764 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-09-12T03:01:05.764 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-09-12T03:01:08.397 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-09-12T03:01:09.076 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-12T03:01:09.158 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T03:01:11.956 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T03:01:12.080 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T03:01:17.198 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T03:01:17.239 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:01:22.371 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:02:02.013 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:02:02.055 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:02:02.459 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:02:02.459 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-12T03:02:02.459 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T03:02:02.459 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T03:02:03.290 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:02:03.290 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:02:03.290 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-09-12T03:02:03.290 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-12T03:02:03.290 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-12T03:02:03.290 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-12T03:02:03.291 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-12T03:02:03.291 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:02:03.291 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-09-12T03:02:03.635 DEBUG:teuthology.orchestra.run.smithi006:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T03:02:03.760 INFO:teuthology.orchestra.run.smithi006.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T03:02:03.760 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T03:02:03.760 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-12T03:02:03.761 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T03:02:03.761 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T03:02:03.761 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-12T03:02:03.761 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi006.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T03:02:03.761 DEBUG:teuthology.orchestra.run.smithi006:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T03:02:03.819 INFO:teuthology.orchestra.run.smithi006.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T03:02:03.820 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T03:02:03.820 DEBUG:teuthology.orchestra.run.smithi006:> sudo rpm -qi grub2-tools 2024-09-12T03:02:03.858 INFO:teuthology.orchestra.run.smithi006.stdout:Name : grub2-tools 2024-09-12T03:02:03.858 INFO:teuthology.orchestra.run.smithi006.stdout:Epoch : 1 2024-09-12T03:02:03.858 INFO:teuthology.orchestra.run.smithi006.stdout:Version : 2.06 2024-09-12T03:02:03.858 INFO:teuthology.orchestra.run.smithi006.stdout:Release : 82.el9 2024-09-12T03:02:03.859 INFO:teuthology.orchestra.run.smithi006.stdout:Architecture: x86_64 2024-09-12T03:02:03.859 INFO:teuthology.orchestra.run.smithi006.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T03:02:03.859 INFO:teuthology.orchestra.run.smithi006.stdout:Group : Unspecified 2024-09-12T03:02:03.859 INFO:teuthology.orchestra.run.smithi006.stdout:Size : 8274670 2024-09-12T03:02:03.859 INFO:teuthology.orchestra.run.smithi006.stdout:License : GPLv3+ 2024-09-12T03:02:03.859 INFO:teuthology.orchestra.run.smithi006.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T03:02:03.859 INFO:teuthology.orchestra.run.smithi006.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T03:02:03.859 INFO:teuthology.orchestra.run.smithi006.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T03:02:03.859 INFO:teuthology.orchestra.run.smithi006.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T03:02:03.859 INFO:teuthology.orchestra.run.smithi006.stdout:Packager : builder@centos.org 2024-09-12T03:02:03.859 INFO:teuthology.orchestra.run.smithi006.stdout:Vendor : CentOS 2024-09-12T03:02:03.859 INFO:teuthology.orchestra.run.smithi006.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T03:02:03.859 INFO:teuthology.orchestra.run.smithi006.stdout:Summary : Support tools for GRUB. 2024-09-12T03:02:03.860 INFO:teuthology.orchestra.run.smithi006.stdout:Description : 2024-09-12T03:02:03.860 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:02:03.860 INFO:teuthology.orchestra.run.smithi006.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T03:02:03.860 INFO:teuthology.orchestra.run.smithi006.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T03:02:03.860 INFO:teuthology.orchestra.run.smithi006.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T03:02:03.860 INFO:teuthology.orchestra.run.smithi006.stdout:hardware devices. 2024-09-12T03:02:03.860 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:02:03.860 INFO:teuthology.orchestra.run.smithi006.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T03:02:03.861 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T03:02:03.861 INFO:teuthology.task.kernel:Updating grub on smithi006 to boot 5.14.0-505.el9.x86_64 2024-09-12T03:02:03.861 DEBUG:teuthology.orchestra.run.smithi006:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T03:02:04.504 INFO:teuthology.orchestra.run.smithi006.stderr:Generating grub configuration file ... 2024-09-12T03:02:05.853 INFO:teuthology.orchestra.run.smithi006.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T03:02:05.892 INFO:teuthology.orchestra.run.smithi006.stderr:done 2024-09-12T03:02:05.894 DEBUG:teuthology.orchestra.run.smithi006:> mktemp 2024-09-12T03:02:05.928 INFO:teuthology.orchestra.run.smithi006.stdout:/tmp/tmp.tBiC6egLv5 2024-09-12T03:02:05.928 DEBUG:teuthology.orchestra.run.smithi006:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.tBiC6egLv5 2024-09-12T03:02:06.001 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0666 /tmp/tmp.tBiC6egLv5 2024-09-12T03:02:06.141 DEBUG:teuthology.orchestra.remote:smithi006:/tmp/tmp.tBiC6egLv5 is 6KB 2024-09-12T03:02:06.190 DEBUG:teuthology.orchestra.run.smithi006:> rm -fr /tmp/tmp.tBiC6egLv5 2024-09-12T03:02:06.204 DEBUG:teuthology.orchestra.run.smithi006:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T03:02:06.267 INFO:teuthology.orchestra.run.smithi006.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T03:02:06.268 INFO:teuthology.orchestra.run.smithi006.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T03:02:06.268 INFO:teuthology.orchestra.run.smithi006.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T03:02:06.268 INFO:teuthology.orchestra.run.smithi006.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-12T03:02:06.269 DEBUG:teuthology.orchestra.run.smithi006:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-12T03:02:06.424 DEBUG:teuthology.orchestra.run.smithi006:> sudo shutdown -r now 2024-09-12T03:02:36.427 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-12T03:02:36.427 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-09-12T03:02:36.428 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:02:54.785 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.6 2024-09-12T03:03:03.786 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-09-12T03:03:03.787 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:04:03.789 DEBUG:teuthology.orchestra.remote:timed out 2024-09-12T03:04:15.792 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-09-12T03:04:15.794 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:04:16.129 DEBUG:teuthology.orchestra.run.smithi006:> true 2024-09-12T03:04:16.517 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-09-12T03:04:16.517 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-12T03:04:16.517 DEBUG:teuthology.orchestra.run.smithi006:> uname -r 2024-09-12T03:04:16.535 INFO:teuthology.orchestra.run.smithi006.stdout:5.14.0-505.el9.x86_64 2024-09-12T03:04:16.535 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-12T03:04:16.535 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T03:04:16.536 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T03:04:17.537 DEBUG:teuthology.parallel:result is None 2024-09-12T03:04:17.537 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-12T03:04:17.544 INFO:teuthology.task.internal:Creating test directory... 2024-09-12T03:04:17.544 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T03:04:17.547 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T03:04:17.550 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T03:04:17.569 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-12T03:04:17.612 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-12T03:04:17.683 INFO:teuthology.task.internal:Creating archive directory... 2024-09-12T03:04:17.683 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T03:04:17.686 DEBUG:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T03:04:17.688 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T03:04:17.731 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-12T03:04:17.737 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-12T03:04:17.737 DEBUG:teuthology.orchestra.run.smithi118:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T03:04:17.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:04:17.777 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T03:04:17.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:04:17.796 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T03:04:17.798 DEBUG:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T03:04:17.820 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T03:04:17.870 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:04:17.880 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:04:17.884 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:04:17.899 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:04:17.901 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:04:17.918 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:04:17.920 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-12T03:04:17.927 INFO:teuthology.task.internal:Configuring sudo... 2024-09-12T03:04:17.927 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T03:04:17.929 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T03:04:17.932 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T03:04:17.959 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-12T03:04:18.028 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-12T03:04:18.028 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T03:04:18.030 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T03:04:18.033 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T03:04:18.049 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:04:18.131 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:04:18.177 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:04:18.263 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:04:18.304 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-09-12T03:04:18.304 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T03:04:18.370 DEBUG:teuthology.orchestra.run.smithi118:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:04:18.413 DEBUG:teuthology.orchestra.run.smithi118:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:04:18.464 DEBUG:teuthology.orchestra.run.smithi118:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:04:18.546 DEBUG:teuthology.orchestra.run.smithi118:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:04:18.589 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-12T03:04:18.589 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T03:04:18.656 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:04:18.702 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:04:18.762 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:04:18.805 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:04:18.849 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-12T03:04:18.849 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T03:04:18.916 DEBUG:teuthology.orchestra.run.smithi006:> sudo service rsyslog restart 2024-09-12T03:04:18.918 DEBUG:teuthology.orchestra.run.smithi118:> sudo service rsyslog restart 2024-09-12T03:04:18.921 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2024-09-12T03:04:18.957 INFO:teuthology.orchestra.run.smithi118.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:04:18.966 INFO:teuthology.orchestra.run.smithi006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:04:18.994 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:04:19.402 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-12T03:04:19.410 INFO:teuthology.task.internal:Starting timer... 2024-09-12T03:04:19.410 INFO:teuthology.run_tasks:Running task pcp... 2024-09-12T03:04:19.467 INFO:teuthology.run_tasks:Running task selinux... 2024-09-12T03:04:19.526 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-12T03:04:19.526 DEBUG:teuthology.orchestra.run.smithi006:> sudo service auditd rotate 2024-09-12T03:04:19.594 INFO:teuthology.orchestra.run.smithi006.stdout:Rotating logs: 2024-09-12T03:04:19.597 DEBUG:teuthology.orchestra.run.smithi118:> sudo service auditd rotate 2024-09-12T03:04:19.666 INFO:teuthology.orchestra.run.smithi118.stdout:Rotating logs: 2024-09-12T03:04:19.668 DEBUG:teuthology.orchestra.run.smithi161:> sudo service auditd rotate 2024-09-12T03:04:19.704 INFO:teuthology.orchestra.run.smithi161.stdout:Rotating logs: 2024-09-12T03:04:19.706 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-12T03:04:19.706 DEBUG:teuthology.orchestra.run.smithi006:> /usr/sbin/getenforce 2024-09-12T03:04:19.731 INFO:teuthology.orchestra.run.smithi006.stdout:Permissive 2024-09-12T03:04:19.731 DEBUG:teuthology.orchestra.run.smithi118:> /usr/sbin/getenforce 2024-09-12T03:04:19.753 INFO:teuthology.orchestra.run.smithi118.stdout:Permissive 2024-09-12T03:04:19.753 DEBUG:teuthology.orchestra.run.smithi161:> /usr/sbin/getenforce 2024-09-12T03:04:19.773 INFO:teuthology.orchestra.run.smithi161.stdout:Permissive 2024-09-12T03:04:19.773 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi006.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi118.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi161.front.sepia.ceph.com': 'permissive'} 2024-09-12T03:04:19.773 DEBUG:teuthology.orchestra.run.smithi006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:04:19.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:04:19.802 DEBUG:teuthology.orchestra.run.smithi118:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:04:19.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:04:19.829 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:04:19.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:04:19.855 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-12T03:04:19.855 DEBUG:teuthology.orchestra.run.smithi006:> sudo /usr/sbin/setenforce permissive 2024-09-12T03:04:19.895 DEBUG:teuthology.orchestra.run.smithi118:> sudo /usr/sbin/setenforce permissive 2024-09-12T03:04:19.941 DEBUG:teuthology.orchestra.run.smithi161:> sudo /usr/sbin/setenforce permissive 2024-09-12T03:04:19.976 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-12T03:04:19.986 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-12T03:04:19.989 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-12T03:04:20.098 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-12T03:04:20.109 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-12T03:04:20.110 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi006.front.sepia.ceph.com,smithi118.front.sepia.ceph.com,smithi161.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-12T03:08:27.520 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi006.front.sepia.ceph.com'), Remote(name='ubuntu@smithi118.front.sepia.ceph.com'), Remote(name='ubuntu@smithi161.front.sepia.ceph.com')] 2024-09-12T03:08:27.522 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-09-12T03:08:27.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:08:27.602 DEBUG:teuthology.orchestra.run.smithi006:> true 2024-09-12T03:08:27.685 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-09-12T03:08:27.686 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-12T03:08:27.686 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:08:27.759 DEBUG:teuthology.orchestra.run.smithi118:> true 2024-09-12T03:08:27.841 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-12T03:08:27.841 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-09-12T03:08:27.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:08:27.919 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-09-12T03:08:28.002 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-09-12T03:08:28.003 INFO:teuthology.run_tasks:Running task clock... 2024-09-12T03:08:28.018 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-12T03:08:28.019 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T03:08:28.019 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:08:28.022 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T03:08:28.022 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:08:28.024 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T03:08:28.025 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:08:28.052 INFO:teuthology.orchestra.run.smithi118.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T03:08:28.055 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T03:08:28.067 INFO:teuthology.orchestra.run.smithi118.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T03:08:28.072 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T03:08:28.074 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T03:08:28.089 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T03:08:28.092 INFO:teuthology.orchestra.run.smithi118.stderr:sudo: ntpd: command not found 2024-09-12T03:08:28.099 INFO:teuthology.orchestra.run.smithi006.stderr:sudo: ntpd: command not found 2024-09-12T03:08:28.103 INFO:teuthology.orchestra.run.smithi118.stdout:506 Cannot talk to daemon 2024-09-12T03:08:28.112 INFO:teuthology.orchestra.run.smithi006.stdout:506 Cannot talk to daemon 2024-09-12T03:08:28.115 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: ntpd: command not found 2024-09-12T03:08:28.117 INFO:teuthology.orchestra.run.smithi118.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T03:08:28.126 INFO:teuthology.orchestra.run.smithi161.stdout:506 Cannot talk to daemon 2024-09-12T03:08:28.127 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T03:08:28.131 INFO:teuthology.orchestra.run.smithi118.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T03:08:28.141 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T03:08:28.142 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T03:08:28.155 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T03:08:28.189 INFO:teuthology.orchestra.run.smithi118.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:08:28.192 INFO:teuthology.orchestra.run.smithi118.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:08:28.192 INFO:teuthology.orchestra.run.smithi118.stdout:=============================================================================== 2024-09-12T03:08:28.192 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:08:28.192 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:08:28.192 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:08:28.193 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:08:28.200 INFO:teuthology.orchestra.run.smithi006.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:08:28.204 INFO:teuthology.orchestra.run.smithi006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:08:28.205 INFO:teuthology.orchestra.run.smithi006.stdout:=============================================================================== 2024-09-12T03:08:28.205 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:08:28.205 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:08:28.205 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:08:28.205 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:08:28.221 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:08:28.224 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:08:28.224 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-09-12T03:08:28.224 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:08:28.224 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:08:28.224 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:08:28.224 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:08:28.225 INFO:teuthology.run_tasks:Running task install... 2024-09-12T03:08:28.235 DEBUG:teuthology.task.install:project ceph 2024-09-12T03:08:28.235 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-12T03:08:28.236 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-12T03:08:28.236 INFO:teuthology.task.install:Using flavor: default 2024-09-12T03:08:28.242 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-12T03:08:28.243 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-12T03:08:28.243 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T03:08:28.244 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:08:28.245 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T03:08:28.246 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:08:28.247 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T03:08:28.248 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:08:28.395 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T03:08:28.395 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T03:08:28.399 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T03:08:28.400 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T03:08:28.404 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T03:08:28.405 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T03:08:28.509 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T03:08:28.509 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-09-12T03:08:28.509 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T03:08:28.515 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T03:08:28.515 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-12T03:08:28.515 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T03:08:28.518 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T03:08:28.518 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-12T03:08:28.518 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T03:08:28.538 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T03:08:28.538 DEBUG:teuthology.orchestra.run.smithi006:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T03:08:28.541 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T03:08:28.541 DEBUG:teuthology.orchestra.run.smithi161:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T03:08:28.544 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T03:08:28.545 DEBUG:teuthology.orchestra.run.smithi118:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T03:08:28.606 DEBUG:teuthology.orchestra.run.smithi006:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T03:08:28.607 DEBUG:teuthology.orchestra.run.smithi161:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T03:08:28.611 DEBUG:teuthology.orchestra.run.smithi118:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T03:08:28.682 DEBUG:teuthology.orchestra.run.smithi161:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T03:08:28.683 DEBUG:teuthology.orchestra.run.smithi006:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T03:08:28.686 DEBUG:teuthology.orchestra.run.smithi118:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T03:08:28.746 INFO:teuthology.orchestra.run.smithi161.stdout:check_obsoletes = 1 2024-09-12T03:08:28.747 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum clean all 2024-09-12T03:08:28.750 INFO:teuthology.orchestra.run.smithi118.stdout:check_obsoletes = 1 2024-09-12T03:08:28.750 INFO:teuthology.orchestra.run.smithi006.stdout:check_obsoletes = 1 2024-09-12T03:08:28.751 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum clean all 2024-09-12T03:08:28.753 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum clean all 2024-09-12T03:08:28.971 INFO:teuthology.orchestra.run.smithi161.stdout:45 files removed 2024-09-12T03:08:28.977 INFO:teuthology.orchestra.run.smithi118.stdout:45 files removed 2024-09-12T03:08:28.992 INFO:teuthology.orchestra.run.smithi006.stdout:45 files removed 2024-09-12T03:08:28.998 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T03:08:29.004 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T03:08:29.020 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T03:08:29.560 INFO:teuthology.orchestra.run.smithi118.stdout:ceph packages for x86_64 299 kB/s | 84 kB 00:00 2024-09-12T03:08:29.567 INFO:teuthology.orchestra.run.smithi161.stdout:ceph packages for x86_64 292 kB/s | 84 kB 00:00 2024-09-12T03:08:29.591 INFO:teuthology.orchestra.run.smithi006.stdout:ceph packages for x86_64 307 kB/s | 84 kB 00:00 2024-09-12T03:08:29.844 INFO:teuthology.orchestra.run.smithi118.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-12T03:08:29.859 INFO:teuthology.orchestra.run.smithi161.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-12T03:08:29.892 INFO:teuthology.orchestra.run.smithi006.stdout:ceph noarch packages 46 kB/s | 12 kB 00:00 2024-09-12T03:08:30.120 INFO:teuthology.orchestra.run.smithi118.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-12T03:08:30.135 INFO:teuthology.orchestra.run.smithi161.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-12T03:08:30.167 INFO:teuthology.orchestra.run.smithi006.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-12T03:08:30.993 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.2 MB 00:00 2024-09-12T03:08:31.426 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - BaseOS 6.8 MB/s | 8.2 MB 00:01 2024-09-12T03:08:32.045 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.2 MB 00:01 2024-09-12T03:08:37.410 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - AppStream 4.2 MB/s | 20 MB 00:04 2024-09-12T03:08:39.927 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-09-12T03:08:42.186 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - CRB 7.9 MB/s | 6.5 MB 00:00 2024-09-12T03:08:43.662 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - Extras packages 90 kB/s | 19 kB 00:00 2024-09-12T03:08:44.474 INFO:teuthology.orchestra.run.smithi006.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-12T03:08:45.495 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-12T03:08:47.020 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 19 kB 00:00 2024-09-12T03:08:47.336 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - AppStream 1.4 MB/s | 20 MB 00:14 2024-09-12T03:08:47.841 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-12T03:08:50.041 INFO:teuthology.orchestra.run.smithi006.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-12T03:08:52.446 INFO:teuthology.orchestra.run.smithi006.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:08:52.446 INFO:teuthology.orchestra.run.smithi006.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:08:52.460 INFO:teuthology.orchestra.run.smithi006.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T03:08:52.461 INFO:teuthology.orchestra.run.smithi006.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T03:08:52.462 INFO:teuthology.orchestra.run.smithi006.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T03:08:52.463 INFO:teuthology.orchestra.run.smithi006.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T03:08:52.465 INFO:teuthology.orchestra.run.smithi006.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T03:08:52.468 INFO:teuthology.orchestra.run.smithi006.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T03:08:52.469 INFO:teuthology.orchestra.run.smithi006.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T03:08:52.537 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-09-12T03:08:52.562 INFO:teuthology.orchestra.run.smithi006.stdout:====================================================================================== 2024-09-12T03:08:52.563 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-09-12T03:08:52.563 INFO:teuthology.orchestra.run.smithi006.stdout:====================================================================================== 2024-09-12T03:08:52.563 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-09-12T03:08:52.563 INFO:teuthology.orchestra.run.smithi006.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T03:08:52.563 INFO:teuthology.orchestra.run.smithi006.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T03:08:52.563 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T03:08:52.563 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T03:08:52.563 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T03:08:52.563 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T03:08:52.563 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T03:08:52.563 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T03:08:52.564 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T03:08:52.564 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T03:08:52.564 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T03:08:52.564 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T03:08:52.564 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T03:08:52.564 INFO:teuthology.orchestra.run.smithi006.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T03:08:52.564 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T03:08:52.564 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T03:08:52.564 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T03:08:52.564 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T03:08:52.564 INFO:teuthology.orchestra.run.smithi006.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T03:08:52.564 INFO:teuthology.orchestra.run.smithi006.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T03:08:52.565 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T03:08:52.565 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T03:08:52.565 INFO:teuthology.orchestra.run.smithi006.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T03:08:52.565 INFO:teuthology.orchestra.run.smithi006.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T03:08:52.565 INFO:teuthology.orchestra.run.smithi006.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T03:08:52.565 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T03:08:52.565 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T03:08:52.565 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T03:08:52.565 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T03:08:52.565 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T03:08:52.565 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T03:08:52.565 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T03:08:52.566 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T03:08:52.566 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T03:08:52.566 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T03:08:52.566 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T03:08:52.566 INFO:teuthology.orchestra.run.smithi006.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T03:08:52.566 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrading: 2024-09-12T03:08:52.566 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T03:08:52.566 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T03:08:52.566 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T03:08:52.566 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T03:08:52.566 INFO:teuthology.orchestra.run.smithi006.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T03:08:52.566 INFO:teuthology.orchestra.run.smithi006.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T03:08:52.566 INFO:teuthology.orchestra.run.smithi006.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T03:08:52.567 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T03:08:52.567 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-09-12T03:08:52.567 INFO:teuthology.orchestra.run.smithi006.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T03:08:52.567 INFO:teuthology.orchestra.run.smithi006.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T03:08:52.567 INFO:teuthology.orchestra.run.smithi006.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T03:08:52.567 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T03:08:52.567 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T03:08:52.567 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T03:08:52.567 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T03:08:52.567 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T03:08:52.567 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T03:08:52.567 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T03:08:52.567 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T03:08:52.568 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T03:08:52.568 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T03:08:52.568 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T03:08:52.568 INFO:teuthology.orchestra.run.smithi006.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T03:08:52.568 INFO:teuthology.orchestra.run.smithi006.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-12T03:08:52.568 INFO:teuthology.orchestra.run.smithi006.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T03:08:52.568 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T03:08:52.568 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T03:08:52.568 INFO:teuthology.orchestra.run.smithi006.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T03:08:52.568 INFO:teuthology.orchestra.run.smithi006.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T03:08:52.568 INFO:teuthology.orchestra.run.smithi006.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T03:08:52.568 INFO:teuthology.orchestra.run.smithi006.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T03:08:52.569 INFO:teuthology.orchestra.run.smithi006.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T03:08:52.569 INFO:teuthology.orchestra.run.smithi006.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T03:08:52.569 INFO:teuthology.orchestra.run.smithi006.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T03:08:52.569 INFO:teuthology.orchestra.run.smithi006.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T03:08:52.569 INFO:teuthology.orchestra.run.smithi006.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T03:08:52.569 INFO:teuthology.orchestra.run.smithi006.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T03:08:52.569 INFO:teuthology.orchestra.run.smithi006.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T03:08:52.569 INFO:teuthology.orchestra.run.smithi006.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T03:08:52.569 INFO:teuthology.orchestra.run.smithi006.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T03:08:52.569 INFO:teuthology.orchestra.run.smithi006.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T03:08:52.569 INFO:teuthology.orchestra.run.smithi006.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T03:08:52.569 INFO:teuthology.orchestra.run.smithi006.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T03:08:52.569 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T03:08:52.569 INFO:teuthology.orchestra.run.smithi006.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T03:08:52.570 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T03:08:52.570 INFO:teuthology.orchestra.run.smithi006.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T03:08:52.570 INFO:teuthology.orchestra.run.smithi006.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T03:08:52.570 INFO:teuthology.orchestra.run.smithi006.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T03:08:52.570 INFO:teuthology.orchestra.run.smithi006.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T03:08:52.570 INFO:teuthology.orchestra.run.smithi006.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T03:08:52.570 INFO:teuthology.orchestra.run.smithi006.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T03:08:52.570 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T03:08:52.571 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T03:08:52.571 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T03:08:52.571 INFO:teuthology.orchestra.run.smithi006.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T03:08:52.571 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T03:08:52.571 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T03:08:52.571 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T03:08:52.571 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T03:08:52.571 INFO:teuthology.orchestra.run.smithi006.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T03:08:52.571 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T03:08:52.571 INFO:teuthology.orchestra.run.smithi006.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T03:08:52.571 INFO:teuthology.orchestra.run.smithi006.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T03:08:52.571 INFO:teuthology.orchestra.run.smithi006.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T03:08:52.571 INFO:teuthology.orchestra.run.smithi006.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T03:08:52.572 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T03:08:52.572 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T03:08:52.572 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T03:08:52.572 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T03:08:52.572 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T03:08:52.572 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T03:08:52.572 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T03:08:52.572 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T03:08:52.572 INFO:teuthology.orchestra.run.smithi006.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T03:08:52.572 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T03:08:52.572 INFO:teuthology.orchestra.run.smithi006.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T03:08:52.573 INFO:teuthology.orchestra.run.smithi006.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T03:08:52.573 INFO:teuthology.orchestra.run.smithi006.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T03:08:52.573 INFO:teuthology.orchestra.run.smithi006.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T03:08:52.573 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T03:08:52.573 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T03:08:52.573 INFO:teuthology.orchestra.run.smithi006.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T03:08:52.573 INFO:teuthology.orchestra.run.smithi006.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T03:08:52.573 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T03:08:52.573 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T03:08:52.573 INFO:teuthology.orchestra.run.smithi006.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T03:08:52.573 INFO:teuthology.orchestra.run.smithi006.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T03:08:52.573 INFO:teuthology.orchestra.run.smithi006.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T03:08:52.573 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T03:08:52.573 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T03:08:52.574 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T03:08:52.574 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T03:08:52.574 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T03:08:52.574 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T03:08:52.574 INFO:teuthology.orchestra.run.smithi006.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T03:08:52.574 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T03:08:52.574 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T03:08:52.574 INFO:teuthology.orchestra.run.smithi006.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T03:08:52.574 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T03:08:52.574 INFO:teuthology.orchestra.run.smithi006.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T03:08:52.574 INFO:teuthology.orchestra.run.smithi006.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T03:08:52.574 INFO:teuthology.orchestra.run.smithi006.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T03:08:52.574 INFO:teuthology.orchestra.run.smithi006.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T03:08:52.575 INFO:teuthology.orchestra.run.smithi006.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T03:08:52.575 INFO:teuthology.orchestra.run.smithi006.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T03:08:52.575 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T03:08:52.575 INFO:teuthology.orchestra.run.smithi006.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T03:08:52.575 INFO:teuthology.orchestra.run.smithi006.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T03:08:52.575 INFO:teuthology.orchestra.run.smithi006.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T03:08:52.575 INFO:teuthology.orchestra.run.smithi006.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T03:08:52.575 INFO:teuthology.orchestra.run.smithi006.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T03:08:52.575 INFO:teuthology.orchestra.run.smithi006.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T03:08:52.575 INFO:teuthology.orchestra.run.smithi006.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T03:08:52.575 INFO:teuthology.orchestra.run.smithi006.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T03:08:52.575 INFO:teuthology.orchestra.run.smithi006.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T03:08:52.575 INFO:teuthology.orchestra.run.smithi006.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T03:08:52.576 INFO:teuthology.orchestra.run.smithi006.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T03:08:52.576 INFO:teuthology.orchestra.run.smithi006.stdout:Installing weak dependencies: 2024-09-12T03:08:52.576 INFO:teuthology.orchestra.run.smithi006.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T03:08:52.576 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:08:52.576 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-09-12T03:08:52.576 INFO:teuthology.orchestra.run.smithi006.stdout:====================================================================================== 2024-09-12T03:08:52.576 INFO:teuthology.orchestra.run.smithi006.stdout:Install 148 Packages 2024-09-12T03:08:52.576 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrade 8 Packages 2024-09-12T03:08:52.576 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:08:52.577 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 226 M 2024-09-12T03:08:52.577 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-09-12T03:08:53.230 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-12T03:08:53.479 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.5 MB 00:02 2024-09-12T03:08:53.525 INFO:teuthology.orchestra.run.smithi006.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 56 kB/s | 6.5 kB 00:00 2024-09-12T03:08:53.767 INFO:teuthology.orchestra.run.smithi006.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 3.6 MB/s | 897 kB 00:00 2024-09-12T03:08:53.859 INFO:teuthology.orchestra.run.smithi006.stdout:(3/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 12 MB/s | 5.4 MB 00:00 2024-09-12T03:08:53.884 INFO:teuthology.orchestra.run.smithi006.stdout:(4/156): ceph-immutable-object-cache-19.1.1-298 1.2 MB/s | 145 kB 00:00 2024-09-12T03:08:53.976 INFO:teuthology.orchestra.run.smithi006.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 20 MB/s | 2.4 MB 00:00 2024-09-12T03:08:54.043 INFO:teuthology.orchestra.run.smithi006.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 9.6 MB/s | 1.5 MB 00:00 2024-09-12T03:08:54.302 INFO:teuthology.orchestra.run.smithi006.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 14 MB/s | 4.7 MB 00:00 2024-09-12T03:08:54.552 INFO:teuthology.orchestra.run.smithi006.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 19 MB/s | 21 MB 00:01 2024-09-12T03:08:54.602 INFO:teuthology.orchestra.run.smithi006.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 508 kB/s | 25 kB 00:00 2024-09-12T03:08:54.929 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - Extras packages 78 kB/s | 19 kB 00:00 2024-09-12T03:08:55.077 INFO:teuthology.orchestra.run.smithi006.stdout:(10/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 16 MB/s | 17 MB 00:01 2024-09-12T03:08:55.128 INFO:teuthology.orchestra.run.smithi006.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 4.5 MB/s | 228 kB 00:00 2024-09-12T03:08:55.195 INFO:teuthology.orchestra.run.smithi006.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 406 kB/s | 27 kB 00:00 2024-09-12T03:08:55.262 INFO:teuthology.orchestra.run.smithi006.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 475 kB/s | 32 kB 00:00 2024-09-12T03:08:55.337 INFO:teuthology.orchestra.run.smithi006.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 9.8 MB/s | 752 kB 00:00 2024-09-12T03:08:55.404 INFO:teuthology.orchestra.run.smithi006.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.4 MB/s | 163 kB 00:00 2024-09-12T03:08:55.457 INFO:teuthology.orchestra.run.smithi161.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:08:55.457 INFO:teuthology.orchestra.run.smithi161.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:08:55.462 INFO:teuthology.orchestra.run.smithi006.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.1 MB/s | 127 kB 00:00 2024-09-12T03:08:55.469 INFO:teuthology.orchestra.run.smithi161.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T03:08:55.469 INFO:teuthology.orchestra.run.smithi161.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T03:08:55.470 INFO:teuthology.orchestra.run.smithi161.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T03:08:55.471 INFO:teuthology.orchestra.run.smithi161.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T03:08:55.473 INFO:teuthology.orchestra.run.smithi161.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T03:08:55.475 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T03:08:55.476 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T03:08:55.539 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-12T03:08:55.567 INFO:teuthology.orchestra.run.smithi161.stdout:====================================================================================== 2024-09-12T03:08:55.567 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-09-12T03:08:55.567 INFO:teuthology.orchestra.run.smithi161.stdout:====================================================================================== 2024-09-12T03:08:55.567 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-09-12T03:08:55.567 INFO:teuthology.orchestra.run.smithi161.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T03:08:55.567 INFO:teuthology.orchestra.run.smithi161.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T03:08:55.567 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T03:08:55.567 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T03:08:55.567 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T03:08:55.567 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T03:08:55.567 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T03:08:55.568 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T03:08:55.568 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T03:08:55.568 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T03:08:55.568 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T03:08:55.568 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T03:08:55.568 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T03:08:55.568 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T03:08:55.568 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T03:08:55.568 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T03:08:55.568 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T03:08:55.568 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T03:08:55.568 INFO:teuthology.orchestra.run.smithi161.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T03:08:55.569 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T03:08:55.569 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T03:08:55.569 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T03:08:55.569 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T03:08:55.569 INFO:teuthology.orchestra.run.smithi161.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T03:08:55.569 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T03:08:55.569 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T03:08:55.569 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T03:08:55.569 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T03:08:55.569 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T03:08:55.569 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T03:08:55.569 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T03:08:55.570 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T03:08:55.570 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T03:08:55.570 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T03:08:55.570 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T03:08:55.570 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T03:08:55.570 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T03:08:55.570 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrading: 2024-09-12T03:08:55.570 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T03:08:55.570 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T03:08:55.570 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T03:08:55.570 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T03:08:55.570 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T03:08:55.570 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T03:08:55.571 INFO:teuthology.orchestra.run.smithi161.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T03:08:55.571 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T03:08:55.571 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-09-12T03:08:55.571 INFO:teuthology.orchestra.run.smithi161.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T03:08:55.571 INFO:teuthology.orchestra.run.smithi161.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T03:08:55.571 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T03:08:55.571 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T03:08:55.571 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T03:08:55.571 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T03:08:55.571 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T03:08:55.571 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T03:08:55.571 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T03:08:55.572 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T03:08:55.572 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T03:08:55.572 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T03:08:55.572 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T03:08:55.572 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T03:08:55.572 INFO:teuthology.orchestra.run.smithi161.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T03:08:55.572 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-12T03:08:55.572 INFO:teuthology.orchestra.run.smithi161.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T03:08:55.572 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T03:08:55.572 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T03:08:55.572 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T03:08:55.572 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T03:08:55.572 INFO:teuthology.orchestra.run.smithi161.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T03:08:55.573 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T03:08:55.573 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T03:08:55.573 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T03:08:55.573 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T03:08:55.573 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T03:08:55.573 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T03:08:55.573 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T03:08:55.573 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T03:08:55.573 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T03:08:55.573 INFO:teuthology.orchestra.run.smithi161.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T03:08:55.573 INFO:teuthology.orchestra.run.smithi161.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T03:08:55.573 INFO:teuthology.orchestra.run.smithi161.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T03:08:55.573 INFO:teuthology.orchestra.run.smithi161.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T03:08:55.574 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T03:08:55.574 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T03:08:55.574 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T03:08:55.574 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T03:08:55.574 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T03:08:55.574 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T03:08:55.574 INFO:teuthology.orchestra.run.smithi161.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T03:08:55.574 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T03:08:55.574 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T03:08:55.574 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T03:08:55.574 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T03:08:55.574 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T03:08:55.575 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T03:08:55.575 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T03:08:55.575 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T03:08:55.575 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T03:08:55.575 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T03:08:55.575 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T03:08:55.575 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T03:08:55.575 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T03:08:55.575 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T03:08:55.575 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T03:08:55.575 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T03:08:55.575 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T03:08:55.576 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T03:08:55.576 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T03:08:55.576 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T03:08:55.576 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T03:08:55.576 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T03:08:55.576 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T03:08:55.576 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T03:08:55.576 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T03:08:55.576 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T03:08:55.576 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T03:08:55.576 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T03:08:55.576 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T03:08:55.577 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T03:08:55.577 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T03:08:55.577 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T03:08:55.577 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T03:08:55.577 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T03:08:55.577 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T03:08:55.577 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T03:08:55.577 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T03:08:55.577 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T03:08:55.577 INFO:teuthology.orchestra.run.smithi161.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T03:08:55.577 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T03:08:55.577 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T03:08:55.577 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T03:08:55.577 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T03:08:55.578 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T03:08:55.578 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T03:08:55.578 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T03:08:55.578 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T03:08:55.578 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T03:08:55.578 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T03:08:55.578 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T03:08:55.578 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T03:08:55.578 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T03:08:55.578 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T03:08:55.578 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T03:08:55.578 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T03:08:55.578 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T03:08:55.579 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T03:08:55.579 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T03:08:55.579 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T03:08:55.579 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T03:08:55.579 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T03:08:55.579 INFO:teuthology.orchestra.run.smithi161.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T03:08:55.579 INFO:teuthology.orchestra.run.smithi161.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T03:08:55.579 INFO:teuthology.orchestra.run.smithi161.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T03:08:55.579 INFO:teuthology.orchestra.run.smithi161.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T03:08:55.579 INFO:teuthology.orchestra.run.smithi161.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T03:08:55.579 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T03:08:55.579 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T03:08:55.579 INFO:teuthology.orchestra.run.smithi161.stdout:Installing weak dependencies: 2024-09-12T03:08:55.580 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T03:08:55.580 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:08:55.580 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-09-12T03:08:55.580 INFO:teuthology.orchestra.run.smithi161.stdout:====================================================================================== 2024-09-12T03:08:55.580 INFO:teuthology.orchestra.run.smithi161.stdout:Install 148 Packages 2024-09-12T03:08:55.580 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrade 8 Packages 2024-09-12T03:08:55.580 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:08:55.581 INFO:teuthology.orchestra.run.smithi006.stdout:(17/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 8.3 MB/s | 11 MB 00:01 2024-09-12T03:08:55.581 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 226 M 2024-09-12T03:08:55.581 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-09-12T03:08:55.613 INFO:teuthology.orchestra.run.smithi006.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 3.3 MB/s | 506 kB 00:00 2024-09-12T03:08:55.663 INFO:teuthology.orchestra.run.smithi006.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 905 kB/s | 45 kB 00:00 2024-09-12T03:08:55.666 INFO:teuthology.orchestra.run.smithi118.stdout:Extra Packages for Enterprise Linux 37 MB/s | 23 MB 00:00 2024-09-12T03:08:55.730 INFO:teuthology.orchestra.run.smithi006.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 2.0 MB/s | 136 kB 00:00 2024-09-12T03:08:55.797 INFO:teuthology.orchestra.run.smithi006.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.4 MB/s | 162 kB 00:00 2024-09-12T03:08:55.872 INFO:teuthology.orchestra.run.smithi006.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 4.2 MB/s | 321 kB 00:00 2024-09-12T03:08:55.948 INFO:teuthology.orchestra.run.smithi006.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 3.9 MB/s | 301 kB 00:00 2024-09-12T03:08:56.031 INFO:teuthology.orchestra.run.smithi006.stdout:(24/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 12 MB/s | 5.3 MB 00:00 2024-09-12T03:08:56.057 INFO:teuthology.orchestra.run.smithi006.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 917 kB/s | 100 kB 00:00 2024-09-12T03:08:56.082 INFO:teuthology.orchestra.run.smithi006.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-12T03:08:56.131 INFO:teuthology.orchestra.run.smithi161.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 51 kB/s | 6.5 kB 00:00 2024-09-12T03:08:56.340 INFO:teuthology.orchestra.run.smithi161.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 4.2 MB/s | 897 kB 00:00 2024-09-12T03:08:56.415 INFO:teuthology.orchestra.run.smithi161.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.9 MB/s | 145 kB 00:00 2024-09-12T03:08:56.523 INFO:teuthology.orchestra.run.smithi161.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 10 MB/s | 5.4 MB 00:00 2024-09-12T03:08:56.524 INFO:teuthology.orchestra.run.smithi006.stdout:(27/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 25 MB/s | 49 MB 00:01 2024-09-12T03:08:56.549 INFO:teuthology.orchestra.run.smithi006.stdout:(28/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 366 kB/s | 171 kB 00:00 2024-09-12T03:08:56.608 INFO:teuthology.orchestra.run.smithi006.stdout:(29/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 5.6 MB/s | 3.1 MB 00:00 2024-09-12T03:08:56.625 INFO:teuthology.orchestra.run.smithi006.stdout:(30/156): ceph-grafana-dashboards-19.1.1-298.gb 318 kB/s | 32 kB 00:00 2024-09-12T03:08:56.650 INFO:teuthology.orchestra.run.smithi006.stdout:(31/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.4 MB/s | 146 kB 00:00 2024-09-12T03:08:56.665 INFO:teuthology.orchestra.run.smithi161.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 9.5 MB/s | 2.4 MB 00:00 2024-09-12T03:08:56.707 INFO:teuthology.orchestra.run.smithi161.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 8.3 MB/s | 1.5 MB 00:00 2024-09-12T03:08:56.717 INFO:teuthology.orchestra.run.smithi006.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 3.7 MB/s | 249 kB 00:00 2024-09-12T03:08:57.142 INFO:teuthology.orchestra.run.smithi006.stdout:(33/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 6.7 MB/s | 3.6 MB 00:00 2024-09-12T03:08:57.226 INFO:teuthology.orchestra.run.smithi006.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 12 MB/s | 7.4 MB 00:00 2024-09-12T03:08:57.251 INFO:teuthology.orchestra.run.smithi006.stdout:(35/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 92 kB/s | 49 kB 00:00 2024-09-12T03:08:57.276 INFO:teuthology.orchestra.run.smithi006.stdout:(36/156): ceph-prometheus-alerts-19.1.1-298.gb5 123 kB/s | 16 kB 00:00 2024-09-12T03:08:57.300 INFO:teuthology.orchestra.run.smithi161.stdout:(7/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 16 MB/s | 21 MB 00:01 2024-09-12T03:08:57.327 INFO:teuthology.orchestra.run.smithi006.stdout:(37/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 2.8 MB/s | 285 kB 00:00 2024-09-12T03:08:57.352 INFO:teuthology.orchestra.run.smithi006.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 243 kB/s | 18 kB 00:00 2024-09-12T03:08:57.375 INFO:teuthology.orchestra.run.smithi161.stdout:(8/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 6.6 MB/s | 4.7 MB 00:00 2024-09-12T03:08:57.385 INFO:teuthology.orchestra.run.smithi006.stdout:(39/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 6.9 MB/s | 765 kB 00:00 2024-09-12T03:08:57.433 INFO:teuthology.orchestra.run.smithi161.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 433 kB/s | 25 kB 00:00 2024-09-12T03:08:57.452 INFO:teuthology.orchestra.run.smithi006.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 642 kB/s | 80 kB 00:00 2024-09-12T03:08:57.511 INFO:teuthology.orchestra.run.smithi006.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-12T03:08:57.553 INFO:teuthology.orchestra.run.smithi006.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-09-12T03:08:57.578 INFO:teuthology.orchestra.run.smithi006.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.5 MB/s | 801 kB 00:00 2024-09-12T03:08:57.603 INFO:teuthology.orchestra.run.smithi006.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-12T03:08:57.645 INFO:teuthology.orchestra.run.smithi006.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-09-12T03:08:57.670 INFO:teuthology.orchestra.run.smithi006.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-09-12T03:08:57.696 INFO:teuthology.orchestra.run.smithi006.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-12T03:08:57.796 INFO:teuthology.orchestra.run.smithi006.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 4.3 MB/s | 1.2 MB 00:00 2024-09-12T03:08:57.808 INFO:teuthology.orchestra.run.smithi161.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 21 MB/s | 11 MB 00:00 2024-09-12T03:08:57.871 INFO:teuthology.orchestra.run.smithi006.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-09-12T03:08:57.962 INFO:teuthology.orchestra.run.smithi006.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 523 kB/s | 22 kB 00:00 2024-09-12T03:08:57.962 INFO:teuthology.orchestra.run.smithi006.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 752 kB/s | 107 kB 00:00 2024-09-12T03:08:57.963 INFO:teuthology.orchestra.run.smithi006.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-12T03:08:57.984 INFO:teuthology.orchestra.run.smithi161.stdout:(11/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 13 MB/s | 17 MB 00:01 2024-09-12T03:08:57.997 INFO:teuthology.orchestra.run.smithi006.stdout:(53/156): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-09-12T03:08:58.009 INFO:teuthology.orchestra.run.smithi161.stdout:(12/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 1.1 MB/s | 228 kB 00:00 2024-09-12T03:08:58.023 INFO:teuthology.orchestra.run.smithi006.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 389 kB/s | 33 kB 00:00 2024-09-12T03:08:58.034 INFO:teuthology.orchestra.run.smithi161.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 538 kB/s | 27 kB 00:00 2024-09-12T03:08:58.048 INFO:teuthology.orchestra.run.smithi006.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-12T03:08:58.059 INFO:teuthology.orchestra.run.smithi161.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 632 kB/s | 32 kB 00:00 2024-09-12T03:08:58.073 INFO:teuthology.orchestra.run.smithi006.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-12T03:08:58.098 INFO:teuthology.orchestra.run.smithi006.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-12T03:08:58.101 INFO:teuthology.orchestra.run.smithi161.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 752 kB 00:00 2024-09-12T03:08:58.124 INFO:teuthology.orchestra.run.smithi006.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-09-12T03:08:58.127 INFO:teuthology.orchestra.run.smithi161.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.4 MB/s | 163 kB 00:00 2024-09-12T03:08:58.157 INFO:teuthology.orchestra.run.smithi006.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-12T03:08:58.160 INFO:teuthology.orchestra.run.smithi161.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.1 MB/s | 127 kB 00:00 2024-09-12T03:08:58.194 INFO:teuthology.orchestra.run.smithi161.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 7.4 MB/s | 506 kB 00:00 2024-09-12T03:08:58.216 INFO:teuthology.orchestra.run.smithi006.stdout:(60/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-09-12T03:08:58.244 INFO:teuthology.orchestra.run.smithi161.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 902 kB/s | 45 kB 00:00 2024-09-12T03:08:58.258 INFO:teuthology.orchestra.run.smithi006.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 244 kB/s | 24 kB 00:00 2024-09-12T03:08:58.283 INFO:teuthology.orchestra.run.smithi006.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 710 kB/s | 47 kB 00:00 2024-09-12T03:08:58.309 INFO:teuthology.orchestra.run.smithi006.stdout:(63/156): libtool-2.4.6-46.el9.x86_64.rpm 3.1 MB/s | 581 kB 00:00 2024-09-12T03:08:58.311 INFO:teuthology.orchestra.run.smithi161.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 2.0 MB/s | 136 kB 00:00 2024-09-12T03:08:58.334 INFO:teuthology.orchestra.run.smithi006.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-12T03:08:58.359 INFO:teuthology.orchestra.run.smithi006.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-12T03:08:58.384 INFO:teuthology.orchestra.run.smithi006.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-09-12T03:08:58.428 INFO:teuthology.orchestra.run.smithi161.stdout:(21/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 20 MB/s | 5.3 MB 00:00 2024-09-12T03:08:58.453 INFO:teuthology.orchestra.run.smithi161.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.1 MB/s | 162 kB 00:00 2024-09-12T03:08:58.479 INFO:teuthology.orchestra.run.smithi161.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 6.3 MB/s | 321 kB 00:00 2024-09-12T03:08:58.485 INFO:teuthology.orchestra.run.smithi006.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 33 MB/s | 4.9 MB 00:00 2024-09-12T03:08:58.512 INFO:teuthology.orchestra.run.smithi161.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.0 MB/s | 301 kB 00:00 2024-09-12T03:08:58.527 INFO:teuthology.orchestra.run.smithi006.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-12T03:08:58.538 INFO:teuthology.orchestra.run.smithi161.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.7 MB/s | 100 kB 00:00 2024-09-12T03:08:58.563 INFO:teuthology.orchestra.run.smithi161.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-12T03:08:58.568 INFO:teuthology.orchestra.run.smithi006.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 11 MB/s | 910 kB 00:00 2024-09-12T03:08:58.594 INFO:teuthology.orchestra.run.smithi006.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-12T03:08:58.622 INFO:teuthology.orchestra.run.smithi161.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.9 MB/s | 171 kB 00:00 2024-09-12T03:08:58.652 INFO:teuthology.orchestra.run.smithi006.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-12T03:08:58.672 INFO:teuthology.orchestra.run.smithi161.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 639 kB/s | 32 kB 00:00 2024-09-12T03:08:58.694 INFO:teuthology.orchestra.run.smithi006.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-09-12T03:08:58.711 INFO:teuthology.orchestra.run.smithi006.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-12T03:08:58.722 INFO:teuthology.orchestra.run.smithi161.stdout:(29/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 17 MB/s | 3.1 MB 00:00 2024-09-12T03:08:58.747 INFO:teuthology.orchestra.run.smithi161.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.9 MB/s | 146 kB 00:00 2024-09-12T03:08:58.753 INFO:teuthology.orchestra.run.smithi006.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-12T03:08:58.837 INFO:teuthology.orchestra.run.smithi006.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-12T03:08:58.862 INFO:teuthology.orchestra.run.smithi006.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-12T03:08:58.895 INFO:teuthology.orchestra.run.smithi006.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-12T03:08:58.921 INFO:teuthology.orchestra.run.smithi006.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-09-12T03:08:58.939 INFO:teuthology.orchestra.run.smithi161.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 16 MB/s | 3.6 MB 00:00 2024-09-12T03:08:58.946 INFO:teuthology.orchestra.run.smithi006.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-12T03:08:58.971 INFO:teuthology.orchestra.run.smithi006.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-12T03:08:58.996 INFO:teuthology.orchestra.run.smithi006.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-09-12T03:08:59.006 INFO:teuthology.orchestra.run.smithi161.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 3.7 MB/s | 249 kB 00:00 2024-09-12T03:08:59.080 INFO:teuthology.orchestra.run.smithi006.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-12T03:08:59.098 INFO:teuthology.orchestra.run.smithi161.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 21 MB/s | 7.4 MB 00:00 2024-09-12T03:08:59.105 INFO:teuthology.orchestra.run.smithi006.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-12T03:08:59.124 INFO:teuthology.orchestra.run.smithi161.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 422 kB/s | 49 kB 00:00 2024-09-12T03:08:59.131 INFO:teuthology.orchestra.run.smithi006.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-12T03:08:59.149 INFO:teuthology.orchestra.run.smithi161.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 329 kB/s | 16 kB 00:00 2024-09-12T03:08:59.156 INFO:teuthology.orchestra.run.smithi006.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-12T03:08:59.181 INFO:teuthology.orchestra.run.smithi006.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-09-12T03:08:59.182 INFO:teuthology.orchestra.run.smithi161.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.8 MB/s | 285 kB 00:00 2024-09-12T03:08:59.207 INFO:teuthology.orchestra.run.smithi006.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-12T03:08:59.224 INFO:teuthology.orchestra.run.smithi161.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 10 MB/s | 765 kB 00:00 2024-09-12T03:08:59.240 INFO:teuthology.orchestra.run.smithi006.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-12T03:08:59.241 INFO:teuthology.orchestra.run.smithi161.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 311 kB/s | 18 kB 00:00 2024-09-12T03:08:59.274 INFO:teuthology.orchestra.run.smithi006.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-09-12T03:08:59.299 INFO:teuthology.orchestra.run.smithi006.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 871 kB/s | 51 kB 00:00 2024-09-12T03:08:59.324 INFO:teuthology.orchestra.run.smithi006.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-12T03:08:59.333 INFO:teuthology.orchestra.run.smithi161.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 736 kB/s | 80 kB 00:00 2024-09-12T03:08:59.411 INFO:teuthology.orchestra.run.smithi161.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 4.6 MB/s | 801 kB 00:00 2024-09-12T03:08:59.416 INFO:teuthology.orchestra.run.smithi006.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-12T03:08:59.442 INFO:teuthology.orchestra.run.smithi006.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 356 kB/s | 42 kB 00:00 2024-09-12T03:08:59.442 INFO:teuthology.orchestra.run.smithi161.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.7 MB/s | 191 kB 00:00 2024-09-12T03:08:59.467 INFO:teuthology.orchestra.run.smithi006.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-09-12T03:08:59.492 INFO:teuthology.orchestra.run.smithi006.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-12T03:08:59.510 INFO:teuthology.orchestra.run.smithi161.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-09-12T03:08:59.517 INFO:teuthology.orchestra.run.smithi006.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-09-12T03:08:59.543 INFO:teuthology.orchestra.run.smithi006.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-12T03:08:59.568 INFO:teuthology.orchestra.run.smithi006.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-12T03:08:59.568 INFO:teuthology.orchestra.run.smithi161.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-12T03:08:59.593 INFO:teuthology.orchestra.run.smithi006.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-12T03:08:59.619 INFO:teuthology.orchestra.run.smithi006.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-12T03:08:59.635 INFO:teuthology.orchestra.run.smithi161.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-12T03:08:59.644 INFO:teuthology.orchestra.run.smithi006.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-12T03:08:59.669 INFO:teuthology.orchestra.run.smithi006.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-09-12T03:08:59.702 INFO:teuthology.orchestra.run.smithi161.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 4.7 MB/s | 1.2 MB 00:00 2024-09-12T03:08:59.727 INFO:teuthology.orchestra.run.smithi161.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-09-12T03:08:59.794 INFO:teuthology.orchestra.run.smithi161.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-12T03:08:59.894 INFO:teuthology.orchestra.run.smithi006.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-12T03:08:59.919 INFO:teuthology.orchestra.run.smithi006.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 81 kB/s | 22 kB 00:00 2024-09-12T03:08:59.986 INFO:teuthology.orchestra.run.smithi006.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.9 MB/s | 548 kB 00:00 2024-09-12T03:09:00.012 INFO:teuthology.orchestra.run.smithi006.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 784 kB/s | 19 kB 00:00 2024-09-12T03:09:00.037 INFO:teuthology.orchestra.run.smithi006.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 2.6 MB/s | 308 kB 00:00 2024-09-12T03:09:00.062 INFO:teuthology.orchestra.run.smithi006.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-12T03:09:00.087 INFO:teuthology.orchestra.run.smithi006.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T03:09:00.113 INFO:teuthology.orchestra.run.smithi006.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-12T03:09:00.138 INFO:teuthology.orchestra.run.smithi006.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-12T03:09:00.172 INFO:teuthology.orchestra.run.smithi006.stdout:(112/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-09-12T03:09:00.230 INFO:teuthology.orchestra.run.smithi006.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-12T03:09:00.247 INFO:teuthology.orchestra.run.smithi006.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-12T03:09:00.264 INFO:teuthology.orchestra.run.smithi006.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-12T03:09:00.281 INFO:teuthology.orchestra.run.smithi006.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 861 kB/s | 14 kB 00:00 2024-09-12T03:09:00.314 INFO:teuthology.orchestra.run.smithi006.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-09-12T03:09:00.328 INFO:teuthology.orchestra.run.smithi161.stdout:(48/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 17 MB/s | 49 MB 00:02 2024-09-12T03:09:00.339 INFO:teuthology.orchestra.run.smithi006.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-12T03:09:00.374 INFO:teuthology.orchestra.run.smithi006.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-12T03:09:00.407 INFO:teuthology.orchestra.run.smithi006.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-12T03:09:00.433 INFO:teuthology.orchestra.run.smithi006.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-12T03:09:00.466 INFO:teuthology.orchestra.run.smithi006.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-12T03:09:00.492 INFO:teuthology.orchestra.run.smithi006.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-12T03:09:00.517 INFO:teuthology.orchestra.run.smithi006.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 712 kB/s | 18 kB 00:00 2024-09-12T03:09:00.542 INFO:teuthology.orchestra.run.smithi006.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 934 kB/s | 23 kB 00:00 2024-09-12T03:09:00.567 INFO:teuthology.orchestra.run.smithi006.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 784 kB/s | 19 kB 00:00 2024-09-12T03:09:00.593 INFO:teuthology.orchestra.run.smithi006.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 787 kB/s | 20 kB 00:00 2024-09-12T03:09:00.668 INFO:teuthology.orchestra.run.smithi006.stdout:(128/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.7 MB/s | 4.4 MB 00:00 2024-09-12T03:09:00.693 INFO:teuthology.orchestra.run.smithi006.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T03:09:00.718 INFO:teuthology.orchestra.run.smithi006.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T03:09:00.752 INFO:teuthology.orchestra.run.smithi006.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-09-12T03:09:00.794 INFO:teuthology.orchestra.run.smithi006.stdout:(132/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.3 MB/s | 2.0 MB 00:00 2024-09-12T03:09:00.795 INFO:teuthology.orchestra.run.smithi161.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 229 kB/s | 107 kB 00:00 2024-09-12T03:09:00.819 INFO:teuthology.orchestra.run.smithi006.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 574 kB/s | 58 kB 00:00 2024-09-12T03:09:00.845 INFO:teuthology.orchestra.run.smithi006.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-12T03:09:00.870 INFO:teuthology.orchestra.run.smithi006.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-12T03:09:00.886 INFO:teuthology.orchestra.run.smithi161.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 594 kB/s | 689 kB 00:01 2024-09-12T03:09:00.895 INFO:teuthology.orchestra.run.smithi006.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-12T03:09:00.912 INFO:teuthology.orchestra.run.smithi161.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 185 kB/s | 22 kB 00:00 2024-09-12T03:09:00.928 INFO:teuthology.orchestra.run.smithi006.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 526 kB/s | 31 kB 00:00 2024-09-12T03:09:00.954 INFO:teuthology.orchestra.run.smithi006.stdout:(138/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-09-12T03:09:00.962 INFO:teuthology.orchestra.run.smithi161.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 808 kB/s | 943 kB 00:01 2024-09-12T03:09:00.979 INFO:teuthology.orchestra.run.smithi006.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-12T03:09:01.004 INFO:teuthology.orchestra.run.smithi161.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-12T03:09:01.005 INFO:teuthology.orchestra.run.smithi006.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-12T03:09:01.029 INFO:teuthology.orchestra.run.smithi161.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 279 kB/s | 33 kB 00:00 2024-09-12T03:09:01.046 INFO:teuthology.orchestra.run.smithi006.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-12T03:09:01.082 INFO:teuthology.orchestra.run.smithi006.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 723 kB/s | 74 kB 00:00 2024-09-12T03:09:01.113 INFO:teuthology.orchestra.run.smithi006.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-09-12T03:09:01.121 INFO:teuthology.orchestra.run.smithi161.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 18 kB 00:00 2024-09-12T03:09:01.123 INFO:teuthology.orchestra.run.smithi118.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-12T03:09:01.138 INFO:teuthology.orchestra.run.smithi006.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-09-12T03:09:01.147 INFO:teuthology.orchestra.run.smithi161.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 95 kB/s | 11 kB 00:00 2024-09-12T03:09:01.164 INFO:teuthology.orchestra.run.smithi006.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-12T03:09:01.198 INFO:teuthology.orchestra.run.smithi006.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 5.7 MB/s | 191 kB 00:00 2024-09-12T03:09:01.223 INFO:teuthology.orchestra.run.smithi006.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.0 MB/s | 427 kB 00:00 2024-09-12T03:09:01.247 INFO:teuthology.orchestra.run.smithi161.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-12T03:09:01.272 INFO:teuthology.orchestra.run.smithi161.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 361 kB/s | 45 kB 00:00 2024-09-12T03:09:01.281 INFO:teuthology.orchestra.run.smithi006.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-12T03:09:01.448 INFO:teuthology.orchestra.run.smithi006.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 14 MB/s | 3.4 MB 00:00 2024-09-12T03:09:01.498 INFO:teuthology.orchestra.run.smithi161.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.6 MB/s | 662 kB 00:00 2024-09-12T03:09:01.590 INFO:teuthology.orchestra.run.smithi006.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 8.6 MB/s | 3.2 MB 00:00 2024-09-12T03:09:01.614 INFO:teuthology.orchestra.run.smithi161.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 210 kB/s | 24 kB 00:00 2024-09-12T03:09:01.657 INFO:teuthology.orchestra.run.smithi006.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 613 kB/s | 41 kB 00:00 2024-09-12T03:09:01.692 INFO:teuthology.orchestra.run.smithi161.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 1.4 MB/s | 581 kB 00:00 2024-09-12T03:09:01.723 INFO:teuthology.orchestra.run.smithi161.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-09-12T03:09:01.749 INFO:teuthology.orchestra.run.smithi006.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 2.7 MB/s | 249 kB 00:00 2024-09-12T03:09:01.807 INFO:teuthology.orchestra.run.smithi161.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-12T03:09:01.825 INFO:teuthology.orchestra.run.smithi006.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 4.8 MB/s | 1.8 MB 00:00 2024-09-12T03:09:01.840 INFO:teuthology.orchestra.run.smithi161.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-12T03:09:01.900 INFO:teuthology.orchestra.run.smithi006.stdout:(154/156): NetworkManager-1.49.90-1.el9.x86_64. 3.8 MB/s | 2.3 MB 00:00 2024-09-12T03:09:01.916 INFO:teuthology.orchestra.run.smithi161.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 354 kB/s | 38 kB 00:00 2024-09-12T03:09:02.017 INFO:teuthology.orchestra.run.smithi006.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 5.1 MB/s | 1.4 MB 00:00 2024-09-12T03:09:02.492 INFO:teuthology.orchestra.run.smithi006.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.6 MB/s | 2.4 MB 00:00 2024-09-12T03:09:02.498 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:09:02.498 INFO:teuthology.orchestra.run.smithi006.stdout:Total 23 MB/s | 226 MB 00:09 2024-09-12T03:09:02.541 INFO:teuthology.orchestra.run.smithi161.stdout:(66/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 7.1 MB/s | 4.4 MB 00:00 2024-09-12T03:09:02.683 INFO:teuthology.orchestra.run.smithi161.stdout:(67/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.7 MB/s | 3.0 MB 00:01 2024-09-12T03:09:02.792 INFO:teuthology.orchestra.run.smithi161.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-09-12T03:09:02.908 INFO:teuthology.orchestra.run.smithi161.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-12T03:09:03.376 INFO:teuthology.orchestra.run.smithi118.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:09:03.377 INFO:teuthology.orchestra.run.smithi118.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:09:03.390 INFO:teuthology.orchestra.run.smithi118.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T03:09:03.392 INFO:teuthology.orchestra.run.smithi118.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T03:09:03.393 INFO:teuthology.orchestra.run.smithi118.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T03:09:03.394 INFO:teuthology.orchestra.run.smithi118.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T03:09:03.396 INFO:teuthology.orchestra.run.smithi118.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T03:09:03.399 INFO:teuthology.orchestra.run.smithi118.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T03:09:03.400 INFO:teuthology.orchestra.run.smithi118.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T03:09:03.463 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-12T03:09:03.485 INFO:teuthology.orchestra.run.smithi118.stdout:====================================================================================== 2024-09-12T03:09:03.485 INFO:teuthology.orchestra.run.smithi118.stdout: Package Arch Version Repository Size 2024-09-12T03:09:03.485 INFO:teuthology.orchestra.run.smithi118.stdout:====================================================================================== 2024-09-12T03:09:03.485 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-09-12T03:09:03.485 INFO:teuthology.orchestra.run.smithi118.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T03:09:03.485 INFO:teuthology.orchestra.run.smithi118.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T03:09:03.485 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T03:09:03.485 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T03:09:03.485 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T03:09:03.485 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T03:09:03.486 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T03:09:03.486 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T03:09:03.486 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T03:09:03.486 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T03:09:03.486 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T03:09:03.486 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T03:09:03.486 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T03:09:03.486 INFO:teuthology.orchestra.run.smithi118.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T03:09:03.486 INFO:teuthology.orchestra.run.smithi118.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T03:09:03.486 INFO:teuthology.orchestra.run.smithi118.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T03:09:03.486 INFO:teuthology.orchestra.run.smithi118.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T03:09:03.486 INFO:teuthology.orchestra.run.smithi118.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T03:09:03.486 INFO:teuthology.orchestra.run.smithi118.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T03:09:03.487 INFO:teuthology.orchestra.run.smithi118.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T03:09:03.487 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T03:09:03.487 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T03:09:03.487 INFO:teuthology.orchestra.run.smithi118.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T03:09:03.487 INFO:teuthology.orchestra.run.smithi118.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T03:09:03.487 INFO:teuthology.orchestra.run.smithi118.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T03:09:03.487 INFO:teuthology.orchestra.run.smithi118.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T03:09:03.487 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T03:09:03.487 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T03:09:03.487 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T03:09:03.487 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T03:09:03.487 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T03:09:03.488 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T03:09:03.488 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T03:09:03.488 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T03:09:03.488 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T03:09:03.488 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T03:09:03.488 INFO:teuthology.orchestra.run.smithi118.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T03:09:03.488 INFO:teuthology.orchestra.run.smithi118.stdout:Upgrading: 2024-09-12T03:09:03.488 INFO:teuthology.orchestra.run.smithi118.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T03:09:03.488 INFO:teuthology.orchestra.run.smithi118.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T03:09:03.488 INFO:teuthology.orchestra.run.smithi118.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T03:09:03.488 INFO:teuthology.orchestra.run.smithi118.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T03:09:03.489 INFO:teuthology.orchestra.run.smithi118.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T03:09:03.489 INFO:teuthology.orchestra.run.smithi118.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T03:09:03.489 INFO:teuthology.orchestra.run.smithi118.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T03:09:03.489 INFO:teuthology.orchestra.run.smithi118.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T03:09:03.489 INFO:teuthology.orchestra.run.smithi118.stdout:Installing dependencies: 2024-09-12T03:09:03.489 INFO:teuthology.orchestra.run.smithi118.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T03:09:03.489 INFO:teuthology.orchestra.run.smithi118.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T03:09:03.489 INFO:teuthology.orchestra.run.smithi118.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T03:09:03.489 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T03:09:03.489 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T03:09:03.489 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T03:09:03.489 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T03:09:03.489 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T03:09:03.490 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T03:09:03.490 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T03:09:03.490 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T03:09:03.490 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T03:09:03.490 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T03:09:03.490 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T03:09:03.490 INFO:teuthology.orchestra.run.smithi118.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T03:09:03.490 INFO:teuthology.orchestra.run.smithi118.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-12T03:09:03.490 INFO:teuthology.orchestra.run.smithi118.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T03:09:03.491 INFO:teuthology.orchestra.run.smithi118.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T03:09:03.491 INFO:teuthology.orchestra.run.smithi118.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T03:09:03.491 INFO:teuthology.orchestra.run.smithi118.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T03:09:03.491 INFO:teuthology.orchestra.run.smithi118.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T03:09:03.491 INFO:teuthology.orchestra.run.smithi118.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T03:09:03.491 INFO:teuthology.orchestra.run.smithi118.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T03:09:03.491 INFO:teuthology.orchestra.run.smithi118.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T03:09:03.491 INFO:teuthology.orchestra.run.smithi118.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T03:09:03.491 INFO:teuthology.orchestra.run.smithi118.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T03:09:03.491 INFO:teuthology.orchestra.run.smithi118.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T03:09:03.491 INFO:teuthology.orchestra.run.smithi118.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T03:09:03.491 INFO:teuthology.orchestra.run.smithi118.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T03:09:03.491 INFO:teuthology.orchestra.run.smithi118.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T03:09:03.491 INFO:teuthology.orchestra.run.smithi118.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T03:09:03.492 INFO:teuthology.orchestra.run.smithi118.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T03:09:03.492 INFO:teuthology.orchestra.run.smithi118.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T03:09:03.492 INFO:teuthology.orchestra.run.smithi118.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T03:09:03.492 INFO:teuthology.orchestra.run.smithi118.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T03:09:03.492 INFO:teuthology.orchestra.run.smithi118.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T03:09:03.492 INFO:teuthology.orchestra.run.smithi118.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T03:09:03.492 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T03:09:03.492 INFO:teuthology.orchestra.run.smithi118.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T03:09:03.492 INFO:teuthology.orchestra.run.smithi118.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T03:09:03.492 INFO:teuthology.orchestra.run.smithi118.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T03:09:03.492 INFO:teuthology.orchestra.run.smithi118.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T03:09:03.492 INFO:teuthology.orchestra.run.smithi118.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T03:09:03.493 INFO:teuthology.orchestra.run.smithi118.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T03:09:03.494 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T03:09:03.494 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T03:09:03.494 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T03:09:03.494 INFO:teuthology.orchestra.run.smithi118.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T03:09:03.494 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T03:09:03.494 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T03:09:03.494 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T03:09:03.494 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T03:09:03.494 INFO:teuthology.orchestra.run.smithi118.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T03:09:03.495 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T03:09:03.495 INFO:teuthology.orchestra.run.smithi118.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T03:09:03.495 INFO:teuthology.orchestra.run.smithi118.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T03:09:03.495 INFO:teuthology.orchestra.run.smithi118.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T03:09:03.495 INFO:teuthology.orchestra.run.smithi118.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T03:09:03.495 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T03:09:03.495 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T03:09:03.495 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T03:09:03.495 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T03:09:03.495 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T03:09:03.495 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T03:09:03.496 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T03:09:03.496 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T03:09:03.496 INFO:teuthology.orchestra.run.smithi118.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T03:09:03.496 INFO:teuthology.orchestra.run.smithi118.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T03:09:03.496 INFO:teuthology.orchestra.run.smithi118.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T03:09:03.496 INFO:teuthology.orchestra.run.smithi118.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T03:09:03.496 INFO:teuthology.orchestra.run.smithi118.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T03:09:03.496 INFO:teuthology.orchestra.run.smithi118.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T03:09:03.496 INFO:teuthology.orchestra.run.smithi118.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T03:09:03.496 INFO:teuthology.orchestra.run.smithi118.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T03:09:03.496 INFO:teuthology.orchestra.run.smithi118.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T03:09:03.497 INFO:teuthology.orchestra.run.smithi118.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T03:09:03.497 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T03:09:03.497 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T03:09:03.497 INFO:teuthology.orchestra.run.smithi118.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T03:09:03.497 INFO:teuthology.orchestra.run.smithi118.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T03:09:03.497 INFO:teuthology.orchestra.run.smithi118.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T03:09:03.497 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T03:09:03.497 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T03:09:03.497 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T03:09:03.497 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T03:09:03.497 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T03:09:03.497 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T03:09:03.498 INFO:teuthology.orchestra.run.smithi118.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T03:09:03.498 INFO:teuthology.orchestra.run.smithi118.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T03:09:03.498 INFO:teuthology.orchestra.run.smithi118.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T03:09:03.498 INFO:teuthology.orchestra.run.smithi118.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T03:09:03.498 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T03:09:03.498 INFO:teuthology.orchestra.run.smithi118.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T03:09:03.498 INFO:teuthology.orchestra.run.smithi118.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T03:09:03.498 INFO:teuthology.orchestra.run.smithi118.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T03:09:03.498 INFO:teuthology.orchestra.run.smithi118.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T03:09:03.498 INFO:teuthology.orchestra.run.smithi118.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T03:09:03.498 INFO:teuthology.orchestra.run.smithi118.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T03:09:03.499 INFO:teuthology.orchestra.run.smithi118.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T03:09:03.499 INFO:teuthology.orchestra.run.smithi118.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T03:09:03.499 INFO:teuthology.orchestra.run.smithi118.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T03:09:03.499 INFO:teuthology.orchestra.run.smithi118.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T03:09:03.499 INFO:teuthology.orchestra.run.smithi118.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T03:09:03.499 INFO:teuthology.orchestra.run.smithi118.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T03:09:03.499 INFO:teuthology.orchestra.run.smithi118.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T03:09:03.499 INFO:teuthology.orchestra.run.smithi118.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T03:09:03.499 INFO:teuthology.orchestra.run.smithi118.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T03:09:03.499 INFO:teuthology.orchestra.run.smithi118.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T03:09:03.499 INFO:teuthology.orchestra.run.smithi118.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T03:09:03.500 INFO:teuthology.orchestra.run.smithi118.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T03:09:03.500 INFO:teuthology.orchestra.run.smithi118.stdout:Installing weak dependencies: 2024-09-12T03:09:03.500 INFO:teuthology.orchestra.run.smithi118.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T03:09:03.500 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-12T03:09:03.500 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-09-12T03:09:03.500 INFO:teuthology.orchestra.run.smithi118.stdout:====================================================================================== 2024-09-12T03:09:03.500 INFO:teuthology.orchestra.run.smithi118.stdout:Install 148 Packages 2024-09-12T03:09:03.500 INFO:teuthology.orchestra.run.smithi118.stdout:Upgrade 8 Packages 2024-09-12T03:09:03.500 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-12T03:09:03.501 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 226 M 2024-09-12T03:09:03.501 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-09-12T03:09:03.559 INFO:teuthology.orchestra.run.smithi161.stdout:(70/156): postgresql-server-13.16-1.el9.x86_64. 8.9 MB/s | 5.8 MB 00:00 2024-09-12T03:09:03.684 INFO:teuthology.orchestra.run.smithi161.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-09-12T03:09:03.762 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-09-12T03:09:03.801 INFO:teuthology.orchestra.run.smithi161.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 750 kB/s | 87 kB 00:00 2024-09-12T03:09:03.926 INFO:teuthology.orchestra.run.smithi161.stdout:(73/156): postgresql-contrib-13.16-1.el9.x86_64 732 kB/s | 910 kB 00:01 2024-09-12T03:09:03.928 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-09-12T03:09:03.928 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-09-12T03:09:04.033 INFO:teuthology.orchestra.run.smithi118.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 50 kB/s | 6.5 kB 00:00 2024-09-12T03:09:04.035 INFO:teuthology.orchestra.run.smithi161.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 567 kB/s | 61 kB 00:00 2024-09-12T03:09:04.333 INFO:teuthology.orchestra.run.smithi118.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.9 MB/s | 897 kB 00:00 2024-09-12T03:09:04.410 INFO:teuthology.orchestra.run.smithi161.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 653 kB/s | 245 kB 00:00 2024-09-12T03:09:04.484 INFO:teuthology.orchestra.run.smithi118.stdout:(3/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 9.3 MB/s | 5.4 MB 00:00 2024-09-12T03:09:04.509 INFO:teuthology.orchestra.run.smithi118.stdout:(4/156): ceph-immutable-object-cache-19.1.1-298 828 kB/s | 145 kB 00:00 2024-09-12T03:09:04.584 INFO:teuthology.orchestra.run.smithi118.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 24 MB/s | 2.4 MB 00:00 2024-09-12T03:09:04.785 INFO:teuthology.orchestra.run.smithi161.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 662 kB/s | 248 kB 00:00 2024-09-12T03:09:04.859 INFO:teuthology.orchestra.run.smithi118.stdout:(6/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 17 MB/s | 4.7 MB 00:00 2024-09-12T03:09:04.885 INFO:teuthology.orchestra.run.smithi161.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-09-12T03:09:04.918 INFO:teuthology.orchestra.run.smithi118.stdout:(7/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 3.7 MB/s | 1.5 MB 00:00 2024-09-12T03:09:04.994 INFO:teuthology.orchestra.run.smithi161.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 177 kB/s | 19 kB 00:00 2024-09-12T03:09:05.135 INFO:teuthology.orchestra.run.smithi118.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 17 MB/s | 21 MB 00:01 2024-09-12T03:09:05.177 INFO:teuthology.orchestra.run.smithi118.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 610 kB/s | 25 kB 00:00 2024-09-12T03:09:05.278 INFO:teuthology.orchestra.run.smithi161.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 608 kB/s | 172 kB 00:00 2024-09-12T03:09:05.345 INFO:teuthology.orchestra.run.smithi161.stdout:(80/156): openblas-openmp-0.3.26-2.el9.x86_64.r 1.4 MB/s | 4.9 MB 00:03 2024-09-12T03:09:05.387 INFO:teuthology.orchestra.run.smithi161.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-09-12T03:09:05.754 INFO:teuthology.orchestra.run.smithi161.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 683 kB/s | 250 kB 00:00 2024-09-12T03:09:06.120 INFO:teuthology.orchestra.run.smithi161.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 606 kB/s | 222 kB 00:00 2024-09-12T03:09:06.144 INFO:teuthology.orchestra.run.smithi118.stdout:(10/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 13 MB/s | 17 MB 00:01 2024-09-12T03:09:06.211 INFO:teuthology.orchestra.run.smithi118.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.4 MB/s | 228 kB 00:00 2024-09-12T03:09:06.261 INFO:teuthology.orchestra.run.smithi118.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 542 kB/s | 27 kB 00:00 2024-09-12T03:09:06.285 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-09-12T03:09:06.285 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-09-12T03:09:06.312 INFO:teuthology.orchestra.run.smithi161.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 403 kB/s | 77 kB 00:00 2024-09-12T03:09:06.387 INFO:teuthology.orchestra.run.smithi118.stdout:(13/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 7.2 MB/s | 11 MB 00:01 2024-09-12T03:09:06.412 INFO:teuthology.orchestra.run.smithi118.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 211 kB/s | 32 kB 00:00 2024-09-12T03:09:06.429 INFO:teuthology.orchestra.run.smithi161.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 363 kB/s | 42 kB 00:00 2024-09-12T03:09:06.462 INFO:teuthology.orchestra.run.smithi118.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 3.2 MB/s | 163 kB 00:00 2024-09-12T03:09:06.496 INFO:teuthology.orchestra.run.smithi118.stdout:(16/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 6.8 MB/s | 752 kB 00:00 2024-09-12T03:09:06.529 INFO:teuthology.orchestra.run.smithi118.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.8 MB/s | 127 kB 00:00 2024-09-12T03:09:06.580 INFO:teuthology.orchestra.run.smithi118.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 5.9 MB/s | 506 kB 00:00 2024-09-12T03:09:06.655 INFO:teuthology.orchestra.run.smithi118.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 600 kB/s | 45 kB 00:00 2024-09-12T03:09:06.731 INFO:teuthology.orchestra.run.smithi118.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.8 MB/s | 136 kB 00:00 2024-09-12T03:09:06.888 INFO:teuthology.orchestra.run.smithi161.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 588 kB/s | 269 kB 00:00 2024-09-12T03:09:07.164 INFO:teuthology.orchestra.run.smithi118.stdout:(21/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 25 MB/s | 49 MB 00:01 2024-09-12T03:09:07.180 INFO:teuthology.orchestra.run.smithi161.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 544 kB/s | 159 kB 00:00 2024-09-12T03:09:07.189 INFO:teuthology.orchestra.run.smithi118.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 353 kB/s | 162 kB 00:00 2024-09-12T03:09:07.216 INFO:teuthology.orchestra.run.smithi118.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 6.3 MB/s | 321 kB 00:00 2024-09-12T03:09:07.290 INFO:teuthology.orchestra.run.smithi118.stdout:(24/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 7.0 MB/s | 5.3 MB 00:00 2024-09-12T03:09:07.315 INFO:teuthology.orchestra.run.smithi118.stdout:(25/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 2.3 MB/s | 301 kB 00:00 2024-09-12T03:09:07.341 INFO:teuthology.orchestra.run.smithi118.stdout:(26/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 793 kB/s | 100 kB 00:00 2024-09-12T03:09:07.366 INFO:teuthology.orchestra.run.smithi118.stdout:(27/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.1 MB/s | 85 kB 00:00 2024-09-12T03:09:07.413 INFO:teuthology.orchestra.run.smithi118.stdout:(28/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.3 MB/s | 171 kB 00:00 2024-09-12T03:09:07.438 INFO:teuthology.orchestra.run.smithi118.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 445 kB/s | 32 kB 00:00 2024-09-12T03:09:07.471 INFO:teuthology.orchestra.run.smithi118.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.4 MB/s | 146 kB 00:00 2024-09-12T03:09:07.630 INFO:teuthology.orchestra.run.smithi118.stdout:(31/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 9.9 MB/s | 3.1 MB 00:00 2024-09-12T03:09:07.647 INFO:teuthology.orchestra.run.smithi161.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 598 kB/s | 279 kB 00:00 2024-09-12T03:09:07.714 INFO:teuthology.orchestra.run.smithi118.stdout:(32/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 13 MB/s | 3.6 MB 00:00 2024-09-12T03:09:07.747 INFO:teuthology.orchestra.run.smithi118.stdout:(33/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.1 MB/s | 249 kB 00:00 2024-09-12T03:09:07.755 INFO:teuthology.orchestra.run.smithi161.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 471 kB/s | 51 kB 00:00 2024-09-12T03:09:07.773 INFO:teuthology.orchestra.run.smithi118.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 841 kB/s | 49 kB 00:00 2024-09-12T03:09:07.798 INFO:teuthology.orchestra.run.smithi118.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 326 kB/s | 16 kB 00:00 2024-09-12T03:09:07.832 INFO:teuthology.orchestra.run.smithi118.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.7 MB/s | 285 kB 00:00 2024-09-12T03:09:07.864 INFO:teuthology.orchestra.run.smithi161.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 496 kB/s | 54 kB 00:00 2024-09-12T03:09:07.916 INFO:teuthology.orchestra.run.smithi118.stdout:(37/156): ceph-mgr-diskprediction-local-19.1.1- 17 MB/s | 7.4 MB 00:00 2024-09-12T03:09:07.941 INFO:teuthology.orchestra.run.smithi118.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 168 kB/s | 18 kB 00:00 2024-09-12T03:09:07.974 INFO:teuthology.orchestra.run.smithi118.stdout:(39/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 4.2 MB/s | 765 kB 00:00 2024-09-12T03:09:08.058 INFO:teuthology.orchestra.run.smithi118.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 562 kB/s | 80 kB 00:00 2024-09-12T03:09:08.092 INFO:teuthology.orchestra.run.smithi118.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.6 MB/s | 191 kB 00:00 2024-09-12T03:09:08.142 INFO:teuthology.orchestra.run.smithi118.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-12T03:09:08.192 INFO:teuthology.orchestra.run.smithi118.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.1 MB/s | 801 kB 00:00 2024-09-12T03:09:08.217 INFO:teuthology.orchestra.run.smithi118.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-12T03:09:08.260 INFO:teuthology.orchestra.run.smithi118.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-09-12T03:09:08.285 INFO:teuthology.orchestra.run.smithi118.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-09-12T03:09:08.327 INFO:teuthology.orchestra.run.smithi118.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 5.2 MB/s | 1.2 MB 00:00 2024-09-12T03:09:08.352 INFO:teuthology.orchestra.run.smithi118.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-12T03:09:08.486 INFO:teuthology.orchestra.run.smithi118.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-09-12T03:09:08.544 INFO:teuthology.orchestra.run.smithi118.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 557 kB/s | 107 kB 00:00 2024-09-12T03:09:08.561 INFO:teuthology.orchestra.run.smithi118.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 288 kB/s | 22 kB 00:00 2024-09-12T03:09:08.620 INFO:teuthology.orchestra.run.smithi118.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 561 kB/s | 33 kB 00:00 2024-09-12T03:09:08.647 INFO:teuthology.orchestra.run.smithi161.stdout:(91/156): python3-babel-2.9.1-2.el9.noarch.rpm 1.2 MB/s | 6.0 MB 00:04 2024-09-12T03:09:08.653 INFO:teuthology.orchestra.run.smithi118.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-12T03:09:08.712 INFO:teuthology.orchestra.run.smithi118.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 2.4 MB/s | 943 kB 00:00 2024-09-12T03:09:08.738 INFO:teuthology.orchestra.run.smithi118.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 217 kB/s | 18 kB 00:00 2024-09-12T03:09:08.773 INFO:teuthology.orchestra.run.smithi161.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 333 kB/s | 42 kB 00:00 2024-09-12T03:09:08.796 INFO:teuthology.orchestra.run.smithi118.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-09-12T03:09:08.821 INFO:teuthology.orchestra.run.smithi118.stdout:(57/156): libaio-devel-0.3.111-13.el9.x86_64.rp 102 kB/s | 11 kB 00:00 2024-09-12T03:09:08.847 INFO:teuthology.orchestra.run.smithi118.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-09-12T03:09:08.872 INFO:teuthology.orchestra.run.smithi118.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 601 kB/s | 45 kB 00:00 2024-09-12T03:09:08.881 INFO:teuthology.orchestra.run.smithi161.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 408 kB/s | 44 kB 00:00 2024-09-12T03:09:08.931 INFO:teuthology.orchestra.run.smithi118.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-09-12T03:09:08.956 INFO:teuthology.orchestra.run.smithi118.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 291 kB/s | 24 kB 00:00 2024-09-12T03:09:08.990 INFO:teuthology.orchestra.run.smithi118.stdout:(62/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.8 MB/s | 662 kB 00:00 2024-09-12T03:09:09.015 INFO:teuthology.orchestra.run.smithi118.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 564 kB/s | 47 kB 00:00 2024-09-12T03:09:09.049 INFO:teuthology.orchestra.run.smithi118.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-12T03:09:09.083 INFO:teuthology.orchestra.run.smithi118.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-09-12T03:09:09.108 INFO:teuthology.orchestra.run.smithi118.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 416 kB/s | 38 kB 00:00 2024-09-12T03:09:09.184 INFO:teuthology.orchestra.run.smithi118.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 37 MB/s | 4.9 MB 00:00 2024-09-12T03:09:09.225 INFO:teuthology.orchestra.run.smithi118.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-12T03:09:09.256 INFO:teuthology.orchestra.run.smithi161.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 589 kB/s | 221 kB 00:00 2024-09-12T03:09:09.267 INFO:teuthology.orchestra.run.smithi118.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 11 MB/s | 910 kB 00:00 2024-09-12T03:09:09.293 INFO:teuthology.orchestra.run.smithi118.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-12T03:09:09.401 INFO:teuthology.orchestra.run.smithi118.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-09-12T03:09:09.415 INFO:teuthology.orchestra.run.smithi161.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-09-12T03:09:09.440 INFO:teuthology.orchestra.run.smithi161.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 195 kB/s | 36 kB 00:00 2024-09-12T03:09:09.485 INFO:teuthology.orchestra.run.smithi118.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 27 MB/s | 5.8 MB 00:00 2024-09-12T03:09:09.524 INFO:teuthology.orchestra.run.smithi161.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-09-12T03:09:09.543 INFO:teuthology.orchestra.run.smithi118.stdout:(73/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 9.6 MB/s | 4.4 MB 00:00 2024-09-12T03:09:09.569 INFO:teuthology.orchestra.run.smithi118.stdout:(74/156): python3-attrs-20.3.0-7.el9.noarch.rpm 523 kB/s | 87 kB 00:00 2024-09-12T03:09:09.616 INFO:teuthology.orchestra.run.smithi161.stdout:(98/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.2 MB/s | 5.1 MB 00:04 2024-09-12T03:09:09.650 INFO:teuthology.orchestra.run.smithi161.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 461 kB/s | 58 kB 00:00 2024-09-12T03:09:09.669 INFO:teuthology.orchestra.run.smithi118.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 6.0 MB 00:00 2024-09-12T03:09:09.695 INFO:teuthology.orchestra.run.smithi118.stdout:(76/156): python3-colorama-0.4.6-3.el9.noarch.r 407 kB/s | 61 kB 00:00 2024-09-12T03:09:09.719 INFO:teuthology.orchestra.run.smithi118.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.6 MB/s | 245 kB 00:00 2024-09-12T03:09:09.733 INFO:teuthology.orchestra.run.smithi161.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 819 kB/s | 96 kB 00:00 2024-09-12T03:09:09.745 INFO:teuthology.orchestra.run.smithi118.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-09-12T03:09:09.758 INFO:teuthology.orchestra.run.smithi161.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 586 kB/s | 64 kB 00:00 2024-09-12T03:09:09.770 INFO:teuthology.orchestra.run.smithi118.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 339 kB/s | 26 kB 00:00 2024-09-12T03:09:09.795 INFO:teuthology.orchestra.run.smithi118.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 253 kB/s | 19 kB 00:00 2024-09-12T03:09:09.821 INFO:teuthology.orchestra.run.smithi118.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-12T03:09:09.838 INFO:teuthology.orchestra.run.smithi118.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 516 kB/s | 35 kB 00:00 2024-09-12T03:09:09.842 INFO:teuthology.orchestra.run.smithi161.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 413 kB/s | 45 kB 00:00 2024-09-12T03:09:09.859 INFO:teuthology.orchestra.run.smithi161.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-09-12T03:09:09.896 INFO:teuthology.orchestra.run.smithi118.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.2 MB/s | 250 kB 00:00 2024-09-12T03:09:09.918 INFO:teuthology.orchestra.run.smithi161.stdout:(104/156): socat-1.7.4.1-6.el9.x86_64.rpm 638 kB/s | 304 kB 00:00 2024-09-12T03:09:09.930 INFO:teuthology.orchestra.run.smithi118.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-09-12T03:09:09.943 INFO:teuthology.orchestra.run.smithi161.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-12T03:09:09.977 INFO:teuthology.orchestra.run.smithi161.stdout:(106/156): gperftools-libs-2.9.1-3.el9.x86_64.r 5.1 MB/s | 308 kB 00:00 2024-09-12T03:09:09.993 INFO:teuthology.orchestra.run.smithi161.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 388 kB/s | 19 kB 00:00 2024-09-12T03:09:09.997 INFO:teuthology.orchestra.run.smithi118.stdout:(85/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 5.1 MB 00:00 2024-09-12T03:09:10.010 INFO:teuthology.orchestra.run.smithi161.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-12T03:09:10.014 INFO:teuthology.orchestra.run.smithi118.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 505 kB/s | 42 kB 00:00 2024-09-12T03:09:10.027 INFO:teuthology.orchestra.run.smithi161.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.9 MB/s | 49 kB 00:00 2024-09-12T03:09:10.039 INFO:teuthology.orchestra.run.smithi118.stdout:(87/156): python3-packaging-20.9-5.el9.noarch.r 541 kB/s | 77 kB 00:00 2024-09-12T03:09:10.069 INFO:teuthology.orchestra.run.smithi161.stdout:(110/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.7 MB/s | 863 kB 00:00 2024-09-12T03:09:10.073 INFO:teuthology.orchestra.run.smithi118.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-09-12T03:09:10.095 INFO:teuthology.orchestra.run.smithi161.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-09-12T03:09:10.098 INFO:teuthology.orchestra.run.smithi118.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 159 kB 00:00 2024-09-12T03:09:10.124 INFO:teuthology.orchestra.run.smithi118.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.2 MB/s | 279 kB 00:00 2024-09-12T03:09:10.128 INFO:teuthology.orchestra.run.smithi161.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-12T03:09:10.149 INFO:teuthology.orchestra.run.smithi118.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-09-12T03:09:10.170 INFO:teuthology.orchestra.run.smithi161.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-12T03:09:10.174 INFO:teuthology.orchestra.run.smithi118.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-09-12T03:09:10.187 INFO:teuthology.orchestra.run.smithi161.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-12T03:09:10.204 INFO:teuthology.orchestra.run.smithi161.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-12T03:09:10.221 INFO:teuthology.orchestra.run.smithi161.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 855 kB/s | 14 kB 00:00 2024-09-12T03:09:10.224 INFO:teuthology.orchestra.run.smithi118.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 554 kB/s | 42 kB 00:00 2024-09-12T03:09:10.250 INFO:teuthology.orchestra.run.smithi118.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 587 kB/s | 44 kB 00:00 2024-09-12T03:09:10.254 INFO:teuthology.orchestra.run.smithi161.stdout:(117/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.1 MB/s | 839 kB 00:00 2024-09-12T03:09:10.280 INFO:teuthology.orchestra.run.smithi161.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-12T03:09:10.292 INFO:teuthology.orchestra.run.smithi118.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-12T03:09:10.313 INFO:teuthology.orchestra.run.smithi161.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-12T03:09:10.317 INFO:teuthology.orchestra.run.smithi118.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 537 kB/s | 36 kB 00:00 2024-09-12T03:09:10.339 INFO:teuthology.orchestra.run.smithi161.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-12T03:09:10.367 INFO:teuthology.orchestra.run.smithi118.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 630 kB/s | 47 kB 00:00 2024-09-12T03:09:10.372 INFO:teuthology.orchestra.run.smithi161.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-12T03:09:10.398 INFO:teuthology.orchestra.run.smithi161.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-12T03:09:10.423 INFO:teuthology.orchestra.run.smithi161.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-09-12T03:09:10.448 INFO:teuthology.orchestra.run.smithi161.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 710 kB/s | 18 kB 00:00 2024-09-12T03:09:10.473 INFO:teuthology.orchestra.run.smithi161.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 927 kB/s | 23 kB 00:00 2024-09-12T03:09:10.499 INFO:teuthology.orchestra.run.smithi161.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 785 kB/s | 19 kB 00:00 2024-09-12T03:09:10.524 INFO:teuthology.orchestra.run.smithi161.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 789 kB/s | 20 kB 00:00 2024-09-12T03:09:10.543 INFO:teuthology.orchestra.run.smithi118.stdout:(98/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2024-09-12T03:09:10.568 INFO:teuthology.orchestra.run.smithi118.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 289 kB/s | 58 kB 00:00 2024-09-12T03:09:10.583 INFO:teuthology.orchestra.run.smithi161.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 18 MB/s | 1.0 MB 00:00 2024-09-12T03:09:10.593 INFO:teuthology.orchestra.run.smithi118.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 1.1 MB/s | 304 kB 00:00 2024-09-12T03:09:10.608 INFO:teuthology.orchestra.run.smithi161.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T03:09:10.618 INFO:teuthology.orchestra.run.smithi118.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.2 MB/s | 96 kB 00:00 2024-09-12T03:09:10.644 INFO:teuthology.orchestra.run.smithi118.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 841 kB/s | 64 kB 00:00 2024-09-12T03:09:10.669 INFO:teuthology.orchestra.run.smithi118.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 594 kB/s | 45 kB 00:00 2024-09-12T03:09:10.675 INFO:teuthology.orchestra.run.smithi161.stdout:(130/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.3 MB/s | 4.4 MB 00:00 2024-09-12T03:09:10.717 INFO:teuthology.orchestra.run.smithi161.stdout:(131/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.4 MB/s | 2.0 MB 00:00 2024-09-12T03:09:10.719 INFO:teuthology.orchestra.run.smithi118.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-12T03:09:10.742 INFO:teuthology.orchestra.run.smithi161.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 588 kB/s | 79 kB 00:00 2024-09-12T03:09:10.745 INFO:teuthology.orchestra.run.smithi118.stdout:(105/156): lua-devel-5.4.4-4.el9.x86_64.rpm 177 kB/s | 22 kB 00:00 2024-09-12T03:09:10.759 INFO:teuthology.orchestra.run.smithi161.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 689 kB/s | 58 kB 00:00 2024-09-12T03:09:10.785 INFO:teuthology.orchestra.run.smithi161.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-12T03:09:10.786 INFO:teuthology.orchestra.run.smithi118.stdout:(106/156): gperftools-libs-2.9.1-3.el9.x86_64.r 4.5 MB/s | 308 kB 00:00 2024-09-12T03:09:10.810 INFO:teuthology.orchestra.run.smithi161.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-12T03:09:10.812 INFO:teuthology.orchestra.run.smithi118.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 290 kB/s | 19 kB 00:00 2024-09-12T03:09:10.835 INFO:teuthology.orchestra.run.smithi161.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-12T03:09:10.837 INFO:teuthology.orchestra.run.smithi118.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-12T03:09:10.860 INFO:teuthology.orchestra.run.smithi161.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-12T03:09:10.862 INFO:teuthology.orchestra.run.smithi118.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T03:09:10.885 INFO:teuthology.orchestra.run.smithi161.stdout:(138/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-09-12T03:09:10.887 INFO:teuthology.orchestra.run.smithi118.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-12T03:09:10.902 INFO:teuthology.orchestra.run.smithi161.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-09-12T03:09:10.913 INFO:teuthology.orchestra.run.smithi118.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-12T03:09:10.919 INFO:teuthology.orchestra.run.smithi161.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-09-12T03:09:10.945 INFO:teuthology.orchestra.run.smithi161.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-09-12T03:09:10.970 INFO:teuthology.orchestra.run.smithi161.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-12T03:09:10.972 INFO:teuthology.orchestra.run.smithi118.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-12T03:09:10.995 INFO:teuthology.orchestra.run.smithi161.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-12T03:09:11.005 INFO:teuthology.orchestra.run.smithi118.stdout:(113/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-09-12T03:09:11.012 INFO:teuthology.orchestra.run.smithi161.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-12T03:09:11.037 INFO:teuthology.orchestra.run.smithi161.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-12T03:09:11.071 INFO:teuthology.orchestra.run.smithi161.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-12T03:09:11.081 INFO:teuthology.orchestra.run.smithi118.stdout:(114/156): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-12T03:09:11.096 INFO:teuthology.orchestra.run.smithi161.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-12T03:09:11.106 INFO:teuthology.orchestra.run.smithi118.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.4 MB/s | 32 kB 00:00 2024-09-12T03:09:11.132 INFO:teuthology.orchestra.run.smithi118.stdout:(116/156): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-09-12T03:09:11.146 INFO:teuthology.orchestra.run.smithi161.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-12T03:09:11.157 INFO:teuthology.orchestra.run.smithi118.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-12T03:09:11.182 INFO:teuthology.orchestra.run.smithi118.stdout:(118/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 246 kB/s | 43 kB 00:00 2024-09-12T03:09:11.207 INFO:teuthology.orchestra.run.smithi118.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-09-12T03:09:11.233 INFO:teuthology.orchestra.run.smithi118.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-09-12T03:09:11.266 INFO:teuthology.orchestra.run.smithi118.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-09-12T03:09:11.292 INFO:teuthology.orchestra.run.smithi118.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.6 MB/s | 220 kB 00:00 2024-09-12T03:09:11.317 INFO:teuthology.orchestra.run.smithi118.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-12T03:09:11.334 INFO:teuthology.orchestra.run.smithi118.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 255 kB/s | 11 kB 00:00 2024-09-12T03:09:11.359 INFO:teuthology.orchestra.run.smithi118.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 423 kB/s | 18 kB 00:00 2024-09-12T03:09:11.384 INFO:teuthology.orchestra.run.smithi118.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-12T03:09:11.410 INFO:teuthology.orchestra.run.smithi118.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-12T03:09:11.435 INFO:teuthology.orchestra.run.smithi118.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-12T03:09:11.460 INFO:teuthology.orchestra.run.smithi118.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T03:09:11.486 INFO:teuthology.orchestra.run.smithi118.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T03:09:11.497 INFO:teuthology.orchestra.run.smithi161.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 6.6 MB/s | 2.3 MB 00:00 2024-09-12T03:09:11.511 INFO:teuthology.orchestra.run.smithi118.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-12T03:09:11.553 INFO:teuthology.orchestra.run.smithi118.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 6.4 MB/s | 272 kB 00:00 2024-09-12T03:09:11.572 INFO:teuthology.orchestra.run.smithi161.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 6.6 MB/s | 3.2 MB 00:00 2024-09-12T03:09:11.586 INFO:teuthology.orchestra.run.smithi118.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-09-12T03:09:11.604 INFO:teuthology.orchestra.run.smithi118.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-12T03:09:11.621 INFO:teuthology.orchestra.run.smithi118.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.7 MB/s | 28 kB 00:00 2024-09-12T03:09:11.630 INFO:teuthology.orchestra.run.smithi161.stdout:(151/156): librados2-19.1.1-298.gb53b6d3e.el9.x 6.1 MB/s | 3.4 MB 00:00 2024-09-12T03:09:11.646 INFO:teuthology.orchestra.run.smithi118.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-12T03:09:11.671 INFO:teuthology.orchestra.run.smithi118.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-12T03:09:11.672 INFO:teuthology.orchestra.run.smithi161.stdout:(152/156): NetworkManager-libnm-1.49.90-1.el9.x 10 MB/s | 1.8 MB 00:00 2024-09-12T03:09:11.696 INFO:teuthology.orchestra.run.smithi118.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T03:09:11.722 INFO:teuthology.orchestra.run.smithi118.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-12T03:09:11.747 INFO:teuthology.orchestra.run.smithi118.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-12T03:09:11.748 INFO:teuthology.orchestra.run.smithi161.stdout:(153/156): NetworkManager-team-1.49.90-1.el9.x8 232 kB/s | 41 kB 00:00 2024-09-12T03:09:11.772 INFO:teuthology.orchestra.run.smithi118.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-12T03:09:11.790 INFO:teuthology.orchestra.run.smithi161.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 12 MB/s | 1.4 MB 00:00 2024-09-12T03:09:11.798 INFO:teuthology.orchestra.run.smithi118.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-12T03:09:11.823 INFO:teuthology.orchestra.run.smithi118.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-12T03:09:11.856 INFO:teuthology.orchestra.run.smithi161.stdout:(155/156): NetworkManager-tui-1.49.90-1.el9.x86 1.1 MB/s | 249 kB 00:00 2024-09-12T03:09:11.865 INFO:teuthology.orchestra.run.smithi118.stdout:(144/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.4 MB/s | 2.0 MB 00:00 2024-09-12T03:09:11.891 INFO:teuthology.orchestra.run.smithi118.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-09-12T03:09:11.924 INFO:teuthology.orchestra.run.smithi118.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-12T03:09:11.950 INFO:teuthology.orchestra.run.smithi118.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-09-12T03:09:11.982 INFO:teuthology.orchestra.run.smithi161.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 10 MB/s | 2.4 MB 00:00 2024-09-12T03:09:11.987 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:09:11.987 INFO:teuthology.orchestra.run.smithi161.stdout:Total 14 MB/s | 226 MB 00:16 2024-09-12T03:09:11.991 INFO:teuthology.orchestra.run.smithi118.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-12T03:09:12.250 INFO:teuthology.orchestra.run.smithi118.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-12T03:09:12.342 INFO:teuthology.orchestra.run.smithi118.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 8.1 MB/s | 3.2 MB 00:00 2024-09-12T03:09:12.418 INFO:teuthology.orchestra.run.smithi118.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 543 kB/s | 41 kB 00:00 2024-09-12T03:09:12.509 INFO:teuthology.orchestra.run.smithi118.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 2.7 MB/s | 249 kB 00:00 2024-09-12T03:09:12.651 INFO:teuthology.orchestra.run.smithi118.stdout:(153/156): NetworkManager-1.49.90-1.el9.x86_64. 3.6 MB/s | 2.3 MB 00:00 2024-09-12T03:09:12.727 INFO:teuthology.orchestra.run.smithi118.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 3.8 MB/s | 1.8 MB 00:00 2024-09-12T03:09:12.760 INFO:teuthology.orchestra.run.smithi118.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 5.4 MB/s | 1.4 MB 00:00 2024-09-12T03:09:12.771 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-09-12T03:09:12.879 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T03:09:12.919 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T03:09:12.972 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T03:09:13.013 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T03:09:13.038 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T03:09:13.093 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T03:09:13.163 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-09-12T03:09:13.314 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-09-12T03:09:13.314 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-09-12T03:09:13.344 INFO:teuthology.orchestra.run.smithi118.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.5 MB/s | 2.4 MB 00:00 2024-09-12T03:09:13.350 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:09:13.350 INFO:teuthology.orchestra.run.smithi118.stdout:Total 23 MB/s | 226 MB 00:09 2024-09-12T03:09:13.647 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T03:09:13.698 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:09:13.807 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:09:13.841 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:09:13.926 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:09:13.954 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:09:14.123 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T03:09:14.184 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T03:09:14.211 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:09:14.318 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:09:14.346 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:09:14.409 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:09:14.438 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:09:14.464 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:09:14.585 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-09-12T03:09:14.651 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:09:14.738 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-09-12T03:09:14.738 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-09-12T03:09:14.806 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:09:15.062 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T03:09:15.120 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-09-12T03:09:15.120 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-09-12T03:09:15.174 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:09:15.255 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:09:15.485 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:09:15.596 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T03:09:15.692 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T03:09:15.727 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T03:09:15.770 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T03:09:15.845 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T03:09:15.922 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T03:09:15.962 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T03:09:16.005 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T03:09:16.038 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T03:09:16.065 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T03:09:16.136 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T03:09:16.188 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T03:09:16.218 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T03:09:16.262 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T03:09:16.340 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T03:09:16.372 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T03:09:16.417 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T03:09:16.444 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T03:09:16.517 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T03:09:16.537 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-09-12T03:09:16.537 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-09-12T03:09:16.559 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T03:09:16.637 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T03:09:16.782 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T03:09:16.837 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T03:09:17.395 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T03:09:17.485 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T03:09:17.519 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T03:09:17.562 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T03:09:17.600 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T03:09:17.651 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T03:09:17.810 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T03:09:17.870 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T03:09:17.926 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T03:09:17.999 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T03:09:18.037 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T03:09:18.073 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T03:09:18.121 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T03:09:18.156 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T03:09:18.200 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T03:09:18.242 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T03:09:18.401 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T03:09:18.443 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T03:09:18.485 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T03:09:18.761 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T03:09:18.793 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T03:09:18.834 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T03:09:18.866 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-12T03:09:18.891 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T03:09:18.985 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T03:09:19.116 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T03:09:19.157 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T03:09:19.185 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T03:09:19.236 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T03:09:19.275 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T03:09:19.315 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T03:09:19.345 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T03:09:19.379 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T03:09:19.412 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T03:09:19.454 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T03:09:19.488 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T03:09:19.582 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T03:09:19.631 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T03:09:19.670 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T03:09:19.778 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T03:09:19.830 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T03:09:20.595 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T03:09:20.643 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T03:09:20.677 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T03:09:20.781 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T03:09:20.840 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T03:09:20.885 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T03:09:20.977 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T03:09:21.012 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T03:09:21.203 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T03:09:21.235 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T03:09:21.321 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T03:09:21.464 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-09-12T03:09:21.560 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T03:09:21.607 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T03:09:21.669 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T03:09:21.703 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T03:09:21.728 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T03:09:21.771 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T03:09:22.014 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T03:09:22.139 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T03:09:22.162 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T03:09:22.271 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:09:22.377 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:09:22.405 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:09:22.496 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:09:22.534 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:09:22.695 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T03:09:22.748 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T03:09:22.776 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:09:22.858 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-09-12T03:09:22.880 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:09:22.910 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:09:22.964 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T03:09:22.965 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:09:22.986 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:09:23.002 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T03:09:23.006 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:09:23.056 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T03:09:23.098 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T03:09:23.196 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:09:23.228 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T03:09:23.322 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:09:23.370 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T03:09:23.437 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T03:09:23.467 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T03:09:23.526 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:09:23.572 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T03:09:23.600 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T03:09:23.605 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:09:23.709 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T03:09:23.725 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T03:09:23.825 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:09:23.833 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:09:23.940 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:09:23.962 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T03:09:23.968 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:09:24.052 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T03:09:24.058 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:09:24.059 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T03:09:24.089 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:09:24.095 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T03:09:24.095 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:09:24.156 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T03:09:24.196 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:09:24.225 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T03:09:24.228 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T03:09:24.236 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T03:09:24.262 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T03:09:24.278 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T03:09:24.280 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T03:09:24.306 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:09:24.320 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T03:09:24.364 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T03:09:24.392 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:09:24.396 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T03:09:24.415 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:09:24.424 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T03:09:24.478 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:09:24.500 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:09:24.503 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T03:09:24.521 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:09:24.718 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:09:24.732 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T03:09:24.767 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T03:09:24.807 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:09:24.835 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T03:09:24.856 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:09:24.878 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:09:24.879 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T03:09:24.908 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T03:09:24.957 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T03:09:24.990 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T03:09:25.033 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T03:09:25.062 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T03:09:25.134 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T03:09:25.185 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T03:09:25.269 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T03:09:25.273 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T03:09:25.374 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:09:25.414 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T03:09:25.450 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:09:25.471 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T03:09:25.688 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:09:25.798 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T03:09:25.901 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T03:09:25.930 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T03:09:25.969 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T03:09:26.008 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T03:09:26.034 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T03:09:26.078 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T03:09:26.092 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T03:09:26.110 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T03:09:26.120 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T03:09:26.154 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T03:09:26.163 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T03:09:26.187 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T03:09:26.201 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T03:09:26.223 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T03:09:26.243 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T03:09:26.292 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T03:09:26.345 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T03:09:26.376 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T03:09:26.420 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T03:09:26.421 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T03:09:26.487 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T03:09:26.496 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T03:09:26.531 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T03:09:26.543 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T03:09:26.574 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T03:09:26.603 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T03:09:26.624 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T03:09:26.655 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T03:09:26.681 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T03:09:26.691 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T03:09:26.734 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T03:09:26.738 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T03:09:26.773 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T03:09:26.809 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T03:09:26.817 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T03:09:26.859 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T03:09:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T03:09:26.997 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T03:09:27.004 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T03:09:27.044 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T03:09:27.195 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T03:09:27.299 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T03:09:27.352 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T03:09:27.367 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:09:27.392 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T03:09:27.405 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:09:27.417 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-12T03:09:27.442 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T03:09:27.517 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T03:09:27.537 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T03:09:27.616 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T03:09:27.646 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T03:09:27.648 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T03:09:27.688 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T03:09:27.693 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T03:09:27.721 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T03:09:27.727 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T03:09:27.755 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T03:09:27.768 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T03:09:27.803 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T03:09:27.843 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T03:09:27.872 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T03:09:27.915 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T03:09:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T03:09:27.964 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T03:09:27.996 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:09:28.007 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T03:09:28.018 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T03:09:28.033 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:09:28.050 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T03:09:28.073 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T03:09:28.135 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:09:28.135 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T03:09:28.135 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T03:09:28.135 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:09:28.138 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T03:09:28.154 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T03:09:28.176 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:09:28.180 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T03:09:28.195 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T03:09:28.220 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T03:09:28.231 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T03:09:28.277 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T03:09:28.313 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T03:09:28.333 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T03:09:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T03:09:28.396 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T03:09:28.400 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T03:09:28.595 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T03:09:28.636 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T03:09:28.678 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T03:09:28.787 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T03:09:28.828 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T03:09:28.868 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T03:09:28.894 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-12T03:09:28.919 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T03:09:28.993 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T03:09:29.130 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T03:09:29.136 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T03:09:29.160 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T03:09:29.176 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T03:09:29.196 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T03:09:29.210 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T03:09:29.239 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T03:09:29.278 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T03:09:29.310 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T03:09:29.312 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T03:09:29.340 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T03:09:29.365 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T03:09:29.374 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T03:09:29.408 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T03:09:29.410 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T03:09:29.450 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T03:09:29.484 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T03:09:29.499 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T03:09:29.537 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T03:09:29.571 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T03:09:29.623 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T03:09:29.654 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T03:09:29.721 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T03:09:29.751 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T03:09:29.757 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T03:09:29.806 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T03:09:29.829 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T03:09:30.497 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T03:09:30.531 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T03:09:30.578 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T03:09:30.620 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T03:09:30.626 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T03:09:30.721 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T03:09:30.775 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T03:09:30.821 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T03:09:30.908 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T03:09:30.940 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T03:09:31.133 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T03:09:31.190 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T03:09:31.283 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T03:09:31.892 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T03:09:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T03:09:31.980 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T03:09:32.008 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T03:09:32.041 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T03:09:32.053 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T03:09:32.363 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T03:09:32.404 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:09:32.495 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:09:32.525 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T03:09:32.561 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T03:09:33.039 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T03:09:33.065 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:09:33.136 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:09:33.166 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T03:09:33.275 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T03:09:33.385 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T03:09:33.413 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T03:09:33.446 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T03:09:33.754 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T03:09:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:09:33.883 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:09:33.931 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T03:09:33.967 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T03:09:34.453 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T03:09:34.513 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:09:34.584 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:09:34.606 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T03:09:35.511 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:09:35.555 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:09:36.102 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:09:36.133 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:09:36.222 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:09:36.222 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T03:09:36.222 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T03:09:36.223 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:09:36.251 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:09:36.932 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:09:36.962 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:09:37.502 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:09:37.549 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:09:37.642 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:09:37.642 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T03:09:37.642 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T03:09:37.642 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-12T03:09:37.683 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:09:40.749 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:09:40.749 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /sys 2024-09-12T03:09:40.749 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /proc 2024-09-12T03:09:40.749 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /mnt 2024-09-12T03:09:40.749 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /var/tmp 2024-09-12T03:09:40.749 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /home 2024-09-12T03:09:40.750 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /root 2024-09-12T03:09:40.750 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /tmp 2024-09-12T03:09:40.750 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:09:40.931 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:09:40.948 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:09:40.948 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:40.948 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T03:09:40.948 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:09:42.476 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:09:42.504 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:09:42.504 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:42.504 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T03:09:42.504 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:09:42.504 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:09:42.505 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:09:42.743 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:09:42.767 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:09:42.767 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:42.767 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T03:09:42.768 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:09:42.768 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:09:42.768 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:09:43.227 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:09:43.252 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:09:43.252 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:43.252 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T03:09:43.252 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:09:43.253 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:09:43.253 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:09:43.366 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T03:09:43.425 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T03:09:43.488 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T03:09:43.695 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T03:09:43.745 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T03:09:43.826 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T03:09:43.917 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T03:09:44.077 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T03:09:44.130 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T03:09:44.168 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T03:09:44.204 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T03:09:44.682 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:09:44.711 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:09:44.743 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T03:09:44.808 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T03:09:44.842 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:09:45.947 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:09:45.986 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:09:47.005 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:09:47.036 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:09:47.147 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:09:47.323 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T03:09:47.355 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:09:47.381 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:09:47.381 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:47.381 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T03:09:47.381 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:09:47.382 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:09:47.382 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:09:47.439 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:09:47.451 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:09:47.553 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T03:09:47.619 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T03:09:47.906 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:09:47.906 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /sys 2024-09-12T03:09:47.906 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /proc 2024-09-12T03:09:47.906 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /mnt 2024-09-12T03:09:47.906 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /var/tmp 2024-09-12T03:09:47.906 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /home 2024-09-12T03:09:47.906 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /root 2024-09-12T03:09:47.906 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /tmp 2024-09-12T03:09:47.906 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:09:48.095 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:09:48.110 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:09:48.110 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:48.110 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T03:09:48.110 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:09:48.596 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T03:09:48.625 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:09:48.650 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:09:48.650 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:48.650 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T03:09:48.650 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:09:48.650 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:09:48.650 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:09:48.692 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:09:48.717 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:09:48.717 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:48.717 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T03:09:48.717 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:09:48.768 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:09:48.792 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:09:48.793 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:48.793 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T03:09:48.793 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:09:48.793 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:09:48.793 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:09:49.110 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:09:49.135 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:09:49.135 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:49.135 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T03:09:49.135 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:09:49.135 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:09:49.135 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:09:49.664 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:09:49.690 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:09:49.690 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:49.690 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T03:09:49.691 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:09:49.691 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:09:49.691 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:09:49.837 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:09:49.837 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /sys 2024-09-12T03:09:49.837 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /proc 2024-09-12T03:09:49.837 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /mnt 2024-09-12T03:09:49.838 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /var/tmp 2024-09-12T03:09:49.838 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /home 2024-09-12T03:09:49.838 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /root 2024-09-12T03:09:49.838 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /tmp 2024-09-12T03:09:49.838 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-12T03:09:49.914 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:09:49.937 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:09:49.937 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:49.937 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T03:09:49.937 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:09:49.937 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:09:49.937 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:09:50.015 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:09:50.032 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:09:50.032 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:50.032 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T03:09:50.032 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-12T03:09:50.373 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:09:50.397 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:09:50.397 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:50.397 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T03:09:50.397 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:09:50.397 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:09:50.397 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:09:50.503 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T03:09:50.561 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T03:09:50.616 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T03:09:50.804 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T03:09:50.857 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T03:09:50.937 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T03:09:51.033 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T03:09:51.199 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T03:09:51.241 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T03:09:51.281 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T03:09:51.317 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T03:09:51.519 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:09:51.546 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:09:51.546 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:51.546 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T03:09:51.546 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:09:51.546 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:09:51.546 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-12T03:09:51.768 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:09:51.778 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:09:51.799 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:09:51.801 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:09:51.802 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:51.802 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T03:09:51.802 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:09:51.802 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:09:51.802 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-12T03:09:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T03:09:51.893 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T03:09:51.922 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:09:52.254 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:09:52.277 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:09:52.277 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:52.277 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T03:09:52.278 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:09:52.278 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:09:52.278 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-12T03:09:52.376 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T03:09:52.432 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T03:09:52.496 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T03:09:52.684 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T03:09:52.738 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T03:09:52.817 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T03:09:52.913 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T03:09:52.974 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:09:53.007 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:09:53.054 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T03:09:53.122 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T03:09:53.154 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T03:09:53.182 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T03:09:53.634 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:09:53.673 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:09:53.704 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T03:09:53.767 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T03:09:53.796 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:09:53.973 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T03:09:53.980 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:09:54.007 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:09:54.014 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T03:09:54.064 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T03:09:54.103 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T03:09:54.111 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:09:54.130 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T03:09:54.282 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T03:09:54.318 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:09:54.331 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T03:09:54.341 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:09:54.341 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:54.341 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T03:09:54.341 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:09:54.341 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:09:54.342 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:09:54.377 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T03:09:54.394 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:09:54.404 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:09:54.407 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T03:09:54.437 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T03:09:54.492 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T03:09:54.503 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T03:09:54.551 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T03:09:54.844 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:09:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:09:55.078 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:09:55.114 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:09:55.136 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:09:55.489 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T03:09:55.529 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:09:55.552 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:09:55.552 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:55.553 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T03:09:55.553 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:09:55.553 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:09:55.553 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:09:55.605 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:09:55.627 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:09:55.628 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:55.628 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T03:09:55.628 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:09:55.672 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:09:55.695 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:09:55.695 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:55.695 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T03:09:55.696 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:09:55.696 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:09:55.696 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:09:55.850 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:09:55.890 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:09:55.991 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:09:55.998 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:09:56.021 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:09:56.021 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:56.021 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T03:09:56.021 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:09:56.021 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:09:56.021 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:09:56.045 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T03:09:56.127 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T03:09:56.159 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T03:09:56.173 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T03:09:56.209 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:09:56.233 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:09:56.233 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:56.233 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T03:09:56.233 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:09:56.233 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:09:56.233 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-12T03:09:56.262 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T03:09:56.284 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:09:56.297 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:09:56.297 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T03:09:56.322 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T03:09:56.356 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T03:09:56.357 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:09:56.394 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T03:09:56.441 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T03:09:56.449 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:09:56.449 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:09:56.533 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:09:56.566 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T03:09:56.572 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:09:56.573 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:09:56.760 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:09:56.760 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:09:56.844 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:09:56.921 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:09:57.378 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T03:09:57.412 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:09:57.436 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:09:57.436 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:57.436 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T03:09:57.436 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:09:57.436 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:09:57.437 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-12T03:09:57.480 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:09:57.503 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:09:57.503 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:57.503 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T03:09:57.503 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-12T03:09:57.555 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:09:57.580 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:09:57.580 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:57.580 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T03:09:57.580 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:09:57.581 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:09:57.581 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-12T03:09:57.881 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:09:57.905 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:09:57.905 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:09:57.905 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T03:09:57.905 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:09:57.905 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:09:57.905 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-12T03:10:00.618 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T03:10:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T03:10:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T03:10:00.732 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T03:10:00.768 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T03:10:00.956 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T03:10:01.015 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T03:10:01.045 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T03:10:01.074 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T03:10:01.121 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T03:10:01.754 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:10:01.793 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:10:01.814 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:10:02.443 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T03:10:02.485 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T03:10:02.525 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T03:10:02.558 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T03:10:02.585 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T03:10:02.594 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:10:02.594 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T03:10:02.594 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T03:10:02.594 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T03:10:02.594 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T03:10:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T03:10:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T03:10:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T03:10:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:10:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:10:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:10:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T03:10:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T03:10:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:10:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T03:10:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T03:10:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T03:10:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T03:10:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T03:10:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T03:10:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T03:10:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T03:10:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T03:10:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T03:10:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T03:10:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T03:10:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T03:10:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T03:10:02.610 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T03:10:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T03:10:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T03:10:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T03:10:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T03:10:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T03:10:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T03:10:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T03:10:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T03:10:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T03:10:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T03:10:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T03:10:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T03:10:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T03:10:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T03:10:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T03:10:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T03:10:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T03:10:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T03:10:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T03:10:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T03:10:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T03:10:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T03:10:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T03:10:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T03:10:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T03:10:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T03:10:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T03:10:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T03:10:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T03:10:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T03:10:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T03:10:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T03:10:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T03:10:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T03:10:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T03:10:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T03:10:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T03:10:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T03:10:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T03:10:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T03:10:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T03:10:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T03:10:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T03:10:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T03:10:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T03:10:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T03:10:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T03:10:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T03:10:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T03:10:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T03:10:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T03:10:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T03:10:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T03:10:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T03:10:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T03:10:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T03:10:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T03:10:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T03:10:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T03:10:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T03:10:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T03:10:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T03:10:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T03:10:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T03:10:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T03:10:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T03:10:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T03:10:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T03:10:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T03:10:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T03:10:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T03:10:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T03:10:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T03:10:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T03:10:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T03:10:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T03:10:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T03:10:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T03:10:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-12T03:10:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T03:10:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T03:10:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T03:10:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T03:10:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T03:10:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T03:10:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T03:10:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T03:10:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T03:10:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T03:10:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T03:10:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T03:10:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T03:10:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T03:10:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T03:10:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T03:10:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T03:10:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T03:10:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T03:10:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T03:10:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T03:10:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T03:10:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T03:10:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T03:10:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T03:10:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T03:10:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T03:10:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T03:10:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T03:10:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T03:10:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T03:10:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T03:10:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T03:10:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T03:10:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T03:10:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T03:10:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T03:10:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T03:10:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T03:10:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T03:10:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T03:10:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T03:10:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T03:10:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T03:10:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T03:10:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T03:10:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T03:10:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T03:10:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T03:10:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T03:10:02.621 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T03:10:02.621 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T03:10:02.621 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T03:10:02.621 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T03:10:02.621 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T03:10:02.621 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T03:10:02.676 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T03:10:02.764 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T03:10:02.765 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T03:10:02.806 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T03:10:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T03:10:02.854 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T03:10:02.884 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T03:10:02.888 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T03:10:02.926 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T03:10:02.947 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T03:10:02.959 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T03:10:02.985 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T03:10:02.985 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:10:03.069 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:10:03.070 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:10:03.162 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:10:03.187 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T03:10:03.191 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:10:03.192 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:10:03.372 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:10:03.372 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:10:03.456 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:10:03.528 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:10:03.575 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:10:03.620 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:10:03.641 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:10:04.478 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T03:10:04.568 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T03:10:04.608 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T03:10:04.690 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T03:10:04.721 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T03:10:04.746 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T03:10:04.789 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T03:10:04.789 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:10:04.882 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:10:04.882 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:10:04.958 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:10:04.982 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T03:10:04.988 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:10:04.988 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:10:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:10:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:10:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:10:05.330 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:10:05.330 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:10:05.330 INFO:teuthology.orchestra.run.smithi006.stdout:Upgraded: 2024-09-12T03:10:05.330 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T03:10:05.330 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T03:10:05.330 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T03:10:05.330 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T03:10:05.331 INFO:teuthology.orchestra.run.smithi006.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.331 INFO:teuthology.orchestra.run.smithi006.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.331 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T03:10:05.331 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T03:10:05.331 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-09-12T03:10:05.331 INFO:teuthology.orchestra.run.smithi006.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T03:10:05.331 INFO:teuthology.orchestra.run.smithi006.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T03:10:05.331 INFO:teuthology.orchestra.run.smithi006.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T03:10:05.331 INFO:teuthology.orchestra.run.smithi006.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T03:10:05.331 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.331 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.331 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.332 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.332 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:05.332 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.332 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.332 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.332 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:05.332 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:05.332 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:05.332 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:05.332 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:05.332 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.332 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.332 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:05.333 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.333 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.333 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.333 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:05.333 INFO:teuthology.orchestra.run.smithi006.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:05.333 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.333 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.333 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:05.333 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T03:10:05.333 INFO:teuthology.orchestra.run.smithi006.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T03:10:05.333 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T03:10:05.333 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T03:10:05.334 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T03:10:05.334 INFO:teuthology.orchestra.run.smithi006.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T03:10:05.334 INFO:teuthology.orchestra.run.smithi006.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-12T03:10:05.334 INFO:teuthology.orchestra.run.smithi006.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T03:10:05.334 INFO:teuthology.orchestra.run.smithi006.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T03:10:05.334 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T03:10:05.334 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T03:10:05.334 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.334 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.334 INFO:teuthology.orchestra.run.smithi006.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.334 INFO:teuthology.orchestra.run.smithi006.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T03:10:05.334 INFO:teuthology.orchestra.run.smithi006.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T03:10:05.334 INFO:teuthology.orchestra.run.smithi006.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T03:10:05.335 INFO:teuthology.orchestra.run.smithi006.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T03:10:05.335 INFO:teuthology.orchestra.run.smithi006.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T03:10:05.335 INFO:teuthology.orchestra.run.smithi006.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.335 INFO:teuthology.orchestra.run.smithi006.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.335 INFO:teuthology.orchestra.run.smithi006.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T03:10:05.335 INFO:teuthology.orchestra.run.smithi006.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.335 INFO:teuthology.orchestra.run.smithi006.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T03:10:05.335 INFO:teuthology.orchestra.run.smithi006.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T03:10:05.335 INFO:teuthology.orchestra.run.smithi006.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T03:10:05.335 INFO:teuthology.orchestra.run.smithi006.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T03:10:05.335 INFO:teuthology.orchestra.run.smithi006.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T03:10:05.335 INFO:teuthology.orchestra.run.smithi006.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T03:10:05.335 INFO:teuthology.orchestra.run.smithi006.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T03:10:05.335 INFO:teuthology.orchestra.run.smithi006.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T03:10:05.336 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T03:10:05.336 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T03:10:05.336 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T03:10:05.336 INFO:teuthology.orchestra.run.smithi006.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T03:10:05.336 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T03:10:05.336 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T03:10:05.336 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T03:10:05.336 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T03:10:05.336 INFO:teuthology.orchestra.run.smithi006.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T03:10:05.336 INFO:teuthology.orchestra.run.smithi006.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T03:10:05.336 INFO:teuthology.orchestra.run.smithi006.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T03:10:05.336 INFO:teuthology.orchestra.run.smithi006.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T03:10:05.337 INFO:teuthology.orchestra.run.smithi006.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T03:10:05.337 INFO:teuthology.orchestra.run.smithi006.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T03:10:05.337 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T03:10:05.337 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.337 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.337 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.337 INFO:teuthology.orchestra.run.smithi006.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T03:10:05.337 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T03:10:05.337 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T03:10:05.337 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T03:10:05.337 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T03:10:05.337 INFO:teuthology.orchestra.run.smithi006.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T03:10:05.337 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T03:10:05.337 INFO:teuthology.orchestra.run.smithi006.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T03:10:05.338 INFO:teuthology.orchestra.run.smithi006.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T03:10:05.338 INFO:teuthology.orchestra.run.smithi006.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T03:10:05.338 INFO:teuthology.orchestra.run.smithi006.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T03:10:05.338 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T03:10:05.338 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T03:10:05.338 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T03:10:05.338 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T03:10:05.338 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T03:10:05.338 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T03:10:05.338 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T03:10:05.338 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T03:10:05.338 INFO:teuthology.orchestra.run.smithi006.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T03:10:05.338 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T03:10:05.338 INFO:teuthology.orchestra.run.smithi006.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T03:10:05.339 INFO:teuthology.orchestra.run.smithi006.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T03:10:05.339 INFO:teuthology.orchestra.run.smithi006.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T03:10:05.339 INFO:teuthology.orchestra.run.smithi006.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T03:10:05.339 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T03:10:05.339 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T03:10:05.339 INFO:teuthology.orchestra.run.smithi006.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T03:10:05.339 INFO:teuthology.orchestra.run.smithi006.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T03:10:05.339 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T03:10:05.339 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T03:10:05.339 INFO:teuthology.orchestra.run.smithi006.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T03:10:05.339 INFO:teuthology.orchestra.run.smithi006.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T03:10:05.339 INFO:teuthology.orchestra.run.smithi006.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T03:10:05.339 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T03:10:05.339 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T03:10:05.340 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T03:10:05.340 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T03:10:05.340 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T03:10:05.340 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T03:10:05.340 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.340 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.340 INFO:teuthology.orchestra.run.smithi006.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T03:10:05.340 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T03:10:05.340 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T03:10:05.340 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.340 INFO:teuthology.orchestra.run.smithi006.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T03:10:05.340 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T03:10:05.340 INFO:teuthology.orchestra.run.smithi006.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T03:10:05.340 INFO:teuthology.orchestra.run.smithi006.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T03:10:05.341 INFO:teuthology.orchestra.run.smithi006.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T03:10:05.341 INFO:teuthology.orchestra.run.smithi006.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T03:10:05.341 INFO:teuthology.orchestra.run.smithi006.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T03:10:05.341 INFO:teuthology.orchestra.run.smithi006.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T03:10:05.341 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T03:10:05.341 INFO:teuthology.orchestra.run.smithi006.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T03:10:05.341 INFO:teuthology.orchestra.run.smithi006.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T03:10:05.341 INFO:teuthology.orchestra.run.smithi006.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T03:10:05.341 INFO:teuthology.orchestra.run.smithi006.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T03:10:05.341 INFO:teuthology.orchestra.run.smithi006.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T03:10:05.341 INFO:teuthology.orchestra.run.smithi006.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T03:10:05.341 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.342 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.342 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:05.342 INFO:teuthology.orchestra.run.smithi006.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T03:10:05.342 INFO:teuthology.orchestra.run.smithi006.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T03:10:05.342 INFO:teuthology.orchestra.run.smithi006.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T03:10:05.342 INFO:teuthology.orchestra.run.smithi006.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T03:10:05.342 INFO:teuthology.orchestra.run.smithi006.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T03:10:05.342 INFO:teuthology.orchestra.run.smithi006.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T03:10:05.342 INFO:teuthology.orchestra.run.smithi006.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T03:10:05.343 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:10:05.343 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-09-12T03:10:05.348 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:10:05.754 DEBUG:teuthology.parallel:result is None 2024-09-12T03:10:09.016 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:10:09.016 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T03:10:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T03:10:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T03:10:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T03:10:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T03:10:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T03:10:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T03:10:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:10:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:10:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:10:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T03:10:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T03:10:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:10:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T03:10:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T03:10:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T03:10:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T03:10:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T03:10:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T03:10:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T03:10:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T03:10:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T03:10:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T03:10:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T03:10:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T03:10:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T03:10:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T03:10:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T03:10:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T03:10:09.019 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T03:10:09.030 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T03:10:09.030 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T03:10:09.030 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T03:10:09.030 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T03:10:09.030 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T03:10:09.030 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T03:10:09.030 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T03:10:09.030 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T03:10:09.030 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T03:10:09.030 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T03:10:09.030 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T03:10:09.031 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T03:10:09.031 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T03:10:09.031 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T03:10:09.031 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T03:10:09.031 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T03:10:09.031 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T03:10:09.031 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T03:10:09.031 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T03:10:09.031 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T03:10:09.031 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T03:10:09.031 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T03:10:09.031 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T03:10:09.031 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T03:10:09.032 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T03:10:09.032 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T03:10:09.032 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T03:10:09.032 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T03:10:09.032 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T03:10:09.032 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T03:10:09.032 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T03:10:09.032 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T03:10:09.032 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T03:10:09.032 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T03:10:09.032 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T03:10:09.032 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T03:10:09.033 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T03:10:09.033 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T03:10:09.033 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T03:10:09.033 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T03:10:09.033 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T03:10:09.033 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T03:10:09.033 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T03:10:09.033 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T03:10:09.033 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T03:10:09.033 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T03:10:09.033 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T03:10:09.033 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T03:10:09.033 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T03:10:09.034 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T03:10:09.034 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T03:10:09.034 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T03:10:09.034 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T03:10:09.034 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T03:10:09.034 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T03:10:09.034 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T03:10:09.034 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T03:10:09.034 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T03:10:09.034 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T03:10:09.034 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T03:10:09.034 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T03:10:09.034 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T03:10:09.034 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T03:10:09.034 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T03:10:09.035 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T03:10:09.035 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T03:10:09.035 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T03:10:09.035 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T03:10:09.035 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T03:10:09.035 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T03:10:09.035 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T03:10:09.035 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T03:10:09.035 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T03:10:09.035 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T03:10:09.035 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T03:10:09.035 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T03:10:09.035 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-12T03:10:09.036 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T03:10:09.036 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T03:10:09.036 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T03:10:09.036 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T03:10:09.036 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T03:10:09.036 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T03:10:09.036 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T03:10:09.036 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T03:10:09.036 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T03:10:09.036 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T03:10:09.036 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T03:10:09.036 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T03:10:09.036 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T03:10:09.037 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T03:10:09.037 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T03:10:09.037 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T03:10:09.037 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T03:10:09.037 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T03:10:09.037 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T03:10:09.037 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T03:10:09.037 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T03:10:09.037 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T03:10:09.037 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T03:10:09.037 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T03:10:09.037 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T03:10:09.037 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T03:10:09.038 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T03:10:09.038 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T03:10:09.038 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T03:10:09.038 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T03:10:09.038 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T03:10:09.038 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T03:10:09.038 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T03:10:09.038 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T03:10:09.038 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T03:10:09.038 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T03:10:09.038 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T03:10:09.038 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T03:10:09.038 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T03:10:09.039 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T03:10:09.039 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T03:10:09.039 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T03:10:09.039 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T03:10:09.039 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T03:10:09.039 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T03:10:09.039 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T03:10:09.039 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T03:10:09.039 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T03:10:09.039 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T03:10:09.039 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T03:10:09.039 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T03:10:09.039 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T03:10:09.039 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T03:10:09.039 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T03:10:09.040 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T03:10:09.040 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T03:10:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:10:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:10:11.539 INFO:teuthology.orchestra.run.smithi161.stdout:Upgraded: 2024-09-12T03:10:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T03:10:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T03:10:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T03:10:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T03:10:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T03:10:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T03:10:11.540 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-09-12T03:10:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T03:10:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T03:10:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T03:10:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T03:10:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:11.540 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.541 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T03:10:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T03:10:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T03:10:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T03:10:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T03:10:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T03:10:11.542 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-12T03:10:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T03:10:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T03:10:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T03:10:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T03:10:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T03:10:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T03:10:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T03:10:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T03:10:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T03:10:11.543 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T03:10:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T03:10:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T03:10:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T03:10:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T03:10:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T03:10:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T03:10:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T03:10:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T03:10:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T03:10:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T03:10:11.544 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T03:10:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T03:10:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T03:10:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T03:10:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T03:10:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T03:10:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T03:10:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T03:10:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T03:10:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T03:10:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T03:10:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T03:10:11.545 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T03:10:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T03:10:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T03:10:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T03:10:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T03:10:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T03:10:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T03:10:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T03:10:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T03:10:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T03:10:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T03:10:11.546 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T03:10:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T03:10:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T03:10:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T03:10:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T03:10:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T03:10:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T03:10:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T03:10:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T03:10:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T03:10:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T03:10:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T03:10:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T03:10:11.547 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T03:10:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T03:10:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T03:10:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T03:10:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T03:10:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T03:10:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T03:10:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T03:10:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T03:10:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T03:10:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T03:10:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T03:10:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T03:10:11.548 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T03:10:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T03:10:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T03:10:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T03:10:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T03:10:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T03:10:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T03:10:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T03:10:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T03:10:11.549 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T03:10:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T03:10:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T03:10:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T03:10:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T03:10:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T03:10:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T03:10:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T03:10:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T03:10:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T03:10:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T03:10:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T03:10:11.550 INFO:teuthology.orchestra.run.smithi161.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T03:10:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T03:10:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T03:10:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T03:10:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T03:10:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T03:10:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T03:10:11.551 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T03:10:11.552 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:10:11.552 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-12T03:10:11.956 DEBUG:teuthology.parallel:result is None 2024-09-12T03:10:13.472 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:10:13.472 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T03:10:13.472 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T03:10:13.472 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T03:10:13.472 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T03:10:13.473 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T03:10:13.473 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T03:10:13.473 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T03:10:13.473 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:10:13.473 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:10:13.473 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:10:13.473 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T03:10:13.473 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T03:10:13.473 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:10:13.473 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T03:10:13.473 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T03:10:13.473 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T03:10:13.473 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T03:10:13.474 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T03:10:13.474 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T03:10:13.474 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T03:10:13.474 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T03:10:13.474 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T03:10:13.474 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T03:10:13.474 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T03:10:13.474 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T03:10:13.474 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T03:10:13.474 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T03:10:13.474 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T03:10:13.474 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T03:10:13.474 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T03:10:13.488 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T03:10:13.488 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T03:10:13.488 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T03:10:13.489 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T03:10:13.489 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T03:10:13.489 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T03:10:13.489 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T03:10:13.489 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T03:10:13.489 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T03:10:13.489 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T03:10:13.489 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T03:10:13.490 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T03:10:13.490 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T03:10:13.490 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T03:10:13.490 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T03:10:13.490 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T03:10:13.490 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T03:10:13.490 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T03:10:13.490 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T03:10:13.490 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T03:10:13.490 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T03:10:13.490 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T03:10:13.490 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T03:10:13.491 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T03:10:13.491 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T03:10:13.491 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T03:10:13.491 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T03:10:13.491 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T03:10:13.491 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T03:10:13.491 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T03:10:13.491 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T03:10:13.491 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T03:10:13.491 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T03:10:13.491 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T03:10:13.491 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T03:10:13.492 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T03:10:13.492 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T03:10:13.492 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T03:10:13.492 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T03:10:13.492 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T03:10:13.492 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T03:10:13.492 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T03:10:13.492 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T03:10:13.492 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T03:10:13.492 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T03:10:13.492 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T03:10:13.492 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T03:10:13.492 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T03:10:13.492 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T03:10:13.493 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T03:10:13.493 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T03:10:13.493 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T03:10:13.493 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T03:10:13.493 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T03:10:13.493 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T03:10:13.493 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T03:10:13.493 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T03:10:13.493 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T03:10:13.493 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T03:10:13.493 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T03:10:13.493 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T03:10:13.493 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T03:10:13.494 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T03:10:13.494 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T03:10:13.494 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T03:10:13.494 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T03:10:13.494 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T03:10:13.494 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T03:10:13.494 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T03:10:13.494 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T03:10:13.494 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T03:10:13.494 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T03:10:13.494 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T03:10:13.494 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T03:10:13.494 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T03:10:13.494 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T03:10:13.495 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-12T03:10:13.495 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T03:10:13.495 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T03:10:13.495 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T03:10:13.495 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T03:10:13.495 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T03:10:13.495 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T03:10:13.495 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T03:10:13.495 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T03:10:13.495 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T03:10:13.495 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T03:10:13.495 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T03:10:13.496 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T03:10:13.496 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T03:10:13.496 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T03:10:13.496 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T03:10:13.496 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T03:10:13.496 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T03:10:13.496 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T03:10:13.496 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T03:10:13.496 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T03:10:13.496 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T03:10:13.496 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T03:10:13.496 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T03:10:13.496 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T03:10:13.497 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T03:10:13.497 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T03:10:13.497 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T03:10:13.497 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T03:10:13.497 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T03:10:13.497 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T03:10:13.497 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T03:10:13.497 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T03:10:13.497 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T03:10:13.497 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T03:10:13.497 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T03:10:13.497 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T03:10:13.497 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T03:10:13.498 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T03:10:13.498 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T03:10:13.498 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T03:10:13.498 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T03:10:13.498 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T03:10:13.498 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T03:10:13.498 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T03:10:13.498 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T03:10:13.498 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T03:10:13.498 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T03:10:13.498 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T03:10:13.498 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T03:10:13.498 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T03:10:13.498 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T03:10:13.499 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T03:10:13.499 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T03:10:13.499 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T03:10:13.499 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T03:10:13.499 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T03:10:16.061 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:10:16.062 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-12T03:10:16.062 INFO:teuthology.orchestra.run.smithi118.stdout:Upgraded: 2024-09-12T03:10:16.062 INFO:teuthology.orchestra.run.smithi118.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T03:10:16.062 INFO:teuthology.orchestra.run.smithi118.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T03:10:16.062 INFO:teuthology.orchestra.run.smithi118.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T03:10:16.062 INFO:teuthology.orchestra.run.smithi118.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T03:10:16.062 INFO:teuthology.orchestra.run.smithi118.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.062 INFO:teuthology.orchestra.run.smithi118.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.062 INFO:teuthology.orchestra.run.smithi118.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T03:10:16.062 INFO:teuthology.orchestra.run.smithi118.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T03:10:16.062 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-09-12T03:10:16.063 INFO:teuthology.orchestra.run.smithi118.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T03:10:16.063 INFO:teuthology.orchestra.run.smithi118.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T03:10:16.063 INFO:teuthology.orchestra.run.smithi118.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T03:10:16.063 INFO:teuthology.orchestra.run.smithi118.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T03:10:16.063 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.063 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.063 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.063 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.063 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:16.063 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.063 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.063 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.064 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:16.064 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:16.064 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:16.064 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:16.064 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:16.064 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.064 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.064 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:16.064 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.064 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.064 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.064 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:16.065 INFO:teuthology.orchestra.run.smithi118.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:16.065 INFO:teuthology.orchestra.run.smithi118.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.065 INFO:teuthology.orchestra.run.smithi118.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.065 INFO:teuthology.orchestra.run.smithi118.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:10:16.065 INFO:teuthology.orchestra.run.smithi118.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T03:10:16.065 INFO:teuthology.orchestra.run.smithi118.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T03:10:16.065 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T03:10:16.065 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T03:10:16.065 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T03:10:16.065 INFO:teuthology.orchestra.run.smithi118.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T03:10:16.065 INFO:teuthology.orchestra.run.smithi118.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-12T03:10:16.065 INFO:teuthology.orchestra.run.smithi118.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T03:10:16.065 INFO:teuthology.orchestra.run.smithi118.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T03:10:16.066 INFO:teuthology.orchestra.run.smithi118.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T03:10:16.066 INFO:teuthology.orchestra.run.smithi118.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T03:10:16.066 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.066 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.066 INFO:teuthology.orchestra.run.smithi118.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.066 INFO:teuthology.orchestra.run.smithi118.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T03:10:16.066 INFO:teuthology.orchestra.run.smithi118.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T03:10:16.066 INFO:teuthology.orchestra.run.smithi118.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T03:10:16.066 INFO:teuthology.orchestra.run.smithi118.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T03:10:16.066 INFO:teuthology.orchestra.run.smithi118.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T03:10:16.066 INFO:teuthology.orchestra.run.smithi118.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.066 INFO:teuthology.orchestra.run.smithi118.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.066 INFO:teuthology.orchestra.run.smithi118.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T03:10:16.066 INFO:teuthology.orchestra.run.smithi118.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.067 INFO:teuthology.orchestra.run.smithi118.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T03:10:16.067 INFO:teuthology.orchestra.run.smithi118.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T03:10:16.067 INFO:teuthology.orchestra.run.smithi118.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T03:10:16.067 INFO:teuthology.orchestra.run.smithi118.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T03:10:16.067 INFO:teuthology.orchestra.run.smithi118.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T03:10:16.067 INFO:teuthology.orchestra.run.smithi118.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T03:10:16.067 INFO:teuthology.orchestra.run.smithi118.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T03:10:16.067 INFO:teuthology.orchestra.run.smithi118.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T03:10:16.067 INFO:teuthology.orchestra.run.smithi118.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T03:10:16.067 INFO:teuthology.orchestra.run.smithi118.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T03:10:16.067 INFO:teuthology.orchestra.run.smithi118.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T03:10:16.067 INFO:teuthology.orchestra.run.smithi118.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T03:10:16.067 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T03:10:16.068 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T03:10:16.068 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T03:10:16.068 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T03:10:16.068 INFO:teuthology.orchestra.run.smithi118.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T03:10:16.068 INFO:teuthology.orchestra.run.smithi118.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T03:10:16.068 INFO:teuthology.orchestra.run.smithi118.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T03:10:16.068 INFO:teuthology.orchestra.run.smithi118.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T03:10:16.068 INFO:teuthology.orchestra.run.smithi118.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T03:10:16.068 INFO:teuthology.orchestra.run.smithi118.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T03:10:16.068 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T03:10:16.068 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.068 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.068 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.069 INFO:teuthology.orchestra.run.smithi118.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T03:10:16.069 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T03:10:16.069 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T03:10:16.069 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T03:10:16.069 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T03:10:16.069 INFO:teuthology.orchestra.run.smithi118.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T03:10:16.069 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T03:10:16.069 INFO:teuthology.orchestra.run.smithi118.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T03:10:16.069 INFO:teuthology.orchestra.run.smithi118.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T03:10:16.069 INFO:teuthology.orchestra.run.smithi118.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T03:10:16.069 INFO:teuthology.orchestra.run.smithi118.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T03:10:16.069 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T03:10:16.069 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T03:10:16.069 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T03:10:16.070 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T03:10:16.070 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T03:10:16.070 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T03:10:16.070 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T03:10:16.070 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T03:10:16.070 INFO:teuthology.orchestra.run.smithi118.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T03:10:16.070 INFO:teuthology.orchestra.run.smithi118.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T03:10:16.070 INFO:teuthology.orchestra.run.smithi118.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T03:10:16.070 INFO:teuthology.orchestra.run.smithi118.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T03:10:16.070 INFO:teuthology.orchestra.run.smithi118.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T03:10:16.070 INFO:teuthology.orchestra.run.smithi118.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T03:10:16.070 INFO:teuthology.orchestra.run.smithi118.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T03:10:16.071 INFO:teuthology.orchestra.run.smithi118.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T03:10:16.071 INFO:teuthology.orchestra.run.smithi118.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T03:10:16.071 INFO:teuthology.orchestra.run.smithi118.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T03:10:16.071 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T03:10:16.071 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T03:10:16.071 INFO:teuthology.orchestra.run.smithi118.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T03:10:16.071 INFO:teuthology.orchestra.run.smithi118.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T03:10:16.071 INFO:teuthology.orchestra.run.smithi118.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T03:10:16.071 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T03:10:16.071 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T03:10:16.071 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T03:10:16.071 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T03:10:16.071 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T03:10:16.071 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T03:10:16.072 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.072 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.072 INFO:teuthology.orchestra.run.smithi118.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T03:10:16.072 INFO:teuthology.orchestra.run.smithi118.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T03:10:16.072 INFO:teuthology.orchestra.run.smithi118.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T03:10:16.072 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.072 INFO:teuthology.orchestra.run.smithi118.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T03:10:16.072 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T03:10:16.072 INFO:teuthology.orchestra.run.smithi118.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T03:10:16.072 INFO:teuthology.orchestra.run.smithi118.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T03:10:16.072 INFO:teuthology.orchestra.run.smithi118.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T03:10:16.072 INFO:teuthology.orchestra.run.smithi118.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T03:10:16.072 INFO:teuthology.orchestra.run.smithi118.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T03:10:16.072 INFO:teuthology.orchestra.run.smithi118.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T03:10:16.073 INFO:teuthology.orchestra.run.smithi118.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T03:10:16.073 INFO:teuthology.orchestra.run.smithi118.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T03:10:16.073 INFO:teuthology.orchestra.run.smithi118.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T03:10:16.073 INFO:teuthology.orchestra.run.smithi118.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T03:10:16.073 INFO:teuthology.orchestra.run.smithi118.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T03:10:16.073 INFO:teuthology.orchestra.run.smithi118.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T03:10:16.073 INFO:teuthology.orchestra.run.smithi118.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T03:10:16.073 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.073 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.073 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:10:16.073 INFO:teuthology.orchestra.run.smithi118.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T03:10:16.073 INFO:teuthology.orchestra.run.smithi118.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T03:10:16.073 INFO:teuthology.orchestra.run.smithi118.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T03:10:16.073 INFO:teuthology.orchestra.run.smithi118.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T03:10:16.074 INFO:teuthology.orchestra.run.smithi118.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T03:10:16.074 INFO:teuthology.orchestra.run.smithi118.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T03:10:16.074 INFO:teuthology.orchestra.run.smithi118.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T03:10:16.074 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-12T03:10:16.074 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-12T03:10:16.507 DEBUG:teuthology.parallel:result is None 2024-09-12T03:10:16.507 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:10:16.657 DEBUG:teuthology.orchestra.run.smithi006:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T03:10:16.684 INFO:teuthology.orchestra.run.smithi006.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T03:10:16.685 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T03:10:16.685 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T03:10:16.687 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:10:16.842 DEBUG:teuthology.orchestra.run.smithi118:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T03:10:16.865 INFO:teuthology.orchestra.run.smithi118.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T03:10:16.865 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T03:10:16.865 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T03:10:16.867 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:10:17.013 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T03:10:17.039 INFO:teuthology.orchestra.run.smithi161.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T03:10:17.039 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T03:10:17.040 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T03:10:17.041 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-12T03:10:17.042 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-09-12T03:10:17.042 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T03:10:17.075 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-12T03:10:17.075 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T03:10:17.105 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-12T03:10:17.105 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T03:10:17.137 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-12T03:10:17.138 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-09-12T03:10:17.138 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T03:10:17.167 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T03:10:17.237 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-12T03:10:17.237 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T03:10:17.265 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T03:10:17.332 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-12T03:10:17.332 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T03:10:17.362 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T03:10:17.430 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-12T03:10:17.430 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-09-12T03:10:17.431 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T03:10:17.459 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T03:10:17.526 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-12T03:10:17.526 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T03:10:17.552 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T03:10:17.619 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-12T03:10:17.619 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T03:10:17.648 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T03:10:17.716 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-12T03:10:17.716 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-09-12T03:10:17.716 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T03:10:17.742 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T03:10:17.810 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-12T03:10:17.810 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T03:10:17.837 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T03:10:17.904 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-12T03:10:17.904 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T03:10:17.933 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T03:10:18.001 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-12T03:10:18.093 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request', 'SLOW_OPS', 'slow request'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-12T03:10:18.093 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:10:18.093 INFO:tasks.cephadm:Cluster fsid is 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:10:18.094 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-12T03:10:18.094 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.6', 'mon.b': '172.21.15.118', 'mon.c': '172.21.15.161'} 2024-09-12T03:10:18.094 INFO:tasks.cephadm:First mon is mon.a on smithi006 2024-09-12T03:10:18.094 INFO:tasks.cephadm:First mgr is x 2024-09-12T03:10:18.094 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-12T03:10:18.094 DEBUG:teuthology.orchestra.run.smithi006:> sudo hostname $(hostname -s) 2024-09-12T03:10:18.121 DEBUG:teuthology.orchestra.run.smithi118:> sudo hostname $(hostname -s) 2024-09-12T03:10:18.147 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname $(hostname -s) 2024-09-12T03:10:18.178 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-12T03:10:18.178 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:10:18.321 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-12T03:10:18.461 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:10:18.462 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T03:10:18.462 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T03:10:18.462 DEBUG:teuthology.orchestra.run.smithi006:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T03:10:18.847 INFO:teuthology.orchestra.run.smithi006.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 03:10 /home/ubuntu/cephtest/cephadm 2024-09-12T03:10:18.848 DEBUG:teuthology.orchestra.run.smithi118:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T03:10:19.267 INFO:teuthology.orchestra.run.smithi118.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 03:10 /home/ubuntu/cephtest/cephadm 2024-09-12T03:10:19.267 DEBUG:teuthology.orchestra.run.smithi161:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T03:10:19.661 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 03:10 /home/ubuntu/cephtest/cephadm 2024-09-12T03:10:19.661 DEBUG:teuthology.orchestra.run.smithi006:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T03:10:19.682 DEBUG:teuthology.orchestra.run.smithi118:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T03:10:19.702 DEBUG:teuthology.orchestra.run.smithi161:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T03:10:19.737 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-12T03:10:19.737 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T03:10:19.739 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T03:10:19.744 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T03:10:21.348 INFO:teuthology.orchestra.run.smithi006.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T03:10:21.401 INFO:teuthology.orchestra.run.smithi161.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T03:10:21.510 INFO:teuthology.orchestra.run.smithi118.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T03:10:50.789 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-12T03:10:50.790 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T03:10:50.790 INFO:teuthology.orchestra.run.smithi118.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T03:10:50.790 INFO:teuthology.orchestra.run.smithi118.stdout: "repo_digests": [ 2024-09-12T03:10:50.790 INFO:teuthology.orchestra.run.smithi118.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T03:10:50.790 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-12T03:10:50.790 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-12T03:10:59.177 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:10:59.177 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T03:10:59.177 INFO:teuthology.orchestra.run.smithi006.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T03:10:59.177 INFO:teuthology.orchestra.run.smithi006.stdout: "repo_digests": [ 2024-09-12T03:10:59.177 INFO:teuthology.orchestra.run.smithi006.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T03:10:59.177 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-09-12T03:10:59.177 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:11:14.158 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-12T03:11:14.158 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T03:11:14.158 INFO:teuthology.orchestra.run.smithi161.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T03:11:14.158 INFO:teuthology.orchestra.run.smithi161.stdout: "repo_digests": [ 2024-09-12T03:11:14.158 INFO:teuthology.orchestra.run.smithi161.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T03:11:14.158 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-12T03:11:14.158 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-12T03:11:14.184 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /etc/ceph 2024-09-12T03:11:14.219 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /etc/ceph 2024-09-12T03:11:14.253 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph 2024-09-12T03:11:14.281 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 777 /etc/ceph 2024-09-12T03:11:14.311 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 777 /etc/ceph 2024-09-12T03:11:14.341 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /etc/ceph 2024-09-12T03:11:14.366 INFO:tasks.cephadm:Writing seed config... 2024-09-12T03:11:14.367 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-12T03:11:14.367 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-12T03:11:14.367 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-12T03:11:14.368 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-12T03:11:14.368 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-12T03:11:14.368 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-12T03:11:14.368 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-12T03:11:14.368 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-12T03:11:14.368 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-12T03:11:14.368 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-12T03:11:14.368 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-12T03:11:14.368 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-12T03:11:14.368 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-12T03:11:14.368 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-12T03:11:14.368 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-12T03:11:14.368 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-12T03:11:14.368 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-12T03:11:14.369 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-12T03:11:14.369 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-12T03:11:14.369 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-12T03:11:14.369 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-12T03:11:14.369 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-12T03:11:14.369 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-12T03:11:14.369 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-12T03:11:14.369 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-12T03:11:14.369 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-12T03:11:14.369 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-12T03:11:14.370 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-12T03:11:14.370 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-12T03:11:14.370 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-12T03:11:14.370 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-12T03:11:14.370 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-12T03:11:14.370 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-12T03:11:14.370 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-12T03:11:14.370 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-12T03:11:14.370 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-12T03:11:14.370 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-12T03:11:14.371 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-12T03:11:14.371 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-12T03:11:14.371 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-12T03:11:14.371 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-12T03:11:14.371 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-12T03:11:14.371 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2024-09-12T03:11:14.372 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-12T03:11:14.372 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-12T03:11:14.372 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-12T03:11:14.372 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-12T03:11:14.372 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2024-09-12T03:11:14.372 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-12T03:11:14.372 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-12T03:11:14.372 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-12T03:11:14.372 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-12T03:11:14.373 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-09-12T03:11:14.373 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-12T03:11:14.392 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 89d99938-70b4-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-12T03:11:14.393 DEBUG:teuthology.orchestra.run.smithi006:mon.a> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mon.a.service 2024-09-12T03:11:14.435 DEBUG:teuthology.orchestra.run.smithi006:mgr.x> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mgr.x.service 2024-09-12T03:11:14.479 INFO:tasks.cephadm:Bootstrapping... 2024-09-12T03:11:14.479 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid 89d99938-70b4-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.6 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:11:14.695 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:11:14.695 INFO:teuthology.orchestra.run.smithi006.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', '89d99938-70b4-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.6', '--skip-admin-label'] 2024-09-12T03:11:14.696 INFO:teuthology.orchestra.run.smithi006.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-12T03:11:14.696 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying podman|docker is present... 2024-09-12T03:11:14.720 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout 5.2.2 2024-09-12T03:11:14.720 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying lvm2 is present... 2024-09-12T03:11:14.720 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying time synchronization is in place... 2024-09-12T03:11:14.728 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T03:11:14.728 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T03:11:14.734 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T03:11:14.734 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-09-12T03:11:14.742 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout enabled 2024-09-12T03:11:14.748 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout active 2024-09-12T03:11:14.748 INFO:teuthology.orchestra.run.smithi006.stdout:Unit chronyd.service is enabled and running 2024-09-12T03:11:14.749 INFO:teuthology.orchestra.run.smithi006.stdout:Repeating the final host check... 2024-09-12T03:11:14.772 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout 5.2.2 2024-09-12T03:11:14.772 INFO:teuthology.orchestra.run.smithi006.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-12T03:11:14.772 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl is present 2024-09-12T03:11:14.772 INFO:teuthology.orchestra.run.smithi006.stdout:lvcreate is present 2024-09-12T03:11:14.778 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T03:11:14.779 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T03:11:14.785 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T03:11:14.785 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-09-12T03:11:14.792 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout enabled 2024-09-12T03:11:14.798 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout active 2024-09-12T03:11:14.798 INFO:teuthology.orchestra.run.smithi006.stdout:Unit chronyd.service is enabled and running 2024-09-12T03:11:14.799 INFO:teuthology.orchestra.run.smithi006.stdout:Host looks OK 2024-09-12T03:11:14.799 INFO:teuthology.orchestra.run.smithi006.stdout:Cluster fsid: 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:11:14.799 INFO:teuthology.orchestra.run.smithi006.stdout:Acquiring lock 139885908702208 on /run/cephadm/89d99938-70b4-11ef-bcea-c7b262605968.lock 2024-09-12T03:11:14.799 INFO:teuthology.orchestra.run.smithi006.stdout:Lock 139885908702208 acquired on /run/cephadm/89d99938-70b4-11ef-bcea-c7b262605968.lock 2024-09-12T03:11:14.800 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying IP 172.21.15.6 port 3300 ... 2024-09-12T03:11:14.800 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying IP 172.21.15.6 port 6789 ... 2024-09-12T03:11:14.800 INFO:teuthology.orchestra.run.smithi006.stdout:Base mon IP(s) is [172.21.15.6:3300, 172.21.15.6:6789], mon addrv is [v2:172.21.15.6:3300,v1:172.21.15.6:6789] 2024-09-12T03:11:14.803 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.6 metric 100 2024-09-12T03:11:14.803 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.6 metric 100 2024-09-12T03:11:14.805 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-12T03:11:14.806 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-12T03:11:14.806 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1771sec hoplimit 64 pref medium 2024-09-12T03:11:14.808 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-12T03:11:14.808 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-12T03:11:14.808 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T03:11:14.808 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-12T03:11:14.809 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15e4/64 scope link 2024-09-12T03:11:14.809 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T03:11:14.809 INFO:teuthology.orchestra.run.smithi006.stdout:Mon IP `172.21.15.6` is in CIDR network `172.21.0.0/20` 2024-09-12T03:11:14.809 INFO:teuthology.orchestra.run.smithi006.stdout:Mon IP `172.21.15.6` is in CIDR network `172.21.0.0/20` 2024-09-12T03:11:14.810 INFO:teuthology.orchestra.run.smithi006.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-12T03:11:14.810 INFO:teuthology.orchestra.run.smithi006.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-12T03:11:14.810 INFO:teuthology.orchestra.run.smithi006.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T03:11:15.428 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T03:11:15.429 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T03:11:15.429 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Getting image source signatures 2024-09-12T03:11:15.429 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-12T03:11:15.429 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-12T03:11:15.429 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T03:11:15.429 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-12T03:11:16.298 INFO:teuthology.orchestra.run.smithi006.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T03:11:16.299 INFO:teuthology.orchestra.run.smithi006.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T03:11:16.299 INFO:teuthology.orchestra.run.smithi006.stdout:Extracting ceph user uid/gid from container image... 2024-09-12T03:11:17.115 INFO:teuthology.orchestra.run.smithi006.stdout:stat: stdout 167 167 2024-09-12T03:11:17.116 INFO:teuthology.orchestra.run.smithi006.stdout:Creating initial keys... 2024-09-12T03:11:17.902 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph-authtool: stdout AQDVW+JmpT4oGRAAkNUK4RirvGDRFrIgW0iFpQ== 2024-09-12T03:11:18.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph-authtool: stdout AQDWW+JmzToZDRAAR65RBTNkVHZPfS3ozCW2WQ== 2024-09-12T03:11:19.821 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph-authtool: stdout AQDXW+Jmk/dfDhAAHRNZBJlcW1aM9Z4NwqDCtQ== 2024-09-12T03:11:19.821 INFO:teuthology.orchestra.run.smithi006.stdout:Creating initial monmap... 2024-09-12T03:11:20.635 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T03:11:20.635 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-12T03:11:20.635 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:11:20.635 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T03:11:20.635 INFO:teuthology.orchestra.run.smithi006.stdout:monmaptool for a [v2:172.21.15.6:3300,v1:172.21.15.6:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T03:11:20.635 INFO:teuthology.orchestra.run.smithi006.stdout:setting min_mon_release = quincy 2024-09-12T03:11:20.636 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: set fsid to 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:11:20.636 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T03:11:20.636 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:11:20.636 INFO:teuthology.orchestra.run.smithi006.stdout:Creating mon... 2024-09-12T03:11:23.544 INFO:teuthology.orchestra.run.smithi006.stdout:create mon.a on 2024-09-12T03:11:24.010 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-12T03:11:24.294 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-12T03:11:25.082 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-89d99938-70b4-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-89d99938-70b4-11ef-bcea-c7b262605968.target. 2024-09-12T03:11:25.082 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-89d99938-70b4-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-89d99938-70b4-11ef-bcea-c7b262605968.target. 2024-09-12T03:11:26.058 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-89d99938-70b4-11ef-bcea-c7b262605968@mon.a 2024-09-12T03:11:26.059 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to reset failed state of unit ceph-89d99938-70b4-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-89d99938-70b4-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-12T03:11:26.315 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-89d99938-70b4-11ef-bcea-c7b262605968.target.wants/ceph-89d99938-70b4-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-89d99938-70b4-11ef-bcea-c7b262605968@.service. 2024-09-12T03:11:27.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:26 smithi006 podman[30602]: 2024-09-12 03:11:26.547967494 +0000 UTC m=+0.016086058 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:11:27.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:27 smithi006 podman[30602]: 2024-09-12 03:11:27.550264639 +0000 UTC m=+1.018383236 container create b1adac5cdbede0f670f23ab4556568f39e430a92f2a1ab0d4b16f15f1a4f1d44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-12T03:11:29.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:29 smithi006 podman[30602]: 2024-09-12 03:11:29.481518427 +0000 UTC m=+2.949636986 container init b1adac5cdbede0f670f23ab4556568f39e430a92f2a1ab0d4b16f15f1a4f1d44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-12T03:11:29.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:29 smithi006 podman[30602]: 2024-09-12 03:11:29.486097323 +0000 UTC m=+2.954215880 container start b1adac5cdbede0f670f23ab4556568f39e430a92f2a1ab0d4b16f15f1a4f1d44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-12T03:11:29.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:29 smithi006 ceph-mon[30622]: mkfs 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:11:29.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:29 smithi006 ceph-mon[30622]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T03:11:30.311 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T03:11:30.311 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2024-09-12T03:11:30.320 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T03:11:30.320 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-09-12T03:11:30.320 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2024-09-12T03:11:30.320 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T03:11:30.320 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mon to start... 2024-09-12T03:11:30.320 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mon... 2024-09-12T03:11:30.568 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:30 smithi006 bash[30602]: b1adac5cdbede0f670f23ab4556568f39e430a92f2a1ab0d4b16f15f1a4f1d44 2024-09-12T03:11:30.569 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:30 smithi006 systemd[1]: Started Ceph mon.a for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:11:30.569 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:30 smithi006 ceph-mon[30622]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T03:11:30.569 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:30 smithi006 ceph-mon[30622]: monmap epoch 1 2024-09-12T03:11:30.569 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:30 smithi006 ceph-mon[30622]: fsid 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:11:30.569 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:30 smithi006 ceph-mon[30622]: last_changed 2024-09-12T03:11:20.144939+0000 2024-09-12T03:11:30.569 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:30 smithi006 ceph-mon[30622]: created 2024-09-12T03:11:20.144939+0000 2024-09-12T03:11:30.569 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:30 smithi006 ceph-mon[30622]: min_mon_release 19 (squid) 2024-09-12T03:11:30.569 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:30 smithi006 ceph-mon[30622]: election_strategy: 1 2024-09-12T03:11:30.569 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:30 smithi006 ceph-mon[30622]: 0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-09-12T03:11:30.569 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:30 smithi006 ceph-mon[30622]: fsmap 2024-09-12T03:11:30.569 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:30 smithi006 ceph-mon[30622]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T03:11:30.569 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:30 smithi006 ceph-mon[30622]: mgrmap e1: no daemons active 2024-09-12T03:11:31.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout cluster: 2024-09-12T03:11:31.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout id: 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:11:31.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-12T03:11:31.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-09-12T03:11:31.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout services: 2024-09-12T03:11:31.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 1.40951s) 2024-09-12T03:11:31.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-12T03:11:31.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-12T03:11:31.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-09-12T03:11:31.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout data: 2024-09-12T03:11:31.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-12T03:11:31.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-12T03:11:31.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-12T03:11:31.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout pgs: 2024-09-12T03:11:31.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-09-12T03:11:31.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.865+0000 7f554dbed640 1 Processor -- start 2024-09-12T03:11:31.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.865+0000 7f554dbed640 1 -- start start 2024-09-12T03:11:31.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.865+0000 7f554dbed640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55481079c0 0x7f5548107dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:31.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.865+0000 7f554dbed640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5548108390 con 0x7f55481079c0 2024-09-12T03:11:31.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.866+0000 7f55477fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55481079c0 0x7f5548107dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:31.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.866+0000 7f55477fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55481079c0 0x7f5548107dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47644/0 (socket says 172.21.15.6:47644) 2024-09-12T03:11:31.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.866+0000 7f55477fe640 1 -- 172.21.15.6:0/884906762 learned_addr learned my addr 172.21.15.6:0/884906762 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:31.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.866+0000 7f55477fe640 1 -- 172.21.15.6:0/884906762 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5548108b20 con 0x7f55481079c0 2024-09-12T03:11:31.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.868+0000 7f55477fe640 1 --2- 172.21.15.6:0/884906762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55481079c0 0x7f5548107dc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5538009820 tx=0x7f553802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410147ef722f615c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:31.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.868+0000 7f55467fc640 1 -- 172.21.15.6:0/884906762 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5538037070 con 0x7f55481079c0 2024-09-12T03:11:31.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.868+0000 7f55467fc640 1 -- 172.21.15.6:0/884906762 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5538032440 con 0x7f55481079c0 2024-09-12T03:11:31.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.868+0000 7f55467fc640 1 -- 172.21.15.6:0/884906762 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5538032740 con 0x7f55481079c0 2024-09-12T03:11:31.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.869+0000 7f554dbed640 1 -- 172.21.15.6:0/884906762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55481079c0 msgr2=0x7f5548107dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:31.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.869+0000 7f554dbed640 1 --2- 172.21.15.6:0/884906762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55481079c0 0x7f5548107dc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5538009820 tx=0x7f553802a270 comp rx=0 tx=0).stop 2024-09-12T03:11:31.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.869+0000 7f554dbed640 1 -- 172.21.15.6:0/884906762 shutdown_connections 2024-09-12T03:11:31.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.869+0000 7f554dbed640 1 --2- 172.21.15.6:0/884906762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55481079c0 0x7f5548107dc0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:31.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.869+0000 7f554dbed640 1 -- 172.21.15.6:0/884906762 >> 172.21.15.6:0/884906762 conn(0x7f55481031b0 msgr2=0x7f55481055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:31.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.869+0000 7f554dbed640 1 -- 172.21.15.6:0/884906762 shutdown_connections 2024-09-12T03:11:31.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.869+0000 7f554dbed640 1 -- 172.21.15.6:0/884906762 wait complete. 2024-09-12T03:11:31.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.870+0000 7f554dbed640 1 Processor -- start 2024-09-12T03:11:31.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.870+0000 7f554dbed640 1 -- start start 2024-09-12T03:11:31.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.870+0000 7f554dbed640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55481079c0 0x7f554807db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:31.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.870+0000 7f554dbed640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f55481090c0 con 0x7f55481079c0 2024-09-12T03:11:31.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.870+0000 7f55477fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55481079c0 0x7f554807db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:31.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.870+0000 7f55477fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55481079c0 0x7f554807db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47660/0 (socket says 172.21.15.6:47660) 2024-09-12T03:11:31.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.870+0000 7f55477fe640 1 -- 172.21.15.6:0/491166594 learned_addr learned my addr 172.21.15.6:0/491166594 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:31.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.871+0000 7f55477fe640 1 -- 172.21.15.6:0/491166594 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554807e090 con 0x7f55481079c0 2024-09-12T03:11:31.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.871+0000 7f55477fe640 1 --2- 172.21.15.6:0/491166594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55481079c0 0x7f554807db50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f55380090b0 tx=0x7f55380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:31.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.871+0000 7f5544ff9640 1 -- 172.21.15.6:0/491166594 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5538041070 con 0x7f55481079c0 2024-09-12T03:11:31.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.871+0000 7f5544ff9640 1 -- 172.21.15.6:0/491166594 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f553803c7a0 con 0x7f55481079c0 2024-09-12T03:11:31.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.871+0000 7f554dbed640 1 -- 172.21.15.6:0/491166594 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554807c280 con 0x7f55481079c0 2024-09-12T03:11:31.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.871+0000 7f5544ff9640 1 -- 172.21.15.6:0/491166594 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5538037040 con 0x7f55481079c0 2024-09-12T03:11:31.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.871+0000 7f554dbed640 1 -- 172.21.15.6:0/491166594 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f554807c7c0 con 0x7f55481079c0 2024-09-12T03:11:31.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.871+0000 7f5544ff9640 1 -- 172.21.15.6:0/491166594 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5538044070 con 0x7f55481079c0 2024-09-12T03:11:31.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.871+0000 7f5544ff9640 1 -- 172.21.15.6:0/491166594 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f553803d3f0 con 0x7f55481079c0 2024-09-12T03:11:31.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.872+0000 7f554dbed640 1 -- 172.21.15.6:0/491166594 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5548109710 con 0x7f55481079c0 2024-09-12T03:11:31.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.875+0000 7f5544ff9640 1 -- 172.21.15.6:0/491166594 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f5538046450 con 0x7f55481079c0 2024-09-12T03:11:31.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.936+0000 7f554dbed640 1 -- 172.21.15.6:0/491166594 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f554807cf20 con 0x7f55481079c0 2024-09-12T03:11:31.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.937+0000 7f5544ff9640 1 -- 172.21.15.6:0/491166594 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+316 (secure 0 0 0) 0x7f5538046630 con 0x7f55481079c0 2024-09-12T03:11:31.454 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.937+0000 7f554dbed640 1 -- 172.21.15.6:0/491166594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55481079c0 msgr2=0x7f554807db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:31.454 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.937+0000 7f554dbed640 1 --2- 172.21.15.6:0/491166594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55481079c0 0x7f554807db50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f55380090b0 tx=0x7f55380046c0 comp rx=0 tx=0).stop 2024-09-12T03:11:31.454 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.938+0000 7f554dbed640 1 -- 172.21.15.6:0/491166594 shutdown_connections 2024-09-12T03:11:31.454 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.938+0000 7f554dbed640 1 --2- 172.21.15.6:0/491166594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55481079c0 0x7f554807db50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:31.454 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.938+0000 7f554dbed640 1 -- 172.21.15.6:0/491166594 >> 172.21.15.6:0/491166594 conn(0x7f55481031b0 msgr2=0x7f554818f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:31.454 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.938+0000 7f554dbed640 1 -- 172.21.15.6:0/491166594 shutdown_connections 2024-09-12T03:11:31.454 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:30.938+0000 7f554dbed640 1 -- 172.21.15.6:0/491166594 wait complete. 2024-09-12T03:11:31.454 INFO:teuthology.orchestra.run.smithi006.stdout:mon is available 2024-09-12T03:11:31.454 INFO:teuthology.orchestra.run.smithi006.stdout:Assimilating anything we can from ceph.conf... 2024-09-12T03:11:31.701 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:31 smithi006 ceph-mon[30622]: from='client.? 172.21.15.6:0/491166594' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-12T03:11:32.456 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-09-12T03:11:32.456 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T03:11:32.456 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout fsid = 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:11:32.456 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.6:3300,v1:172.21.15.6:6789] 2024-09-12T03:11:32.457 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T03:11:32.457 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T03:11:32.457 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T03:11:32.457 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T03:11:32.457 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-09-12T03:11:32.457 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T03:11:32.457 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T03:11:32.457 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-09-12T03:11:32.457 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T03:11:32.457 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-09-12T03:11:32.457 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T03:11:32.457 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T03:11:32.457 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T03:11:32.457 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T03:11:32.458 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T03:11:32.458 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.900+0000 7f754e5d1640 1 Processor -- start 2024-09-12T03:11:32.458 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.900+0000 7f754e5d1640 1 -- start start 2024-09-12T03:11:32.458 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.901+0000 7f754e5d1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75481061f0 0x7f75481065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:32.458 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.901+0000 7f754e5d1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f754807be50 con 0x7f75481061f0 2024-09-12T03:11:32.458 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.901+0000 7f7547fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75481061f0 0x7f75481065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:32.458 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.901+0000 7f7547fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75481061f0 0x7f75481065f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47666/0 (socket says 172.21.15.6:47666) 2024-09-12T03:11:32.458 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.901+0000 7f7547fff640 1 -- 172.21.15.6:0/2397468548 learned_addr learned my addr 172.21.15.6:0/2397468548 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:32.458 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.901+0000 7f7547fff640 1 -- 172.21.15.6:0/2397468548 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7548106b30 con 0x7f75481061f0 2024-09-12T03:11:32.458 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.902+0000 7f7547fff640 1 --2- 172.21.15.6:0/2397468548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75481061f0 0x7f75481065f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7538009820 tx=0x7f753802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2ea2c056c4503a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:32.458 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.902+0000 7f7546ffd640 1 -- 172.21.15.6:0/2397468548 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7538037070 con 0x7f75481061f0 2024-09-12T03:11:32.458 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.902+0000 7f7546ffd640 1 -- 172.21.15.6:0/2397468548 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7538032440 con 0x7f75481061f0 2024-09-12T03:11:32.459 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.902+0000 7f7546ffd640 1 -- 172.21.15.6:0/2397468548 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7538032740 con 0x7f75481061f0 2024-09-12T03:11:32.459 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.903+0000 7f754e5d1640 1 -- 172.21.15.6:0/2397468548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75481061f0 msgr2=0x7f75481065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:32.459 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.903+0000 7f754e5d1640 1 --2- 172.21.15.6:0/2397468548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75481061f0 0x7f75481065f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7538009820 tx=0x7f753802a270 comp rx=0 tx=0).stop 2024-09-12T03:11:32.459 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.903+0000 7f754e5d1640 1 -- 172.21.15.6:0/2397468548 shutdown_connections 2024-09-12T03:11:32.459 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.903+0000 7f754e5d1640 1 --2- 172.21.15.6:0/2397468548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75481061f0 0x7f75481065f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:32.459 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.903+0000 7f754e5d1640 1 -- 172.21.15.6:0/2397468548 >> 172.21.15.6:0/2397468548 conn(0x7f7548102000 msgr2=0x7f7548104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:32.459 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.903+0000 7f754e5d1640 1 -- 172.21.15.6:0/2397468548 shutdown_connections 2024-09-12T03:11:32.459 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.903+0000 7f754e5d1640 1 -- 172.21.15.6:0/2397468548 wait complete. 2024-09-12T03:11:32.459 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.903+0000 7f754e5d1640 1 Processor -- start 2024-09-12T03:11:32.459 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.904+0000 7f754e5d1640 1 -- start start 2024-09-12T03:11:32.459 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.904+0000 7f754e5d1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75481061f0 0x7f754819d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:32.459 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.904+0000 7f754e5d1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f75481070d0 con 0x7f75481061f0 2024-09-12T03:11:32.459 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.904+0000 7f7547fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75481061f0 0x7f754819d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:32.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.904+0000 7f7547fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75481061f0 0x7f754819d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47668/0 (socket says 172.21.15.6:47668) 2024-09-12T03:11:32.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.904+0000 7f7547fff640 1 -- 172.21.15.6:0/231810687 learned_addr learned my addr 172.21.15.6:0/231810687 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:32.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.904+0000 7f7547fff640 1 -- 172.21.15.6:0/231810687 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754819d830 con 0x7f75481061f0 2024-09-12T03:11:32.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.904+0000 7f7547fff640 1 --2- 172.21.15.6:0/231810687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75481061f0 0x7f754819d2f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f75380090b0 tx=0x7f75380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:32.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.905+0000 7f75457fa640 1 -- 172.21.15.6:0/231810687 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7538041070 con 0x7f75481061f0 2024-09-12T03:11:32.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.905+0000 7f75457fa640 1 -- 172.21.15.6:0/231810687 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f753803c7a0 con 0x7f75481061f0 2024-09-12T03:11:32.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.905+0000 7f754e5d1640 1 -- 172.21.15.6:0/231810687 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754819dac0 con 0x7f75481061f0 2024-09-12T03:11:32.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.905+0000 7f75457fa640 1 -- 172.21.15.6:0/231810687 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7538037040 con 0x7f75481061f0 2024-09-12T03:11:32.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.905+0000 7f754e5d1640 1 -- 172.21.15.6:0/231810687 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75481a07b0 con 0x7f75481061f0 2024-09-12T03:11:32.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.905+0000 7f75457fa640 1 -- 172.21.15.6:0/231810687 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7538044070 con 0x7f75481061f0 2024-09-12T03:11:32.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.905+0000 7f75457fa640 1 -- 172.21.15.6:0/231810687 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f753803d3f0 con 0x7f75481061f0 2024-09-12T03:11:32.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.906+0000 7f754e5d1640 1 -- 172.21.15.6:0/231810687 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75481065f0 con 0x7f75481061f0 2024-09-12T03:11:32.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.908+0000 7f75457fa640 1 -- 172.21.15.6:0/231810687 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f7538046450 con 0x7f75481061f0 2024-09-12T03:11:32.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.968+0000 7f754e5d1640 1 -- 172.21.15.6:0/231810687 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f75481a1e90 con 0x7f75481061f0 2024-09-12T03:11:32.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.971+0000 7f75457fa640 1 -- 172.21.15.6:0/231810687 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+452 (secure 0 0 0) 0x7f7538046630 con 0x7f75481061f0 2024-09-12T03:11:32.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.972+0000 7f754e5d1640 1 -- 172.21.15.6:0/231810687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75481061f0 msgr2=0x7f754819d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:32.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.972+0000 7f754e5d1640 1 --2- 172.21.15.6:0/231810687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75481061f0 0x7f754819d2f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f75380090b0 tx=0x7f75380046c0 comp rx=0 tx=0).stop 2024-09-12T03:11:32.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.972+0000 7f754e5d1640 1 -- 172.21.15.6:0/231810687 shutdown_connections 2024-09-12T03:11:32.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.972+0000 7f754e5d1640 1 --2- 172.21.15.6:0/231810687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75481061f0 0x7f754819d2f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:32.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.972+0000 7f754e5d1640 1 -- 172.21.15.6:0/231810687 >> 172.21.15.6:0/231810687 conn(0x7f7548102000 msgr2=0x7f75481938d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:32.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.973+0000 7f754e5d1640 1 -- 172.21.15.6:0/231810687 shutdown_connections 2024-09-12T03:11:32.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:31.973+0000 7f754e5d1640 1 -- 172.21.15.6:0/231810687 wait complete. 2024-09-12T03:11:32.461 INFO:teuthology.orchestra.run.smithi006.stdout:Generating new minimal ceph.conf... 2024-09-12T03:11:32.783 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:32 smithi006 ceph-mon[30622]: from='client.? 172.21.15.6:0/231810687' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T03:11:32.783 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:32 smithi006 ceph-mon[30622]: from='client.? 172.21.15.6:0/231810687' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-12T03:11:33.477 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.862+0000 7fdbf2968640 1 Processor -- start 2024-09-12T03:11:33.477 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.862+0000 7fdbf2968640 1 -- start start 2024-09-12T03:11:33.477 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.863+0000 7fdbf2968640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbec106890 0x7fdbec106c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:33.477 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.863+0000 7fdbf2968640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec107260 con 0x7fdbec106890 2024-09-12T03:11:33.477 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.863+0000 7fdbebfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbec106890 0x7fdbec106c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:33.477 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.863+0000 7fdbebfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbec106890 0x7fdbec106c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47676/0 (socket says 172.21.15.6:47676) 2024-09-12T03:11:33.477 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.863+0000 7fdbebfff640 1 -- 172.21.15.6:0/3410149727 learned_addr learned my addr 172.21.15.6:0/3410149727 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:33.478 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.863+0000 7fdbebfff640 1 -- 172.21.15.6:0/3410149727 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbec107a40 con 0x7fdbec106890 2024-09-12T03:11:33.478 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.864+0000 7fdbebfff640 1 --2- 172.21.15.6:0/3410149727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbec106890 0x7fdbec106c90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009820 tx=0x7fdbdc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8cfde5ecae4b5d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:33.478 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.864+0000 7fdbeaffd640 1 -- 172.21.15.6:0/3410149727 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbdc037070 con 0x7fdbec106890 2024-09-12T03:11:33.478 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.864+0000 7fdbeaffd640 1 -- 172.21.15.6:0/3410149727 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdbdc032900 con 0x7fdbec106890 2024-09-12T03:11:33.478 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.864+0000 7fdbeaffd640 1 -- 172.21.15.6:0/3410149727 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbdc030850 con 0x7fdbec106890 2024-09-12T03:11:33.478 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.865+0000 7fdbf2968640 1 -- 172.21.15.6:0/3410149727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbec106890 msgr2=0x7fdbec106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:33.478 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.865+0000 7fdbf2968640 1 --2- 172.21.15.6:0/3410149727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbec106890 0x7fdbec106c90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009820 tx=0x7fdbdc02a270 comp rx=0 tx=0).stop 2024-09-12T03:11:33.478 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.865+0000 7fdbf2968640 1 -- 172.21.15.6:0/3410149727 shutdown_connections 2024-09-12T03:11:33.478 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.865+0000 7fdbf2968640 1 --2- 172.21.15.6:0/3410149727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbec106890 0x7fdbec106c90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.478 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.865+0000 7fdbf2968640 1 -- 172.21.15.6:0/3410149727 >> 172.21.15.6:0/3410149727 conn(0x7fdbec102000 msgr2=0x7fdbec104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:33.478 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.865+0000 7fdbf2968640 1 -- 172.21.15.6:0/3410149727 shutdown_connections 2024-09-12T03:11:33.478 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.865+0000 7fdbf2968640 1 -- 172.21.15.6:0/3410149727 wait complete. 2024-09-12T03:11:33.479 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.865+0000 7fdbf2968640 1 Processor -- start 2024-09-12T03:11:33.479 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.865+0000 7fdbf2968640 1 -- start start 2024-09-12T03:11:33.479 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.866+0000 7fdbf2968640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbec106890 0x7fdbec07ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:33.479 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.866+0000 7fdbf2968640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec10adf0 con 0x7fdbec106890 2024-09-12T03:11:33.479 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.866+0000 7fdbebfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbec106890 0x7fdbec07ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:33.479 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.866+0000 7fdbebfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbec106890 0x7fdbec07ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47682/0 (socket says 172.21.15.6:47682) 2024-09-12T03:11:33.479 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.866+0000 7fdbebfff640 1 -- 172.21.15.6:0/4195141856 learned_addr learned my addr 172.21.15.6:0/4195141856 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:33.479 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.866+0000 7fdbebfff640 1 -- 172.21.15.6:0/4195141856 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbec19e6e0 con 0x7fdbec106890 2024-09-12T03:11:33.479 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.866+0000 7fdbebfff640 1 --2- 172.21.15.6:0/4195141856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbec106890 0x7fdbec07ba30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009940 tx=0x7fdbdc0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:33.479 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.866+0000 7fdbe97fa640 1 -- 172.21.15.6:0/4195141856 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbdc042070 con 0x7fdbec106890 2024-09-12T03:11:33.479 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.866+0000 7fdbe97fa640 1 -- 172.21.15.6:0/4195141856 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdbdc032be0 con 0x7fdbec106890 2024-09-12T03:11:33.479 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.866+0000 7fdbf2968640 1 -- 172.21.15.6:0/4195141856 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbec19e8b0 con 0x7fdbec106890 2024-09-12T03:11:33.480 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.866+0000 7fdbe97fa640 1 -- 172.21.15.6:0/4195141856 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbdc037040 con 0x7fdbec106890 2024-09-12T03:11:33.480 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.867+0000 7fdbf2968640 1 -- 172.21.15.6:0/4195141856 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbec19ec50 con 0x7fdbec106890 2024-09-12T03:11:33.480 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.867+0000 7fdbe97fa640 1 -- 172.21.15.6:0/4195141856 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdbdc045070 con 0x7fdbec106890 2024-09-12T03:11:33.480 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.867+0000 7fdbe97fa640 1 -- 172.21.15.6:0/4195141856 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdbdc047a30 con 0x7fdbec106890 2024-09-12T03:11:33.480 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.867+0000 7fdbf2968640 1 -- 172.21.15.6:0/4195141856 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbec106d10 con 0x7fdbec106890 2024-09-12T03:11:33.480 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.870+0000 7fdbe97fa640 1 -- 172.21.15.6:0/4195141856 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fdbdc032d90 con 0x7fdbec106890 2024-09-12T03:11:33.480 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.929+0000 7fdbf2968640 1 -- 172.21.15.6:0/4195141856 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fdbec19f090 con 0x7fdbec106890 2024-09-12T03:11:33.480 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.929+0000 7fdbe97fa640 1 -- 172.21.15.6:0/4195141856 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7fdbdc03d430 con 0x7fdbec106890 2024-09-12T03:11:33.480 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.930+0000 7fdbf2968640 1 -- 172.21.15.6:0/4195141856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbec106890 msgr2=0x7fdbec07ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:33.480 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.930+0000 7fdbf2968640 1 --2- 172.21.15.6:0/4195141856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbec106890 0x7fdbec07ba30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009940 tx=0x7fdbdc0326e0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.480 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.930+0000 7fdbf2968640 1 -- 172.21.15.6:0/4195141856 shutdown_connections 2024-09-12T03:11:33.480 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.930+0000 7fdbf2968640 1 --2- 172.21.15.6:0/4195141856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbec106890 0x7fdbec07ba30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.480 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.930+0000 7fdbf2968640 1 -- 172.21.15.6:0/4195141856 >> 172.21.15.6:0/4195141856 conn(0x7fdbec102000 msgr2=0x7fdbec102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:33.481 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.930+0000 7fdbf2968640 1 -- 172.21.15.6:0/4195141856 shutdown_connections 2024-09-12T03:11:33.481 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:32.930+0000 7fdbf2968640 1 -- 172.21.15.6:0/4195141856 wait complete. 2024-09-12T03:11:33.481 INFO:teuthology.orchestra.run.smithi006.stdout:Restarting the monitor... 2024-09-12T03:11:33.732 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:33 smithi006 systemd[1]: Stopping Ceph mon.a for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:11:33.732 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:33 smithi006 ceph-mon[30622]: from='client.? 172.21.15.6:0/4195141856' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:34.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:33 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-a[30618]: 2024-09-12T03:11:33.730+0000 7f8e647a4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:11:34.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:33 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-a[30618]: 2024-09-12T03:11:33.730+0000 7f8e647a4640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-12T03:11:34.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:33 smithi006 podman[30864]: 2024-09-12 03:11:33.906660806 +0000 UTC m=+0.257299801 container died b1adac5cdbede0f670f23ab4556568f39e430a92f2a1ab0d4b16f15f1a4f1d44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-12T03:11:34.415 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:34 smithi006 podman[30864]: 2024-09-12 03:11:34.232008475 +0000 UTC m=+0.582647469 container cleanup b1adac5cdbede0f670f23ab4556568f39e430a92f2a1ab0d4b16f15f1a4f1d44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731) 2024-09-12T03:11:34.415 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:34 smithi006 bash[30864]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-a 2024-09-12T03:11:34.688 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:34 smithi006 podman[30877]: 2024-09-12 03:11:34.416237788 +0000 UTC m=+0.509241234 container remove b1adac5cdbede0f670f23ab4556568f39e430a92f2a1ab0d4b16f15f1a4f1d44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-12T03:11:34.688 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:34 smithi006 systemd[1]: ceph-89d99938-70b4-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-12T03:11:34.688 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:34 smithi006 systemd[1]: Stopped Ceph mon.a for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:11:34.688 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:34 smithi006 systemd[1]: Starting Ceph mon.a for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:11:34.965 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:34 smithi006 podman[30971]: 2024-09-12 03:11:34.858819234 +0000 UTC m=+0.101215009 container create 29ec88f97057c5e91ed7f19e183beb5e3552940502247b6e1d99ce0a0195e70c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:11:34.965 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:34 smithi006 podman[30971]: 2024-09-12 03:11:34.774154089 +0000 UTC m=+0.016549834 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:11:35.123 INFO:teuthology.orchestra.run.smithi006.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-12T03:11:35.355 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 podman[30971]: 2024-09-12 03:11:35.06154359 +0000 UTC m=+0.303939323 container init 29ec88f97057c5e91ed7f19e183beb5e3552940502247b6e1d99ce0a0195e70c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T03:11:35.355 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 podman[30971]: 2024-09-12 03:11:35.064800567 +0000 UTC m=+0.307196300 container start 29ec88f97057c5e91ed7f19e183beb5e3552940502247b6e1d99ce0a0195e70c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-12T03:11:35.355 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T03:11:35.355 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T03:11:35.355 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: pidfile_write: ignore empty --pid-file 2024-09-12T03:11:35.355 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: load: jerasure load: lrc 2024-09-12T03:11:35.355 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: RocksDB version: 7.9.2 2024-09-12T03:11:35.356 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Git sha 0 2024-09-12T03:11:35.356 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T03:11:35.356 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: DB SUMMARY 2024-09-12T03:11:35.356 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: DB Session ID: CMRQLR53NKHH3GXHNS2K 2024-09-12T03:11:35.356 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: CURRENT file: CURRENT 2024-09-12T03:11:35.356 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T03:11:35.356 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-12T03:11:35.356 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-12T03:11:35.356 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 107477 ; 2024-09-12T03:11:35.356 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.error_if_exists: 0 2024-09-12T03:11:35.356 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.create_if_missing: 0 2024-09-12T03:11:35.356 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.paranoid_checks: 1 2024-09-12T03:11:35.356 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T03:11:35.356 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T03:11:35.357 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T03:11:35.357 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.env: 0x563a2a59d940 2024-09-12T03:11:35.357 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T03:11:35.357 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.info_log: 0x563a2c2796e0 2024-09-12T03:11:35.357 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T03:11:35.357 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.statistics: (nil) 2024-09-12T03:11:35.358 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.use_fsync: 0 2024-09-12T03:11:35.358 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_log_file_size: 0 2024-09-12T03:11:35.358 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T03:11:35.358 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T03:11:35.358 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T03:11:35.358 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T03:11:35.358 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.allow_fallocate: 1 2024-09-12T03:11:35.358 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T03:11:35.358 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T03:11:35.358 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.use_direct_reads: 0 2024-09-12T03:11:35.358 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T03:11:35.359 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T03:11:35.359 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.db_log_dir: 2024-09-12T03:11:35.359 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.wal_dir: 2024-09-12T03:11:35.359 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T03:11:35.359 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T03:11:35.359 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T03:11:35.359 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T03:11:35.359 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T03:11:35.359 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T03:11:35.359 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T03:11:35.359 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T03:11:35.359 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.write_buffer_manager: 0x563a2c288dc0 2024-09-12T03:11:35.359 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T03:11:35.359 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T03:11:35.360 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T03:11:35.360 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T03:11:35.360 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T03:11:35.360 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T03:11:35.360 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T03:11:35.360 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T03:11:35.360 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.unordered_write: 0 2024-09-12T03:11:35.360 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T03:11:35.360 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T03:11:35.360 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T03:11:35.360 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T03:11:35.360 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.row_cache: None 2024-09-12T03:11:35.360 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.wal_filter: None 2024-09-12T03:11:35.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T03:11:35.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T03:11:35.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.two_write_queues: 0 2024-09-12T03:11:35.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T03:11:35.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.wal_compression: 0 2024-09-12T03:11:35.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.atomic_flush: 0 2024-09-12T03:11:35.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T03:11:35.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T03:11:35.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T03:11:35.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.log_readahead_size: 0 2024-09-12T03:11:35.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T03:11:35.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T03:11:35.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T03:11:35.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T03:11:35.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T03:11:35.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T03:11:35.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T03:11:35.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_background_jobs: 2 2024-09-12T03:11:35.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_background_compactions: -1 2024-09-12T03:11:35.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_subcompactions: 1 2024-09-12T03:11:35.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T03:11:35.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T03:11:35.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T03:11:35.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T03:11:35.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T03:11:35.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T03:11:35.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T03:11:35.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T03:11:35.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_open_files: -1 2024-09-12T03:11:35.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T03:11:35.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T03:11:35.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T03:11:35.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T03:11:35.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_background_flushes: -1 2024-09-12T03:11:35.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Compression algorithms supported: 2024-09-12T03:11:35.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: kZSTD supported: 0 2024-09-12T03:11:35.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: kXpressCompression supported: 0 2024-09-12T03:11:35.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: kBZip2Compression supported: 0 2024-09-12T03:11:35.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T03:11:35.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: kLZ4Compression supported: 1 2024-09-12T03:11:35.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: kZlibCompression supported: 1 2024-09-12T03:11:35.364 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T03:11:35.364 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: kSnappyCompression supported: 1 2024-09-12T03:11:35.364 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T03:11:35.364 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T03:11:35.364 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-12T03:11:35.364 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T03:11:35.364 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T03:11:35.364 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.merge_operator: 2024-09-12T03:11:35.364 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compaction_filter: None 2024-09-12T03:11:35.364 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T03:11:35.364 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T03:11:35.364 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T03:11:35.364 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T03:11:35.365 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563a2c279320) 2024-09-12T03:11:35.365 INFO:journalctl@ceph.mon.a.smithi006.stdout: cache_index_and_filter_blocks: 1 2024-09-12T03:11:35.365 INFO:journalctl@ceph.mon.a.smithi006.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T03:11:35.365 INFO:journalctl@ceph.mon.a.smithi006.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T03:11:35.365 INFO:journalctl@ceph.mon.a.smithi006.stdout: pin_top_level_index_and_filter: 1 2024-09-12T03:11:35.365 INFO:journalctl@ceph.mon.a.smithi006.stdout: index_type: 0 2024-09-12T03:11:35.365 INFO:journalctl@ceph.mon.a.smithi006.stdout: data_block_index_type: 0 2024-09-12T03:11:35.365 INFO:journalctl@ceph.mon.a.smithi006.stdout: index_shortening: 1 2024-09-12T03:11:35.365 INFO:journalctl@ceph.mon.a.smithi006.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T03:11:35.365 INFO:journalctl@ceph.mon.a.smithi006.stdout: checksum: 4 2024-09-12T03:11:35.365 INFO:journalctl@ceph.mon.a.smithi006.stdout: no_block_cache: 0 2024-09-12T03:11:35.365 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_cache: 0x563a2c26d1f0 2024-09-12T03:11:35.365 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_cache_name: BinnedLRUCache 2024-09-12T03:11:35.366 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_cache_options: 2024-09-12T03:11:35.366 INFO:journalctl@ceph.mon.a.smithi006.stdout: capacity : 536870912 2024-09-12T03:11:35.366 INFO:journalctl@ceph.mon.a.smithi006.stdout: num_shard_bits : 4 2024-09-12T03:11:35.366 INFO:journalctl@ceph.mon.a.smithi006.stdout: strict_capacity_limit : 0 2024-09-12T03:11:35.366 INFO:journalctl@ceph.mon.a.smithi006.stdout: high_pri_pool_ratio: 0.000 2024-09-12T03:11:35.366 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_cache_compressed: (nil) 2024-09-12T03:11:35.366 INFO:journalctl@ceph.mon.a.smithi006.stdout: persistent_cache: (nil) 2024-09-12T03:11:35.366 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_size: 4096 2024-09-12T03:11:35.366 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_size_deviation: 10 2024-09-12T03:11:35.366 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_restart_interval: 16 2024-09-12T03:11:35.366 INFO:journalctl@ceph.mon.a.smithi006.stdout: index_block_restart_interval: 1 2024-09-12T03:11:35.366 INFO:journalctl@ceph.mon.a.smithi006.stdout: metadata_block_size: 4096 2024-09-12T03:11:35.366 INFO:journalctl@ceph.mon.a.smithi006.stdout: partition_filters: 0 2024-09-12T03:11:35.366 INFO:journalctl@ceph.mon.a.smithi006.stdout: use_delta_encoding: 1 2024-09-12T03:11:35.367 INFO:journalctl@ceph.mon.a.smithi006.stdout: filter_policy: bloomfilter 2024-09-12T03:11:35.367 INFO:journalctl@ceph.mon.a.smithi006.stdout: whole_key_filtering: 1 2024-09-12T03:11:35.367 INFO:journalctl@ceph.mon.a.smithi006.stdout: verify_compression: 0 2024-09-12T03:11:35.367 INFO:journalctl@ceph.mon.a.smithi006.stdout: read_amp_bytes_per_bit: 0 2024-09-12T03:11:35.367 INFO:journalctl@ceph.mon.a.smithi006.stdout: format_version: 5 2024-09-12T03:11:35.367 INFO:journalctl@ceph.mon.a.smithi006.stdout: enable_index_compression: 1 2024-09-12T03:11:35.367 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_align: 0 2024-09-12T03:11:35.367 INFO:journalctl@ceph.mon.a.smithi006.stdout: max_auto_readahead_size: 262144 2024-09-12T03:11:35.367 INFO:journalctl@ceph.mon.a.smithi006.stdout: prepopulate_block_cache: 0 2024-09-12T03:11:35.367 INFO:journalctl@ceph.mon.a.smithi006.stdout: initial_auto_readahead_size: 8192 2024-09-12T03:11:35.367 INFO:journalctl@ceph.mon.a.smithi006.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T03:11:35.367 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T03:11:35.367 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T03:11:35.367 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compression: NoCompression 2024-09-12T03:11:35.368 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T03:11:35.368 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T03:11:35.368 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T03:11:35.368 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.num_levels: 7 2024-09-12T03:11:35.368 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T03:11:35.368 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T03:11:35.368 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T03:11:35.368 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T03:11:35.368 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T03:11:35.368 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T03:11:35.368 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T03:11:35.368 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T03:11:35.368 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T03:11:35.369 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T03:11:35.369 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T03:11:35.369 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T03:11:35.369 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T03:11:35.369 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T03:11:35.369 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T03:11:35.369 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T03:11:35.369 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T03:11:35.369 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T03:11:35.369 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T03:11:35.369 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T03:11:35.369 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T03:11:35.369 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T03:11:35.369 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T03:11:35.370 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T03:11:35.370 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T03:11:35.370 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T03:11:35.370 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T03:11:35.370 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T03:11:35.370 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T03:11:35.370 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T03:11:35.370 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T03:11:35.370 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T03:11:35.371 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T03:11:35.371 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T03:11:35.371 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T03:11:35.371 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T03:11:35.371 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T03:11:35.371 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T03:11:35.371 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T03:11:35.371 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T03:11:35.372 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T03:11:35.372 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T03:11:35.372 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T03:11:35.372 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T03:11:35.372 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T03:11:35.372 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T03:11:35.372 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T03:11:35.372 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T03:11:35.372 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T03:11:35.372 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T03:11:35.372 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T03:11:35.372 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T03:11:35.372 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T03:11:35.372 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T03:11:35.373 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.inplace_update_support: 0 2024-09-12T03:11:35.373 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T03:11:35.373 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T03:11:35.373 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T03:11:35.373 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T03:11:35.373 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.bloom_locality: 0 2024-09-12T03:11:35.373 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.max_successive_merges: 0 2024-09-12T03:11:35.373 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T03:11:35.373 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T03:11:35.373 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T03:11:35.373 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T03:11:35.373 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.ttl: 2592000 2024-09-12T03:11:35.373 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T03:11:35.374 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T03:11:35.374 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T03:11:35.374 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.enable_blob_files: false 2024-09-12T03:11:35.374 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.min_blob_size: 0 2024-09-12T03:11:35.374 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T03:11:35.374 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T03:11:35.374 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T03:11:35.374 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T03:11:35.374 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T03:11:35.374 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T03:11:35.374 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T03:11:35.374 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T03:11:35.374 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-12T03:11:35.374 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-12T03:11:35.375 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8ed6c575-9a9a-4789-9d24-fb1eb67ecaa3 2024-09-12T03:11:35.375 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726110695091754, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-12T03:11:35.375 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-12T03:11:35.375 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726110695092904, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 102201, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 342, "table_properties": {"data_size": 100086, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14818, "raw_average_key_size": 49, "raw_value_size": 92070, "raw_average_value_size": 310, "num_data_blocks": 12, "num_entries": 297, "num_filter_entries": 297, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726110695, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8ed6c575-9a9a-4789-9d24-fb1eb67ecaa3", "db_session_id": "CMRQLR53NKHH3GXHNS2K", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-12T03:11:35.375 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726110695093004, "job": 1, "event": "recovery_finished"} 2024-09-12T03:11:35.375 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-12T03:11:35.375 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T03:11:35.375 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563a2c29ae00 2024-09-12T03:11:35.375 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: DB pointer 0x563a2c396000 2024-09-12T03:11:35.375 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T03:11:35.375 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T03:11:35.375 INFO:journalctl@ceph.mon.a.smithi006.stdout: ** DB Stats ** 2024-09-12T03:11:35.375 INFO:journalctl@ceph.mon.a.smithi006.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T03:11:35.375 INFO:journalctl@ceph.mon.a.smithi006.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T03:11:35.376 INFO:journalctl@ceph.mon.a.smithi006.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T03:11:35.376 INFO:journalctl@ceph.mon.a.smithi006.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T03:11:35.376 INFO:journalctl@ceph.mon.a.smithi006.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T03:11:35.376 INFO:journalctl@ceph.mon.a.smithi006.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T03:11:35.376 INFO:journalctl@ceph.mon.a.smithi006.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T03:11:35.376 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-09-12T03:11:35.376 INFO:journalctl@ceph.mon.a.smithi006.stdout: ** Compaction Stats [default] ** 2024-09-12T03:11:35.376 INFO:journalctl@ceph.mon.a.smithi006.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T03:11:35.376 INFO:journalctl@ceph.mon.a.smithi006.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T03:11:35.376 INFO:journalctl@ceph.mon.a.smithi006.stdout: L0 2/0 101.67 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T03:11:35.376 INFO:journalctl@ceph.mon.a.smithi006.stdout: Sum 2/0 101.67 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T03:11:35.376 INFO:journalctl@ceph.mon.a.smithi006.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T03:11:35.376 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-09-12T03:11:35.376 INFO:journalctl@ceph.mon.a.smithi006.stdout: ** Compaction Stats [default] ** 2024-09-12T03:11:35.377 INFO:journalctl@ceph.mon.a.smithi006.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T03:11:35.377 INFO:journalctl@ceph.mon.a.smithi006.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T03:11:35.377 INFO:journalctl@ceph.mon.a.smithi006.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 133.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T03:11:35.377 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-09-12T03:11:35.377 INFO:journalctl@ceph.mon.a.smithi006.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T03:11:35.377 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-09-12T03:11:35.377 INFO:journalctl@ceph.mon.a.smithi006.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T03:11:35.377 INFO:journalctl@ceph.mon.a.smithi006.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T03:11:35.377 INFO:journalctl@ceph.mon.a.smithi006.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T03:11:35.377 INFO:journalctl@ceph.mon.a.smithi006.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T03:11:35.377 INFO:journalctl@ceph.mon.a.smithi006.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T03:11:35.377 INFO:journalctl@ceph.mon.a.smithi006.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T03:11:35.377 INFO:journalctl@ceph.mon.a.smithi006.stdout: Cumulative compaction: 0.00 GB write, 29.20 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T03:11:35.377 INFO:journalctl@ceph.mon.a.smithi006.stdout: Interval compaction: 0.00 GB write, 29.20 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T03:11:35.378 INFO:journalctl@ceph.mon.a.smithi006.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T03:11:35.378 INFO:journalctl@ceph.mon.a.smithi006.stdout: Block cache BinnedLRUCache@0x563a2c26d1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-12T03:11:35.378 INFO:journalctl@ceph.mon.a.smithi006.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-12T03:11:35.378 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-09-12T03:11:35.378 INFO:journalctl@ceph.mon.a.smithi006.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T03:11:35.378 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: starting mon.a rank 0 at public addrs [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] at bind addrs [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:11:35.378 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: mon.a@-1(???) e1 preinit fsid 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:11:35.378 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: mon.a@-1(???).mds e1 new map 2024-09-12T03:11:35.378 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: mon.a@-1(???).mds e1 print_map 2024-09-12T03:11:35.378 INFO:journalctl@ceph.mon.a.smithi006.stdout: e1 2024-09-12T03:11:35.378 INFO:journalctl@ceph.mon.a.smithi006.stdout: btime 2024-09-12T03:11:29:530327+0000 2024-09-12T03:11:35.378 INFO:journalctl@ceph.mon.a.smithi006.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T03:11:35.379 INFO:journalctl@ceph.mon.a.smithi006.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T03:11:35.379 INFO:journalctl@ceph.mon.a.smithi006.stdout: legacy client fscid: -1 2024-09-12T03:11:35.379 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-09-12T03:11:35.379 INFO:journalctl@ceph.mon.a.smithi006.stdout: No filesystems configured 2024-09-12T03:11:35.379 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T03:11:35.379 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:11:35.379 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:11:35.379 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:11:35.379 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-12T03:11:35.379 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 bash[30971]: 29ec88f97057c5e91ed7f19e183beb5e3552940502247b6e1d99ce0a0195e70c 2024-09-12T03:11:35.379 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 systemd[1]: Started Ceph mon.a for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:11:35.379 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T03:11:35.379 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: monmap epoch 1 2024-09-12T03:11:35.379 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: fsid 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:11:35.379 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: last_changed 2024-09-12T03:11:20.144939+0000 2024-09-12T03:11:35.380 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: created 2024-09-12T03:11:20.144939+0000 2024-09-12T03:11:35.380 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: min_mon_release 19 (squid) 2024-09-12T03:11:35.380 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: election_strategy: 1 2024-09-12T03:11:35.380 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: 0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-09-12T03:11:35.380 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: fsmap 2024-09-12T03:11:35.380 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T03:11:35.380 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:35 smithi006 ceph-mon[30991]: mgrmap e1: no daemons active 2024-09-12T03:11:37.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:36 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1763942952' entity='client.admin' 2024-09-12T03:11:39.190 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.538+0000 7fd890597640 1 Processor -- start 2024-09-12T03:11:39.194 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.538+0000 7fd890597640 1 -- start start 2024-09-12T03:11:39.194 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.538+0000 7fd890597640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd888107e30 0x7fd888108230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.194 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.538+0000 7fd890597640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd888108800 con 0x7fd888107e30 2024-09-12T03:11:39.194 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.539+0000 7fd88e333640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd888107e30 0x7fd888108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.194 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.539+0000 7fd88e333640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd888107e30 0x7fd888108230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47690/0 (socket says 172.21.15.6:47690) 2024-09-12T03:11:39.194 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.539+0000 7fd88e333640 1 -- 172.21.15.6:0/734737415 learned_addr learned my addr 172.21.15.6:0/734737415 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:39.194 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.539+0000 7fd88e333640 1 -- 172.21.15.6:0/734737415 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd888106a40 con 0x7fd888107e30 2024-09-12T03:11:39.194 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.541+0000 7fd88e333640 1 --2- 172.21.15.6:0/734737415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd888107e30 0x7fd888108230 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd884009820 tx=0x7fd88402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3d9a1ccfac1410 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:39.195 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.541+0000 7fd88d331640 1 -- 172.21.15.6:0/734737415 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd884037070 con 0x7fd888107e30 2024-09-12T03:11:39.195 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.541+0000 7fd88d331640 1 -- 172.21.15.6:0/734737415 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd884032900 con 0x7fd888107e30 2024-09-12T03:11:39.195 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.541+0000 7fd88d331640 1 -- 172.21.15.6:0/734737415 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd884030850 con 0x7fd888107e30 2024-09-12T03:11:39.195 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.542+0000 7fd890597640 1 -- 172.21.15.6:0/734737415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd888107e30 msgr2=0x7fd888108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:39.195 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.542+0000 7fd890597640 1 --2- 172.21.15.6:0/734737415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd888107e30 0x7fd888108230 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd884009820 tx=0x7fd88402a270 comp rx=0 tx=0).stop 2024-09-12T03:11:39.195 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.542+0000 7fd890597640 1 -- 172.21.15.6:0/734737415 shutdown_connections 2024-09-12T03:11:39.195 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.542+0000 7fd890597640 1 --2- 172.21.15.6:0/734737415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd888107e30 0x7fd888108230 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.195 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.542+0000 7fd890597640 1 -- 172.21.15.6:0/734737415 >> 172.21.15.6:0/734737415 conn(0x7fd8881017a0 msgr2=0x7fd888103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:39.195 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.542+0000 7fd890597640 1 -- 172.21.15.6:0/734737415 shutdown_connections 2024-09-12T03:11:39.195 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.542+0000 7fd890597640 1 -- 172.21.15.6:0/734737415 wait complete. 2024-09-12T03:11:39.195 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.542+0000 7fd890597640 1 Processor -- start 2024-09-12T03:11:39.195 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.543+0000 7fd890597640 1 -- start start 2024-09-12T03:11:39.196 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.543+0000 7fd890597640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd888107e30 0x7fd88819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.196 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.543+0000 7fd890597640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd88810ae00 con 0x7fd888107e30 2024-09-12T03:11:39.196 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.543+0000 7fd88e333640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd888107e30 0x7fd88819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.196 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.543+0000 7fd88e333640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd888107e30 0x7fd88819ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47704/0 (socket says 172.21.15.6:47704) 2024-09-12T03:11:39.196 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.543+0000 7fd88e333640 1 -- 172.21.15.6:0/1763942952 learned_addr learned my addr 172.21.15.6:0/1763942952 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:39.196 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.543+0000 7fd88e333640 1 -- 172.21.15.6:0/1763942952 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88819bf50 con 0x7fd888107e30 2024-09-12T03:11:39.196 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.543+0000 7fd88e333640 1 --2- 172.21.15.6:0/1763942952 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd888107e30 0x7fd88819ba10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd884009940 tx=0x7fd8840326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:39.196 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.543+0000 7fd87f7fe640 1 -- 172.21.15.6:0/1763942952 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd884042070 con 0x7fd888107e30 2024-09-12T03:11:39.196 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.543+0000 7fd87f7fe640 1 -- 172.21.15.6:0/1763942952 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd884032ca0 con 0x7fd888107e30 2024-09-12T03:11:39.196 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.543+0000 7fd890597640 1 -- 172.21.15.6:0/1763942952 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88819c1e0 con 0x7fd888107e30 2024-09-12T03:11:39.196 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.543+0000 7fd87f7fe640 1 -- 172.21.15.6:0/1763942952 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd884037040 con 0x7fd888107e30 2024-09-12T03:11:39.196 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.543+0000 7fd890597640 1 -- 172.21.15.6:0/1763942952 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88819eed0 con 0x7fd888107e30 2024-09-12T03:11:39.197 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.544+0000 7fd87f7fe640 1 -- 172.21.15.6:0/1763942952 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd884045070 con 0x7fd888107e30 2024-09-12T03:11:39.197 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.544+0000 7fd87f7fe640 1 -- 172.21.15.6:0/1763942952 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd8840479e0 con 0x7fd888107e30 2024-09-12T03:11:39.197 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.544+0000 7fd890597640 1 -- 172.21.15.6:0/1763942952 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8881082b0 con 0x7fd888107e30 2024-09-12T03:11:39.197 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.547+0000 7fd87f7fe640 1 -- 172.21.15.6:0/1763942952 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd88403d9f0 con 0x7fd888107e30 2024-09-12T03:11:39.197 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.608+0000 7fd890597640 1 -- 172.21.15.6:0/1763942952 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fd88802d9b0 con 0x7fd888107e30 2024-09-12T03:11:39.197 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.613+0000 7fd87f7fe640 1 -- 172.21.15.6:0/1763942952 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fd884047be0 con 0x7fd888107e30 2024-09-12T03:11:39.197 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.614+0000 7fd890597640 1 -- 172.21.15.6:0/1763942952 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd888107e30 msgr2=0x7fd88819ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:39.197 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.614+0000 7fd890597640 1 --2- 172.21.15.6:0/1763942952 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd888107e30 0x7fd88819ba10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd884009940 tx=0x7fd8840326e0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.197 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.614+0000 7fd890597640 1 -- 172.21.15.6:0/1763942952 shutdown_connections 2024-09-12T03:11:39.197 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.614+0000 7fd890597640 1 --2- 172.21.15.6:0/1763942952 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd888107e30 0x7fd88819ba10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.197 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.614+0000 7fd890597640 1 -- 172.21.15.6:0/1763942952 >> 172.21.15.6:0/1763942952 conn(0x7fd8881017a0 msgr2=0x7fd8881026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:39.197 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.614+0000 7fd890597640 1 -- 172.21.15.6:0/1763942952 shutdown_connections 2024-09-12T03:11:39.198 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:35.614+0000 7fd890597640 1 -- 172.21.15.6:0/1763942952 wait complete. 2024-09-12T03:11:39.198 INFO:teuthology.orchestra.run.smithi006.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-12T03:11:39.212 INFO:teuthology.orchestra.run.smithi006.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:11:39.212 INFO:teuthology.orchestra.run.smithi006.stdout:Creating mgr... 2024-09-12T03:11:39.212 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-12T03:11:39.213 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-12T03:11:39.470 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-89d99938-70b4-11ef-bcea-c7b262605968@mgr.x 2024-09-12T03:11:39.470 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to reset failed state of unit ceph-89d99938-70b4-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-89d99938-70b4-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-12T03:11:39.622 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-89d99938-70b4-11ef-bcea-c7b262605968.target.wants/ceph-89d99938-70b4-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-89d99938-70b4-11ef-bcea-c7b262605968@.service. 2024-09-12T03:11:39.852 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:39 smithi006 systemd[1]: Starting Ceph mgr.x for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:11:40.111 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:39 smithi006 podman[31205]: 2024-09-12 03:11:39.755915735 +0000 UTC m=+0.016093717 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:11:40.112 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:39 smithi006 podman[31205]: 2024-09-12 03:11:39.881982528 +0000 UTC m=+0.142160495 container create 7792be15ececa21c77a936256cce69d498d0d2e60a52c2af05b04f6a8ccc6a7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-12T03:11:40.112 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:40 smithi006 podman[31205]: 2024-09-12 03:11:40.075949961 +0000 UTC m=+0.336127927 container init 7792be15ececa21c77a936256cce69d498d0d2e60a52c2af05b04f6a8ccc6a7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-09-12T03:11:40.112 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:40 smithi006 podman[31205]: 2024-09-12 03:11:40.078731669 +0000 UTC m=+0.338909629 container start 7792be15ececa21c77a936256cce69d498d0d2e60a52c2af05b04f6a8ccc6a7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-12T03:11:40.189 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T03:11:40.190 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2024-09-12T03:11:40.197 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T03:11:40.197 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-09-12T03:11:40.197 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2024-09-12T03:11:40.197 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T03:11:40.206 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T03:11:40.206 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2024-09-12T03:11:40.214 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T03:11:40.214 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-09-12T03:11:40.214 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2024-09-12T03:11:40.214 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-12T03:11:40.214 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr to start... 2024-09-12T03:11:40.214 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr... 2024-09-12T03:11:40.460 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:40 smithi006 bash[31205]: 7792be15ececa21c77a936256cce69d498d0d2e60a52c2af05b04f6a8ccc6a7e 2024-09-12T03:11:40.461 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:40 smithi006 systemd[1]: Started Ceph mgr.x for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:11:40.461 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:40 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:40.203+0000 7fcff4db3040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:11:40.461 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:40 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:40.266+0000 7fcff4db3040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:11:40.461 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:40 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:40.424+0000 7fcff4db3040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:11:40.738 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:40 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:40.496+0000 7fcff4db3040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:11:40.739 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:40 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:40.653+0000 7fcff4db3040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:11:41.014 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:40 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:40.809+0000 7fcff4db3040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:11:41.015 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:40 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/4020203547' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T03:11:41.324 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:41 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:41.144+0000 7fcff4db3040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:11:41.324 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:41 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:41.209+0000 7fcff4db3040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:11:41.326 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-09-12T03:11:41.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:11:41.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsid": "89d99938-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:11:41.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T03:11:41.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T03:11:41.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T03:11:41.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T03:11:41.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:11:41.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T03:11:41.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T03:11:41.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 0 2024-09-12T03:11:41.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:11:41.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T03:11:41.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T03:11:41.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:11:41.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-12T03:11:41.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T03:11:41.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:11:41.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T03:11:41.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T03:11:41.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:11:41.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T03:11:41.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:11:41.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T03:11:41.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T03:11:41.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T03:11:41.338 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T03:11:41.338 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T03:11:41.338 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T03:11:41.338 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:11:41.338 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T03:11:41.338 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T03:11:41.338 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T03:11:41.338 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T03:11:41.338 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T03:11:41.338 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T03:11:41.339 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T03:11:41.339 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T03:11:41.339 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T03:11:41.339 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:11:41.339 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T03:11:41.339 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:11:41.339 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T03:11:29:530327+0000", 2024-09-12T03:11:41.339 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T03:11:41.339 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T03:11:41.339 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:11:41.339 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T03:11:41.339 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T03:11:41.339 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T03:11:41.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T03:11:41.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T03:11:41.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T03:11:41.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T03:11:41.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:11:41.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:11:41.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:11:41.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T03:11:41.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:11:41.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T03:11:29.531090+0000", 2024-09-12T03:11:41.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:11:41.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:11:41.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T03:11:41.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:11:41.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.667+0000 7f66190ce640 1 Processor -- start 2024-09-12T03:11:41.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.668+0000 7f66190ce640 1 -- start start 2024-09-12T03:11:41.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.668+0000 7f66190ce640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6614106890 0x7f6614106c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:41.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.668+0000 7f66190ce640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6614107260 con 0x7f6614106890 2024-09-12T03:11:41.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.668+0000 7f6612d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6614106890 0x7f6614106c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:41.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.668+0000 7f6612d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6614106890 0x7f6614106c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43262/0 (socket says 172.21.15.6:43262) 2024-09-12T03:11:41.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.668+0000 7f6612d9d640 1 -- 172.21.15.6:0/3636249934 learned_addr learned my addr 172.21.15.6:0/3636249934 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:41.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.669+0000 7f6612d9d640 1 -- 172.21.15.6:0/3636249934 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6614107a40 con 0x7f6614106890 2024-09-12T03:11:41.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.669+0000 7f6612d9d640 1 --2- 172.21.15.6:0/3636249934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6614106890 0x7f6614106c90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6604009820 tx=0x7f660402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ebc06fd1a04b01 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:41.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.669+0000 7f661259c640 1 -- 172.21.15.6:0/3636249934 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6604037070 con 0x7f6614106890 2024-09-12T03:11:41.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.669+0000 7f661259c640 1 -- 172.21.15.6:0/3636249934 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6604032900 con 0x7f6614106890 2024-09-12T03:11:41.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.670+0000 7f661259c640 1 -- 172.21.15.6:0/3636249934 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6604030850 con 0x7f6614106890 2024-09-12T03:11:41.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.670+0000 7f66190ce640 1 -- 172.21.15.6:0/3636249934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6614106890 msgr2=0x7f6614106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:41.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.670+0000 7f66190ce640 1 --2- 172.21.15.6:0/3636249934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6614106890 0x7f6614106c90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6604009820 tx=0x7f660402a270 comp rx=0 tx=0).stop 2024-09-12T03:11:41.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.670+0000 7f66190ce640 1 -- 172.21.15.6:0/3636249934 shutdown_connections 2024-09-12T03:11:41.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.670+0000 7f66190ce640 1 --2- 172.21.15.6:0/3636249934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6614106890 0x7f6614106c90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:41.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.670+0000 7f66190ce640 1 -- 172.21.15.6:0/3636249934 >> 172.21.15.6:0/3636249934 conn(0x7f6614102000 msgr2=0x7f6614104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:41.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.670+0000 7f66190ce640 1 -- 172.21.15.6:0/3636249934 shutdown_connections 2024-09-12T03:11:41.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.670+0000 7f66190ce640 1 -- 172.21.15.6:0/3636249934 wait complete. 2024-09-12T03:11:41.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.671+0000 7f66190ce640 1 Processor -- start 2024-09-12T03:11:41.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.671+0000 7f66190ce640 1 -- start start 2024-09-12T03:11:41.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.671+0000 7f66190ce640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6614106890 0x7f661419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:41.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.672+0000 7f66190ce640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f661407b130 con 0x7f6614106890 2024-09-12T03:11:41.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.672+0000 7f6612d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6614106890 0x7f661419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:41.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.672+0000 7f6612d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6614106890 0x7f661419db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43268/0 (socket says 172.21.15.6:43268) 2024-09-12T03:11:41.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.672+0000 7f6612d9d640 1 -- 172.21.15.6:0/4020203547 learned_addr learned my addr 172.21.15.6:0/4020203547 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:41.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.672+0000 7f6612d9d640 1 -- 172.21.15.6:0/4020203547 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661419e0c0 con 0x7f6614106890 2024-09-12T03:11:41.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.672+0000 7f6612d9d640 1 --2- 172.21.15.6:0/4020203547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6614106890 0x7f661419db80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f66040097f0 tx=0x7f6604030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:41.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.672+0000 7f6610d99640 1 -- 172.21.15.6:0/4020203547 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6604042070 con 0x7f6614106890 2024-09-12T03:11:41.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.672+0000 7f6610d99640 1 -- 172.21.15.6:0/4020203547 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6604032ca0 con 0x7f6614106890 2024-09-12T03:11:41.346 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.672+0000 7f66190ce640 1 -- 172.21.15.6:0/4020203547 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661419e350 con 0x7f6614106890 2024-09-12T03:11:41.346 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.672+0000 7f6610d99640 1 -- 172.21.15.6:0/4020203547 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6604037040 con 0x7f6614106890 2024-09-12T03:11:41.346 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.673+0000 7f66190ce640 1 -- 172.21.15.6:0/4020203547 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66141a1040 con 0x7f6614106890 2024-09-12T03:11:41.346 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.673+0000 7f6610d99640 1 -- 172.21.15.6:0/4020203547 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6604045070 con 0x7f6614106890 2024-09-12T03:11:41.346 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.673+0000 7f6610d99640 1 -- 172.21.15.6:0/4020203547 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6604047ae0 con 0x7f6614106890 2024-09-12T03:11:41.346 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.674+0000 7f66190ce640 1 -- 172.21.15.6:0/4020203547 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6614106c90 con 0x7f6614106890 2024-09-12T03:11:41.346 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.676+0000 7f6610d99640 1 -- 172.21.15.6:0/4020203547 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f660403da60 con 0x7f6614106890 2024-09-12T03:11:41.346 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.738+0000 7f66190ce640 1 -- 172.21.15.6:0/4020203547 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f66141976b0 con 0x7f6614106890 2024-09-12T03:11:41.346 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.738+0000 7f6610d99640 1 -- 172.21.15.6:0/4020203547 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f660403d430 con 0x7f6614106890 2024-09-12T03:11:41.346 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.739+0000 7f66190ce640 1 -- 172.21.15.6:0/4020203547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6614106890 msgr2=0x7f661419db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:41.346 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.739+0000 7f66190ce640 1 --2- 172.21.15.6:0/4020203547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6614106890 0x7f661419db80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f66040097f0 tx=0x7f6604030ec0 comp rx=0 tx=0).stop 2024-09-12T03:11:41.346 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.739+0000 7f66190ce640 1 -- 172.21.15.6:0/4020203547 shutdown_connections 2024-09-12T03:11:41.346 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.740+0000 7f66190ce640 1 --2- 172.21.15.6:0/4020203547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6614106890 0x7f661419db80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:41.346 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.740+0000 7f66190ce640 1 -- 172.21.15.6:0/4020203547 >> 172.21.15.6:0/4020203547 conn(0x7f6614102000 msgr2=0x7f6614104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:41.347 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.740+0000 7f66190ce640 1 -- 172.21.15.6:0/4020203547 shutdown_connections 2024-09-12T03:11:41.347 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:40.740+0000 7f66190ce640 1 -- 172.21.15.6:0/4020203547 wait complete. 2024-09-12T03:11:41.347 INFO:teuthology.orchestra.run.smithi006.stdout:mgr not available, waiting (1/15)... 2024-09-12T03:11:41.611 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:41 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:41.404+0000 7fcff4db3040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:11:42.611 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:42 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:42.331+0000 7fcff4db3040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:11:42.612 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:42 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:42.402+0000 7fcff4db3040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:11:43.437 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:43 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:43.159+0000 7fcff4db3040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:11:43.438 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:43 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:43.279+0000 7fcff4db3040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:11:43.734 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:43 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:43.437+0000 7fcff4db3040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:11:43.734 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:43 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:43.502+0000 7fcff4db3040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:11:43.734 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:43 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:43.661+0000 7fcff4db3040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:11:43.734 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:43 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:43.734+0000 7fcff4db3040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:11:44.039 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:43 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/109043465' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T03:11:44.040 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:43 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:43.799+0000 7fcff4db3040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:11:44.040 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:43 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:43.970+0000 7fcff4db3040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:11:44.040 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:44 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:44.039+0000 7fcff4db3040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:11:44.321 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:44 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:44.109+0000 7fcff4db3040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:11:44.321 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:44 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:44.196+0000 7fcff4db3040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:11:44.321 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:44 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:44.259+0000 7fcff4db3040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:11:44.321 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:44 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:44.321+0000 7fcff4db3040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:11:44.770 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-09-12T03:11:44.770 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:11:44.770 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsid": "89d99938-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:11:44.770 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T03:11:44.770 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T03:11:44.770 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T03:11:44.770 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T03:11:44.770 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:11:44.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T03:11:44.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T03:11:44.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 0 2024-09-12T03:11:44.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:11:44.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T03:11:44.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T03:11:44.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:11:44.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-12T03:11:44.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T03:11:44.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:11:44.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T03:11:44.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T03:11:44.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:11:44.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T03:11:44.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:11:44.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T03:11:44.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T03:11:44.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T03:11:44.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T03:11:44.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T03:11:44.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T03:11:44.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:11:44.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T03:11:44.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T03:11:44.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T03:11:44.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T03:11:44.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T03:11:44.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T03:11:44.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T03:11:44.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T03:11:44.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T03:11:44.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:11:44.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T03:11:44.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:11:44.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T03:11:29:530327+0000", 2024-09-12T03:11:44.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T03:11:44.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T03:11:44.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:11:44.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T03:11:44.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T03:11:44.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T03:11:44.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T03:11:44.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T03:11:44.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T03:11:44.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T03:11:44.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:11:44.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:11:44.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:11:44.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T03:11:44.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:11:44.775 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T03:11:29.531090+0000", 2024-09-12T03:11:44.775 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:11:44.775 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:11:44.775 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T03:11:44.775 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:11:44.775 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.746+0000 7f3e4bb34640 1 Processor -- start 2024-09-12T03:11:44.775 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.746+0000 7f3e4bb34640 1 -- start start 2024-09-12T03:11:44.775 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.747+0000 7f3e4bb34640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e44108400 0x7f3e4410a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:44.775 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.747+0000 7f3e4bb34640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4407be40 con 0x7f3e44108400 2024-09-12T03:11:44.775 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.747+0000 7f3e498d0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e44108400 0x7f3e4410a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:44.775 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.747+0000 7f3e498d0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e44108400 0x7f3e4410a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43270/0 (socket says 172.21.15.6:43270) 2024-09-12T03:11:44.775 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.747+0000 7f3e498d0640 1 -- 172.21.15.6:0/2381670553 learned_addr learned my addr 172.21.15.6:0/2381670553 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:44.775 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.747+0000 7f3e498d0640 1 -- 172.21.15.6:0/2381670553 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4410ad30 con 0x7f3e44108400 2024-09-12T03:11:44.775 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.748+0000 7f3e498d0640 1 --2- 172.21.15.6:0/2381670553 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e44108400 0x7f3e4410a7f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009820 tx=0x7f3e4002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40931587ee89d880 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:44.776 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.748+0000 7f3e488ce640 1 -- 172.21.15.6:0/2381670553 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e40037070 con 0x7f3e44108400 2024-09-12T03:11:44.776 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.748+0000 7f3e488ce640 1 -- 172.21.15.6:0/2381670553 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3e40032900 con 0x7f3e44108400 2024-09-12T03:11:44.776 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.748+0000 7f3e488ce640 1 -- 172.21.15.6:0/2381670553 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e40030850 con 0x7f3e44108400 2024-09-12T03:11:44.776 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.749+0000 7f3e4bb34640 1 -- 172.21.15.6:0/2381670553 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e44108400 msgr2=0x7f3e4410a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:44.776 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.749+0000 7f3e4bb34640 1 --2- 172.21.15.6:0/2381670553 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e44108400 0x7f3e4410a7f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009820 tx=0x7f3e4002a270 comp rx=0 tx=0).stop 2024-09-12T03:11:44.776 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.749+0000 7f3e4bb34640 1 -- 172.21.15.6:0/2381670553 shutdown_connections 2024-09-12T03:11:44.776 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.749+0000 7f3e4bb34640 1 --2- 172.21.15.6:0/2381670553 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e44108400 0x7f3e4410a7f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:44.776 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.749+0000 7f3e4bb34640 1 -- 172.21.15.6:0/2381670553 >> 172.21.15.6:0/2381670553 conn(0x7f3e44102000 msgr2=0x7f3e44104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:44.776 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.749+0000 7f3e4bb34640 1 -- 172.21.15.6:0/2381670553 shutdown_connections 2024-09-12T03:11:44.776 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.749+0000 7f3e4bb34640 1 -- 172.21.15.6:0/2381670553 wait complete. 2024-09-12T03:11:44.777 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.749+0000 7f3e4bb34640 1 Processor -- start 2024-09-12T03:11:44.777 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.749+0000 7f3e4bb34640 1 -- start start 2024-09-12T03:11:44.777 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.749+0000 7f3e4bb34640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e44108400 0x7f3e4419b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:44.777 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.749+0000 7f3e4bb34640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4410bd70 con 0x7f3e44108400 2024-09-12T03:11:44.777 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.750+0000 7f3e498d0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e44108400 0x7f3e4419b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:44.777 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.750+0000 7f3e498d0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e44108400 0x7f3e4419b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43284/0 (socket says 172.21.15.6:43284) 2024-09-12T03:11:44.777 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.750+0000 7f3e498d0640 1 -- 172.21.15.6:0/109043465 learned_addr learned my addr 172.21.15.6:0/109043465 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:44.777 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.750+0000 7f3e498d0640 1 -- 172.21.15.6:0/109043465 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4419beb0 con 0x7f3e44108400 2024-09-12T03:11:44.777 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.750+0000 7f3e498d0640 1 --2- 172.21.15.6:0/109043465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e44108400 0x7f3e4419b970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3e40031000 tx=0x7f3e400326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:44.777 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.750+0000 7f3e3affd640 1 -- 172.21.15.6:0/109043465 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e40042070 con 0x7f3e44108400 2024-09-12T03:11:44.777 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.750+0000 7f3e3affd640 1 -- 172.21.15.6:0/109043465 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3e40032be0 con 0x7f3e44108400 2024-09-12T03:11:44.778 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.750+0000 7f3e3affd640 1 -- 172.21.15.6:0/109043465 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e40037040 con 0x7f3e44108400 2024-09-12T03:11:44.778 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.750+0000 7f3e4bb34640 1 -- 172.21.15.6:0/109043465 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e4419c140 con 0x7f3e44108400 2024-09-12T03:11:44.778 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.750+0000 7f3e4bb34640 1 -- 172.21.15.6:0/109043465 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e4419ee30 con 0x7f3e44108400 2024-09-12T03:11:44.778 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.751+0000 7f3e3affd640 1 -- 172.21.15.6:0/109043465 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3e40045070 con 0x7f3e44108400 2024-09-12T03:11:44.778 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.751+0000 7f3e3affd640 1 -- 172.21.15.6:0/109043465 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3e40047b70 con 0x7f3e44108400 2024-09-12T03:11:44.778 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.751+0000 7f3e4bb34640 1 -- 172.21.15.6:0/109043465 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e44108860 con 0x7f3e44108400 2024-09-12T03:11:44.778 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.753+0000 7f3e3affd640 1 -- 172.21.15.6:0/109043465 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f3e4004c030 con 0x7f3e44108400 2024-09-12T03:11:44.778 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.816+0000 7f3e4bb34640 1 -- 172.21.15.6:0/109043465 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3e4419f000 con 0x7f3e44108400 2024-09-12T03:11:44.778 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.816+0000 7f3e3affd640 1 -- 172.21.15.6:0/109043465 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f3e40047e10 con 0x7f3e44108400 2024-09-12T03:11:44.778 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.817+0000 7f3e4bb34640 1 -- 172.21.15.6:0/109043465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e44108400 msgr2=0x7f3e4419b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:44.778 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.817+0000 7f3e4bb34640 1 --2- 172.21.15.6:0/109043465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e44108400 0x7f3e4419b970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3e40031000 tx=0x7f3e400326e0 comp rx=0 tx=0).stop 2024-09-12T03:11:44.779 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.817+0000 7f3e4bb34640 1 -- 172.21.15.6:0/109043465 shutdown_connections 2024-09-12T03:11:44.779 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.817+0000 7f3e4bb34640 1 --2- 172.21.15.6:0/109043465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e44108400 0x7f3e4419b970 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:44.779 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.817+0000 7f3e4bb34640 1 -- 172.21.15.6:0/109043465 >> 172.21.15.6:0/109043465 conn(0x7f3e44102000 msgr2=0x7f3e44102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:44.779 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.817+0000 7f3e4bb34640 1 -- 172.21.15.6:0/109043465 shutdown_connections 2024-09-12T03:11:44.779 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:43.817+0000 7f3e4bb34640 1 -- 172.21.15.6:0/109043465 wait complete. 2024-09-12T03:11:44.779 INFO:teuthology.orchestra.run.smithi006.stdout:mgr not available, waiting (2/15)... 2024-09-12T03:11:45.612 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:45 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:45.153+0000 7fcff4db3040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:11:45.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:45 smithi006 ceph-mon[30991]: Activating manager daemon x 2024-09-12T03:11:45.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:45 smithi006 ceph-mon[30991]: mgrmap e2: x(active, starting, since 0.00350859s) 2024-09-12T03:11:45.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:45 smithi006 ceph-mon[30991]: from='mgr.14100 172.21.15.6:0/4268270992' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T03:11:45.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:45 smithi006 ceph-mon[30991]: from='mgr.14100 172.21.15.6:0/4268270992' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T03:11:45.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:45 smithi006 ceph-mon[30991]: from='mgr.14100 172.21.15.6:0/4268270992' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T03:11:45.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:45 smithi006 ceph-mon[30991]: from='mgr.14100 172.21.15.6:0/4268270992' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:11:45.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:45 smithi006 ceph-mon[30991]: from='mgr.14100 172.21.15.6:0/4268270992' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T03:11:45.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:45 smithi006 ceph-mon[30991]: Manager daemon x is now available 2024-09-12T03:11:45.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:45 smithi006 ceph-mon[30991]: from='mgr.14100 172.21.15.6:0/4268270992' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:11:45.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:45 smithi006 ceph-mon[30991]: from='mgr.14100 172.21.15.6:0/4268270992' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:11:45.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:45 smithi006 ceph-mon[30991]: from='mgr.14100 172.21.15.6:0/4268270992' entity='mgr.x' 2024-09-12T03:11:45.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:45 smithi006 ceph-mon[30991]: from='mgr.14100 172.21.15.6:0/4268270992' entity='mgr.x' 2024-09-12T03:11:45.614 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:45 smithi006 ceph-mon[30991]: from='mgr.14100 172.21.15.6:0/4268270992' entity='mgr.x' 2024-09-12T03:11:47.442 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:47 smithi006 ceph-mon[30991]: mgrmap e3: x(active, since 1.00693s) 2024-09-12T03:11:47.982 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-09-12T03:11:47.982 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:11:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsid": "89d99938-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:11:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T03:11:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T03:11:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T03:11:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T03:11:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:11:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T03:11:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T03:11:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 0 2024-09-12T03:11:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:11:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T03:11:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T03:11:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:11:47.984 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-09-12T03:11:47.984 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T03:11:47.984 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:11:47.984 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T03:11:47.984 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T03:11:47.984 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:11:47.984 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T03:11:47.984 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:11:47.984 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T03:11:47.984 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T03:11:47.984 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T03:11:47.984 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T03:11:47.984 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T03:11:47.985 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T03:11:47.985 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:11:47.985 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T03:11:47.985 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T03:11:47.985 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T03:11:47.985 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T03:11:47.985 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T03:11:47.985 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T03:11:47.985 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T03:11:47.985 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T03:11:47.986 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T03:11:47.986 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:11:47.986 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T03:11:47.986 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:11:47.986 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T03:11:29:530327+0000", 2024-09-12T03:11:47.986 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T03:11:47.986 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T03:11:47.986 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:11:47.986 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T03:11:47.986 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T03:11:47.986 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T03:11:47.986 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T03:11:47.986 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T03:11:47.987 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T03:11:47.987 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T03:11:47.987 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:11:47.987 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:11:47.987 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:11:47.987 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T03:11:47.987 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:11:47.987 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T03:11:29.531090+0000", 2024-09-12T03:11:47.987 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:11:47.987 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:11:47.987 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T03:11:47.987 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:11:47.988 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.188+0000 7f59e52ed640 1 Processor -- start 2024-09-12T03:11:47.988 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.188+0000 7f59e52ed640 1 -- start start 2024-09-12T03:11:47.988 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.188+0000 7f59e52ed640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e0106230 0x7f59e0106630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:47.988 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.188+0000 7f59e52ed640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f59e007be00 con 0x7f59e0106230 2024-09-12T03:11:47.988 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.189+0000 7f59deffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e0106230 0x7f59e0106630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:47.988 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.189+0000 7f59deffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e0106230 0x7f59e0106630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43334/0 (socket says 172.21.15.6:43334) 2024-09-12T03:11:47.988 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.189+0000 7f59deffd640 1 -- 172.21.15.6:0/1434360635 learned_addr learned my addr 172.21.15.6:0/1434360635 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:47.988 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.189+0000 7f59deffd640 1 -- 172.21.15.6:0/1434360635 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e0106b70 con 0x7f59e0106230 2024-09-12T03:11:47.988 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.190+0000 7f59deffd640 1 --2- 172.21.15.6:0/1434360635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e0106230 0x7f59e0106630 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009820 tx=0x7f59d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd65e0b521878ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:47.988 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.190+0000 7f59ddffb640 1 -- 172.21.15.6:0/1434360635 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59d0037070 con 0x7f59e0106230 2024-09-12T03:11:47.988 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.190+0000 7f59ddffb640 1 -- 172.21.15.6:0/1434360635 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f59d0032900 con 0x7f59e0106230 2024-09-12T03:11:47.988 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.190+0000 7f59ddffb640 1 -- 172.21.15.6:0/1434360635 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59d0030850 con 0x7f59e0106230 2024-09-12T03:11:47.988 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.190+0000 7f59e52ed640 1 -- 172.21.15.6:0/1434360635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e0106230 msgr2=0x7f59e0106630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:47.989 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.190+0000 7f59e52ed640 1 --2- 172.21.15.6:0/1434360635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e0106230 0x7f59e0106630 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009820 tx=0x7f59d002a270 comp rx=0 tx=0).stop 2024-09-12T03:11:47.989 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.190+0000 7f59e52ed640 1 -- 172.21.15.6:0/1434360635 shutdown_connections 2024-09-12T03:11:47.989 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.190+0000 7f59e52ed640 1 --2- 172.21.15.6:0/1434360635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e0106230 0x7f59e0106630 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:47.989 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.190+0000 7f59e52ed640 1 -- 172.21.15.6:0/1434360635 >> 172.21.15.6:0/1434360635 conn(0x7f59e0102000 msgr2=0x7f59e0104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:47.989 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.191+0000 7f59e52ed640 1 -- 172.21.15.6:0/1434360635 shutdown_connections 2024-09-12T03:11:47.989 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.191+0000 7f59e52ed640 1 -- 172.21.15.6:0/1434360635 wait complete. 2024-09-12T03:11:47.989 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.191+0000 7f59e52ed640 1 Processor -- start 2024-09-12T03:11:47.989 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.191+0000 7f59e52ed640 1 -- start start 2024-09-12T03:11:47.989 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.192+0000 7f59e52ed640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e0106230 0x7f59e019db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:47.989 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.192+0000 7f59e52ed640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f59e0107c70 con 0x7f59e0106230 2024-09-12T03:11:47.989 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.192+0000 7f59deffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e0106230 0x7f59e019db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:47.989 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.192+0000 7f59deffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e0106230 0x7f59e019db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43340/0 (socket says 172.21.15.6:43340) 2024-09-12T03:11:47.989 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.192+0000 7f59deffd640 1 -- 172.21.15.6:0/3191657828 learned_addr learned my addr 172.21.15.6:0/3191657828 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:47.990 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.192+0000 7f59deffd640 1 -- 172.21.15.6:0/3191657828 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e019e070 con 0x7f59e0106230 2024-09-12T03:11:47.990 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.192+0000 7f59deffd640 1 --2- 172.21.15.6:0/3191657828 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e0106230 0x7f59e019db30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f59d0031000 tx=0x7f59d00326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:47.990 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.192+0000 7f59c7fff640 1 -- 172.21.15.6:0/3191657828 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59d0042070 con 0x7f59e0106230 2024-09-12T03:11:47.990 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.192+0000 7f59c7fff640 1 -- 172.21.15.6:0/3191657828 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f59d0032be0 con 0x7f59e0106230 2024-09-12T03:11:47.990 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.192+0000 7f59e52ed640 1 -- 172.21.15.6:0/3191657828 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e019e300 con 0x7f59e0106230 2024-09-12T03:11:47.990 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.192+0000 7f59c7fff640 1 -- 172.21.15.6:0/3191657828 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59d0037040 con 0x7f59e0106230 2024-09-12T03:11:47.990 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.193+0000 7f59e52ed640 1 -- 172.21.15.6:0/3191657828 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e01a0ff0 con 0x7f59e0106230 2024-09-12T03:11:47.990 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.193+0000 7f59c7fff640 1 -- 172.21.15.6:0/3191657828 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f59d0045070 con 0x7f59e0106230 2024-09-12T03:11:47.990 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.194+0000 7f59e52ed640 1 -- 172.21.15.6:0/3191657828 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59e0106630 con 0x7f59e0106230 2024-09-12T03:11:47.990 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.194+0000 7f59c7fff640 1 --2- 172.21.15.6:0/3191657828 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f59c8038f70 0x7f59c803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:47.990 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.194+0000 7f59c7fff640 1 -- 172.21.15.6:0/3191657828 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f59d006bfb0 con 0x7f59e0106230 2024-09-12T03:11:47.990 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.194+0000 7f59de7fc640 1 --2- 172.21.15.6:0/3191657828 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f59c8038f70 0x7f59c803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:47.990 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.194+0000 7f59de7fc640 1 --2- 172.21.15.6:0/3191657828 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f59c8038f70 0x7f59c803b430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f59d40098c0 tx=0x7f59d4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:47.991 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.199+0000 7f59c7fff640 1 -- 172.21.15.6:0/3191657828 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f59d003a540 con 0x7f59e0106230 2024-09-12T03:11:47.991 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.442+0000 7f59e52ed640 1 -- 172.21.15.6:0/3191657828 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f59e002d9f0 con 0x7f59e0106230 2024-09-12T03:11:47.991 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.442+0000 7f59c7fff640 1 -- 172.21.15.6:0/3191657828 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f59d004c030 con 0x7f59e0106230 2024-09-12T03:11:47.991 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.444+0000 7f59e52ed640 1 -- 172.21.15.6:0/3191657828 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f59c8038f70 msgr2=0x7f59c803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:47.991 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.444+0000 7f59e52ed640 1 --2- 172.21.15.6:0/3191657828 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f59c8038f70 0x7f59c803b430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f59d40098c0 tx=0x7f59d4006db0 comp rx=0 tx=0).stop 2024-09-12T03:11:47.991 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.444+0000 7f59e52ed640 1 -- 172.21.15.6:0/3191657828 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e0106230 msgr2=0x7f59e019db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:47.991 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.444+0000 7f59e52ed640 1 --2- 172.21.15.6:0/3191657828 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e0106230 0x7f59e019db30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f59d0031000 tx=0x7f59d00326e0 comp rx=0 tx=0).stop 2024-09-12T03:11:47.991 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.444+0000 7f59e52ed640 1 -- 172.21.15.6:0/3191657828 shutdown_connections 2024-09-12T03:11:47.991 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.444+0000 7f59e52ed640 1 --2- 172.21.15.6:0/3191657828 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f59c8038f70 0x7f59c803b430 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:47.991 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.444+0000 7f59e52ed640 1 --2- 172.21.15.6:0/3191657828 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e0106230 0x7f59e019db30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:47.991 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.444+0000 7f59e52ed640 1 -- 172.21.15.6:0/3191657828 >> 172.21.15.6:0/3191657828 conn(0x7f59e0102000 msgr2=0x7f59e01947d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:47.991 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.444+0000 7f59e52ed640 1 -- 172.21.15.6:0/3191657828 shutdown_connections 2024-09-12T03:11:47.991 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:47.444+0000 7f59e52ed640 1 -- 172.21.15.6:0/3191657828 wait complete. 2024-09-12T03:11:47.991 INFO:teuthology.orchestra.run.smithi006.stdout:mgr is available 2024-09-12T03:11:48.567 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:48 smithi006 ceph-mon[30991]: mgrmap e4: x(active, since 2s) 2024-09-12T03:11:48.567 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:48 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3191657828' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T03:11:49.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:49 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1880245716' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T03:11:49.218 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-09-12T03:11:49.218 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T03:11:49.218 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout fsid = 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:11:49.218 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.6:3300,v1:172.21.15.6:6789] 2024-09-12T03:11:49.218 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T03:11:49.219 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T03:11:49.219 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T03:11:49.219 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T03:11:49.219 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-09-12T03:11:49.219 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T03:11:49.219 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T03:11:49.219 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-09-12T03:11:49.219 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T03:11:49.219 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-09-12T03:11:49.219 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T03:11:49.219 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T03:11:49.219 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T03:11:49.219 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T03:11:49.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T03:11:49.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.379+0000 7fd210722640 1 Processor -- start 2024-09-12T03:11:49.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.379+0000 7fd210722640 1 -- start start 2024-09-12T03:11:49.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.380+0000 7fd210722640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2081083c0 0x7fd20810a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:49.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.380+0000 7fd210722640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd20807be90 con 0x7fd2081083c0 2024-09-12T03:11:49.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.380+0000 7fd20e4be640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2081083c0 0x7fd20810a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:49.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.380+0000 7fd20e4be640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2081083c0 0x7fd20810a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43342/0 (socket says 172.21.15.6:43342) 2024-09-12T03:11:49.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.380+0000 7fd20e4be640 1 -- 172.21.15.6:0/3788864358 learned_addr learned my addr 172.21.15.6:0/3788864358 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:49.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.380+0000 7fd20e4be640 1 -- 172.21.15.6:0/3788864358 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20810acf0 con 0x7fd2081083c0 2024-09-12T03:11:49.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.381+0000 7fd20e4be640 1 --2- 172.21.15.6:0/3788864358 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2081083c0 0x7fd20810a7b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd204009820 tx=0x7fd20402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961d374e57c18db7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:49.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.381+0000 7fd20d4bc640 1 -- 172.21.15.6:0/3788864358 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd204037070 con 0x7fd2081083c0 2024-09-12T03:11:49.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.381+0000 7fd20d4bc640 1 -- 172.21.15.6:0/3788864358 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd204032900 con 0x7fd2081083c0 2024-09-12T03:11:49.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.381+0000 7fd20d4bc640 1 -- 172.21.15.6:0/3788864358 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd204030850 con 0x7fd2081083c0 2024-09-12T03:11:49.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.382+0000 7fd210722640 1 -- 172.21.15.6:0/3788864358 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2081083c0 msgr2=0x7fd20810a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:49.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.382+0000 7fd210722640 1 --2- 172.21.15.6:0/3788864358 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2081083c0 0x7fd20810a7b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd204009820 tx=0x7fd20402a270 comp rx=0 tx=0).stop 2024-09-12T03:11:49.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.382+0000 7fd210722640 1 -- 172.21.15.6:0/3788864358 shutdown_connections 2024-09-12T03:11:49.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.382+0000 7fd210722640 1 --2- 172.21.15.6:0/3788864358 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2081083c0 0x7fd20810a7b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.382+0000 7fd210722640 1 -- 172.21.15.6:0/3788864358 >> 172.21.15.6:0/3788864358 conn(0x7fd208102000 msgr2=0x7fd208104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:49.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.382+0000 7fd210722640 1 -- 172.21.15.6:0/3788864358 shutdown_connections 2024-09-12T03:11:49.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.382+0000 7fd210722640 1 -- 172.21.15.6:0/3788864358 wait complete. 2024-09-12T03:11:49.223 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.382+0000 7fd210722640 1 Processor -- start 2024-09-12T03:11:49.223 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.383+0000 7fd210722640 1 -- start start 2024-09-12T03:11:49.223 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.383+0000 7fd210722640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2081083c0 0x7fd208107c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:49.223 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.383+0000 7fd210722640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd20810bd90 con 0x7fd2081083c0 2024-09-12T03:11:49.223 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.383+0000 7fd20e4be640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2081083c0 0x7fd208107c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:49.223 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.383+0000 7fd20e4be640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2081083c0 0x7fd208107c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43352/0 (socket says 172.21.15.6:43352) 2024-09-12T03:11:49.223 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.383+0000 7fd20e4be640 1 -- 172.21.15.6:0/1880245716 learned_addr learned my addr 172.21.15.6:0/1880245716 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:49.224 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.383+0000 7fd20e4be640 1 -- 172.21.15.6:0/1880245716 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd208108180 con 0x7fd2081083c0 2024-09-12T03:11:49.224 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.383+0000 7fd20e4be640 1 --2- 172.21.15.6:0/1880245716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2081083c0 0x7fd208107c40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd2040097f0 tx=0x7fd204030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:49.224 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.383+0000 7fd1ff7fe640 1 -- 172.21.15.6:0/1880245716 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd204042070 con 0x7fd2081083c0 2024-09-12T03:11:49.224 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.383+0000 7fd210722640 1 -- 172.21.15.6:0/1880245716 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd208106370 con 0x7fd2081083c0 2024-09-12T03:11:49.224 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.384+0000 7fd1ff7fe640 1 -- 172.21.15.6:0/1880245716 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd204032ca0 con 0x7fd2081083c0 2024-09-12T03:11:49.224 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.384+0000 7fd1ff7fe640 1 -- 172.21.15.6:0/1880245716 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd204037040 con 0x7fd2081083c0 2024-09-12T03:11:49.224 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.384+0000 7fd210722640 1 -- 172.21.15.6:0/1880245716 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2081068b0 con 0x7fd2081083c0 2024-09-12T03:11:49.224 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.384+0000 7fd1ff7fe640 1 -- 172.21.15.6:0/1880245716 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fd204045070 con 0x7fd2081083c0 2024-09-12T03:11:49.225 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.385+0000 7fd1ff7fe640 1 --2- 172.21.15.6:0/1880245716 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7fd1f4038f20 0x7fd1f403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:49.225 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.385+0000 7fd1ff7fe640 1 -- 172.21.15.6:0/1880245716 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd20406bee0 con 0x7fd2081083c0 2024-09-12T03:11:49.225 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.385+0000 7fd210722640 1 -- 172.21.15.6:0/1880245716 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd208108820 con 0x7fd2081083c0 2024-09-12T03:11:49.225 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.385+0000 7fd20dcbd640 1 --2- 172.21.15.6:0/1880245716 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7fd1f4038f20 0x7fd1f403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:49.225 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.385+0000 7fd20dcbd640 1 --2- 172.21.15.6:0/1880245716 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7fd1f4038f20 0x7fd1f403b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80098c0 tx=0x7fd1f8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:49.226 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.389+0000 7fd1ff7fe640 1 -- 172.21.15.6:0/1880245716 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd20409b1b0 con 0x7fd2081083c0 2024-09-12T03:11:49.226 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.567+0000 7fd210722640 1 -- 172.21.15.6:0/1880245716 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd208107050 con 0x7fd2081083c0 2024-09-12T03:11:49.226 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.568+0000 7fd1ff7fe640 1 -- 172.21.15.6:0/1880245716 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+452 (secure 0 0 0) 0x7fd204045360 con 0x7fd2081083c0 2024-09-12T03:11:49.226 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.570+0000 7fd210722640 1 -- 172.21.15.6:0/1880245716 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7fd1f4038f20 msgr2=0x7fd1f403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:49.226 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.570+0000 7fd210722640 1 --2- 172.21.15.6:0/1880245716 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7fd1f4038f20 0x7fd1f403b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80098c0 tx=0x7fd1f8006db0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.226 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.570+0000 7fd210722640 1 -- 172.21.15.6:0/1880245716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2081083c0 msgr2=0x7fd208107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:49.227 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.570+0000 7fd210722640 1 --2- 172.21.15.6:0/1880245716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2081083c0 0x7fd208107c40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd2040097f0 tx=0x7fd204030ec0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.227 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.570+0000 7fd210722640 1 -- 172.21.15.6:0/1880245716 shutdown_connections 2024-09-12T03:11:49.227 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.570+0000 7fd210722640 1 --2- 172.21.15.6:0/1880245716 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7fd1f4038f20 0x7fd1f403b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.227 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.570+0000 7fd210722640 1 --2- 172.21.15.6:0/1880245716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2081083c0 0x7fd208107c40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.227 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.570+0000 7fd210722640 1 -- 172.21.15.6:0/1880245716 >> 172.21.15.6:0/1880245716 conn(0x7fd208102000 msgr2=0x7fd208102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:49.227 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.570+0000 7fd210722640 1 -- 172.21.15.6:0/1880245716 shutdown_connections 2024-09-12T03:11:49.228 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:48.570+0000 7fd210722640 1 -- 172.21.15.6:0/1880245716 wait complete. 2024-09-12T03:11:49.228 INFO:teuthology.orchestra.run.smithi006.stdout:Enabling cephadm module... 2024-09-12T03:11:50.473 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:50 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: ignoring --setuser ceph since I am not root 2024-09-12T03:11:50.474 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:50 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: ignoring --setgroup ceph since I am not root 2024-09-12T03:11:50.474 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:50 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:50.310+0000 7fd50a443040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:11:50.474 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:50 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:50.369+0000 7fd50a443040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:11:50.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:50 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1320033292' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-12T03:11:50.751 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:50 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:50.527+0000 7fd50a443040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:11:50.751 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:50 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:50.596+0000 7fd50a443040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:11:50.751 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:50 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:50.750+0000 7fd50a443040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:11:50.781 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.596+0000 7f77934a1640 1 Processor -- start 2024-09-12T03:11:50.781 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.597+0000 7f77934a1640 1 -- start start 2024-09-12T03:11:50.781 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.597+0000 7f77934a1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f778c07ddc0 0x7f778c07c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:50.781 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.597+0000 7f77934a1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f778c07c800 con 0x7f778c07ddc0 2024-09-12T03:11:50.781 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.597+0000 7f779123d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f778c07ddc0 0x7f778c07c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:50.781 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.597+0000 7f779123d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f778c07ddc0 0x7f778c07c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33322/0 (socket says 172.21.15.6:33322) 2024-09-12T03:11:50.781 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.597+0000 7f779123d640 1 -- 172.21.15.6:0/538148400 learned_addr learned my addr 172.21.15.6:0/538148400 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:50.781 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.597+0000 7f779123d640 1 -- 172.21.15.6:0/538148400 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778c07c980 con 0x7f778c07ddc0 2024-09-12T03:11:50.782 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.598+0000 7f779123d640 1 --2- 172.21.15.6:0/538148400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f778c07ddc0 0x7f778c07c2c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7788009820 tx=0x7f778802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa0340f2cf8375 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:50.782 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.598+0000 7f7783fff640 1 -- 172.21.15.6:0/538148400 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7788037070 con 0x7f778c07ddc0 2024-09-12T03:11:50.782 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.598+0000 7f7783fff640 1 -- 172.21.15.6:0/538148400 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7788032900 con 0x7f778c07ddc0 2024-09-12T03:11:50.782 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.598+0000 7f7783fff640 1 -- 172.21.15.6:0/538148400 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7788030850 con 0x7f778c07ddc0 2024-09-12T03:11:50.782 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.599+0000 7f77934a1640 1 -- 172.21.15.6:0/538148400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f778c07ddc0 msgr2=0x7f778c07c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:50.782 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.599+0000 7f77934a1640 1 --2- 172.21.15.6:0/538148400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f778c07ddc0 0x7f778c07c2c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7788009820 tx=0x7f778802a270 comp rx=0 tx=0).stop 2024-09-12T03:11:50.782 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.599+0000 7f77934a1640 1 -- 172.21.15.6:0/538148400 shutdown_connections 2024-09-12T03:11:50.782 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.599+0000 7f77934a1640 1 --2- 172.21.15.6:0/538148400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f778c07ddc0 0x7f778c07c2c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:50.782 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.599+0000 7f77934a1640 1 -- 172.21.15.6:0/538148400 >> 172.21.15.6:0/538148400 conn(0x7f778c1031b0 msgr2=0x7f778c1055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:50.782 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.599+0000 7f77934a1640 1 -- 172.21.15.6:0/538148400 shutdown_connections 2024-09-12T03:11:50.782 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.599+0000 7f77934a1640 1 -- 172.21.15.6:0/538148400 wait complete. 2024-09-12T03:11:50.782 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.600+0000 7f77934a1640 1 Processor -- start 2024-09-12T03:11:50.782 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.600+0000 7f77934a1640 1 -- start start 2024-09-12T03:11:50.782 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.600+0000 7f77934a1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f778c07ddc0 0x7f778c19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:50.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.600+0000 7f77934a1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f778c10df90 con 0x7f778c07ddc0 2024-09-12T03:11:50.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.600+0000 7f779123d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f778c07ddc0 0x7f778c19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:50.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.600+0000 7f779123d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f778c07ddc0 0x7f778c19dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33336/0 (socket says 172.21.15.6:33336) 2024-09-12T03:11:50.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.600+0000 7f779123d640 1 -- 172.21.15.6:0/1320033292 learned_addr learned my addr 172.21.15.6:0/1320033292 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:50.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.600+0000 7f779123d640 1 -- 172.21.15.6:0/1320033292 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778c19e000 con 0x7f778c07ddc0 2024-09-12T03:11:50.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.600+0000 7f779123d640 1 --2- 172.21.15.6:0/1320033292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f778c07ddc0 0x7f778c19dac0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f77880097f0 tx=0x7f7788030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:50.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.601+0000 7f77827fc640 1 -- 172.21.15.6:0/1320033292 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7788037040 con 0x7f778c07ddc0 2024-09-12T03:11:50.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.601+0000 7f77827fc640 1 -- 172.21.15.6:0/1320033292 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7788040070 con 0x7f778c07ddc0 2024-09-12T03:11:50.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.601+0000 7f77827fc640 1 -- 172.21.15.6:0/1320033292 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f778803a390 con 0x7f778c07ddc0 2024-09-12T03:11:50.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.601+0000 7f77934a1640 1 -- 172.21.15.6:0/1320033292 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f778c19e290 con 0x7f778c07ddc0 2024-09-12T03:11:50.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.601+0000 7f77934a1640 1 -- 172.21.15.6:0/1320033292 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f778c1a0f80 con 0x7f778c07ddc0 2024-09-12T03:11:50.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.601+0000 7f77827fc640 1 -- 172.21.15.6:0/1320033292 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f778803b430 con 0x7f778c07ddc0 2024-09-12T03:11:50.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.602+0000 7f77934a1640 1 -- 172.21.15.6:0/1320033292 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f778c07c340 con 0x7f778c07ddc0 2024-09-12T03:11:50.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.602+0000 7f77827fc640 1 --2- 172.21.15.6:0/1320033292 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f7778038f20 0x7f777803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:50.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.602+0000 7f77827fc640 1 -- 172.21.15.6:0/1320033292 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f778806c0e0 con 0x7f778c07ddc0 2024-09-12T03:11:50.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.602+0000 7f7790a3c640 1 --2- 172.21.15.6:0/1320033292 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f7778038f20 0x7f777803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:50.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.602+0000 7f7790a3c640 1 --2- 172.21.15.6:0/1320033292 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f7778038f20 0x7f777803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f777c0098c0 tx=0x7f777c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:50.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.607+0000 7f77827fc640 1 -- 172.21.15.6:0/1320033292 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f778806be40 con 0x7f778c07ddc0 2024-09-12T03:11:50.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:49.835+0000 7f77934a1640 1 -- 172.21.15.6:0/1320033292 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f778c1a1150 con 0x7f778c07ddc0 2024-09-12T03:11:50.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:50.166+0000 7f77827fc640 1 -- 172.21.15.6:0/1320033292 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f7788040ce0 con 0x7f778c07ddc0 2024-09-12T03:11:50.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:50.167+0000 7f77827fc640 1 -- 172.21.15.6:0/1320033292 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f7788032900 con 0x7f778c07ddc0 2024-09-12T03:11:50.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:50.168+0000 7f77934a1640 1 -- 172.21.15.6:0/1320033292 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f7778038f20 msgr2=0x7f777803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:50.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:50.168+0000 7f77934a1640 1 --2- 172.21.15.6:0/1320033292 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f7778038f20 0x7f777803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f777c0098c0 tx=0x7f777c006db0 comp rx=0 tx=0).stop 2024-09-12T03:11:50.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:50.168+0000 7f77934a1640 1 -- 172.21.15.6:0/1320033292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f778c07ddc0 msgr2=0x7f778c19dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:50.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:50.168+0000 7f77934a1640 1 --2- 172.21.15.6:0/1320033292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f778c07ddc0 0x7f778c19dac0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f77880097f0 tx=0x7f7788030d80 comp rx=0 tx=0).stop 2024-09-12T03:11:50.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:50.169+0000 7f77934a1640 1 -- 172.21.15.6:0/1320033292 shutdown_connections 2024-09-12T03:11:50.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:50.169+0000 7f77934a1640 1 --2- 172.21.15.6:0/1320033292 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f7778038f20 0x7f777803b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:50.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:50.169+0000 7f77934a1640 1 --2- 172.21.15.6:0/1320033292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f778c07ddc0 0x7f778c19dac0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:50.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:50.169+0000 7f77934a1640 1 -- 172.21.15.6:0/1320033292 >> 172.21.15.6:0/1320033292 conn(0x7f778c1031b0 msgr2=0x7f778c103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:50.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:50.169+0000 7f77934a1640 1 -- 172.21.15.6:0/1320033292 shutdown_connections 2024-09-12T03:11:50.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:50.169+0000 7f77934a1640 1 -- 172.21.15.6:0/1320033292 wait complete. 2024-09-12T03:11:51.030 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:50 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:50.905+0000 7fd50a443040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:11:51.306 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:51 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1320033292' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-12T03:11:51.306 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:51 smithi006 ceph-mon[30991]: mgrmap e5: x(active, since 5s) 2024-09-12T03:11:51.306 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:51 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:51.241+0000 7fd50a443040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:11:51.307 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:51 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:51.306+0000 7fd50a443040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:11:51.611 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:51 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:51.504+0000 7fd50a443040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:11:52.119 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:11:52.119 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-12T03:11:52.119 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T03:11:52.119 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T03:11:52.119 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T03:11:52.119 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:11:52.119 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.206+0000 7f9539bde640 1 Processor -- start 2024-09-12T03:11:52.119 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.206+0000 7f9539bde640 1 -- start start 2024-09-12T03:11:52.120 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.206+0000 7f9539bde640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9534106270 0x7f9534106670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:52.120 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.206+0000 7f9539bde640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f953407bdc0 con 0x7f9534106270 2024-09-12T03:11:52.120 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.206+0000 7f95337fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9534106270 0x7f9534106670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:52.120 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.206+0000 7f95337fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9534106270 0x7f9534106670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33354/0 (socket says 172.21.15.6:33354) 2024-09-12T03:11:52.120 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.206+0000 7f95337fe640 1 -- 172.21.15.6:0/1534013736 learned_addr learned my addr 172.21.15.6:0/1534013736 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:52.120 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.206+0000 7f95337fe640 1 -- 172.21.15.6:0/1534013736 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9534106bb0 con 0x7f9534106270 2024-09-12T03:11:52.120 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.207+0000 7f95337fe640 1 --2- 172.21.15.6:0/1534013736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9534106270 0x7f9534106670 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9524009820 tx=0x7f952402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8a4e97e433e889 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:52.120 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.207+0000 7f9532ffd640 1 -- 172.21.15.6:0/1534013736 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9524037070 con 0x7f9534106270 2024-09-12T03:11:52.120 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.207+0000 7f9532ffd640 1 -- 172.21.15.6:0/1534013736 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9524032900 con 0x7f9534106270 2024-09-12T03:11:52.120 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.207+0000 7f9532ffd640 1 -- 172.21.15.6:0/1534013736 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9524030850 con 0x7f9534106270 2024-09-12T03:11:52.120 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.208+0000 7f9539bde640 1 -- 172.21.15.6:0/1534013736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9534106270 msgr2=0x7f9534106670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:52.120 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.208+0000 7f9539bde640 1 --2- 172.21.15.6:0/1534013736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9534106270 0x7f9534106670 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9524009820 tx=0x7f952402a270 comp rx=0 tx=0).stop 2024-09-12T03:11:52.121 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.208+0000 7f9539bde640 1 -- 172.21.15.6:0/1534013736 shutdown_connections 2024-09-12T03:11:52.121 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.208+0000 7f9539bde640 1 --2- 172.21.15.6:0/1534013736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9534106270 0x7f9534106670 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.121 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.208+0000 7f9539bde640 1 -- 172.21.15.6:0/1534013736 >> 172.21.15.6:0/1534013736 conn(0x7f9534102000 msgr2=0x7f9534104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:52.121 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.208+0000 7f9539bde640 1 -- 172.21.15.6:0/1534013736 shutdown_connections 2024-09-12T03:11:52.121 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.208+0000 7f9539bde640 1 -- 172.21.15.6:0/1534013736 wait complete. 2024-09-12T03:11:52.121 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.209+0000 7f9539bde640 1 Processor -- start 2024-09-12T03:11:52.121 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.209+0000 7f9539bde640 1 -- start start 2024-09-12T03:11:52.121 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.209+0000 7f9539bde640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9534106270 0x7f953419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:52.121 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.209+0000 7f9539bde640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9534107b30 con 0x7f9534106270 2024-09-12T03:11:52.121 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.210+0000 7f95337fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9534106270 0x7f953419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:52.121 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.210+0000 7f95337fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9534106270 0x7f953419db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33370/0 (socket says 172.21.15.6:33370) 2024-09-12T03:11:52.122 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.210+0000 7f95337fe640 1 -- 172.21.15.6:0/1044281186 learned_addr learned my addr 172.21.15.6:0/1044281186 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:52.122 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.210+0000 7f95337fe640 1 -- 172.21.15.6:0/1044281186 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953419e0a0 con 0x7f9534106270 2024-09-12T03:11:52.122 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.210+0000 7f95337fe640 1 --2- 172.21.15.6:0/1044281186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9534106270 0x7f953419db60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9524031000 tx=0x7f95240326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:52.122 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.210+0000 7f95317fa640 1 -- 172.21.15.6:0/1044281186 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9524042070 con 0x7f9534106270 2024-09-12T03:11:52.122 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.210+0000 7f95317fa640 1 -- 172.21.15.6:0/1044281186 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9524032be0 con 0x7f9534106270 2024-09-12T03:11:52.122 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.210+0000 7f9539bde640 1 -- 172.21.15.6:0/1044281186 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f953419e330 con 0x7f9534106270 2024-09-12T03:11:52.122 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.210+0000 7f95317fa640 1 -- 172.21.15.6:0/1044281186 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9524037040 con 0x7f9534106270 2024-09-12T03:11:52.122 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.210+0000 7f9539bde640 1 -- 172.21.15.6:0/1044281186 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95341a1020 con 0x7f9534106270 2024-09-12T03:11:52.122 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.211+0000 7f95317fa640 1 -- 172.21.15.6:0/1044281186 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f9524045070 con 0x7f9534106270 2024-09-12T03:11:52.122 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.211+0000 7f9539bde640 1 -- 172.21.15.6:0/1044281186 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9534108200 con 0x7f9534106270 2024-09-12T03:11:52.122 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.212+0000 7f95317fa640 1 --2- 172.21.15.6:0/1044281186 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f9520038f70 0x7f952003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:52.122 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.212+0000 7f95317fa640 1 -- 172.21.15.6:0/1044281186 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f952406bf20 con 0x7f9534106270 2024-09-12T03:11:52.122 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.212+0000 7f952bfff640 1 -- 172.21.15.6:0/1044281186 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f9520038f70 msgr2=0x7f952003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/836933784 2024-09-12T03:11:52.123 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.212+0000 7f952bfff640 1 --2- 172.21.15.6:0/1044281186 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f9520038f70 0x7f952003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:11:52.123 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.216+0000 7f95317fa640 1 -- 172.21.15.6:0/1044281186 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9524032d90 con 0x7f9534106270 2024-09-12T03:11:52.123 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.413+0000 7f952bfff640 1 -- 172.21.15.6:0/1044281186 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f9520038f70 msgr2=0x7f952003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/836933784 2024-09-12T03:11:52.123 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.413+0000 7f952bfff640 1 --2- 172.21.15.6:0/1044281186 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f9520038f70 0x7f952003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T03:11:52.123 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.440+0000 7f9539bde640 1 -- 172.21.15.6:0/1044281186 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f953402da40 con 0x7f9534106270 2024-09-12T03:11:52.123 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.440+0000 7f95317fa640 1 -- 172.21.15.6:0/1044281186 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f952406b940 con 0x7f9534106270 2024-09-12T03:11:52.123 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.442+0000 7f9539bde640 1 -- 172.21.15.6:0/1044281186 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f9520038f70 msgr2=0x7f952003b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:11:52.123 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.442+0000 7f9539bde640 1 --2- 172.21.15.6:0/1044281186 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f9520038f70 0x7f952003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.123 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.443+0000 7f9539bde640 1 -- 172.21.15.6:0/1044281186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9534106270 msgr2=0x7f953419db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:52.123 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.443+0000 7f9539bde640 1 --2- 172.21.15.6:0/1044281186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9534106270 0x7f953419db60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9524031000 tx=0x7f95240326e0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.123 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.443+0000 7f9539bde640 1 -- 172.21.15.6:0/1044281186 shutdown_connections 2024-09-12T03:11:52.123 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.443+0000 7f9539bde640 1 --2- 172.21.15.6:0/1044281186 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f9520038f70 0x7f952003b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.124 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.443+0000 7f9539bde640 1 --2- 172.21.15.6:0/1044281186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9534106270 0x7f953419db60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.124 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.443+0000 7f9539bde640 1 -- 172.21.15.6:0/1044281186 >> 172.21.15.6:0/1044281186 conn(0x7f9534102000 msgr2=0x7f9534194820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:52.124 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.443+0000 7f9539bde640 1 -- 172.21.15.6:0/1044281186 shutdown_connections 2024-09-12T03:11:52.124 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:51.443+0000 7f9539bde640 1 -- 172.21.15.6:0/1044281186 wait complete. 2024-09-12T03:11:52.124 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for the mgr to restart... 2024-09-12T03:11:52.124 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr epoch 5... 2024-09-12T03:11:52.236 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:52 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1044281186' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T03:11:52.500 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:52 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:52.445+0000 7fd50a443040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:11:52.787 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:52 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:52.516+0000 7fd50a443040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:11:53.542 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:53 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:53.271+0000 7fd50a443040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:11:53.542 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:53 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:53.392+0000 7fd50a443040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:11:53.834 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:53 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:53.542+0000 7fd50a443040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:11:53.835 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:53 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:53.606+0000 7fd50a443040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:11:53.835 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:53 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:53.763+0000 7fd50a443040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:11:53.835 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:53 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:53.834+0000 7fd50a443040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:11:54.111 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:53 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:53.897+0000 7fd50a443040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:11:54.112 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:54 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:54.064+0000 7fd50a443040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:11:54.412 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:54 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:54.132+0000 7fd50a443040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:11:54.413 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:54 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:54.202+0000 7fd50a443040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:11:54.413 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:54 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:54.288+0000 7fd50a443040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:11:54.413 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:54 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:54.350+0000 7fd50a443040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:11:54.413 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:54 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:54.412+0000 7fd50a443040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:11:55.611 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:55 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:55.241+0000 7fd50a443040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:11:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:55 smithi006 ceph-mon[30991]: Active manager daemon x restarted 2024-09-12T03:11:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:55 smithi006 ceph-mon[30991]: Activating manager daemon x 2024-09-12T03:11:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:55 smithi006 ceph-mon[30991]: osdmap e2: 0 total, 0 up, 0 in 2024-09-12T03:11:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:55 smithi006 ceph-mon[30991]: mgrmap e6: x(active, starting, since 0.00340811s) 2024-09-12T03:11:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:55 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:11:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:55 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T03:11:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:55 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T03:11:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:55 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T03:11:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:55 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T03:11:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:55 smithi006 ceph-mon[30991]: Manager daemon x is now available 2024-09-12T03:11:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:55 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' 2024-09-12T03:11:55.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:55 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' 2024-09-12T03:11:55.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:55 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:55.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:55 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:55.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:55 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:11:55.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:55 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:11:56.839 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:11:56.839 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-12T03:11:56.839 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T03:11:56.839 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:11:56.839 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.581+0000 7f93a9847640 1 Processor -- start 2024-09-12T03:11:56.839 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.581+0000 7f93a9847640 1 -- start start 2024-09-12T03:11:56.839 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.581+0000 7f93a9847640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93a4106220 0x7f93a4106620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:56.840 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.581+0000 7f93a9847640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f93a407be10 con 0x7f93a4106220 2024-09-12T03:11:56.840 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.581+0000 7f93a2ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93a4106220 0x7f93a4106620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:56.840 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.581+0000 7f93a2ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93a4106220 0x7f93a4106620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33386/0 (socket says 172.21.15.6:33386) 2024-09-12T03:11:56.840 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.581+0000 7f93a2ffd640 1 -- 172.21.15.6:0/2665549366 learned_addr learned my addr 172.21.15.6:0/2665549366 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:56.840 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.582+0000 7f93a2ffd640 1 -- 172.21.15.6:0/2665549366 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a4106b60 con 0x7f93a4106220 2024-09-12T03:11:56.840 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.582+0000 7f93a2ffd640 1 --2- 172.21.15.6:0/2665549366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93a4106220 0x7f93a4106620 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9394009820 tx=0x7f939402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8351c46acd5efca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:56.840 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.582+0000 7f93a27fc640 1 -- 172.21.15.6:0/2665549366 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9394037070 con 0x7f93a4106220 2024-09-12T03:11:56.840 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.583+0000 7f93a27fc640 1 -- 172.21.15.6:0/2665549366 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9394032900 con 0x7f93a4106220 2024-09-12T03:11:56.840 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.583+0000 7f93a27fc640 1 -- 172.21.15.6:0/2665549366 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9394030850 con 0x7f93a4106220 2024-09-12T03:11:56.840 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.583+0000 7f93a9847640 1 -- 172.21.15.6:0/2665549366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93a4106220 msgr2=0x7f93a4106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:56.840 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.583+0000 7f93a9847640 1 --2- 172.21.15.6:0/2665549366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93a4106220 0x7f93a4106620 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9394009820 tx=0x7f939402a270 comp rx=0 tx=0).stop 2024-09-12T03:11:56.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.583+0000 7f93a9847640 1 -- 172.21.15.6:0/2665549366 shutdown_connections 2024-09-12T03:11:56.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.583+0000 7f93a9847640 1 --2- 172.21.15.6:0/2665549366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93a4106220 0x7f93a4106620 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:56.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.583+0000 7f93a9847640 1 -- 172.21.15.6:0/2665549366 >> 172.21.15.6:0/2665549366 conn(0x7f93a4101fb0 msgr2=0x7f93a4104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:56.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.583+0000 7f93a9847640 1 -- 172.21.15.6:0/2665549366 shutdown_connections 2024-09-12T03:11:56.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.583+0000 7f93a9847640 1 -- 172.21.15.6:0/2665549366 wait complete. 2024-09-12T03:11:56.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.584+0000 7f93a9847640 1 Processor -- start 2024-09-12T03:11:56.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.584+0000 7f93a9847640 1 -- start start 2024-09-12T03:11:56.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.584+0000 7f93a9847640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93a4106220 0x7f93a419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:56.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.584+0000 7f93a9847640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f93a4107c60 con 0x7f93a4106220 2024-09-12T03:11:56.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.584+0000 7f93a2ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93a4106220 0x7f93a419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:56.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.584+0000 7f93a2ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93a4106220 0x7f93a419db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33396/0 (socket says 172.21.15.6:33396) 2024-09-12T03:11:56.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.584+0000 7f93a2ffd640 1 -- 172.21.15.6:0/386994781 learned_addr learned my addr 172.21.15.6:0/386994781 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:56.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.585+0000 7f93a2ffd640 1 -- 172.21.15.6:0/386994781 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a419e0a0 con 0x7f93a4106220 2024-09-12T03:11:56.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.585+0000 7f93a2ffd640 1 --2- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93a4106220 0x7f93a419db60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f93940097f0 tx=0x7f9394030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:56.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.585+0000 7f93a8845640 1 -- 172.21.15.6:0/386994781 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9394042070 con 0x7f93a4106220 2024-09-12T03:11:56.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.585+0000 7f93a8845640 1 -- 172.21.15.6:0/386994781 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9394032ca0 con 0x7f93a4106220 2024-09-12T03:11:56.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.585+0000 7f93a9847640 1 -- 172.21.15.6:0/386994781 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a419e330 con 0x7f93a4106220 2024-09-12T03:11:56.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.585+0000 7f93a8845640 1 -- 172.21.15.6:0/386994781 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9394037040 con 0x7f93a4106220 2024-09-12T03:11:56.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.585+0000 7f93a9847640 1 -- 172.21.15.6:0/386994781 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a41a1020 con 0x7f93a4106220 2024-09-12T03:11:56.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.586+0000 7f93a8845640 1 -- 172.21.15.6:0/386994781 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f9394045070 con 0x7f93a4106220 2024-09-12T03:11:56.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.586+0000 7f93a8845640 1 --2- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f9390038fc0 0x7f939003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:56.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.586+0000 7f93a8845640 1 -- 172.21.15.6:0/386994781 --> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f939003bb90 con 0x7f9390038fc0 2024-09-12T03:11:56.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.586+0000 7f93a8845640 1 -- 172.21.15.6:0/386994781 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f939406c030 con 0x7f93a4106220 2024-09-12T03:11:56.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.586+0000 7f939bfff640 1 -- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f9390038fc0 msgr2=0x7f939003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/836933784 2024-09-12T03:11:56.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.586+0000 7f939bfff640 1 --2- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f9390038fc0 0x7f939003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:11:56.843 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.787+0000 7f939bfff640 1 -- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f9390038fc0 msgr2=0x7f939003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/836933784 2024-09-12T03:11:56.843 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:52.787+0000 7f939bfff640 1 --2- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f9390038fc0 0x7f939003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T03:11:56.843 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:53.188+0000 7f939bfff640 1 -- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f9390038fc0 msgr2=0x7f939003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/836933784 2024-09-12T03:11:56.843 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:53.188+0000 7f939bfff640 1 --2- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f9390038fc0 0x7f939003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T03:11:56.843 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:53.989+0000 7f939bfff640 1 -- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f9390038fc0 msgr2=0x7f939003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/836933784 2024-09-12T03:11:56.843 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:53.989+0000 7f939bfff640 1 --2- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f9390038fc0 0x7f939003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T03:11:56.843 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:55.245+0000 7f93a8845640 1 -- 172.21.15.6:0/386994781 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f939406abb0 con 0x7f93a4106220 2024-09-12T03:11:56.843 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:55.245+0000 7f93a8845640 1 -- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f9390038fc0 msgr2=0x7f939003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:11:56.843 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:55.245+0000 7f93a8845640 1 --2- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f9390038fc0 0x7f939003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:56.843 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:56.246+0000 7f93a8845640 1 -- 172.21.15.6:0/386994781 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f939403c720 con 0x7f93a4106220 2024-09-12T03:11:56.844 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:56.246+0000 7f93a8845640 1 --2- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f939003c8f0 0x7f939003ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:56.844 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:56.246+0000 7f93a8845640 1 -- 172.21.15.6:0/386994781 --> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f939400af30 con 0x7f939003c8f0 2024-09-12T03:11:56.844 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:56.247+0000 7f939bfff640 1 --2- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f939003c8f0 0x7f939003ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:56.844 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:56.247+0000 7f939bfff640 1 --2- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f939003c8f0 0x7f939003ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f938c003980 tx=0x7f938c0092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:56.844 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:56.248+0000 7f93a8845640 1 -- 172.21.15.6:0/386994781 <== mgr.14118 v2:172.21.15.6:6800/1684477653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f939400af30 con 0x7f939003c8f0 2024-09-12T03:11:56.844 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:56.255+0000 7f93a9847640 1 -- 172.21.15.6:0/386994781 --> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f93a4106620 con 0x7f939003c8f0 2024-09-12T03:11:56.844 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:56.255+0000 7f93a8845640 1 -- 172.21.15.6:0/386994781 <== mgr.14118 v2:172.21.15.6:6800/1684477653 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f93a4106620 con 0x7f939003c8f0 2024-09-12T03:11:56.844 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:56.256+0000 7f93a9847640 1 -- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f939003c8f0 msgr2=0x7f939003ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:56.844 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:56.256+0000 7f93a9847640 1 --2- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f939003c8f0 0x7f939003ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f938c003980 tx=0x7f938c0092b0 comp rx=0 tx=0).stop 2024-09-12T03:11:56.844 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:56.256+0000 7f93a9847640 1 -- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93a4106220 msgr2=0x7f93a419db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:56.845 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:56.256+0000 7f93a9847640 1 --2- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93a4106220 0x7f93a419db60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f93940097f0 tx=0x7f9394030ec0 comp rx=0 tx=0).stop 2024-09-12T03:11:56.845 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:56.256+0000 7f93a9847640 1 -- 172.21.15.6:0/386994781 shutdown_connections 2024-09-12T03:11:56.845 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:56.256+0000 7f93a9847640 1 --2- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f939003c8f0 0x7f939003ece0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:56.845 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:56.256+0000 7f93a9847640 1 --2- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:6800/836933784,v1:172.21.15.6:6801/836933784] conn(0x7f9390038fc0 0x7f939003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:56.845 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:56.256+0000 7f93a9847640 1 --2- 172.21.15.6:0/386994781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93a4106220 0x7f93a419db60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:56.845 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:56.256+0000 7f93a9847640 1 -- 172.21.15.6:0/386994781 >> 172.21.15.6:0/386994781 conn(0x7f93a4101fb0 msgr2=0x7f93a4194820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:56.845 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:56.256+0000 7f93a9847640 1 -- 172.21.15.6:0/386994781 shutdown_connections 2024-09-12T03:11:56.845 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:56.256+0000 7f93a9847640 1 -- 172.21.15.6:0/386994781 wait complete. 2024-09-12T03:11:56.845 INFO:teuthology.orchestra.run.smithi006.stdout:mgr epoch 5 is available 2024-09-12T03:11:56.845 INFO:teuthology.orchestra.run.smithi006.stdout:Setting orchestrator backend to cephadm... 2024-09-12T03:11:57.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-mon[30991]: Found migration_current of "None". Setting to last migration. 2024-09-12T03:11:57.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' 2024-09-12T03:11:57.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' 2024-09-12T03:11:57.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-mon[30991]: mgrmap e7: x(active, since 1.00544s) 2024-09-12T03:11:57.362 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:57.210+0000 7fd476fa8640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:03:11:57] ENGINE Error in HTTPServer.serve 2024-09-12T03:11:57.362 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: Traceback (most recent call last): 2024-09-12T03:11:57.362 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:11:57.362 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: self._connections.run(self.expiration_interval) 2024-09-12T03:11:57.362 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:11:57.363 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: self._run(expiration_interval) 2024-09-12T03:11:57.363 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:11:57.363 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:11:57.363 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:11:57.363 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:11:57.363 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:11:57.363 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: s = self.context.wrap_socket( 2024-09-12T03:11:57.363 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:11:57.363 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: return self.sslsocket_class._create( 2024-09-12T03:11:57.363 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:11:57.363 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: self.do_handshake() 2024-09-12T03:11:57.363 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:11:57.363 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: self._sslobj.do_handshake() 2024-09-12T03:11:57.364 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:11:57.364 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:11:58.068 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.279+0000 7fcc3d51b640 1 Processor -- start 2024-09-12T03:11:58.068 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.279+0000 7fcc3d51b640 1 -- start start 2024-09-12T03:11:58.069 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.279+0000 7fcc3d51b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc38106250 0x7fcc38106650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:58.069 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.279+0000 7fcc3d51b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3807bde0 con 0x7fcc38106250 2024-09-12T03:11:58.069 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.280+0000 7fcc36ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc38106250 0x7fcc38106650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:58.069 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.280+0000 7fcc36ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc38106250 0x7fcc38106650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33468/0 (socket says 172.21.15.6:33468) 2024-09-12T03:11:58.069 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.280+0000 7fcc36ffd640 1 -- 172.21.15.6:0/95102150 learned_addr learned my addr 172.21.15.6:0/95102150 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:58.069 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.280+0000 7fcc36ffd640 1 -- 172.21.15.6:0/95102150 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc38106b90 con 0x7fcc38106250 2024-09-12T03:11:58.069 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.281+0000 7fcc36ffd640 1 --2- 172.21.15.6:0/95102150 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc38106250 0x7fcc38106650 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcc28009e80 tx=0x7fcc2802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483500a7fea9793a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:58.069 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.281+0000 7fcc35ffb640 1 -- 172.21.15.6:0/95102150 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc28036070 con 0x7fcc38106250 2024-09-12T03:11:58.069 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.281+0000 7fcc35ffb640 1 -- 172.21.15.6:0/95102150 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcc280318c0 con 0x7fcc38106250 2024-09-12T03:11:58.069 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.281+0000 7fcc35ffb640 1 -- 172.21.15.6:0/95102150 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc28007ab0 con 0x7fcc38106250 2024-09-12T03:11:58.069 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.281+0000 7fcc3d51b640 1 -- 172.21.15.6:0/95102150 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc38106250 msgr2=0x7fcc38106650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:58.069 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.281+0000 7fcc3d51b640 1 --2- 172.21.15.6:0/95102150 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc38106250 0x7fcc38106650 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcc28009e80 tx=0x7fcc2802a7a0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.069 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.281+0000 7fcc3d51b640 1 -- 172.21.15.6:0/95102150 shutdown_connections 2024-09-12T03:11:58.070 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.281+0000 7fcc3d51b640 1 --2- 172.21.15.6:0/95102150 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc38106250 0x7fcc38106650 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.070 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.281+0000 7fcc3d51b640 1 -- 172.21.15.6:0/95102150 >> 172.21.15.6:0/95102150 conn(0x7fcc38102000 msgr2=0x7fcc38104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:58.070 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.281+0000 7fcc3d51b640 1 -- 172.21.15.6:0/95102150 shutdown_connections 2024-09-12T03:11:58.070 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.281+0000 7fcc3d51b640 1 -- 172.21.15.6:0/95102150 wait complete. 2024-09-12T03:11:58.070 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.282+0000 7fcc3d51b640 1 Processor -- start 2024-09-12T03:11:58.070 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.282+0000 7fcc3d51b640 1 -- start start 2024-09-12T03:11:58.070 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.282+0000 7fcc3d51b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc38106250 0x7fcc3819db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:58.070 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.282+0000 7fcc3d51b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38107c90 con 0x7fcc38106250 2024-09-12T03:11:58.070 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.283+0000 7fcc36ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc38106250 0x7fcc3819db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:58.070 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.283+0000 7fcc36ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc38106250 0x7fcc3819db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33472/0 (socket says 172.21.15.6:33472) 2024-09-12T03:11:58.070 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.283+0000 7fcc36ffd640 1 -- 172.21.15.6:0/2391686626 learned_addr learned my addr 172.21.15.6:0/2391686626 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:58.070 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.283+0000 7fcc36ffd640 1 -- 172.21.15.6:0/2391686626 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3819e080 con 0x7fcc38106250 2024-09-12T03:11:58.070 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.283+0000 7fcc36ffd640 1 --2- 172.21.15.6:0/2391686626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc38106250 0x7fcc3819db40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcc28031ca0 tx=0x7fcc28031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:58.070 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.283+0000 7fcc1ffff640 1 -- 172.21.15.6:0/2391686626 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc28036050 con 0x7fcc38106250 2024-09-12T03:11:58.071 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.283+0000 7fcc1ffff640 1 -- 172.21.15.6:0/2391686626 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcc28042070 con 0x7fcc38106250 2024-09-12T03:11:58.071 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.283+0000 7fcc3d51b640 1 -- 172.21.15.6:0/2391686626 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3819e310 con 0x7fcc38106250 2024-09-12T03:11:58.071 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.283+0000 7fcc1ffff640 1 -- 172.21.15.6:0/2391686626 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc2803d940 con 0x7fcc38106250 2024-09-12T03:11:58.071 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.283+0000 7fcc3d51b640 1 -- 172.21.15.6:0/2391686626 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc381a1000 con 0x7fcc38106250 2024-09-12T03:11:58.071 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.284+0000 7fcc1ffff640 1 -- 172.21.15.6:0/2391686626 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fcc2803dbf0 con 0x7fcc38106250 2024-09-12T03:11:58.071 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.284+0000 7fcc1ffff640 1 --2- 172.21.15.6:0/2391686626 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fcc20038ec0 0x7fcc2003b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:58.071 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.284+0000 7fcc1ffff640 1 -- 172.21.15.6:0/2391686626 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcc28007e00 con 0x7fcc38106250 2024-09-12T03:11:58.071 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.284+0000 7fcc3d51b640 1 -- 172.21.15.6:0/2391686626 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc38106650 con 0x7fcc38106250 2024-09-12T03:11:58.071 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.284+0000 7fcc367fc640 1 --2- 172.21.15.6:0/2391686626 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fcc20038ec0 0x7fcc2003b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:58.071 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.285+0000 7fcc367fc640 1 --2- 172.21.15.6:0/2391686626 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fcc20038ec0 0x7fcc2003b380 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0098c0 tx=0x7fcc2c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:58.071 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.290+0000 7fcc1ffff640 1 -- 172.21.15.6:0/2391686626 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc28045360 con 0x7fcc38106250 2024-09-12T03:11:58.071 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.480+0000 7fcc3d51b640 1 -- 172.21.15.6:0/2391686626 --> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fcc3802d9f0 con 0x7fcc20038ec0 2024-09-12T03:11:58.072 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.485+0000 7fcc1ffff640 1 -- 172.21.15.6:0/2391686626 <== mgr.14118 v2:172.21.15.6:6800/1684477653 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcc3802d9f0 con 0x7fcc20038ec0 2024-09-12T03:11:58.072 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.487+0000 7fcc3d51b640 1 -- 172.21.15.6:0/2391686626 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fcc20038ec0 msgr2=0x7fcc2003b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:58.072 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.487+0000 7fcc3d51b640 1 --2- 172.21.15.6:0/2391686626 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fcc20038ec0 0x7fcc2003b380 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0098c0 tx=0x7fcc2c006db0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.072 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.487+0000 7fcc3d51b640 1 -- 172.21.15.6:0/2391686626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc38106250 msgr2=0x7fcc3819db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:58.072 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.487+0000 7fcc3d51b640 1 --2- 172.21.15.6:0/2391686626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc38106250 0x7fcc3819db40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcc28031ca0 tx=0x7fcc28031cd0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.072 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.487+0000 7fcc3d51b640 1 -- 172.21.15.6:0/2391686626 shutdown_connections 2024-09-12T03:11:58.072 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.487+0000 7fcc3d51b640 1 --2- 172.21.15.6:0/2391686626 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fcc20038ec0 0x7fcc2003b380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.072 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.487+0000 7fcc3d51b640 1 --2- 172.21.15.6:0/2391686626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc38106250 0x7fcc3819db40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.072 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.487+0000 7fcc3d51b640 1 -- 172.21.15.6:0/2391686626 >> 172.21.15.6:0/2391686626 conn(0x7fcc38102000 msgr2=0x7fcc381947a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:58.072 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.487+0000 7fcc3d51b640 1 -- 172.21.15.6:0/2391686626 shutdown_connections 2024-09-12T03:11:58.073 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:57.487+0000 7fcc3d51b640 1 -- 172.21.15.6:0/2391686626 wait complete. 2024-09-12T03:11:58.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:58 smithi006 ceph-mon[30991]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T03:11:58.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:58 smithi006 ceph-mon[30991]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T03:11:58.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:58 smithi006 ceph-mon[30991]: [12/Sep/2024:03:11:57] ENGINE Bus STARTING 2024-09-12T03:11:58.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:58 smithi006 ceph-mon[30991]: [12/Sep/2024:03:11:57] ENGINE Serving on https://172.21.15.6:7150 2024-09-12T03:11:58.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:58 smithi006 ceph-mon[30991]: [12/Sep/2024:03:11:57] ENGINE Error in HTTPServer.serve 2024-09-12T03:11:58.362 INFO:journalctl@ceph.mon.a.smithi006.stdout: Traceback (most recent call last): 2024-09-12T03:11:58.362 INFO:journalctl@ceph.mon.a.smithi006.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:11:58.362 INFO:journalctl@ceph.mon.a.smithi006.stdout: self._connections.run(self.expiration_interval) 2024-09-12T03:11:58.362 INFO:journalctl@ceph.mon.a.smithi006.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:11:58.362 INFO:journalctl@ceph.mon.a.smithi006.stdout: self._run(expiration_interval) 2024-09-12T03:11:58.362 INFO:journalctl@ceph.mon.a.smithi006.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:11:58.362 INFO:journalctl@ceph.mon.a.smithi006.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:11:58.362 INFO:journalctl@ceph.mon.a.smithi006.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:11:58.363 INFO:journalctl@ceph.mon.a.smithi006.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:11:58.363 INFO:journalctl@ceph.mon.a.smithi006.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:11:58.363 INFO:journalctl@ceph.mon.a.smithi006.stdout: s = self.context.wrap_socket( 2024-09-12T03:11:58.363 INFO:journalctl@ceph.mon.a.smithi006.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:11:58.363 INFO:journalctl@ceph.mon.a.smithi006.stdout: return self.sslsocket_class._create( 2024-09-12T03:11:58.363 INFO:journalctl@ceph.mon.a.smithi006.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:11:58.363 INFO:journalctl@ceph.mon.a.smithi006.stdout: self.do_handshake() 2024-09-12T03:11:58.363 INFO:journalctl@ceph.mon.a.smithi006.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:11:58.363 INFO:journalctl@ceph.mon.a.smithi006.stdout: self._sslobj.do_handshake() 2024-09-12T03:11:58.363 INFO:journalctl@ceph.mon.a.smithi006.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:11:58.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:58 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:58.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:58 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' 2024-09-12T03:11:58.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:58 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:59.258 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:59 smithi006 ceph-mon[30991]: [12/Sep/2024:03:11:57] ENGINE Serving on http://172.21.15.6:8765 2024-09-12T03:11:59.258 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:59 smithi006 ceph-mon[30991]: [12/Sep/2024:03:11:57] ENGINE Bus STARTED 2024-09-12T03:11:59.258 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:59 smithi006 ceph-mon[30991]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:11:59.259 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:11:59 smithi006 ceph-mon[30991]: mgrmap e8: x(active, since 2s) 2024-09-12T03:11:59.265 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-12T03:11:59.265 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.502+0000 7f0805aee640 1 Processor -- start 2024-09-12T03:11:59.266 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.502+0000 7f0805aee640 1 -- start start 2024-09-12T03:11:59.266 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.503+0000 7f0805aee640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0800106200 0x7f0800106600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:59.266 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.503+0000 7f0805aee640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f080007be30 con 0x7f0800106200 2024-09-12T03:11:59.266 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.503+0000 7f07ff7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0800106200 0x7f0800106600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:59.266 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.503+0000 7f07ff7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0800106200 0x7f0800106600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33476/0 (socket says 172.21.15.6:33476) 2024-09-12T03:11:59.266 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.503+0000 7f07ff7fe640 1 -- 172.21.15.6:0/2037958611 learned_addr learned my addr 172.21.15.6:0/2037958611 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:59.266 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.503+0000 7f07ff7fe640 1 -- 172.21.15.6:0/2037958611 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0800106b40 con 0x7f0800106200 2024-09-12T03:11:59.266 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.504+0000 7f07ff7fe640 1 --2- 172.21.15.6:0/2037958611 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0800106200 0x7f0800106600 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009820 tx=0x7f07f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9c2ea7dd08412b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:59.266 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.504+0000 7f07feffd640 1 -- 172.21.15.6:0/2037958611 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07f0037070 con 0x7f0800106200 2024-09-12T03:11:59.266 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.504+0000 7f07feffd640 1 -- 172.21.15.6:0/2037958611 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f07f0032900 con 0x7f0800106200 2024-09-12T03:11:59.266 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.504+0000 7f07feffd640 1 -- 172.21.15.6:0/2037958611 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07f0030850 con 0x7f0800106200 2024-09-12T03:11:59.266 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.505+0000 7f0805aee640 1 -- 172.21.15.6:0/2037958611 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0800106200 msgr2=0x7f0800106600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:59.267 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.505+0000 7f0805aee640 1 --2- 172.21.15.6:0/2037958611 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0800106200 0x7f0800106600 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009820 tx=0x7f07f002a270 comp rx=0 tx=0).stop 2024-09-12T03:11:59.267 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.505+0000 7f0805aee640 1 -- 172.21.15.6:0/2037958611 shutdown_connections 2024-09-12T03:11:59.267 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.505+0000 7f0805aee640 1 --2- 172.21.15.6:0/2037958611 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0800106200 0x7f0800106600 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:59.267 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.505+0000 7f0805aee640 1 -- 172.21.15.6:0/2037958611 >> 172.21.15.6:0/2037958611 conn(0x7f0800101ff0 msgr2=0x7f0800104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:59.267 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.505+0000 7f0805aee640 1 -- 172.21.15.6:0/2037958611 shutdown_connections 2024-09-12T03:11:59.267 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.505+0000 7f0805aee640 1 -- 172.21.15.6:0/2037958611 wait complete. 2024-09-12T03:11:59.267 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.505+0000 7f0805aee640 1 Processor -- start 2024-09-12T03:11:59.267 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.505+0000 7f0805aee640 1 -- start start 2024-09-12T03:11:59.267 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.506+0000 7f0805aee640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0800106200 0x7f080019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:59.267 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.506+0000 7f0805aee640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0800107c40 con 0x7f0800106200 2024-09-12T03:11:59.267 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.506+0000 7f07ff7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0800106200 0x7f080019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:59.267 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.506+0000 7f07ff7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0800106200 0x7f080019db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33486/0 (socket says 172.21.15.6:33486) 2024-09-12T03:11:59.267 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.506+0000 7f07ff7fe640 1 -- 172.21.15.6:0/3378289946 learned_addr learned my addr 172.21.15.6:0/3378289946 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:11:59.268 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.506+0000 7f07ff7fe640 1 -- 172.21.15.6:0/3378289946 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080019e090 con 0x7f0800106200 2024-09-12T03:11:59.268 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.506+0000 7f07ff7fe640 1 --2- 172.21.15.6:0/3378289946 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0800106200 0x7f080019db50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f07f0031000 tx=0x7f07f00326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:59.268 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.506+0000 7f07fd7fa640 1 -- 172.21.15.6:0/3378289946 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07f0042070 con 0x7f0800106200 2024-09-12T03:11:59.268 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.506+0000 7f07fd7fa640 1 -- 172.21.15.6:0/3378289946 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f07f0032be0 con 0x7f0800106200 2024-09-12T03:11:59.268 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.506+0000 7f0805aee640 1 -- 172.21.15.6:0/3378289946 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f080019e320 con 0x7f0800106200 2024-09-12T03:11:59.268 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.507+0000 7f0805aee640 1 -- 172.21.15.6:0/3378289946 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08001a1010 con 0x7f0800106200 2024-09-12T03:11:59.268 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.507+0000 7f07fd7fa640 1 -- 172.21.15.6:0/3378289946 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07f0037040 con 0x7f0800106200 2024-09-12T03:11:59.268 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.507+0000 7f07fd7fa640 1 -- 172.21.15.6:0/3378289946 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f07f0045070 con 0x7f0800106200 2024-09-12T03:11:59.268 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.507+0000 7f0805aee640 1 -- 172.21.15.6:0/3378289946 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0800106600 con 0x7f0800106200 2024-09-12T03:11:59.268 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.508+0000 7f07fd7fa640 1 --2- 172.21.15.6:0/3378289946 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f07ec038fc0 0x7f07ec03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:59.268 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.508+0000 7f07fd7fa640 1 -- 172.21.15.6:0/3378289946 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f07f006bf20 con 0x7f0800106200 2024-09-12T03:11:59.268 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.508+0000 7f07f7fff640 1 --2- 172.21.15.6:0/3378289946 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f07ec038fc0 0x7f07ec03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:59.269 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.508+0000 7f07f7fff640 1 --2- 172.21.15.6:0/3378289946 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f07ec038fc0 0x7f07ec03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f07e80098c0 tx=0x7f07e8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:59.269 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.512+0000 7f07fd7fa640 1 -- 172.21.15.6:0/3378289946 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f07f0045360 con 0x7f0800106200 2024-09-12T03:11:59.269 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.691+0000 7f0805aee640 1 -- 172.21.15.6:0/3378289946 --> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f080002dab0 con 0x7f07ec038fc0 2024-09-12T03:11:59.269 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.692+0000 7f07fd7fa640 1 -- 172.21.15.6:0/3378289946 <== mgr.14118 v2:172.21.15.6:6800/1684477653 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f080002dab0 con 0x7f07ec038fc0 2024-09-12T03:11:59.269 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.695+0000 7f0805aee640 1 -- 172.21.15.6:0/3378289946 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f07ec038fc0 msgr2=0x7f07ec03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:59.269 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.695+0000 7f0805aee640 1 --2- 172.21.15.6:0/3378289946 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f07ec038fc0 0x7f07ec03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f07e80098c0 tx=0x7f07e8006db0 comp rx=0 tx=0).stop 2024-09-12T03:11:59.269 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.695+0000 7f0805aee640 1 -- 172.21.15.6:0/3378289946 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0800106200 msgr2=0x7f080019db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:59.269 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.695+0000 7f0805aee640 1 --2- 172.21.15.6:0/3378289946 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0800106200 0x7f080019db50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f07f0031000 tx=0x7f07f00326e0 comp rx=0 tx=0).stop 2024-09-12T03:11:59.269 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.695+0000 7f0805aee640 1 -- 172.21.15.6:0/3378289946 shutdown_connections 2024-09-12T03:11:59.269 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.695+0000 7f0805aee640 1 --2- 172.21.15.6:0/3378289946 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f07ec038fc0 0x7f07ec03b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:59.269 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.695+0000 7f0805aee640 1 --2- 172.21.15.6:0/3378289946 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0800106200 0x7f080019db50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:59.269 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.695+0000 7f0805aee640 1 -- 172.21.15.6:0/3378289946 >> 172.21.15.6:0/3378289946 conn(0x7f0800101ff0 msgr2=0x7f08001947f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:59.269 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.695+0000 7f0805aee640 1 -- 172.21.15.6:0/3378289946 shutdown_connections 2024-09-12T03:11:59.269 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:58.695+0000 7f0805aee640 1 -- 172.21.15.6:0/3378289946 wait complete. 2024-09-12T03:11:59.270 INFO:teuthology.orchestra.run.smithi006.stdout:Generating ssh key... 2024-09-12T03:12:00.101 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: Generating public/private rsa key pair. 2024-09-12T03:12:00.101 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: Your identification has been saved in /tmp/tmp1r_vs2ru/key 2024-09-12T03:12:00.101 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: Your public key has been saved in /tmp/tmp1r_vs2ru/key.pub 2024-09-12T03:12:00.101 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: The key fingerprint is: 2024-09-12T03:12:00.101 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: SHA256:WQbc2kkMElg0vQla4RpliIXUFCy/krJQyvgJuhG2EuY ceph-89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:12:00.102 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: The key's randomart image is: 2024-09-12T03:12:00.102 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: +---[RSA 3072]----+ 2024-09-12T03:12:00.102 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: | ..B+*X=o+ | 2024-09-12T03:12:00.102 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: | + =+ooo.+ | 2024-09-12T03:12:00.102 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: | o.o.. *o. | 2024-09-12T03:12:00.102 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: | . oo ++o | 2024-09-12T03:12:00.102 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: |== ... S | 2024-09-12T03:12:00.102 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: |X++ . | 2024-09-12T03:12:00.102 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: |=E o | 2024-09-12T03:12:00.102 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: |+.o | 2024-09-12T03:12:00.102 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: |.. | 2024-09-12T03:12:00.102 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:11:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: +----[SHA256]-----+ 2024-09-12T03:12:00.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:00 smithi006 ceph-mon[30991]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:00.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:00 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' 2024-09-12T03:12:00.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:00 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' 2024-09-12T03:12:00.613 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.632+0000 7fcb18442640 1 Processor -- start 2024-09-12T03:12:00.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.632+0000 7fcb18442640 1 -- start start 2024-09-12T03:12:00.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.632+0000 7fcb18442640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb1007ddc0 0x7fcb1007c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:00.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.632+0000 7fcb18442640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1007c800 con 0x7fcb1007ddc0 2024-09-12T03:12:00.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.632+0000 7fcb161de640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb1007ddc0 0x7fcb1007c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:00.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.632+0000 7fcb161de640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb1007ddc0 0x7fcb1007c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51080/0 (socket says 172.21.15.6:51080) 2024-09-12T03:12:00.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.632+0000 7fcb161de640 1 -- 172.21.15.6:0/623952954 learned_addr learned my addr 172.21.15.6:0/623952954 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:00.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.633+0000 7fcb161de640 1 -- 172.21.15.6:0/623952954 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1007c980 con 0x7fcb1007ddc0 2024-09-12T03:12:00.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.633+0000 7fcb161de640 1 --2- 172.21.15.6:0/623952954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb1007ddc0 0x7fcb1007c2c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009820 tx=0x7fcb0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb20431019b4f72 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:00.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.633+0000 7fcb151dc640 1 -- 172.21.15.6:0/623952954 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb0c037070 con 0x7fcb1007ddc0 2024-09-12T03:12:00.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.633+0000 7fcb151dc640 1 -- 172.21.15.6:0/623952954 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcb0c032900 con 0x7fcb1007ddc0 2024-09-12T03:12:00.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.633+0000 7fcb151dc640 1 -- 172.21.15.6:0/623952954 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb0c030850 con 0x7fcb1007ddc0 2024-09-12T03:12:00.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.634+0000 7fcb18442640 1 -- 172.21.15.6:0/623952954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb1007ddc0 msgr2=0x7fcb1007c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:00.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.634+0000 7fcb18442640 1 --2- 172.21.15.6:0/623952954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb1007ddc0 0x7fcb1007c2c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009820 tx=0x7fcb0c02a270 comp rx=0 tx=0).stop 2024-09-12T03:12:00.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.634+0000 7fcb18442640 1 -- 172.21.15.6:0/623952954 shutdown_connections 2024-09-12T03:12:00.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.634+0000 7fcb18442640 1 --2- 172.21.15.6:0/623952954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb1007ddc0 0x7fcb1007c2c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.634+0000 7fcb18442640 1 -- 172.21.15.6:0/623952954 >> 172.21.15.6:0/623952954 conn(0x7fcb101031b0 msgr2=0x7fcb101055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:00.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.634+0000 7fcb18442640 1 -- 172.21.15.6:0/623952954 shutdown_connections 2024-09-12T03:12:00.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.634+0000 7fcb18442640 1 -- 172.21.15.6:0/623952954 wait complete. 2024-09-12T03:12:00.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.635+0000 7fcb18442640 1 Processor -- start 2024-09-12T03:12:00.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.635+0000 7fcb18442640 1 -- start start 2024-09-12T03:12:00.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.635+0000 7fcb18442640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb1007ddc0 0x7fcb1019dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:00.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.635+0000 7fcb18442640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1010df90 con 0x7fcb1007ddc0 2024-09-12T03:12:00.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.635+0000 7fcb161de640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb1007ddc0 0x7fcb1019dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:00.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.635+0000 7fcb161de640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb1007ddc0 0x7fcb1019dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51092/0 (socket says 172.21.15.6:51092) 2024-09-12T03:12:00.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.635+0000 7fcb161de640 1 -- 172.21.15.6:0/2760932806 learned_addr learned my addr 172.21.15.6:0/2760932806 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:00.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.635+0000 7fcb161de640 1 -- 172.21.15.6:0/2760932806 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1019e020 con 0x7fcb1007ddc0 2024-09-12T03:12:00.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.635+0000 7fcb161de640 1 --2- 172.21.15.6:0/2760932806 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb1007ddc0 0x7fcb1019dae0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c032ce0 tx=0x7fcb0c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:00.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.636+0000 7fcb077fe640 1 -- 172.21.15.6:0/2760932806 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb0c042070 con 0x7fcb1007ddc0 2024-09-12T03:12:00.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.636+0000 7fcb077fe640 1 -- 172.21.15.6:0/2760932806 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcb0c0308c0 con 0x7fcb1007ddc0 2024-09-12T03:12:00.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.636+0000 7fcb18442640 1 -- 172.21.15.6:0/2760932806 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1019e2b0 con 0x7fcb1007ddc0 2024-09-12T03:12:00.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.636+0000 7fcb077fe640 1 -- 172.21.15.6:0/2760932806 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb0c037040 con 0x7fcb1007ddc0 2024-09-12T03:12:00.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.636+0000 7fcb18442640 1 -- 172.21.15.6:0/2760932806 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb101a0fa0 con 0x7fcb1007ddc0 2024-09-12T03:12:00.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.637+0000 7fcb077fe640 1 -- 172.21.15.6:0/2760932806 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fcb0c045070 con 0x7fcb1007ddc0 2024-09-12T03:12:00.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.637+0000 7fcb18442640 1 -- 172.21.15.6:0/2760932806 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb1007c340 con 0x7fcb1007ddc0 2024-09-12T03:12:00.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.637+0000 7fcb077fe640 1 --2- 172.21.15.6:0/2760932806 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fcafc038f70 0x7fcafc03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:00.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.637+0000 7fcb077fe640 1 -- 172.21.15.6:0/2760932806 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcb0c06c170 con 0x7fcb1007ddc0 2024-09-12T03:12:00.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.637+0000 7fcb159dd640 1 --2- 172.21.15.6:0/2760932806 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fcafc038f70 0x7fcafc03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:00.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.637+0000 7fcb159dd640 1 --2- 172.21.15.6:0/2760932806 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fcafc038f70 0x7fcafc03b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcb000098c0 tx=0x7fcb00006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:00.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.642+0000 7fcb077fe640 1 -- 172.21.15.6:0/2760932806 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb0c045520 con 0x7fcb1007ddc0 2024-09-12T03:12:00.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.818+0000 7fcb18442640 1 -- 172.21.15.6:0/2760932806 --> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fcb10104400 con 0x7fcafc038f70 2024-09-12T03:12:00.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.987+0000 7fcb077fe640 1 -- 172.21.15.6:0/2760932806 <== mgr.14118 v2:172.21.15.6:6800/1684477653 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcb10104400 con 0x7fcafc038f70 2024-09-12T03:12:00.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.988+0000 7fcb18442640 1 -- 172.21.15.6:0/2760932806 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fcafc038f70 msgr2=0x7fcafc03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:00.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.989+0000 7fcb18442640 1 --2- 172.21.15.6:0/2760932806 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fcafc038f70 0x7fcafc03b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcb000098c0 tx=0x7fcb00006db0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.989+0000 7fcb18442640 1 -- 172.21.15.6:0/2760932806 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb1007ddc0 msgr2=0x7fcb1019dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:00.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.989+0000 7fcb18442640 1 --2- 172.21.15.6:0/2760932806 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb1007ddc0 0x7fcb1019dae0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c032ce0 tx=0x7fcb0c032d10 comp rx=0 tx=0).stop 2024-09-12T03:12:00.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.989+0000 7fcb18442640 1 -- 172.21.15.6:0/2760932806 shutdown_connections 2024-09-12T03:12:00.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.989+0000 7fcb18442640 1 --2- 172.21.15.6:0/2760932806 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fcafc038f70 0x7fcafc03b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.989+0000 7fcb18442640 1 --2- 172.21.15.6:0/2760932806 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb1007ddc0 0x7fcb1019dae0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.618 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.989+0000 7fcb18442640 1 -- 172.21.15.6:0/2760932806 >> 172.21.15.6:0/2760932806 conn(0x7fcb101031b0 msgr2=0x7fcb10103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:00.618 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.989+0000 7fcb18442640 1 -- 172.21.15.6:0/2760932806 shutdown_connections 2024-09-12T03:12:00.618 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:11:59.989+0000 7fcb18442640 1 -- 172.21.15.6:0/2760932806 wait complete. 2024-09-12T03:12:01.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:01 smithi006 ceph-mon[30991]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:01.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:01 smithi006 ceph-mon[30991]: Generating ssh key... 2024-09-12T03:12:01.737 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyyWudC1H8NyjplBuWJYyTGyCmOlQK9SLFIXzd+u4UcDVx3IqI5XWQqhwA9fx0ksZSLWYMp8mNXfFeVxGUupTk6aDeXYXsyOpOqRV7gGBZtgewf0xl2fBLdOxDHuP0denmuHNqJXXRpM+uJz9wT5mtuIU9jF1+EjgU5euAZPq/DEr+WrR7fOuwnrca/Rx+L37kgkSsw6eUE55qBMe/tAP8j4pT1B40FArlrcmmrnR13VZG6kIRsFQqhnmAbbi9Hsu02ZBWBSnnXcQyBE64BsX2/PiFF2kZEXe7pBfH0bLmUJKY7jUrqLTX2aGhOYOaBE0SI2fYbaMEuKAjuTDpEQjGJm7ANhRi9a43rPMllur6FOdWTnWj0bCMKiCjcx4vD136RT+oHNtqPTNz31rUi6GFMwbxMduPPnaST3F9vjAZhRYRCa5kjwUwSnXqTfNrVEd159zk4ZQd62bMNwgb/ciOPn8+ucq92c89M2kGmTi5BrH4Ezn+GMqOXB8m4vO2CRs= ceph-89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:12:01.737 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.015+0000 7fbc8238e640 1 Processor -- start 2024-09-12T03:12:01.737 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.015+0000 7fbc8238e640 1 -- start start 2024-09-12T03:12:01.737 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.015+0000 7fbc8238e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc7c106230 0x7fbc7c106630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:01.737 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.015+0000 7fbc8238e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c07be10 con 0x7fbc7c106230 2024-09-12T03:12:01.737 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.015+0000 7fbc7bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc7c106230 0x7fbc7c106630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:01.737 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.015+0000 7fbc7bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc7c106230 0x7fbc7c106630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51108/0 (socket says 172.21.15.6:51108) 2024-09-12T03:12:01.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.015+0000 7fbc7bfff640 1 -- 172.21.15.6:0/1748419835 learned_addr learned my addr 172.21.15.6:0/1748419835 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:01.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.015+0000 7fbc7bfff640 1 -- 172.21.15.6:0/1748419835 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7c106b70 con 0x7fbc7c106230 2024-09-12T03:12:01.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.016+0000 7fbc7bfff640 1 --2- 172.21.15.6:0/1748419835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc7c106230 0x7fbc7c106630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c009820 tx=0x7fbc6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4ec166b9bdb9ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:01.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.016+0000 7fbc7b7fe640 1 -- 172.21.15.6:0/1748419835 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc6c037070 con 0x7fbc7c106230 2024-09-12T03:12:01.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.016+0000 7fbc7b7fe640 1 -- 172.21.15.6:0/1748419835 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc6c032900 con 0x7fbc7c106230 2024-09-12T03:12:01.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.016+0000 7fbc7b7fe640 1 -- 172.21.15.6:0/1748419835 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc6c030850 con 0x7fbc7c106230 2024-09-12T03:12:01.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.017+0000 7fbc8238e640 1 -- 172.21.15.6:0/1748419835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc7c106230 msgr2=0x7fbc7c106630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:01.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.017+0000 7fbc8238e640 1 --2- 172.21.15.6:0/1748419835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc7c106230 0x7fbc7c106630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c009820 tx=0x7fbc6c02a270 comp rx=0 tx=0).stop 2024-09-12T03:12:01.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.017+0000 7fbc8238e640 1 -- 172.21.15.6:0/1748419835 shutdown_connections 2024-09-12T03:12:01.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.017+0000 7fbc8238e640 1 --2- 172.21.15.6:0/1748419835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc7c106230 0x7fbc7c106630 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:01.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.017+0000 7fbc8238e640 1 -- 172.21.15.6:0/1748419835 >> 172.21.15.6:0/1748419835 conn(0x7fbc7c102000 msgr2=0x7fbc7c104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:01.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.017+0000 7fbc8238e640 1 -- 172.21.15.6:0/1748419835 shutdown_connections 2024-09-12T03:12:01.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.017+0000 7fbc8238e640 1 -- 172.21.15.6:0/1748419835 wait complete. 2024-09-12T03:12:01.739 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.018+0000 7fbc8238e640 1 Processor -- start 2024-09-12T03:12:01.739 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.018+0000 7fbc8238e640 1 -- start start 2024-09-12T03:12:01.739 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.018+0000 7fbc8238e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc7c106230 0x7fbc7c19db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:01.739 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.018+0000 7fbc8238e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c107c70 con 0x7fbc7c106230 2024-09-12T03:12:01.739 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.018+0000 7fbc7bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc7c106230 0x7fbc7c19db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:01.739 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.018+0000 7fbc7bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc7c106230 0x7fbc7c19db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51118/0 (socket says 172.21.15.6:51118) 2024-09-12T03:12:01.739 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.018+0000 7fbc7bfff640 1 -- 172.21.15.6:0/2789136334 learned_addr learned my addr 172.21.15.6:0/2789136334 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:01.739 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.019+0000 7fbc7bfff640 1 -- 172.21.15.6:0/2789136334 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7c19e060 con 0x7fbc7c106230 2024-09-12T03:12:01.739 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.019+0000 7fbc7bfff640 1 --2- 172.21.15.6:0/2789136334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc7c106230 0x7fbc7c19db20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0097f0 tx=0x7fbc6c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:01.739 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.019+0000 7fbc79ffb640 1 -- 172.21.15.6:0/2789136334 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc6c042070 con 0x7fbc7c106230 2024-09-12T03:12:01.739 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.019+0000 7fbc79ffb640 1 -- 172.21.15.6:0/2789136334 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc6c032ca0 con 0x7fbc7c106230 2024-09-12T03:12:01.739 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.019+0000 7fbc8238e640 1 -- 172.21.15.6:0/2789136334 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc7c19e2f0 con 0x7fbc7c106230 2024-09-12T03:12:01.739 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.019+0000 7fbc79ffb640 1 -- 172.21.15.6:0/2789136334 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc6c037040 con 0x7fbc7c106230 2024-09-12T03:12:01.740 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.019+0000 7fbc8238e640 1 -- 172.21.15.6:0/2789136334 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc7c1a0fe0 con 0x7fbc7c106230 2024-09-12T03:12:01.740 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.020+0000 7fbc79ffb640 1 -- 172.21.15.6:0/2789136334 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbc6c045070 con 0x7fbc7c106230 2024-09-12T03:12:01.740 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.020+0000 7fbc8238e640 1 -- 172.21.15.6:0/2789136334 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc7c106630 con 0x7fbc7c106230 2024-09-12T03:12:01.740 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.020+0000 7fbc79ffb640 1 --2- 172.21.15.6:0/2789136334 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fbc68038fc0 0x7fbc6803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:01.740 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.020+0000 7fbc79ffb640 1 -- 172.21.15.6:0/2789136334 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbc6c06c0c0 con 0x7fbc7c106230 2024-09-12T03:12:01.740 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.020+0000 7fbc73fff640 1 --2- 172.21.15.6:0/2789136334 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fbc68038fc0 0x7fbc6803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:01.740 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.021+0000 7fbc73fff640 1 --2- 172.21.15.6:0/2789136334 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fbc68038fc0 0x7fbc6803b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbc640098c0 tx=0x7fbc64006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:01.740 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.025+0000 7fbc79ffb640 1 -- 172.21.15.6:0/2789136334 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc6c03add0 con 0x7fbc7c106230 2024-09-12T03:12:01.740 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.206+0000 7fbc8238e640 1 -- 172.21.15.6:0/2789136334 --> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fbc7c02dab0 con 0x7fbc68038fc0 2024-09-12T03:12:01.740 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.207+0000 7fbc79ffb640 1 -- 172.21.15.6:0/2789136334 <== mgr.14118 v2:172.21.15.6:6800/1684477653 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fbc7c02dab0 con 0x7fbc68038fc0 2024-09-12T03:12:01.740 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.209+0000 7fbc8238e640 1 -- 172.21.15.6:0/2789136334 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fbc68038fc0 msgr2=0x7fbc6803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:01.740 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.209+0000 7fbc8238e640 1 --2- 172.21.15.6:0/2789136334 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fbc68038fc0 0x7fbc6803b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbc640098c0 tx=0x7fbc64006db0 comp rx=0 tx=0).stop 2024-09-12T03:12:01.741 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.209+0000 7fbc8238e640 1 -- 172.21.15.6:0/2789136334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc7c106230 msgr2=0x7fbc7c19db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:01.741 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.209+0000 7fbc8238e640 1 --2- 172.21.15.6:0/2789136334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc7c106230 0x7fbc7c19db20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0097f0 tx=0x7fbc6c030ec0 comp rx=0 tx=0).stop 2024-09-12T03:12:01.741 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.209+0000 7fbc8238e640 1 -- 172.21.15.6:0/2789136334 shutdown_connections 2024-09-12T03:12:01.741 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.209+0000 7fbc8238e640 1 --2- 172.21.15.6:0/2789136334 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fbc68038fc0 0x7fbc6803b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:01.741 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.209+0000 7fbc8238e640 1 --2- 172.21.15.6:0/2789136334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc7c106230 0x7fbc7c19db20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:01.741 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.209+0000 7fbc8238e640 1 -- 172.21.15.6:0/2789136334 >> 172.21.15.6:0/2789136334 conn(0x7fbc7c102000 msgr2=0x7fbc7c1947e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:01.741 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.210+0000 7fbc8238e640 1 -- 172.21.15.6:0/2789136334 shutdown_connections 2024-09-12T03:12:01.741 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:01.210+0000 7fbc8238e640 1 -- 172.21.15.6:0/2789136334 wait complete. 2024-09-12T03:12:01.741 INFO:teuthology.orchestra.run.smithi006.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:12:01.741 INFO:teuthology.orchestra.run.smithi006.stdout:Adding key to root@localhost authorized_keys... 2024-09-12T03:12:01.746 INFO:teuthology.orchestra.run.smithi006.stdout:Adding host smithi006... 2024-09-12T03:12:02.188 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:02 smithi006 ceph-mon[30991]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:03.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:03 smithi006 ceph-mon[30991]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi006", "addr": "172.21.15.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:04.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:04 smithi006 ceph-mon[30991]: Deploying cephadm binary to smithi006 2024-09-12T03:12:06.869 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Added host 'smithi006' with addr '172.21.15.6' 2024-09-12T03:12:06.869 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.188+0000 7fb41e369640 1 Processor -- start 2024-09-12T03:12:06.870 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.188+0000 7fb41e369640 1 -- start start 2024-09-12T03:12:06.870 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.188+0000 7fb41e369640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb41807dda0 0x7fb41807c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:06.870 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.188+0000 7fb41e369640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb41807c7e0 con 0x7fb41807dda0 2024-09-12T03:12:06.870 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.188+0000 7fb417fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb41807dda0 0x7fb41807c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:06.870 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.188+0000 7fb417fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb41807dda0 0x7fb41807c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51128/0 (socket says 172.21.15.6:51128) 2024-09-12T03:12:06.870 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.188+0000 7fb417fff640 1 -- 172.21.15.6:0/3801655604 learned_addr learned my addr 172.21.15.6:0/3801655604 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:06.870 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.189+0000 7fb417fff640 1 -- 172.21.15.6:0/3801655604 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41807c960 con 0x7fb41807dda0 2024-09-12T03:12:06.870 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.189+0000 7fb417fff640 1 --2- 172.21.15.6:0/3801655604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb41807dda0 0x7fb41807c2a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb408009820 tx=0x7fb40802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48591fe4051634b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:06.870 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.190+0000 7fb416ffd640 1 -- 172.21.15.6:0/3801655604 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb408037070 con 0x7fb41807dda0 2024-09-12T03:12:06.870 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.190+0000 7fb416ffd640 1 -- 172.21.15.6:0/3801655604 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb408032900 con 0x7fb41807dda0 2024-09-12T03:12:06.870 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.190+0000 7fb416ffd640 1 -- 172.21.15.6:0/3801655604 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb408030850 con 0x7fb41807dda0 2024-09-12T03:12:06.870 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.190+0000 7fb41e369640 1 -- 172.21.15.6:0/3801655604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb41807dda0 msgr2=0x7fb41807c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:06.870 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.190+0000 7fb41e369640 1 --2- 172.21.15.6:0/3801655604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb41807dda0 0x7fb41807c2a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb408009820 tx=0x7fb40802a270 comp rx=0 tx=0).stop 2024-09-12T03:12:06.871 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.190+0000 7fb41e369640 1 -- 172.21.15.6:0/3801655604 shutdown_connections 2024-09-12T03:12:06.871 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.190+0000 7fb41e369640 1 --2- 172.21.15.6:0/3801655604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb41807dda0 0x7fb41807c2a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:06.871 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.190+0000 7fb41e369640 1 -- 172.21.15.6:0/3801655604 >> 172.21.15.6:0/3801655604 conn(0x7fb4181031b0 msgr2=0x7fb4181055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:06.871 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.190+0000 7fb41e369640 1 -- 172.21.15.6:0/3801655604 shutdown_connections 2024-09-12T03:12:06.871 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.190+0000 7fb41e369640 1 -- 172.21.15.6:0/3801655604 wait complete. 2024-09-12T03:12:06.871 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.191+0000 7fb41e369640 1 Processor -- start 2024-09-12T03:12:06.871 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.191+0000 7fb41e369640 1 -- start start 2024-09-12T03:12:06.871 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.191+0000 7fb41e369640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb41807dda0 0x7fb41819daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:06.872 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.191+0000 7fb41e369640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb41810dfb0 con 0x7fb41807dda0 2024-09-12T03:12:06.872 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.191+0000 7fb417fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb41807dda0 0x7fb41819daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:06.872 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.192+0000 7fb417fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb41807dda0 0x7fb41819daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51130/0 (socket says 172.21.15.6:51130) 2024-09-12T03:12:06.872 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.192+0000 7fb417fff640 1 -- 172.21.15.6:0/1004399123 learned_addr learned my addr 172.21.15.6:0/1004399123 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:06.872 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.192+0000 7fb417fff640 1 -- 172.21.15.6:0/1004399123 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41819e030 con 0x7fb41807dda0 2024-09-12T03:12:06.872 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.192+0000 7fb417fff640 1 --2- 172.21.15.6:0/1004399123 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb41807dda0 0x7fb41819daf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb4080097f0 tx=0x7fb408030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:06.872 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.192+0000 7fb4157fa640 1 -- 172.21.15.6:0/1004399123 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb408037040 con 0x7fb41807dda0 2024-09-12T03:12:06.872 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.192+0000 7fb41e369640 1 -- 172.21.15.6:0/1004399123 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41819e2c0 con 0x7fb41807dda0 2024-09-12T03:12:06.873 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.192+0000 7fb4157fa640 1 -- 172.21.15.6:0/1004399123 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb408040070 con 0x7fb41807dda0 2024-09-12T03:12:06.873 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.192+0000 7fb4157fa640 1 -- 172.21.15.6:0/1004399123 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb40803a390 con 0x7fb41807dda0 2024-09-12T03:12:06.873 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.192+0000 7fb41e369640 1 -- 172.21.15.6:0/1004399123 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4181a0fb0 con 0x7fb41807dda0 2024-09-12T03:12:06.873 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.193+0000 7fb4157fa640 1 -- 172.21.15.6:0/1004399123 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb40803a5f0 con 0x7fb41807dda0 2024-09-12T03:12:06.873 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.193+0000 7fb4157fa640 1 --2- 172.21.15.6:0/1004399123 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fb400038f70 0x7fb40003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:06.874 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.193+0000 7fb41e369640 1 -- 172.21.15.6:0/1004399123 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb41807c320 con 0x7fb41807dda0 2024-09-12T03:12:06.874 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.193+0000 7fb4157fa640 1 -- 172.21.15.6:0/1004399123 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb40806c4e0 con 0x7fb41807dda0 2024-09-12T03:12:06.874 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.193+0000 7fb4177fe640 1 --2- 172.21.15.6:0/1004399123 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fb400038f70 0x7fb40003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:06.874 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.194+0000 7fb4177fe640 1 --2- 172.21.15.6:0/1004399123 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fb400038f70 0x7fb40003b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0098c0 tx=0x7fb40c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:06.874 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.198+0000 7fb4157fa640 1 -- 172.21.15.6:0/1004399123 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb40804a200 con 0x7fb41807dda0 2024-09-12T03:12:06.874 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:02.377+0000 7fb41e369640 1 -- 172.21.15.6:0/1004399123 --> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi006", "addr": "172.21.15.6", "target": ["mon-mgr", ""]}) -- 0x7fb4181043e0 con 0x7fb400038f70 2024-09-12T03:12:06.874 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:06.250+0000 7fb4157fa640 1 -- 172.21.15.6:0/1004399123 <== mgr.14118 v2:172.21.15.6:6800/1684477653 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7fb4181043e0 con 0x7fb400038f70 2024-09-12T03:12:06.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:06.253+0000 7fb41e369640 1 -- 172.21.15.6:0/1004399123 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fb400038f70 msgr2=0x7fb40003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:06.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:06.253+0000 7fb41e369640 1 --2- 172.21.15.6:0/1004399123 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fb400038f70 0x7fb40003b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0098c0 tx=0x7fb40c006db0 comp rx=0 tx=0).stop 2024-09-12T03:12:06.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:06.253+0000 7fb41e369640 1 -- 172.21.15.6:0/1004399123 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb41807dda0 msgr2=0x7fb41819daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:06.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:06.253+0000 7fb41e369640 1 --2- 172.21.15.6:0/1004399123 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb41807dda0 0x7fb41819daf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb4080097f0 tx=0x7fb408030d80 comp rx=0 tx=0).stop 2024-09-12T03:12:06.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:06.253+0000 7fb41e369640 1 -- 172.21.15.6:0/1004399123 shutdown_connections 2024-09-12T03:12:06.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:06.253+0000 7fb41e369640 1 --2- 172.21.15.6:0/1004399123 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fb400038f70 0x7fb40003b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:06.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:06.253+0000 7fb41e369640 1 --2- 172.21.15.6:0/1004399123 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb41807dda0 0x7fb41819daf0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:06.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:06.253+0000 7fb41e369640 1 -- 172.21.15.6:0/1004399123 >> 172.21.15.6:0/1004399123 conn(0x7fb4181031b0 msgr2=0x7fb418103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:06.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:06.253+0000 7fb41e369640 1 -- 172.21.15.6:0/1004399123 shutdown_connections 2024-09-12T03:12:06.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:06.253+0000 7fb41e369640 1 -- 172.21.15.6:0/1004399123 wait complete. 2024-09-12T03:12:06.876 INFO:teuthology.orchestra.run.smithi006.stdout:Deploying unmanaged mon service... 2024-09-12T03:12:07.500 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:07 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' 2024-09-12T03:12:07.500 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:07 smithi006 ceph-mon[30991]: Added host smithi006 2024-09-12T03:12:07.500 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:07 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:08.279 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-12T03:12:08.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.276+0000 7fe1b58b4640 1 Processor -- start 2024-09-12T03:12:08.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.276+0000 7fe1b58b4640 1 -- start start 2024-09-12T03:12:08.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.276+0000 7fe1b58b4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1b007de00 0x7fe1b007c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:08.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.276+0000 7fe1b58b4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b007c840 con 0x7fe1b007de00 2024-09-12T03:12:08.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.276+0000 7fe1aeffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1b007de00 0x7fe1b007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:08.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.276+0000 7fe1aeffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1b007de00 0x7fe1b007c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51136/0 (socket says 172.21.15.6:51136) 2024-09-12T03:12:08.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.276+0000 7fe1aeffd640 1 -- 172.21.15.6:0/799154654 learned_addr learned my addr 172.21.15.6:0/799154654 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:08.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.277+0000 7fe1aeffd640 1 -- 172.21.15.6:0/799154654 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b007c9c0 con 0x7fe1b007de00 2024-09-12T03:12:08.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.277+0000 7fe1aeffd640 1 --2- 172.21.15.6:0/799154654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1b007de00 0x7fe1b007c300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009820 tx=0x7fe1a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bdf9dac7c7c3f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:08.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.278+0000 7fe1ae7fc640 1 -- 172.21.15.6:0/799154654 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1a0037070 con 0x7fe1b007de00 2024-09-12T03:12:08.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.278+0000 7fe1ae7fc640 1 -- 172.21.15.6:0/799154654 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe1a0032900 con 0x7fe1b007de00 2024-09-12T03:12:08.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.278+0000 7fe1ae7fc640 1 -- 172.21.15.6:0/799154654 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1a0030850 con 0x7fe1b007de00 2024-09-12T03:12:08.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.278+0000 7fe1b58b4640 1 -- 172.21.15.6:0/799154654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1b007de00 msgr2=0x7fe1b007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:08.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.278+0000 7fe1b58b4640 1 --2- 172.21.15.6:0/799154654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1b007de00 0x7fe1b007c300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009820 tx=0x7fe1a002a270 comp rx=0 tx=0).stop 2024-09-12T03:12:08.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.278+0000 7fe1b58b4640 1 -- 172.21.15.6:0/799154654 shutdown_connections 2024-09-12T03:12:08.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.278+0000 7fe1b58b4640 1 --2- 172.21.15.6:0/799154654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1b007de00 0x7fe1b007c300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:08.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.278+0000 7fe1b58b4640 1 -- 172.21.15.6:0/799154654 >> 172.21.15.6:0/799154654 conn(0x7fe1b01031b0 msgr2=0x7fe1b01055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:08.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.278+0000 7fe1b58b4640 1 -- 172.21.15.6:0/799154654 shutdown_connections 2024-09-12T03:12:08.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.278+0000 7fe1b58b4640 1 -- 172.21.15.6:0/799154654 wait complete. 2024-09-12T03:12:08.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.279+0000 7fe1b58b4640 1 Processor -- start 2024-09-12T03:12:08.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.279+0000 7fe1b58b4640 1 -- start start 2024-09-12T03:12:08.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.279+0000 7fe1b58b4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1b007de00 0x7fe1b019db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:08.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.279+0000 7fe1b58b4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b010dfb0 con 0x7fe1b007de00 2024-09-12T03:12:08.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.279+0000 7fe1aeffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1b007de00 0x7fe1b019db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:08.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.279+0000 7fe1aeffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1b007de00 0x7fe1b019db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51138/0 (socket says 172.21.15.6:51138) 2024-09-12T03:12:08.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.279+0000 7fe1aeffd640 1 -- 172.21.15.6:0/565383224 learned_addr learned my addr 172.21.15.6:0/565383224 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:08.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.279+0000 7fe1aeffd640 1 -- 172.21.15.6:0/565383224 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b019e060 con 0x7fe1b007de00 2024-09-12T03:12:08.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.279+0000 7fe1aeffd640 1 --2- 172.21.15.6:0/565383224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1b007de00 0x7fe1b019db20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00097f0 tx=0x7fe1a0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:08.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.280+0000 7fe1b48b2640 1 -- 172.21.15.6:0/565383224 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1a0037040 con 0x7fe1b007de00 2024-09-12T03:12:08.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.280+0000 7fe1b48b2640 1 -- 172.21.15.6:0/565383224 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe1a0040070 con 0x7fe1b007de00 2024-09-12T03:12:08.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.280+0000 7fe1b58b4640 1 -- 172.21.15.6:0/565383224 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b019e2f0 con 0x7fe1b007de00 2024-09-12T03:12:08.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.280+0000 7fe1b48b2640 1 -- 172.21.15.6:0/565383224 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1a003a390 con 0x7fe1b007de00 2024-09-12T03:12:08.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.280+0000 7fe1b58b4640 1 -- 172.21.15.6:0/565383224 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b01a0fe0 con 0x7fe1b007de00 2024-09-12T03:12:08.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.280+0000 7fe1b48b2640 1 -- 172.21.15.6:0/565383224 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe1a003a5a0 con 0x7fe1b007de00 2024-09-12T03:12:08.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.281+0000 7fe1b58b4640 1 -- 172.21.15.6:0/565383224 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1b007c380 con 0x7fe1b007de00 2024-09-12T03:12:08.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.281+0000 7fe1b48b2640 1 --2- 172.21.15.6:0/565383224 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fe19c038fc0 0x7fe19c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:08.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.281+0000 7fe1b48b2640 1 -- 172.21.15.6:0/565383224 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe1a006c5a0 con 0x7fe1b007de00 2024-09-12T03:12:08.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.281+0000 7fe1a7fff640 1 --2- 172.21.15.6:0/565383224 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fe19c038fc0 0x7fe19c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:08.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.281+0000 7fe1a7fff640 1 --2- 172.21.15.6:0/565383224 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fe19c038fc0 0x7fe19c03b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe1980098c0 tx=0x7fe198006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:08.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.286+0000 7fe1b48b2640 1 -- 172.21.15.6:0/565383224 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe1a0043360 con 0x7fe1b007de00 2024-09-12T03:12:08.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.500+0000 7fe1b58b4640 1 -- 172.21.15.6:0/565383224 --> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe1b01043d0 con 0x7fe19c038fc0 2024-09-12T03:12:08.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.505+0000 7fe1b48b2640 1 -- 172.21.15.6:0/565383224 <== mgr.14118 v2:172.21.15.6:6800/1684477653 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe1b01043d0 con 0x7fe19c038fc0 2024-09-12T03:12:08.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.507+0000 7fe1b58b4640 1 -- 172.21.15.6:0/565383224 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fe19c038fc0 msgr2=0x7fe19c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:08.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.507+0000 7fe1b58b4640 1 --2- 172.21.15.6:0/565383224 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fe19c038fc0 0x7fe19c03b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe1980098c0 tx=0x7fe198006db0 comp rx=0 tx=0).stop 2024-09-12T03:12:08.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.507+0000 7fe1b58b4640 1 -- 172.21.15.6:0/565383224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1b007de00 msgr2=0x7fe1b019db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:08.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.507+0000 7fe1b58b4640 1 --2- 172.21.15.6:0/565383224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1b007de00 0x7fe1b019db20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00097f0 tx=0x7fe1a0030d80 comp rx=0 tx=0).stop 2024-09-12T03:12:08.284 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.507+0000 7fe1b58b4640 1 -- 172.21.15.6:0/565383224 shutdown_connections 2024-09-12T03:12:08.284 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.507+0000 7fe1b58b4640 1 --2- 172.21.15.6:0/565383224 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7fe19c038fc0 0x7fe19c03b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:08.284 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.507+0000 7fe1b58b4640 1 --2- 172.21.15.6:0/565383224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1b007de00 0x7fe1b019db20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:08.284 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.507+0000 7fe1b58b4640 1 -- 172.21.15.6:0/565383224 >> 172.21.15.6:0/565383224 conn(0x7fe1b01031b0 msgr2=0x7fe1b0103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:08.284 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.507+0000 7fe1b58b4640 1 -- 172.21.15.6:0/565383224 shutdown_connections 2024-09-12T03:12:08.284 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:07.507+0000 7fe1b58b4640 1 -- 172.21.15.6:0/565383224 wait complete. 2024-09-12T03:12:08.284 INFO:teuthology.orchestra.run.smithi006.stdout:Deploying unmanaged mgr service... 2024-09-12T03:12:08.809 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:08 smithi006 ceph-mon[30991]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:08.809 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:08 smithi006 ceph-mon[30991]: Saving service mon spec with placement count:5 2024-09-12T03:12:08.809 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:08 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' 2024-09-12T03:12:10.217 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:10 smithi006 ceph-mon[30991]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:10.217 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:10 smithi006 ceph-mon[30991]: Saving service mgr spec with placement count:2 2024-09-12T03:12:10.218 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:10 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' 2024-09-12T03:12:10.218 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:10 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' 2024-09-12T03:12:10.224 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-12T03:12:10.224 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.899+0000 7f8977d8c640 1 Processor -- start 2024-09-12T03:12:10.224 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.899+0000 7f8977d8c640 1 -- start start 2024-09-12T03:12:10.225 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.899+0000 7f8977d8c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f897007dde0 0x7f897007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:10.225 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.899+0000 7f8977d8c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f897007c820 con 0x7f897007dde0 2024-09-12T03:12:10.225 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.899+0000 7f8975b28640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f897007dde0 0x7f897007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:10.225 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.899+0000 7f8975b28640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f897007dde0 0x7f897007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51150/0 (socket says 172.21.15.6:51150) 2024-09-12T03:12:10.225 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.899+0000 7f8975b28640 1 -- 172.21.15.6:0/239998300 learned_addr learned my addr 172.21.15.6:0/239998300 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:10.225 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.900+0000 7f8975b28640 1 -- 172.21.15.6:0/239998300 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897007c9a0 con 0x7f897007dde0 2024-09-12T03:12:10.225 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.900+0000 7f8975b28640 1 --2- 172.21.15.6:0/239998300 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f897007dde0 0x7f897007c2e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f896c009820 tx=0x7f896c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9562a934aba078 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:10.225 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.901+0000 7f8974b26640 1 -- 172.21.15.6:0/239998300 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f896c037070 con 0x7f897007dde0 2024-09-12T03:12:10.225 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.901+0000 7f8974b26640 1 -- 172.21.15.6:0/239998300 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f896c032900 con 0x7f897007dde0 2024-09-12T03:12:10.225 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.901+0000 7f8974b26640 1 -- 172.21.15.6:0/239998300 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f896c030850 con 0x7f897007dde0 2024-09-12T03:12:10.225 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.901+0000 7f8977d8c640 1 -- 172.21.15.6:0/239998300 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f897007dde0 msgr2=0x7f897007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:10.225 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.901+0000 7f8977d8c640 1 --2- 172.21.15.6:0/239998300 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f897007dde0 0x7f897007c2e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f896c009820 tx=0x7f896c02a270 comp rx=0 tx=0).stop 2024-09-12T03:12:10.225 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.901+0000 7f8977d8c640 1 -- 172.21.15.6:0/239998300 shutdown_connections 2024-09-12T03:12:10.226 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.901+0000 7f8977d8c640 1 --2- 172.21.15.6:0/239998300 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f897007dde0 0x7f897007c2e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.226 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.901+0000 7f8977d8c640 1 -- 172.21.15.6:0/239998300 >> 172.21.15.6:0/239998300 conn(0x7f89701031b0 msgr2=0x7f89701055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:10.226 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.901+0000 7f8977d8c640 1 -- 172.21.15.6:0/239998300 shutdown_connections 2024-09-12T03:12:10.226 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.901+0000 7f8977d8c640 1 -- 172.21.15.6:0/239998300 wait complete. 2024-09-12T03:12:10.226 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.902+0000 7f8977d8c640 1 Processor -- start 2024-09-12T03:12:10.226 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.902+0000 7f8977d8c640 1 -- start start 2024-09-12T03:12:10.226 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.902+0000 7f8977d8c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f897007dde0 0x7f897019dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:10.226 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.902+0000 7f8977d8c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f897010df70 con 0x7f897007dde0 2024-09-12T03:12:10.226 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.902+0000 7f8975b28640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f897007dde0 0x7f897019dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:10.226 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.902+0000 7f8975b28640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f897007dde0 0x7f897019dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51156/0 (socket says 172.21.15.6:51156) 2024-09-12T03:12:10.226 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.902+0000 7f8975b28640 1 -- 172.21.15.6:0/4289195611 learned_addr learned my addr 172.21.15.6:0/4289195611 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:10.226 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.903+0000 7f8975b28640 1 -- 172.21.15.6:0/4289195611 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897019e020 con 0x7f897007dde0 2024-09-12T03:12:10.226 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.903+0000 7f8975b28640 1 --2- 172.21.15.6:0/4289195611 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f897007dde0 0x7f897019dae0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f896c032ce0 tx=0x7f896c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:10.227 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.903+0000 7f8966ffd640 1 -- 172.21.15.6:0/4289195611 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f896c042070 con 0x7f897007dde0 2024-09-12T03:12:10.227 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.903+0000 7f8966ffd640 1 -- 172.21.15.6:0/4289195611 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f896c0308c0 con 0x7f897007dde0 2024-09-12T03:12:10.227 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.903+0000 7f8966ffd640 1 -- 172.21.15.6:0/4289195611 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f896c037040 con 0x7f897007dde0 2024-09-12T03:12:10.227 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.903+0000 7f8977d8c640 1 -- 172.21.15.6:0/4289195611 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f897019e2b0 con 0x7f897007dde0 2024-09-12T03:12:10.227 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.903+0000 7f8977d8c640 1 -- 172.21.15.6:0/4289195611 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89701a0fa0 con 0x7f897007dde0 2024-09-12T03:12:10.227 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.904+0000 7f8966ffd640 1 -- 172.21.15.6:0/4289195611 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f896c045070 con 0x7f897007dde0 2024-09-12T03:12:10.227 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.904+0000 7f8966ffd640 1 --2- 172.21.15.6:0/4289195611 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f895c038fc0 0x7f895c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:10.227 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.904+0000 7f8977d8c640 1 -- 172.21.15.6:0/4289195611 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f897007c360 con 0x7f897007dde0 2024-09-12T03:12:10.227 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.904+0000 7f8966ffd640 1 -- 172.21.15.6:0/4289195611 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f896c06bfd0 con 0x7f897007dde0 2024-09-12T03:12:10.227 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.905+0000 7f8975327640 1 --2- 172.21.15.6:0/4289195611 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f895c038fc0 0x7f895c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:10.227 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.905+0000 7f8975327640 1 --2- 172.21.15.6:0/4289195611 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f895c038fc0 0x7f895c03b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f89600098c0 tx=0x7f8960006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:10.227 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:08.909+0000 7f8966ffd640 1 -- 172.21.15.6:0/4289195611 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f896c04b170 con 0x7f897007dde0 2024-09-12T03:12:10.227 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:09.099+0000 7f8977d8c640 1 -- 172.21.15.6:0/4289195611 --> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f89701043c0 con 0x7f895c038fc0 2024-09-12T03:12:10.228 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:09.103+0000 7f8966ffd640 1 -- 172.21.15.6:0/4289195611 <== mgr.14118 v2:172.21.15.6:6800/1684477653 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f89701043c0 con 0x7f895c038fc0 2024-09-12T03:12:10.228 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:09.105+0000 7f8977d8c640 1 -- 172.21.15.6:0/4289195611 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f895c038fc0 msgr2=0x7f895c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:10.228 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:09.105+0000 7f8977d8c640 1 --2- 172.21.15.6:0/4289195611 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f895c038fc0 0x7f895c03b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f89600098c0 tx=0x7f8960006db0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.228 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:09.105+0000 7f8977d8c640 1 -- 172.21.15.6:0/4289195611 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f897007dde0 msgr2=0x7f897019dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:10.228 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:09.105+0000 7f8977d8c640 1 --2- 172.21.15.6:0/4289195611 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f897007dde0 0x7f897019dae0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f896c032ce0 tx=0x7f896c032d10 comp rx=0 tx=0).stop 2024-09-12T03:12:10.228 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:09.106+0000 7f8977d8c640 1 -- 172.21.15.6:0/4289195611 shutdown_connections 2024-09-12T03:12:10.228 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:09.106+0000 7f8977d8c640 1 --2- 172.21.15.6:0/4289195611 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f895c038fc0 0x7f895c03b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.228 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:09.106+0000 7f8977d8c640 1 --2- 172.21.15.6:0/4289195611 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f897007dde0 0x7f897019dae0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.228 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:09.106+0000 7f8977d8c640 1 -- 172.21.15.6:0/4289195611 >> 172.21.15.6:0/4289195611 conn(0x7f89701031b0 msgr2=0x7f8970103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:10.228 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:09.106+0000 7f8977d8c640 1 -- 172.21.15.6:0/4289195611 shutdown_connections 2024-09-12T03:12:10.228 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:09.106+0000 7f8977d8c640 1 -- 172.21.15.6:0/4289195611 wait complete. 2024-09-12T03:12:11.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.630+0000 7f699110b640 1 Processor -- start 2024-09-12T03:12:11.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.630+0000 7f699110b640 1 -- start start 2024-09-12T03:12:11.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.630+0000 7f699110b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698c1079c0 0x7f698c107dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:11.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.630+0000 7f699110b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f698c108390 con 0x7f698c1079c0 2024-09-12T03:12:11.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.630+0000 7f698ad9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698c1079c0 0x7f698c107dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:11.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.630+0000 7f698ad9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698c1079c0 0x7f698c107dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54248/0 (socket says 172.21.15.6:54248) 2024-09-12T03:12:11.448 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.630+0000 7f698ad9d640 1 -- 172.21.15.6:0/2896476183 learned_addr learned my addr 172.21.15.6:0/2896476183 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:11.448 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.631+0000 7f698ad9d640 1 -- 172.21.15.6:0/2896476183 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c108b20 con 0x7f698c1079c0 2024-09-12T03:12:11.448 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.631+0000 7f698ad9d640 1 --2- 172.21.15.6:0/2896476183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698c1079c0 0x7f698c107dc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6984009820 tx=0x7f698402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8962db434d43d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:11.448 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.632+0000 7f6989d9b640 1 -- 172.21.15.6:0/2896476183 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6984037070 con 0x7f698c1079c0 2024-09-12T03:12:11.448 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.632+0000 7f6989d9b640 1 -- 172.21.15.6:0/2896476183 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6984032920 con 0x7f698c1079c0 2024-09-12T03:12:11.448 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.632+0000 7f6989d9b640 1 -- 172.21.15.6:0/2896476183 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6984030810 con 0x7f698c1079c0 2024-09-12T03:12:11.448 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.632+0000 7f699110b640 1 -- 172.21.15.6:0/2896476183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698c1079c0 msgr2=0x7f698c107dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:11.448 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.632+0000 7f699110b640 1 --2- 172.21.15.6:0/2896476183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698c1079c0 0x7f698c107dc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6984009820 tx=0x7f698402a270 comp rx=0 tx=0).stop 2024-09-12T03:12:11.448 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.632+0000 7f699110b640 1 -- 172.21.15.6:0/2896476183 shutdown_connections 2024-09-12T03:12:11.448 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.632+0000 7f699110b640 1 --2- 172.21.15.6:0/2896476183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698c1079c0 0x7f698c107dc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:11.448 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.632+0000 7f699110b640 1 -- 172.21.15.6:0/2896476183 >> 172.21.15.6:0/2896476183 conn(0x7f698c1031b0 msgr2=0x7f698c1055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:11.448 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.632+0000 7f699110b640 1 -- 172.21.15.6:0/2896476183 shutdown_connections 2024-09-12T03:12:11.448 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.632+0000 7f699110b640 1 -- 172.21.15.6:0/2896476183 wait complete. 2024-09-12T03:12:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.633+0000 7f699110b640 1 Processor -- start 2024-09-12T03:12:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.633+0000 7f699110b640 1 -- start start 2024-09-12T03:12:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.633+0000 7f699110b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698c1079c0 0x7f698c07db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.633+0000 7f699110b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f698c10bed0 con 0x7f698c1079c0 2024-09-12T03:12:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.634+0000 7f698ad9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698c1079c0 0x7f698c07db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.634+0000 7f698ad9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698c1079c0 0x7f698c07db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54264/0 (socket says 172.21.15.6:54264) 2024-09-12T03:12:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.634+0000 7f698ad9d640 1 -- 172.21.15.6:0/4096283220 learned_addr learned my addr 172.21.15.6:0/4096283220 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.634+0000 7f698ad9d640 1 -- 172.21.15.6:0/4096283220 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c07e090 con 0x7f698c1079c0 2024-09-12T03:12:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.634+0000 7f698ad9d640 1 --2- 172.21.15.6:0/4096283220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698c1079c0 0x7f698c07db50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6984030560 tx=0x7f6984032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.634+0000 7f6983fff640 1 -- 172.21.15.6:0/4096283220 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6984037040 con 0x7f698c1079c0 2024-09-12T03:12:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.634+0000 7f6983fff640 1 -- 172.21.15.6:0/4096283220 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6984040070 con 0x7f698c1079c0 2024-09-12T03:12:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.634+0000 7f699110b640 1 -- 172.21.15.6:0/4096283220 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698c07c280 con 0x7f698c1079c0 2024-09-12T03:12:11.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.634+0000 7f6983fff640 1 -- 172.21.15.6:0/4096283220 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f698403a740 con 0x7f698c1079c0 2024-09-12T03:12:11.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.634+0000 7f699110b640 1 -- 172.21.15.6:0/4096283220 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f698c07c7c0 con 0x7f698c1079c0 2024-09-12T03:12:11.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.635+0000 7f6983fff640 1 -- 172.21.15.6:0/4096283220 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f698403a9b0 con 0x7f698c1079c0 2024-09-12T03:12:11.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.635+0000 7f699110b640 1 -- 172.21.15.6:0/4096283220 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f698c10c050 con 0x7f698c1079c0 2024-09-12T03:12:11.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.635+0000 7f6983fff640 1 --2- 172.21.15.6:0/4096283220 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f6978038f70 0x7f697803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:11.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.635+0000 7f6983fff640 1 -- 172.21.15.6:0/4096283220 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f698406c5b0 con 0x7f698c1079c0 2024-09-12T03:12:11.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.636+0000 7f698a59c640 1 --2- 172.21.15.6:0/4096283220 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f6978038f70 0x7f697803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:11.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.636+0000 7f698a59c640 1 --2- 172.21.15.6:0/4096283220 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f6978038f70 0x7f697803b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f69740098c0 tx=0x7f6974006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:11.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.640+0000 7f6983fff640 1 -- 172.21.15.6:0/4096283220 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6984041270 con 0x7f698c1079c0 2024-09-12T03:12:11.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.824+0000 7f699110b640 1 -- 172.21.15.6:0/4096283220 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f698c107e40 con 0x7f698c1079c0 2024-09-12T03:12:11.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.829+0000 7f6983fff640 1 -- 172.21.15.6:0/4096283220 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f69840328f0 con 0x7f698c1079c0 2024-09-12T03:12:11.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.832+0000 7f699110b640 1 -- 172.21.15.6:0/4096283220 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f6978038f70 msgr2=0x7f697803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:11.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.832+0000 7f699110b640 1 --2- 172.21.15.6:0/4096283220 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f6978038f70 0x7f697803b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f69740098c0 tx=0x7f6974006db0 comp rx=0 tx=0).stop 2024-09-12T03:12:11.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.832+0000 7f699110b640 1 -- 172.21.15.6:0/4096283220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698c1079c0 msgr2=0x7f698c07db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:11.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.832+0000 7f699110b640 1 --2- 172.21.15.6:0/4096283220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698c1079c0 0x7f698c07db50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6984030560 tx=0x7f6984032670 comp rx=0 tx=0).stop 2024-09-12T03:12:11.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.832+0000 7f699110b640 1 -- 172.21.15.6:0/4096283220 shutdown_connections 2024-09-12T03:12:11.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.832+0000 7f699110b640 1 --2- 172.21.15.6:0/4096283220 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f6978038f70 0x7f697803b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:11.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.832+0000 7f699110b640 1 --2- 172.21.15.6:0/4096283220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698c1079c0 0x7f698c07db50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:11.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.832+0000 7f699110b640 1 -- 172.21.15.6:0/4096283220 >> 172.21.15.6:0/4096283220 conn(0x7f698c1031b0 msgr2=0x7f698c103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:11.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.832+0000 7f699110b640 1 -- 172.21.15.6:0/4096283220 shutdown_connections 2024-09-12T03:12:11.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:10.832+0000 7f699110b640 1 -- 172.21.15.6:0/4096283220 wait complete. 2024-09-12T03:12:11.826 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:11 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' 2024-09-12T03:12:11.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:11 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/4096283220' entity='client.admin' 2024-09-12T03:12:13.047 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.845+0000 7f7c554e5640 1 Processor -- start 2024-09-12T03:12:13.047 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.845+0000 7f7c554e5640 1 -- start start 2024-09-12T03:12:13.047 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.845+0000 7f7c554e5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c50107960 0x7f7c50107d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:13.047 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.845+0000 7f7c554e5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7c50108330 con 0x7f7c50107960 2024-09-12T03:12:13.047 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.845+0000 7f7c4effd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c50107960 0x7f7c50107d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:13.048 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.845+0000 7f7c4effd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c50107960 0x7f7c50107d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54270/0 (socket says 172.21.15.6:54270) 2024-09-12T03:12:13.048 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.845+0000 7f7c4effd640 1 -- 172.21.15.6:0/1087672024 learned_addr learned my addr 172.21.15.6:0/1087672024 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:13.048 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.846+0000 7f7c4effd640 1 -- 172.21.15.6:0/1087672024 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c50108b10 con 0x7f7c50107960 2024-09-12T03:12:13.048 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.846+0000 7f7c4effd640 1 --2- 172.21.15.6:0/1087672024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c50107960 0x7f7c50107d60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009a30 tx=0x7f7c4002a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd57833f23abda1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:13.048 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.847+0000 7f7c4dffb640 1 -- 172.21.15.6:0/1087672024 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c40036070 con 0x7f7c50107960 2024-09-12T03:12:13.048 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.847+0000 7f7c4dffb640 1 -- 172.21.15.6:0/1087672024 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c4002fb20 con 0x7f7c50107960 2024-09-12T03:12:13.048 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.847+0000 7f7c4dffb640 1 -- 172.21.15.6:0/1087672024 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c40031cb0 con 0x7f7c50107960 2024-09-12T03:12:13.048 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.847+0000 7f7c554e5640 1 -- 172.21.15.6:0/1087672024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c50107960 msgr2=0x7f7c50107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:13.049 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.847+0000 7f7c554e5640 1 --2- 172.21.15.6:0/1087672024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c50107960 0x7f7c50107d60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009a30 tx=0x7f7c4002a470 comp rx=0 tx=0).stop 2024-09-12T03:12:13.049 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.847+0000 7f7c554e5640 1 -- 172.21.15.6:0/1087672024 shutdown_connections 2024-09-12T03:12:13.049 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.847+0000 7f7c554e5640 1 --2- 172.21.15.6:0/1087672024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c50107960 0x7f7c50107d60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.049 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.847+0000 7f7c554e5640 1 -- 172.21.15.6:0/1087672024 >> 172.21.15.6:0/1087672024 conn(0x7f7c50103150 msgr2=0x7f7c50105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:13.049 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.848+0000 7f7c554e5640 1 -- 172.21.15.6:0/1087672024 shutdown_connections 2024-09-12T03:12:13.049 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.848+0000 7f7c554e5640 1 -- 172.21.15.6:0/1087672024 wait complete. 2024-09-12T03:12:13.049 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.848+0000 7f7c554e5640 1 Processor -- start 2024-09-12T03:12:13.049 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.848+0000 7f7c554e5640 1 -- start start 2024-09-12T03:12:13.049 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.848+0000 7f7c554e5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c50107960 0x7f7c5019b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:13.049 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.849+0000 7f7c554e5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5010bec0 con 0x7f7c50107960 2024-09-12T03:12:13.050 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.849+0000 7f7c4effd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c50107960 0x7f7c5019b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:13.050 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.849+0000 7f7c4effd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c50107960 0x7f7c5019b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54272/0 (socket says 172.21.15.6:54272) 2024-09-12T03:12:13.050 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.849+0000 7f7c4effd640 1 -- 172.21.15.6:0/3919985244 learned_addr learned my addr 172.21.15.6:0/3919985244 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:13.050 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.849+0000 7f7c4effd640 1 -- 172.21.15.6:0/3919985244 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5019be40 con 0x7f7c50107960 2024-09-12T03:12:13.050 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.849+0000 7f7c4effd640 1 --2- 172.21.15.6:0/3919985244 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c50107960 0x7f7c5019b900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009b50 tx=0x7f7c4002f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:13.050 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.849+0000 7f7c37fff640 1 -- 172.21.15.6:0/3919985244 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c40040020 con 0x7f7c50107960 2024-09-12T03:12:13.050 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.849+0000 7f7c37fff640 1 -- 172.21.15.6:0/3919985244 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c4003c900 con 0x7f7c50107960 2024-09-12T03:12:13.050 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.849+0000 7f7c554e5640 1 -- 172.21.15.6:0/3919985244 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5019c0d0 con 0x7f7c50107960 2024-09-12T03:12:13.050 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.849+0000 7f7c37fff640 1 -- 172.21.15.6:0/3919985244 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c40036040 con 0x7f7c50107960 2024-09-12T03:12:13.050 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.850+0000 7f7c554e5640 1 -- 172.21.15.6:0/3919985244 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c5019edc0 con 0x7f7c50107960 2024-09-12T03:12:13.050 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.851+0000 7f7c37fff640 1 -- 172.21.15.6:0/3919985244 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7c40043020 con 0x7f7c50107960 2024-09-12T03:12:13.050 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.851+0000 7f7c554e5640 1 -- 172.21.15.6:0/3919985244 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c5010c040 con 0x7f7c50107960 2024-09-12T03:12:13.051 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.851+0000 7f7c37fff640 1 --2- 172.21.15.6:0/3919985244 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f7c38038f70 0x7f7c3803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:13.051 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.851+0000 7f7c37fff640 1 -- 172.21.15.6:0/3919985244 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7c4006cf40 con 0x7f7c50107960 2024-09-12T03:12:13.051 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.851+0000 7f7c4e7fc640 1 --2- 172.21.15.6:0/3919985244 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f7c38038f70 0x7f7c3803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:13.051 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.851+0000 7f7c4e7fc640 1 --2- 172.21.15.6:0/3919985244 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f7c38038f70 0x7f7c3803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7c440098c0 tx=0x7f7c44006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:13.051 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:11.856+0000 7f7c37fff640 1 -- 172.21.15.6:0/3919985244 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7c4004a020 con 0x7f7c50107960 2024-09-12T03:12:13.051 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:12.039+0000 7f7c554e5640 1 -- 172.21.15.6:0/3919985244 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f7c50107de0 con 0x7f7c50107960 2024-09-12T03:12:13.051 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:12.042+0000 7f7c37fff640 1 -- 172.21.15.6:0/3919985244 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f7c400361e0 con 0x7f7c50107960 2024-09-12T03:12:13.051 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:12.045+0000 7f7c554e5640 1 -- 172.21.15.6:0/3919985244 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f7c38038f70 msgr2=0x7f7c3803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:13.051 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:12.045+0000 7f7c554e5640 1 --2- 172.21.15.6:0/3919985244 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f7c38038f70 0x7f7c3803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7c440098c0 tx=0x7f7c44006db0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.051 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:12.045+0000 7f7c554e5640 1 -- 172.21.15.6:0/3919985244 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c50107960 msgr2=0x7f7c5019b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:13.051 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:12.045+0000 7f7c554e5640 1 --2- 172.21.15.6:0/3919985244 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c50107960 0x7f7c5019b900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009b50 tx=0x7f7c4002f740 comp rx=0 tx=0).stop 2024-09-12T03:12:13.051 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:12.045+0000 7f7c554e5640 1 -- 172.21.15.6:0/3919985244 shutdown_connections 2024-09-12T03:12:13.051 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:12.045+0000 7f7c554e5640 1 --2- 172.21.15.6:0/3919985244 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f7c38038f70 0x7f7c3803b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.052 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:12.045+0000 7f7c554e5640 1 --2- 172.21.15.6:0/3919985244 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c50107960 0x7f7c5019b900 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.052 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:12.045+0000 7f7c554e5640 1 -- 172.21.15.6:0/3919985244 >> 172.21.15.6:0/3919985244 conn(0x7f7c50103150 msgr2=0x7f7c50103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:13.052 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:12.045+0000 7f7c554e5640 1 -- 172.21.15.6:0/3919985244 shutdown_connections 2024-09-12T03:12:13.052 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:12.045+0000 7f7c554e5640 1 -- 172.21.15.6:0/3919985244 wait complete. 2024-09-12T03:12:13.052 INFO:teuthology.orchestra.run.smithi006.stdout:Enabling the dashboard module... 2024-09-12T03:12:13.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:13 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3919985244' entity='client.admin' 2024-09-12T03:12:13.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:13 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' 2024-09-12T03:12:14.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:14 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2861641330' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-12T03:12:14.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:14 smithi006 ceph-mon[30991]: from='mgr.14118 172.21.15.6:0/1513060919' entity='mgr.x' 2024-09-12T03:12:14.362 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:14 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: ignoring --setuser ceph since I am not root 2024-09-12T03:12:14.362 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:14 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: ignoring --setgroup ceph since I am not root 2024-09-12T03:12:14.362 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:14 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:14.209+0000 7f8b394f7040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:12:14.362 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:14 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:14.268+0000 7f8b394f7040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:12:14.624 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.425+0000 7f1229b26640 1 Processor -- start 2024-09-12T03:12:14.625 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.425+0000 7f1229b26640 1 -- start start 2024-09-12T03:12:14.625 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.425+0000 7f1229b26640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f122407ddc0 0x7f122407c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:14.625 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.425+0000 7f1229b26640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f122407c800 con 0x7f122407ddc0 2024-09-12T03:12:14.625 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.426+0000 7f12237fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f122407ddc0 0x7f122407c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:14.625 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.426+0000 7f12237fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f122407ddc0 0x7f122407c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54276/0 (socket says 172.21.15.6:54276) 2024-09-12T03:12:14.625 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.426+0000 7f12237fe640 1 -- 172.21.15.6:0/2342026727 learned_addr learned my addr 172.21.15.6:0/2342026727 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:14.625 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.426+0000 7f12237fe640 1 -- 172.21.15.6:0/2342026727 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122407c980 con 0x7f122407ddc0 2024-09-12T03:12:14.625 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.427+0000 7f12237fe640 1 --2- 172.21.15.6:0/2342026727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f122407ddc0 0x7f122407c2c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1214009820 tx=0x7f121402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4728f830890ef46 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:14.625 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.427+0000 7f12227fc640 1 -- 172.21.15.6:0/2342026727 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1214037070 con 0x7f122407ddc0 2024-09-12T03:12:14.625 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.427+0000 7f12227fc640 1 -- 172.21.15.6:0/2342026727 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1214032920 con 0x7f122407ddc0 2024-09-12T03:12:14.625 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.427+0000 7f12227fc640 1 -- 172.21.15.6:0/2342026727 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1214030810 con 0x7f122407ddc0 2024-09-12T03:12:14.626 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.427+0000 7f1229b26640 1 -- 172.21.15.6:0/2342026727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f122407ddc0 msgr2=0x7f122407c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:14.626 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.427+0000 7f1229b26640 1 --2- 172.21.15.6:0/2342026727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f122407ddc0 0x7f122407c2c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1214009820 tx=0x7f121402a270 comp rx=0 tx=0).stop 2024-09-12T03:12:14.626 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.427+0000 7f1229b26640 1 -- 172.21.15.6:0/2342026727 shutdown_connections 2024-09-12T03:12:14.626 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.427+0000 7f1229b26640 1 --2- 172.21.15.6:0/2342026727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f122407ddc0 0x7f122407c2c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.626 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.427+0000 7f1229b26640 1 -- 172.21.15.6:0/2342026727 >> 172.21.15.6:0/2342026727 conn(0x7f12241031b0 msgr2=0x7f12241055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:14.626 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.427+0000 7f1229b26640 1 -- 172.21.15.6:0/2342026727 shutdown_connections 2024-09-12T03:12:14.626 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.427+0000 7f1229b26640 1 -- 172.21.15.6:0/2342026727 wait complete. 2024-09-12T03:12:14.626 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.428+0000 7f1229b26640 1 Processor -- start 2024-09-12T03:12:14.626 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.428+0000 7f1229b26640 1 -- start start 2024-09-12T03:12:14.626 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.428+0000 7f1229b26640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f122407ddc0 0x7f122419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:14.626 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.428+0000 7f1229b26640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f122410df90 con 0x7f122407ddc0 2024-09-12T03:12:14.626 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.429+0000 7f12237fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f122407ddc0 0x7f122419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:14.626 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.429+0000 7f12237fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f122407ddc0 0x7f122419dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54284/0 (socket says 172.21.15.6:54284) 2024-09-12T03:12:14.627 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.429+0000 7f12237fe640 1 -- 172.21.15.6:0/2861641330 learned_addr learned my addr 172.21.15.6:0/2861641330 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:14.627 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.429+0000 7f12237fe640 1 -- 172.21.15.6:0/2861641330 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122419e020 con 0x7f122407ddc0 2024-09-12T03:12:14.627 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.429+0000 7f12237fe640 1 --2- 172.21.15.6:0/2861641330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f122407ddc0 0x7f122419dae0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f12140097f0 tx=0x7f12140327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:14.627 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.429+0000 7f1220ff9640 1 -- 172.21.15.6:0/2861641330 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1214037040 con 0x7f122407ddc0 2024-09-12T03:12:14.627 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.429+0000 7f1220ff9640 1 -- 172.21.15.6:0/2861641330 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1214030cc0 con 0x7f122407ddc0 2024-09-12T03:12:14.627 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.429+0000 7f1229b26640 1 -- 172.21.15.6:0/2861641330 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122419e2b0 con 0x7f122407ddc0 2024-09-12T03:12:14.627 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.429+0000 7f1220ff9640 1 -- 172.21.15.6:0/2861641330 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f121403a6b0 con 0x7f122407ddc0 2024-09-12T03:12:14.627 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.429+0000 7f1229b26640 1 -- 172.21.15.6:0/2861641330 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12241a0fa0 con 0x7f122407ddc0 2024-09-12T03:12:14.627 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.430+0000 7f1220ff9640 1 -- 172.21.15.6:0/2861641330 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f121403a850 con 0x7f122407ddc0 2024-09-12T03:12:14.627 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.430+0000 7f1220ff9640 1 --2- 172.21.15.6:0/2861641330 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f120c038fc0 0x7f120c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:14.627 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.430+0000 7f1220ff9640 1 -- 172.21.15.6:0/2861641330 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f121406c7e0 con 0x7f122407ddc0 2024-09-12T03:12:14.627 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.430+0000 7f1229b26640 1 -- 172.21.15.6:0/2861641330 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f122407c340 con 0x7f122407ddc0 2024-09-12T03:12:14.628 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.430+0000 7f1222ffd640 1 --2- 172.21.15.6:0/2861641330 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f120c038fc0 0x7f120c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:14.628 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.431+0000 7f1222ffd640 1 --2- 172.21.15.6:0/2861641330 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f120c038fc0 0x7f120c03b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f12180098c0 tx=0x7f1218006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:14.628 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.435+0000 7f1220ff9640 1 -- 172.21.15.6:0/2861641330 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f12140371e0 con 0x7f122407ddc0 2024-09-12T03:12:14.628 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:13.662+0000 7f1229b26640 1 -- 172.21.15.6:0/2861641330 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f12241a14d0 con 0x7f122407ddc0 2024-09-12T03:12:14.628 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:14.047+0000 7f1220ff9640 1 -- 172.21.15.6:0/2861641330 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f121402cb80 con 0x7f122407ddc0 2024-09-12T03:12:14.628 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:14.048+0000 7f1220ff9640 1 -- 172.21.15.6:0/2861641330 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f121403baf0 con 0x7f122407ddc0 2024-09-12T03:12:14.628 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:14.050+0000 7f1229b26640 1 -- 172.21.15.6:0/2861641330 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f120c038fc0 msgr2=0x7f120c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:14.628 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:14.050+0000 7f1229b26640 1 --2- 172.21.15.6:0/2861641330 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f120c038fc0 0x7f120c03b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f12180098c0 tx=0x7f1218006db0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.628 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:14.050+0000 7f1229b26640 1 -- 172.21.15.6:0/2861641330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f122407ddc0 msgr2=0x7f122419dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:14.628 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:14.050+0000 7f1229b26640 1 --2- 172.21.15.6:0/2861641330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f122407ddc0 0x7f122419dae0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f12140097f0 tx=0x7f12140327b0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.628 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:14.050+0000 7f1229b26640 1 -- 172.21.15.6:0/2861641330 shutdown_connections 2024-09-12T03:12:14.628 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:14.050+0000 7f1229b26640 1 --2- 172.21.15.6:0/2861641330 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f120c038fc0 0x7f120c03b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.628 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:14.050+0000 7f1229b26640 1 --2- 172.21.15.6:0/2861641330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f122407ddc0 0x7f122419dae0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.629 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:14.050+0000 7f1229b26640 1 -- 172.21.15.6:0/2861641330 >> 172.21.15.6:0/2861641330 conn(0x7f12241031b0 msgr2=0x7f1224103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:14.629 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:14.050+0000 7f1229b26640 1 -- 172.21.15.6:0/2861641330 shutdown_connections 2024-09-12T03:12:14.629 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:14.050+0000 7f1229b26640 1 -- 172.21.15.6:0/2861641330 wait complete. 2024-09-12T03:12:14.651 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:14 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:14.424+0000 7f8b394f7040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:12:14.652 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:14 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:14.494+0000 7f8b394f7040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:12:14.652 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:14 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:14.651+0000 7f8b394f7040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:12:14.905 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:14 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:14.800+0000 7f8b394f7040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:12:15.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:15 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2861641330' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-12T03:12:15.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:15 smithi006 ceph-mon[30991]: mgrmap e9: x(active, since 18s) 2024-09-12T03:12:15.213 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:15 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:15.148+0000 7f8b394f7040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:12:15.214 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:15 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:15.213+0000 7f8b394f7040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:12:15.611 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:15 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:15.415+0000 7f8b394f7040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:12:16.236 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:16 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3173709644' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T03:12:16.243 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:12:16.243 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-12T03:12:16.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T03:12:16.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T03:12:16.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T03:12:16.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:12:16.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.058+0000 7f39f9254640 1 Processor -- start 2024-09-12T03:12:16.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.058+0000 7f39f9254640 1 -- start start 2024-09-12T03:12:16.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.058+0000 7f39f9254640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f4108400 0x7f39f410a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:16.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.058+0000 7f39f9254640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f39f407be40 con 0x7f39f4108400 2024-09-12T03:12:16.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.059+0000 7f39f2d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f4108400 0x7f39f410a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:16.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.059+0000 7f39f2d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f4108400 0x7f39f410a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54306/0 (socket says 172.21.15.6:54306) 2024-09-12T03:12:16.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.059+0000 7f39f2d9d640 1 -- 172.21.15.6:0/3261275967 learned_addr learned my addr 172.21.15.6:0/3261275967 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:16.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.059+0000 7f39f2d9d640 1 -- 172.21.15.6:0/3261275967 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f410ad30 con 0x7f39f4108400 2024-09-12T03:12:16.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.060+0000 7f39f2d9d640 1 --2- 172.21.15.6:0/3261275967 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f4108400 0x7f39f410a7f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009820 tx=0x7f39e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbceb742918c16cc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:16.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.060+0000 7f39f259c640 1 -- 172.21.15.6:0/3261275967 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39e4037070 con 0x7f39f4108400 2024-09-12T03:12:16.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.060+0000 7f39f259c640 1 -- 172.21.15.6:0/3261275967 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e4032920 con 0x7f39f4108400 2024-09-12T03:12:16.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.060+0000 7f39f259c640 1 -- 172.21.15.6:0/3261275967 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39e4030810 con 0x7f39f4108400 2024-09-12T03:12:16.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.061+0000 7f39f9254640 1 -- 172.21.15.6:0/3261275967 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f4108400 msgr2=0x7f39f410a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:16.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.061+0000 7f39f9254640 1 --2- 172.21.15.6:0/3261275967 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f4108400 0x7f39f410a7f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009820 tx=0x7f39e402a270 comp rx=0 tx=0).stop 2024-09-12T03:12:16.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.061+0000 7f39f9254640 1 -- 172.21.15.6:0/3261275967 shutdown_connections 2024-09-12T03:12:16.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.061+0000 7f39f9254640 1 --2- 172.21.15.6:0/3261275967 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f4108400 0x7f39f410a7f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:16.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.061+0000 7f39f9254640 1 -- 172.21.15.6:0/3261275967 >> 172.21.15.6:0/3261275967 conn(0x7f39f4102000 msgr2=0x7f39f4104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:16.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.061+0000 7f39f9254640 1 -- 172.21.15.6:0/3261275967 shutdown_connections 2024-09-12T03:12:16.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.061+0000 7f39f9254640 1 -- 172.21.15.6:0/3261275967 wait complete. 2024-09-12T03:12:16.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.062+0000 7f39f9254640 1 Processor -- start 2024-09-12T03:12:16.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.062+0000 7f39f9254640 1 -- start start 2024-09-12T03:12:16.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.062+0000 7f39f9254640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f4108400 0x7f39f419b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:16.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.062+0000 7f39f9254640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f39f410bdb0 con 0x7f39f4108400 2024-09-12T03:12:16.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.062+0000 7f39f2d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f4108400 0x7f39f419b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:16.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.062+0000 7f39f2d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f4108400 0x7f39f419b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54308/0 (socket says 172.21.15.6:54308) 2024-09-12T03:12:16.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.062+0000 7f39f2d9d640 1 -- 172.21.15.6:0/3173709644 learned_addr learned my addr 172.21.15.6:0/3173709644 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:16.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.063+0000 7f39f2d9d640 1 -- 172.21.15.6:0/3173709644 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f419bea0 con 0x7f39f4108400 2024-09-12T03:12:16.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.063+0000 7f39f2d9d640 1 --2- 172.21.15.6:0/3173709644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f4108400 0x7f39f419b960 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f39e4030420 tx=0x7f39e4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:16.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.063+0000 7f39f0d99640 1 -- 172.21.15.6:0/3173709644 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39e4037040 con 0x7f39f4108400 2024-09-12T03:12:16.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.063+0000 7f39f0d99640 1 -- 172.21.15.6:0/3173709644 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e4040070 con 0x7f39f4108400 2024-09-12T03:12:16.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.063+0000 7f39f9254640 1 -- 172.21.15.6:0/3173709644 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f419c130 con 0x7f39f4108400 2024-09-12T03:12:16.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.063+0000 7f39f0d99640 1 -- 172.21.15.6:0/3173709644 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39e403a6e0 con 0x7f39f4108400 2024-09-12T03:12:16.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.063+0000 7f39f9254640 1 -- 172.21.15.6:0/3173709644 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f419ee20 con 0x7f39f4108400 2024-09-12T03:12:16.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.064+0000 7f39f0d99640 1 -- 172.21.15.6:0/3173709644 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f39e403a9e0 con 0x7f39f4108400 2024-09-12T03:12:16.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.064+0000 7f39f9254640 1 -- 172.21.15.6:0/3173709644 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f4108860 con 0x7f39f4108400 2024-09-12T03:12:16.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.064+0000 7f39f0d99640 1 --2- 172.21.15.6:0/3173709644 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f39e0038fc0 0x7f39e003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:16.247 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.065+0000 7f39f0d99640 1 -- 172.21.15.6:0/3173709644 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f39e406c720 con 0x7f39f4108400 2024-09-12T03:12:16.247 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.065+0000 7f39ebfff640 1 -- 172.21.15.6:0/3173709644 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f39e0038fc0 msgr2=0x7f39e003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/1684477653 2024-09-12T03:12:16.247 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.065+0000 7f39ebfff640 1 --2- 172.21.15.6:0/3173709644 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f39e0038fc0 0x7f39e003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:12:16.247 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.069+0000 7f39f0d99640 1 -- 172.21.15.6:0/3173709644 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39e4042e10 con 0x7f39f4108400 2024-09-12T03:12:16.247 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.266+0000 7f39ebfff640 1 -- 172.21.15.6:0/3173709644 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f39e0038fc0 msgr2=0x7f39e003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/1684477653 2024-09-12T03:12:16.247 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.266+0000 7f39ebfff640 1 --2- 172.21.15.6:0/3173709644 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f39e0038fc0 0x7f39e003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T03:12:16.247 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.301+0000 7f39f9254640 1 -- 172.21.15.6:0/3173709644 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f39f419f410 con 0x7f39f4108400 2024-09-12T03:12:16.247 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.301+0000 7f39f0d99640 1 -- 172.21.15.6:0/3173709644 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f39e406c070 con 0x7f39f4108400 2024-09-12T03:12:16.247 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.304+0000 7f39f9254640 1 -- 172.21.15.6:0/3173709644 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f39e0038fc0 msgr2=0x7f39e003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:12:16.247 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.304+0000 7f39f9254640 1 --2- 172.21.15.6:0/3173709644 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f39e0038fc0 0x7f39e003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:16.247 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.304+0000 7f39f9254640 1 -- 172.21.15.6:0/3173709644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f4108400 msgr2=0x7f39f419b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:16.247 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.304+0000 7f39f9254640 1 --2- 172.21.15.6:0/3173709644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f4108400 0x7f39f419b960 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f39e4030420 tx=0x7f39e4032670 comp rx=0 tx=0).stop 2024-09-12T03:12:16.248 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.304+0000 7f39f9254640 1 -- 172.21.15.6:0/3173709644 shutdown_connections 2024-09-12T03:12:16.248 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.304+0000 7f39f9254640 1 --2- 172.21.15.6:0/3173709644 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f39e0038fc0 0x7f39e003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:16.248 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.304+0000 7f39f9254640 1 --2- 172.21.15.6:0/3173709644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f4108400 0x7f39f419b960 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:16.248 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.304+0000 7f39f9254640 1 -- 172.21.15.6:0/3173709644 >> 172.21.15.6:0/3173709644 conn(0x7f39f4102000 msgr2=0x7f39f4102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:16.248 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.304+0000 7f39f9254640 1 -- 172.21.15.6:0/3173709644 shutdown_connections 2024-09-12T03:12:16.248 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:15.304+0000 7f39f9254640 1 -- 172.21.15.6:0/3173709644 wait complete. 2024-09-12T03:12:16.248 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for the mgr to restart... 2024-09-12T03:12:16.248 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr epoch 9... 2024-09-12T03:12:16.580 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:16 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:16.351+0000 7f8b394f7040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:12:16.581 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:16 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:16.422+0000 7f8b394f7040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:12:17.457 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:17 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:17.184+0000 7f8b394f7040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:12:17.457 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:17 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:17.307+0000 7f8b394f7040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:12:17.748 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:17 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:17.457+0000 7f8b394f7040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:12:17.748 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:17 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:17.520+0000 7f8b394f7040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:12:17.748 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:17 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:17.677+0000 7f8b394f7040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:12:17.749 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:17 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:17.748+0000 7f8b394f7040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:12:18.045 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:17 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:17.811+0000 7f8b394f7040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:12:18.233 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:17 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:17.976+0000 7f8b394f7040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:12:18.233 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:18 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:18.044+0000 7f8b394f7040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:12:18.325 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:18 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:18.114+0000 7f8b394f7040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:12:18.325 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:18 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:18.200+0000 7f8b394f7040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:12:18.325 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:18 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:18.262+0000 7f8b394f7040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:12:18.325 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:18 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:18.324+0000 7f8b394f7040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:12:19.418 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:19 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:19.153+0000 7f8b394f7040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:12:19.418 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:19 smithi006 ceph-mon[30991]: Active manager daemon x restarted 2024-09-12T03:12:19.418 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:19 smithi006 ceph-mon[30991]: Activating manager daemon x 2024-09-12T03:12:19.418 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:19 smithi006 ceph-mon[30991]: osdmap e3: 0 total, 0 up, 0 in 2024-09-12T03:12:19.419 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:19 smithi006 ceph-mon[30991]: mgrmap e10: x(active, starting, since 0.00344261s) 2024-09-12T03:12:19.419 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:12:19.419 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T03:12:19.419 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T03:12:19.419 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T03:12:19.419 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T03:12:19.419 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:19 smithi006 ceph-mon[30991]: Manager daemon x is now available 2024-09-12T03:12:19.419 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:19.419 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:12:19.419 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:12:21.123 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:12:21.124 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-12T03:12:21.124 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T03:12:21.124 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:12:21.124 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.661+0000 7f676a102640 1 Processor -- start 2024-09-12T03:12:21.124 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.661+0000 7f676a102640 1 -- start start 2024-09-12T03:12:21.124 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.661+0000 7f676a102640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67641084c0 0x7f676410a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.124 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.661+0000 7f676a102640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f676407bd80 con 0x7f67641084c0 2024-09-12T03:12:21.124 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.661+0000 7f67637fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67641084c0 0x7f676410a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:21.124 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.661+0000 7f67637fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67641084c0 0x7f676410a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54314/0 (socket says 172.21.15.6:54314) 2024-09-12T03:12:21.124 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.661+0000 7f67637fe640 1 -- 172.21.15.6:0/3524147652 learned_addr learned my addr 172.21.15.6:0/3524147652 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:21.125 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.661+0000 7f67637fe640 1 -- 172.21.15.6:0/3524147652 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676410adf0 con 0x7f67641084c0 2024-09-12T03:12:21.125 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.662+0000 7f67637fe640 1 --2- 172.21.15.6:0/3524147652 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67641084c0 0x7f676410a8b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6754009820 tx=0x7f675402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13d071ad64a5655 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:21.125 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.662+0000 7f6762ffd640 1 -- 172.21.15.6:0/3524147652 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6754037070 con 0x7f67641084c0 2024-09-12T03:12:21.125 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.662+0000 7f6762ffd640 1 -- 172.21.15.6:0/3524147652 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6754032920 con 0x7f67641084c0 2024-09-12T03:12:21.125 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.662+0000 7f6762ffd640 1 -- 172.21.15.6:0/3524147652 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6754030810 con 0x7f67641084c0 2024-09-12T03:12:21.125 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.663+0000 7f676a102640 1 -- 172.21.15.6:0/3524147652 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67641084c0 msgr2=0x7f676410a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:21.125 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.663+0000 7f676a102640 1 --2- 172.21.15.6:0/3524147652 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67641084c0 0x7f676410a8b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6754009820 tx=0x7f675402a270 comp rx=0 tx=0).stop 2024-09-12T03:12:21.125 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.663+0000 7f676a102640 1 -- 172.21.15.6:0/3524147652 shutdown_connections 2024-09-12T03:12:21.125 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.663+0000 7f676a102640 1 --2- 172.21.15.6:0/3524147652 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67641084c0 0x7f676410a8b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.125 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.663+0000 7f676a102640 1 -- 172.21.15.6:0/3524147652 >> 172.21.15.6:0/3524147652 conn(0x7f6764102000 msgr2=0x7f6764104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:21.125 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.663+0000 7f676a102640 1 -- 172.21.15.6:0/3524147652 shutdown_connections 2024-09-12T03:12:21.125 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.663+0000 7f676a102640 1 -- 172.21.15.6:0/3524147652 wait complete. 2024-09-12T03:12:21.125 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.664+0000 7f676a102640 1 Processor -- start 2024-09-12T03:12:21.126 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.664+0000 7f676a102640 1 -- start start 2024-09-12T03:12:21.126 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.664+0000 7f676a102640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67641084c0 0x7f676419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.126 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.664+0000 7f676a102640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f676410bdb0 con 0x7f67641084c0 2024-09-12T03:12:21.126 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.664+0000 7f67637fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67641084c0 0x7f676419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:21.126 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.664+0000 7f67637fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67641084c0 0x7f676419b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54328/0 (socket says 172.21.15.6:54328) 2024-09-12T03:12:21.126 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.664+0000 7f67637fe640 1 -- 172.21.15.6:0/2274726464 learned_addr learned my addr 172.21.15.6:0/2274726464 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:21.126 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.665+0000 7f67637fe640 1 -- 172.21.15.6:0/2274726464 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676419bed0 con 0x7f67641084c0 2024-09-12T03:12:21.126 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.665+0000 7f67637fe640 1 --2- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67641084c0 0x7f676419b990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f67540097f0 tx=0x7f67540327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:21.126 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.665+0000 7f67617fa640 1 -- 172.21.15.6:0/2274726464 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6754037040 con 0x7f67641084c0 2024-09-12T03:12:21.126 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.665+0000 7f67617fa640 1 -- 172.21.15.6:0/2274726464 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6754030c40 con 0x7f67641084c0 2024-09-12T03:12:21.126 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.665+0000 7f676a102640 1 -- 172.21.15.6:0/2274726464 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676419c160 con 0x7f67641084c0 2024-09-12T03:12:21.126 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.665+0000 7f67617fa640 1 -- 172.21.15.6:0/2274726464 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f675403a5d0 con 0x7f67641084c0 2024-09-12T03:12:21.127 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.665+0000 7f676a102640 1 -- 172.21.15.6:0/2274726464 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f676419ee50 con 0x7f67641084c0 2024-09-12T03:12:21.127 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.666+0000 7f67617fa640 1 -- 172.21.15.6:0/2274726464 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f675403a810 con 0x7f67641084c0 2024-09-12T03:12:21.127 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.666+0000 7f67617fa640 1 --2- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f6750038fc0 0x7f675003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.127 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.666+0000 7f67617fa640 1 -- 172.21.15.6:0/2274726464 --> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f675003bb90 con 0x7f6750038fc0 2024-09-12T03:12:21.127 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.666+0000 7f67617fa640 1 -- 172.21.15.6:0/2274726464 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f675406c510 con 0x7f67641084c0 2024-09-12T03:12:21.127 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.666+0000 7f675affd640 1 -- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f6750038fc0 msgr2=0x7f675003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/1684477653 2024-09-12T03:12:21.127 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.666+0000 7f675affd640 1 --2- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f6750038fc0 0x7f675003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:12:21.127 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.867+0000 7f675affd640 1 -- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f6750038fc0 msgr2=0x7f675003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/1684477653 2024-09-12T03:12:21.127 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:16.867+0000 7f675affd640 1 --2- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f6750038fc0 0x7f675003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T03:12:21.127 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:17.268+0000 7f675affd640 1 -- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f6750038fc0 msgr2=0x7f675003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/1684477653 2024-09-12T03:12:21.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:17.268+0000 7f675affd640 1 --2- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f6750038fc0 0x7f675003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T03:12:21.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:18.069+0000 7f675affd640 1 -- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f6750038fc0 msgr2=0x7f675003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/1684477653 2024-09-12T03:12:21.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:18.069+0000 7f675affd640 1 --2- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f6750038fc0 0x7f675003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T03:12:21.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:19.156+0000 7f67617fa640 1 -- 172.21.15.6:0/2274726464 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f675406afc0 con 0x7f67641084c0 2024-09-12T03:12:21.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:19.156+0000 7f67617fa640 1 -- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f6750038fc0 msgr2=0x7f675003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:12:21.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:19.156+0000 7f67617fa640 1 --2- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f6750038fc0 0x7f675003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:20.158+0000 7f67617fa640 1 -- 172.21.15.6:0/2274726464 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f6754044920 con 0x7f67641084c0 2024-09-12T03:12:21.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:20.158+0000 7f67617fa640 1 --2- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f675003ca80 0x7f675003ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:20.158+0000 7f67617fa640 1 -- 172.21.15.6:0/2274726464 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6754004490 con 0x7f675003ca80 2024-09-12T03:12:21.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:20.158+0000 7f675affd640 1 --2- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f675003ca80 0x7f675003ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:21.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:20.159+0000 7f675affd640 1 --2- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f675003ca80 0x7f675003ee70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f674c003980 tx=0x7f674c006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:21.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:20.160+0000 7f67617fa640 1 -- 172.21.15.6:0/2274726464 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f6754004490 con 0x7f675003ca80 2024-09-12T03:12:21.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:20.167+0000 7f676a102640 1 -- 172.21.15.6:0/2274726464 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f6764108920 con 0x7f675003ca80 2024-09-12T03:12:21.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:20.167+0000 7f67617fa640 1 -- 172.21.15.6:0/2274726464 <== mgr.14150 v2:172.21.15.6:6800/5477767 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f6764108920 con 0x7f675003ca80 2024-09-12T03:12:21.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:20.167+0000 7f676a102640 1 -- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f675003ca80 msgr2=0x7f675003ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:21.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:20.167+0000 7f676a102640 1 --2- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f675003ca80 0x7f675003ee70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f674c003980 tx=0x7f674c006e10 comp rx=0 tx=0).stop 2024-09-12T03:12:21.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:20.167+0000 7f676a102640 1 -- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67641084c0 msgr2=0x7f676419b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:21.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:20.167+0000 7f676a102640 1 --2- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67641084c0 0x7f676419b990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f67540097f0 tx=0x7f67540327b0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:20.168+0000 7f676a102640 1 -- 172.21.15.6:0/2274726464 shutdown_connections 2024-09-12T03:12:21.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:20.168+0000 7f676a102640 1 --2- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f675003ca80 0x7f675003ee70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:20.168+0000 7f676a102640 1 --2- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:6800/1684477653,v1:172.21.15.6:6801/1684477653] conn(0x7f6750038fc0 0x7f675003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:20.168+0000 7f676a102640 1 --2- 172.21.15.6:0/2274726464 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67641084c0 0x7f676419b990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:20.168+0000 7f676a102640 1 -- 172.21.15.6:0/2274726464 >> 172.21.15.6:0/2274726464 conn(0x7f6764102000 msgr2=0x7f6764102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:21.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:20.168+0000 7f676a102640 1 -- 172.21.15.6:0/2274726464 shutdown_connections 2024-09-12T03:12:21.130 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:20.168+0000 7f676a102640 1 -- 172.21.15.6:0/2274726464 wait complete. 2024-09-12T03:12:21.130 INFO:teuthology.orchestra.run.smithi006.stdout:mgr epoch 9 is available 2024-09-12T03:12:21.130 INFO:teuthology.orchestra.run.smithi006.stdout:Generating a dashboard self-signed certificate... 2024-09-12T03:12:21.407 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-mon[30991]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T03:12:21.407 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-mon[30991]: mgrmap e11: x(active, since 1.00619s) 2024-09-12T03:12:21.407 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-mon[30991]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T03:12:21.407 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:21.733 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:21.406+0000 7f8aa47cb640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:03:12:21] ENGINE Error in HTTPServer.serve 2024-09-12T03:12:21.733 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: Traceback (most recent call last): 2024-09-12T03:12:21.733 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:12:21.733 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: self._connections.run(self.expiration_interval) 2024-09-12T03:12:21.733 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:12:21.733 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: self._run(expiration_interval) 2024-09-12T03:12:21.733 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:12:21.733 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:12:21.733 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:12:21.734 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:12:21.734 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:12:21.734 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: s = self.context.wrap_socket( 2024-09-12T03:12:21.734 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:12:21.734 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: return self.sslsocket_class._create( 2024-09-12T03:12:21.734 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:12:21.734 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: self.do_handshake() 2024-09-12T03:12:21.734 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:12:21.734 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: self._sslobj.do_handshake() 2024-09-12T03:12:21.734 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:12:21.734 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:12:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:12:22.759 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:22 smithi006 ceph-mon[30991]: [12/Sep/2024:03:12:21] ENGINE Bus STARTING 2024-09-12T03:12:22.759 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:22 smithi006 ceph-mon[30991]: [12/Sep/2024:03:12:21] ENGINE Serving on https://172.21.15.6:7150 2024-09-12T03:12:22.760 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:22 smithi006 ceph-mon[30991]: [12/Sep/2024:03:12:21] ENGINE Error in HTTPServer.serve 2024-09-12T03:12:22.760 INFO:journalctl@ceph.mon.a.smithi006.stdout: Traceback (most recent call last): 2024-09-12T03:12:22.760 INFO:journalctl@ceph.mon.a.smithi006.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:12:22.760 INFO:journalctl@ceph.mon.a.smithi006.stdout: self._connections.run(self.expiration_interval) 2024-09-12T03:12:22.760 INFO:journalctl@ceph.mon.a.smithi006.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:12:22.760 INFO:journalctl@ceph.mon.a.smithi006.stdout: self._run(expiration_interval) 2024-09-12T03:12:22.760 INFO:journalctl@ceph.mon.a.smithi006.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:12:22.760 INFO:journalctl@ceph.mon.a.smithi006.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:12:22.760 INFO:journalctl@ceph.mon.a.smithi006.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:12:22.760 INFO:journalctl@ceph.mon.a.smithi006.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:12:22.760 INFO:journalctl@ceph.mon.a.smithi006.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:12:22.760 INFO:journalctl@ceph.mon.a.smithi006.stdout: s = self.context.wrap_socket( 2024-09-12T03:12:22.761 INFO:journalctl@ceph.mon.a.smithi006.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:12:22.761 INFO:journalctl@ceph.mon.a.smithi006.stdout: return self.sslsocket_class._create( 2024-09-12T03:12:22.761 INFO:journalctl@ceph.mon.a.smithi006.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:12:22.761 INFO:journalctl@ceph.mon.a.smithi006.stdout: self.do_handshake() 2024-09-12T03:12:22.761 INFO:journalctl@ceph.mon.a.smithi006.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:12:22.761 INFO:journalctl@ceph.mon.a.smithi006.stdout: self._sslobj.do_handshake() 2024-09-12T03:12:22.761 INFO:journalctl@ceph.mon.a.smithi006.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:12:22.761 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:22 smithi006 ceph-mon[30991]: [12/Sep/2024:03:12:21] ENGINE Serving on http://172.21.15.6:8765 2024-09-12T03:12:22.761 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:22 smithi006 ceph-mon[30991]: [12/Sep/2024:03:12:21] ENGINE Bus STARTED 2024-09-12T03:12:22.761 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:22 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:22.761 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:22 smithi006 ceph-mon[30991]: mgrmap e12: x(active, since 2s) 2024-09-12T03:12:22.761 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:22 smithi006 ceph-mon[30991]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:22.761 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:22 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:22.762 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:22 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:23.319 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-12T03:12:23.319 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.601+0000 7ffa66943640 1 Processor -- start 2024-09-12T03:12:23.319 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.601+0000 7ffa66943640 1 -- start start 2024-09-12T03:12:23.319 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.601+0000 7ffa66943640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa60108400 0x7ffa6010a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:23.319 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.601+0000 7ffa66943640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6007be40 con 0x7ffa60108400 2024-09-12T03:12:23.319 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.601+0000 7ffa5ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa60108400 0x7ffa6010a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:23.320 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.601+0000 7ffa5ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa60108400 0x7ffa6010a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57796/0 (socket says 172.21.15.6:57796) 2024-09-12T03:12:23.320 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.601+0000 7ffa5ffff640 1 -- 172.21.15.6:0/4120381633 learned_addr learned my addr 172.21.15.6:0/4120381633 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:23.320 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.602+0000 7ffa5ffff640 1 -- 172.21.15.6:0/4120381633 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6010ad30 con 0x7ffa60108400 2024-09-12T03:12:23.320 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.602+0000 7ffa5ffff640 1 --2- 172.21.15.6:0/4120381633 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa60108400 0x7ffa6010a7f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009820 tx=0x7ffa5002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4972aa606855f355 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:23.320 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.603+0000 7ffa5effd640 1 -- 172.21.15.6:0/4120381633 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa50037070 con 0x7ffa60108400 2024-09-12T03:12:23.320 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.603+0000 7ffa5effd640 1 -- 172.21.15.6:0/4120381633 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa50032920 con 0x7ffa60108400 2024-09-12T03:12:23.320 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.603+0000 7ffa5effd640 1 -- 172.21.15.6:0/4120381633 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa50030810 con 0x7ffa60108400 2024-09-12T03:12:23.320 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.603+0000 7ffa66943640 1 -- 172.21.15.6:0/4120381633 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa60108400 msgr2=0x7ffa6010a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:23.320 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.603+0000 7ffa66943640 1 --2- 172.21.15.6:0/4120381633 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa60108400 0x7ffa6010a7f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009820 tx=0x7ffa5002a270 comp rx=0 tx=0).stop 2024-09-12T03:12:23.320 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.603+0000 7ffa66943640 1 -- 172.21.15.6:0/4120381633 shutdown_connections 2024-09-12T03:12:23.320 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.603+0000 7ffa66943640 1 --2- 172.21.15.6:0/4120381633 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa60108400 0x7ffa6010a7f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.320 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.603+0000 7ffa66943640 1 -- 172.21.15.6:0/4120381633 >> 172.21.15.6:0/4120381633 conn(0x7ffa60102000 msgr2=0x7ffa60104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:23.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.603+0000 7ffa66943640 1 -- 172.21.15.6:0/4120381633 shutdown_connections 2024-09-12T03:12:23.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.604+0000 7ffa66943640 1 -- 172.21.15.6:0/4120381633 wait complete. 2024-09-12T03:12:23.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.604+0000 7ffa66943640 1 Processor -- start 2024-09-12T03:12:23.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.604+0000 7ffa66943640 1 -- start start 2024-09-12T03:12:23.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.605+0000 7ffa66943640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa60108400 0x7ffa60107c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:23.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.605+0000 7ffa66943640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6010bdb0 con 0x7ffa60108400 2024-09-12T03:12:23.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.605+0000 7ffa5ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa60108400 0x7ffa60107c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:23.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.605+0000 7ffa5ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa60108400 0x7ffa60107c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57802/0 (socket says 172.21.15.6:57802) 2024-09-12T03:12:23.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.605+0000 7ffa5ffff640 1 -- 172.21.15.6:0/697115235 learned_addr learned my addr 172.21.15.6:0/697115235 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:23.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.605+0000 7ffa5ffff640 1 -- 172.21.15.6:0/697115235 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa601081c0 con 0x7ffa60108400 2024-09-12T03:12:23.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.605+0000 7ffa5ffff640 1 --2- 172.21.15.6:0/697115235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa60108400 0x7ffa60107c80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ffa500097f0 tx=0x7ffa500327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:23.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.605+0000 7ffa5d7fa640 1 -- 172.21.15.6:0/697115235 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa50037040 con 0x7ffa60108400 2024-09-12T03:12:23.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.605+0000 7ffa5d7fa640 1 -- 172.21.15.6:0/697115235 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa50030c40 con 0x7ffa60108400 2024-09-12T03:12:23.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.605+0000 7ffa66943640 1 -- 172.21.15.6:0/697115235 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa601063b0 con 0x7ffa60108400 2024-09-12T03:12:23.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.605+0000 7ffa5d7fa640 1 -- 172.21.15.6:0/697115235 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa5003a5d0 con 0x7ffa60108400 2024-09-12T03:12:23.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.605+0000 7ffa66943640 1 -- 172.21.15.6:0/697115235 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa601068f0 con 0x7ffa60108400 2024-09-12T03:12:23.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.606+0000 7ffa5d7fa640 1 -- 172.21.15.6:0/697115235 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7ffa5003a7c0 con 0x7ffa60108400 2024-09-12T03:12:23.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.606+0000 7ffa5d7fa640 1 --2- 172.21.15.6:0/697115235 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ffa48038ef0 0x7ffa4803b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:23.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.606+0000 7ffa5d7fa640 1 -- 172.21.15.6:0/697115235 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ffa5006d380 con 0x7ffa60108400 2024-09-12T03:12:23.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.607+0000 7ffa5f7fe640 1 --2- 172.21.15.6:0/697115235 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ffa48038ef0 0x7ffa4803b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:23.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.607+0000 7ffa66943640 1 -- 172.21.15.6:0/697115235 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa60108860 con 0x7ffa60108400 2024-09-12T03:12:23.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.607+0000 7ffa5f7fe640 1 --2- 172.21.15.6:0/697115235 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ffa48038ef0 0x7ffa4803b3b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffa540098c0 tx=0x7ffa54006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:23.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.612+0000 7ffa5d7fa640 1 -- 172.21.15.6:0/697115235 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa500371e0 con 0x7ffa60108400 2024-09-12T03:12:23.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.732+0000 7ffa5d7fa640 1 -- 172.21.15.6:0/697115235 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7ffa50044920 con 0x7ffa60108400 2024-09-12T03:12:23.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.797+0000 7ffa66943640 1 -- 172.21.15.6:0/697115235 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7ffa60106fe0 con 0x7ffa48038ef0 2024-09-12T03:12:23.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.914+0000 7ffa5d7fa640 1 -- 172.21.15.6:0/697115235 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7ffa60106fe0 con 0x7ffa48038ef0 2024-09-12T03:12:23.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.916+0000 7ffa66943640 1 -- 172.21.15.6:0/697115235 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ffa48038ef0 msgr2=0x7ffa4803b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:23.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.916+0000 7ffa66943640 1 --2- 172.21.15.6:0/697115235 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ffa48038ef0 0x7ffa4803b3b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffa540098c0 tx=0x7ffa54006db0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.916+0000 7ffa66943640 1 -- 172.21.15.6:0/697115235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa60108400 msgr2=0x7ffa60107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:23.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.916+0000 7ffa66943640 1 --2- 172.21.15.6:0/697115235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa60108400 0x7ffa60107c80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ffa500097f0 tx=0x7ffa500327b0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.917+0000 7ffa66943640 1 -- 172.21.15.6:0/697115235 shutdown_connections 2024-09-12T03:12:23.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.917+0000 7ffa66943640 1 --2- 172.21.15.6:0/697115235 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ffa48038ef0 0x7ffa4803b3b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.917+0000 7ffa66943640 1 --2- 172.21.15.6:0/697115235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa60108400 0x7ffa60107c80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.917+0000 7ffa66943640 1 -- 172.21.15.6:0/697115235 >> 172.21.15.6:0/697115235 conn(0x7ffa60102000 msgr2=0x7ffa60102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:23.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.917+0000 7ffa66943640 1 -- 172.21.15.6:0/697115235 shutdown_connections 2024-09-12T03:12:23.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:21.917+0000 7ffa66943640 1 -- 172.21.15.6:0/697115235 wait complete. 2024-09-12T03:12:23.323 INFO:teuthology.orchestra.run.smithi006.stdout:Creating initial admin user... 2024-09-12T03:12:24.184 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:24 smithi006 ceph-mon[30991]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:24.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$hlM.clAkmXozPebkTd1fPeyqniUcqkugmZ3/iBxbO6apO.Qv/LfES", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726110744, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-12T03:12:24.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.731+0000 7f2ad4cc8640 1 Processor -- start 2024-09-12T03:12:24.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.731+0000 7f2ad4cc8640 1 -- start start 2024-09-12T03:12:24.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.732+0000 7f2ad4cc8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ad01079c0 0x7f2ad0107dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:24.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.732+0000 7f2ad4cc8640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad0108390 con 0x7f2ad01079c0 2024-09-12T03:12:24.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.732+0000 7f2ace59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ad01079c0 0x7f2ad0107dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:24.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.732+0000 7f2ace59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ad01079c0 0x7f2ad0107dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57808/0 (socket says 172.21.15.6:57808) 2024-09-12T03:12:24.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.732+0000 7f2ace59c640 1 -- 172.21.15.6:0/3295033316 learned_addr learned my addr 172.21.15.6:0/3295033316 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:24.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.732+0000 7f2ace59c640 1 -- 172.21.15.6:0/3295033316 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad0108b20 con 0x7f2ad01079c0 2024-09-12T03:12:24.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.733+0000 7f2ace59c640 1 --2- 172.21.15.6:0/3295033316 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ad01079c0 0x7f2ad0107dc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009e80 tx=0x7f2ac002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb97b041d90e8d49 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:24.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.733+0000 7f2acd59a640 1 -- 172.21.15.6:0/3295033316 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ac0036070 con 0x7f2ad01079c0 2024-09-12T03:12:24.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.733+0000 7f2acd59a640 1 -- 172.21.15.6:0/3295033316 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ac00318e0 con 0x7f2ad01079c0 2024-09-12T03:12:24.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.733+0000 7f2acd59a640 1 -- 172.21.15.6:0/3295033316 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ac0007a00 con 0x7f2ad01079c0 2024-09-12T03:12:24.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.734+0000 7f2ad4cc8640 1 -- 172.21.15.6:0/3295033316 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ad01079c0 msgr2=0x7f2ad0107dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:24.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.734+0000 7f2ad4cc8640 1 --2- 172.21.15.6:0/3295033316 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ad01079c0 0x7f2ad0107dc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009e80 tx=0x7f2ac002a7a0 comp rx=0 tx=0).stop 2024-09-12T03:12:24.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.734+0000 7f2ad4cc8640 1 -- 172.21.15.6:0/3295033316 shutdown_connections 2024-09-12T03:12:24.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.734+0000 7f2ad4cc8640 1 --2- 172.21.15.6:0/3295033316 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ad01079c0 0x7f2ad0107dc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:24.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.734+0000 7f2ad4cc8640 1 -- 172.21.15.6:0/3295033316 >> 172.21.15.6:0/3295033316 conn(0x7f2ad01031b0 msgr2=0x7f2ad01055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:24.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.734+0000 7f2ad4cc8640 1 -- 172.21.15.6:0/3295033316 shutdown_connections 2024-09-12T03:12:24.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.734+0000 7f2ad4cc8640 1 -- 172.21.15.6:0/3295033316 wait complete. 2024-09-12T03:12:24.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.735+0000 7f2ad4cc8640 1 Processor -- start 2024-09-12T03:12:24.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.735+0000 7f2ad4cc8640 1 -- start start 2024-09-12T03:12:24.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.735+0000 7f2ad4cc8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ad01079c0 0x7f2ad019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:24.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.735+0000 7f2ad4cc8640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad010bed0 con 0x7f2ad01079c0 2024-09-12T03:12:24.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.735+0000 7f2ace59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ad01079c0 0x7f2ad019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:24.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.735+0000 7f2ace59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ad01079c0 0x7f2ad019b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57822/0 (socket says 172.21.15.6:57822) 2024-09-12T03:12:24.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.735+0000 7f2ace59c640 1 -- 172.21.15.6:0/1750865449 learned_addr learned my addr 172.21.15.6:0/1750865449 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:24.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.735+0000 7f2ace59c640 1 -- 172.21.15.6:0/1750865449 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad019be30 con 0x7f2ad01079c0 2024-09-12T03:12:24.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.736+0000 7f2ace59c640 1 --2- 172.21.15.6:0/1750865449 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ad01079c0 0x7f2ad019b8f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0031cc0 tx=0x7f2ac0031cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:24.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.736+0000 7f2ac77fe640 1 -- 172.21.15.6:0/1750865449 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ac0036050 con 0x7f2ad01079c0 2024-09-12T03:12:24.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.736+0000 7f2ad4cc8640 1 -- 172.21.15.6:0/1750865449 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad019c0c0 con 0x7f2ad01079c0 2024-09-12T03:12:24.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.736+0000 7f2ac77fe640 1 -- 172.21.15.6:0/1750865449 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ac0042070 con 0x7f2ad01079c0 2024-09-12T03:12:24.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.736+0000 7f2ac77fe640 1 -- 172.21.15.6:0/1750865449 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ac003d8e0 con 0x7f2ad01079c0 2024-09-12T03:12:24.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.736+0000 7f2ad4cc8640 1 -- 172.21.15.6:0/1750865449 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad019edb0 con 0x7f2ad01079c0 2024-09-12T03:12:24.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.737+0000 7f2ac77fe640 1 -- 172.21.15.6:0/1750865449 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f2ac003db90 con 0x7f2ad01079c0 2024-09-12T03:12:24.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.737+0000 7f2ad4cc8640 1 -- 172.21.15.6:0/1750865449 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad010c050 con 0x7f2ad01079c0 2024-09-12T03:12:24.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.737+0000 7f2ac77fe640 1 --2- 172.21.15.6:0/1750865449 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2abc039010 0x7f2abc03b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:24.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.737+0000 7f2ac77fe640 1 -- 172.21.15.6:0/1750865449 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2ac006d720 con 0x7f2ad01079c0 2024-09-12T03:12:24.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.737+0000 7f2acdd9b640 1 --2- 172.21.15.6:0/1750865449 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2abc039010 0x7f2abc03b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:24.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.738+0000 7f2acdd9b640 1 --2- 172.21.15.6:0/1750865449 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2abc039010 0x7f2abc03b4d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80098c0 tx=0x7f2ab8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:24.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.742+0000 7f2ac77fe640 1 -- 172.21.15.6:0/1750865449 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ac0044310 con 0x7f2ad01079c0 2024-09-12T03:12:24.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:23.934+0000 7f2ad4cc8640 1 -- 172.21.15.6:0/1750865449 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f2ad0107e40 con 0x7f2abc039010 2024-09-12T03:12:24.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:24.182+0000 7f2ac77fe640 1 -- 172.21.15.6:0/1750865449 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f2ad0107e40 con 0x7f2abc039010 2024-09-12T03:12:24.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:24.184+0000 7f2ad4cc8640 1 -- 172.21.15.6:0/1750865449 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2abc039010 msgr2=0x7f2abc03b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:24.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:24.184+0000 7f2ad4cc8640 1 --2- 172.21.15.6:0/1750865449 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2abc039010 0x7f2abc03b4d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80098c0 tx=0x7f2ab8006db0 comp rx=0 tx=0).stop 2024-09-12T03:12:24.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:24.184+0000 7f2ad4cc8640 1 -- 172.21.15.6:0/1750865449 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ad01079c0 msgr2=0x7f2ad019b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:24.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:24.184+0000 7f2ad4cc8640 1 --2- 172.21.15.6:0/1750865449 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ad01079c0 0x7f2ad019b8f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0031cc0 tx=0x7f2ac0031cf0 comp rx=0 tx=0).stop 2024-09-12T03:12:24.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:24.184+0000 7f2ad4cc8640 1 -- 172.21.15.6:0/1750865449 shutdown_connections 2024-09-12T03:12:24.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:24.184+0000 7f2ad4cc8640 1 --2- 172.21.15.6:0/1750865449 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2abc039010 0x7f2abc03b4d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:24.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:24.184+0000 7f2ad4cc8640 1 --2- 172.21.15.6:0/1750865449 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ad01079c0 0x7f2ad019b8f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:24.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:24.184+0000 7f2ad4cc8640 1 -- 172.21.15.6:0/1750865449 >> 172.21.15.6:0/1750865449 conn(0x7f2ad01031b0 msgr2=0x7f2ad0103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:24.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:24.184+0000 7f2ad4cc8640 1 -- 172.21.15.6:0/1750865449 shutdown_connections 2024-09-12T03:12:24.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:24.184+0000 7f2ad4cc8640 1 -- 172.21.15.6:0/1750865449 wait complete. 2024-09-12T03:12:24.787 INFO:teuthology.orchestra.run.smithi006.stdout:Fetching dashboard port number... 2024-09-12T03:12:25.338 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:25 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:25.943 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 8443 2024-09-12T03:12:25.943 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.152+0000 7f9435947640 1 Processor -- start 2024-09-12T03:12:25.943 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.152+0000 7f9435947640 1 -- start start 2024-09-12T03:12:25.943 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.153+0000 7f9435947640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f943007ddc0 0x7f943007c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.943 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.153+0000 7f9435947640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f943007c800 con 0x7f943007ddc0 2024-09-12T03:12:25.944 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.153+0000 7f942effd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f943007ddc0 0x7f943007c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.945 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.153+0000 7f942effd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f943007ddc0 0x7f943007c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57838/0 (socket says 172.21.15.6:57838) 2024-09-12T03:12:25.945 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.153+0000 7f942effd640 1 -- 172.21.15.6:0/1733301346 learned_addr learned my addr 172.21.15.6:0/1733301346 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:25.945 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.153+0000 7f942effd640 1 -- 172.21.15.6:0/1733301346 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f943007c980 con 0x7f943007ddc0 2024-09-12T03:12:25.945 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.154+0000 7f942effd640 1 --2- 172.21.15.6:0/1733301346 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f943007ddc0 0x7f943007c2c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9420009820 tx=0x7f942002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a0d76706b89e91 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:25.945 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.154+0000 7f942dffb640 1 -- 172.21.15.6:0/1733301346 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9420037070 con 0x7f943007ddc0 2024-09-12T03:12:25.945 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.154+0000 7f942dffb640 1 -- 172.21.15.6:0/1733301346 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9420032920 con 0x7f943007ddc0 2024-09-12T03:12:25.945 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.154+0000 7f942dffb640 1 -- 172.21.15.6:0/1733301346 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9420030810 con 0x7f943007ddc0 2024-09-12T03:12:25.945 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.155+0000 7f9435947640 1 -- 172.21.15.6:0/1733301346 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f943007ddc0 msgr2=0x7f943007c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:25.945 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.155+0000 7f9435947640 1 --2- 172.21.15.6:0/1733301346 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f943007ddc0 0x7f943007c2c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9420009820 tx=0x7f942002a270 comp rx=0 tx=0).stop 2024-09-12T03:12:25.945 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.155+0000 7f9435947640 1 -- 172.21.15.6:0/1733301346 shutdown_connections 2024-09-12T03:12:25.945 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.155+0000 7f9435947640 1 --2- 172.21.15.6:0/1733301346 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f943007ddc0 0x7f943007c2c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.945 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.155+0000 7f9435947640 1 -- 172.21.15.6:0/1733301346 >> 172.21.15.6:0/1733301346 conn(0x7f94301031b0 msgr2=0x7f94301055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:25.945 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.155+0000 7f9435947640 1 -- 172.21.15.6:0/1733301346 shutdown_connections 2024-09-12T03:12:25.946 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.155+0000 7f9435947640 1 -- 172.21.15.6:0/1733301346 wait complete. 2024-09-12T03:12:25.946 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.155+0000 7f9435947640 1 Processor -- start 2024-09-12T03:12:25.946 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.156+0000 7f9435947640 1 -- start start 2024-09-12T03:12:25.946 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.156+0000 7f9435947640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f943007ddc0 0x7f943019dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.946 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.156+0000 7f9435947640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f943010df90 con 0x7f943007ddc0 2024-09-12T03:12:25.946 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.156+0000 7f942effd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f943007ddc0 0x7f943019dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.946 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.156+0000 7f942effd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f943007ddc0 0x7f943019dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57840/0 (socket says 172.21.15.6:57840) 2024-09-12T03:12:25.946 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.156+0000 7f942effd640 1 -- 172.21.15.6:0/964996580 learned_addr learned my addr 172.21.15.6:0/964996580 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:25.946 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.156+0000 7f942effd640 1 -- 172.21.15.6:0/964996580 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f943019e020 con 0x7f943007ddc0 2024-09-12T03:12:25.946 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.157+0000 7f942effd640 1 --2- 172.21.15.6:0/964996580 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f943007ddc0 0x7f943019dae0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9420030380 tx=0x7f9420032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:25.946 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.157+0000 7f9434945640 1 -- 172.21.15.6:0/964996580 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9420037040 con 0x7f943007ddc0 2024-09-12T03:12:25.946 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.157+0000 7f9435947640 1 -- 172.21.15.6:0/964996580 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f943019e2b0 con 0x7f943007ddc0 2024-09-12T03:12:25.946 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.157+0000 7f9434945640 1 -- 172.21.15.6:0/964996580 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9420040070 con 0x7f943007ddc0 2024-09-12T03:12:25.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.157+0000 7f9434945640 1 -- 172.21.15.6:0/964996580 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f942003a7b0 con 0x7f943007ddc0 2024-09-12T03:12:25.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.157+0000 7f9435947640 1 -- 172.21.15.6:0/964996580 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94301a0fa0 con 0x7f943007ddc0 2024-09-12T03:12:25.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.158+0000 7f9434945640 1 -- 172.21.15.6:0/964996580 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f942003aa30 con 0x7f943007ddc0 2024-09-12T03:12:25.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.158+0000 7f9435947640 1 -- 172.21.15.6:0/964996580 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f943007c340 con 0x7f943007ddc0 2024-09-12T03:12:25.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.158+0000 7f9434945640 1 --2- 172.21.15.6:0/964996580 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9418038fc0 0x7f941803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.158+0000 7f9434945640 1 -- 172.21.15.6:0/964996580 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f942006c780 con 0x7f943007ddc0 2024-09-12T03:12:25.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.158+0000 7f942e7fc640 1 --2- 172.21.15.6:0/964996580 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9418038fc0 0x7f941803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.159+0000 7f942e7fc640 1 --2- 172.21.15.6:0/964996580 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9418038fc0 0x7f941803b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f94240098c0 tx=0x7f9424006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:25.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.163+0000 7f9434945640 1 -- 172.21.15.6:0/964996580 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9420044c70 con 0x7f943007ddc0 2024-09-12T03:12:25.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.183+0000 7f9434945640 1 -- 172.21.15.6:0/964996580 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f942006ed00 con 0x7f943007ddc0 2024-09-12T03:12:25.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.337+0000 7f9435947640 1 -- 172.21.15.6:0/964996580 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f94301a1170 con 0x7f943007ddc0 2024-09-12T03:12:25.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.338+0000 7f9434945640 1 -- 172.21.15.6:0/964996580 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f942004a200 con 0x7f943007ddc0 2024-09-12T03:12:25.948 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.340+0000 7f9435947640 1 -- 172.21.15.6:0/964996580 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9418038fc0 msgr2=0x7f941803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:25.948 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.340+0000 7f9435947640 1 --2- 172.21.15.6:0/964996580 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9418038fc0 0x7f941803b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f94240098c0 tx=0x7f9424006db0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.948 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.340+0000 7f9435947640 1 -- 172.21.15.6:0/964996580 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f943007ddc0 msgr2=0x7f943019dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:25.948 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.340+0000 7f9435947640 1 --2- 172.21.15.6:0/964996580 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f943007ddc0 0x7f943019dae0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9420030380 tx=0x7f9420032670 comp rx=0 tx=0).stop 2024-09-12T03:12:25.948 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.340+0000 7f9435947640 1 -- 172.21.15.6:0/964996580 shutdown_connections 2024-09-12T03:12:25.948 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.340+0000 7f9435947640 1 --2- 172.21.15.6:0/964996580 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9418038fc0 0x7f941803b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.948 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.340+0000 7f9435947640 1 --2- 172.21.15.6:0/964996580 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f943007ddc0 0x7f943019dae0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.948 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.340+0000 7f9435947640 1 -- 172.21.15.6:0/964996580 >> 172.21.15.6:0/964996580 conn(0x7f94301031b0 msgr2=0x7f9430103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:25.948 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.340+0000 7f9435947640 1 -- 172.21.15.6:0/964996580 shutdown_connections 2024-09-12T03:12:25.948 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:25.340+0000 7f9435947640 1 -- 172.21.15.6:0/964996580 wait complete. 2024-09-12T03:12:25.949 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T03:12:25.949 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2024-09-12T03:12:25.955 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T03:12:25.955 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-09-12T03:12:25.955 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2024-09-12T03:12:25.955 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-12T03:12:25.957 INFO:teuthology.orchestra.run.smithi006.stdout:Ceph Dashboard is now available at: 2024-09-12T03:12:25.957 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:12:25.957 INFO:teuthology.orchestra.run.smithi006.stdout: URL: https://smithi006.front.sepia.ceph.com:8443/ 2024-09-12T03:12:25.957 INFO:teuthology.orchestra.run.smithi006.stdout: User: admin 2024-09-12T03:12:25.957 INFO:teuthology.orchestra.run.smithi006.stdout: Password: eumqxlndf7 2024-09-12T03:12:25.957 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:12:25.957 INFO:teuthology.orchestra.run.smithi006.stdout:Saving cluster configuration to /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/config directory 2024-09-12T03:12:26.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:26 smithi006 ceph-mon[30991]: mgrmap e13: x(active, since 6s) 2024-09-12T03:12:26.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:26 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/964996580' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-12T03:12:27.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.359+0000 7faeb9fdd640 1 Processor -- start 2024-09-12T03:12:27.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.359+0000 7faeb9fdd640 1 -- start start 2024-09-12T03:12:27.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.359+0000 7faeb9fdd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faeb407de00 0x7faeb407c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:27.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.359+0000 7faeb9fdd640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7faeb407c840 con 0x7faeb407de00 2024-09-12T03:12:27.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.359+0000 7faeb37fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faeb407de00 0x7faeb407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:27.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.359+0000 7faeb37fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faeb407de00 0x7faeb407c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57842/0 (socket says 172.21.15.6:57842) 2024-09-12T03:12:27.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.359+0000 7faeb37fe640 1 -- 172.21.15.6:0/3528201086 learned_addr learned my addr 172.21.15.6:0/3528201086 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:27.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.360+0000 7faeb37fe640 1 -- 172.21.15.6:0/3528201086 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb407c9c0 con 0x7faeb407de00 2024-09-12T03:12:27.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.360+0000 7faeb37fe640 1 --2- 172.21.15.6:0/3528201086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faeb407de00 0x7faeb407c300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faea4009820 tx=0x7faea402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ab886838634adf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:27.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.360+0000 7faeb27fc640 1 -- 172.21.15.6:0/3528201086 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faea4037070 con 0x7faeb407de00 2024-09-12T03:12:27.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.360+0000 7faeb27fc640 1 -- 172.21.15.6:0/3528201086 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faea4032920 con 0x7faeb407de00 2024-09-12T03:12:27.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.361+0000 7faeb27fc640 1 -- 172.21.15.6:0/3528201086 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faea4030810 con 0x7faeb407de00 2024-09-12T03:12:27.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.361+0000 7faeb9fdd640 1 -- 172.21.15.6:0/3528201086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faeb407de00 msgr2=0x7faeb407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:27.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.361+0000 7faeb9fdd640 1 --2- 172.21.15.6:0/3528201086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faeb407de00 0x7faeb407c300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faea4009820 tx=0x7faea402a270 comp rx=0 tx=0).stop 2024-09-12T03:12:27.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.361+0000 7faeb9fdd640 1 -- 172.21.15.6:0/3528201086 shutdown_connections 2024-09-12T03:12:27.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.361+0000 7faeb9fdd640 1 --2- 172.21.15.6:0/3528201086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faeb407de00 0x7faeb407c300 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:27.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.361+0000 7faeb9fdd640 1 -- 172.21.15.6:0/3528201086 >> 172.21.15.6:0/3528201086 conn(0x7faeb41031b0 msgr2=0x7faeb41055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:27.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.361+0000 7faeb9fdd640 1 -- 172.21.15.6:0/3528201086 shutdown_connections 2024-09-12T03:12:27.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.361+0000 7faeb9fdd640 1 -- 172.21.15.6:0/3528201086 wait complete. 2024-09-12T03:12:27.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.362+0000 7faeb9fdd640 1 Processor -- start 2024-09-12T03:12:27.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.362+0000 7faeb9fdd640 1 -- start start 2024-09-12T03:12:27.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.362+0000 7faeb9fdd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faeb407de00 0x7faeb419db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:27.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.362+0000 7faeb9fdd640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7faeb410dfb0 con 0x7faeb407de00 2024-09-12T03:12:27.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.362+0000 7faeb37fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faeb407de00 0x7faeb419db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:27.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.362+0000 7faeb37fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faeb407de00 0x7faeb419db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57850/0 (socket says 172.21.15.6:57850) 2024-09-12T03:12:27.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.362+0000 7faeb37fe640 1 -- 172.21.15.6:0/3503247449 learned_addr learned my addr 172.21.15.6:0/3503247449 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:27.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.363+0000 7faeb37fe640 1 -- 172.21.15.6:0/3503247449 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb419e040 con 0x7faeb407de00 2024-09-12T03:12:27.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.363+0000 7faeb37fe640 1 --2- 172.21.15.6:0/3503247449 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faeb407de00 0x7faeb419db00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faea40097f0 tx=0x7faea40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:27.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.363+0000 7faeb0ff9640 1 -- 172.21.15.6:0/3503247449 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faea4037040 con 0x7faeb407de00 2024-09-12T03:12:27.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.363+0000 7faeb0ff9640 1 -- 172.21.15.6:0/3503247449 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faea4030cc0 con 0x7faeb407de00 2024-09-12T03:12:27.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.363+0000 7faeb9fdd640 1 -- 172.21.15.6:0/3503247449 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb419e2d0 con 0x7faeb407de00 2024-09-12T03:12:27.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.363+0000 7faeb0ff9640 1 -- 172.21.15.6:0/3503247449 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faea403a6b0 con 0x7faeb407de00 2024-09-12T03:12:27.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.363+0000 7faeb9fdd640 1 -- 172.21.15.6:0/3503247449 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb41a0fc0 con 0x7faeb407de00 2024-09-12T03:12:27.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.364+0000 7faeb0ff9640 1 -- 172.21.15.6:0/3503247449 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7faea403a850 con 0x7faeb407de00 2024-09-12T03:12:27.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.364+0000 7faeb0ff9640 1 --2- 172.21.15.6:0/3503247449 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7faea00390b0 0x7faea003b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:27.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.364+0000 7faeb0ff9640 1 -- 172.21.15.6:0/3503247449 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faea406c690 con 0x7faeb407de00 2024-09-12T03:12:27.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.364+0000 7faeb9fdd640 1 -- 172.21.15.6:0/3503247449 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeb407c380 con 0x7faeb407de00 2024-09-12T03:12:27.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.364+0000 7faeb2ffd640 1 --2- 172.21.15.6:0/3503247449 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7faea00390b0 0x7faea003b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:27.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.365+0000 7faeb2ffd640 1 --2- 172.21.15.6:0/3503247449 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7faea00390b0 0x7faea003b570 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0098c0 tx=0x7fae9c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:27.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.369+0000 7faeb0ff9640 1 -- 172.21.15.6:0/3503247449 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faea4039760 con 0x7faeb407de00 2024-09-12T03:12:27.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.636+0000 7faeb9fdd640 1 -- 172.21.15.6:0/3503247449 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7faeb41a1190 con 0x7faeb407de00 2024-09-12T03:12:27.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.639+0000 7faeb0ff9640 1 -- 172.21.15.6:0/3503247449 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7faea40371e0 con 0x7faeb407de00 2024-09-12T03:12:27.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-12T03:12:27.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.641+0000 7faeb9fdd640 1 -- 172.21.15.6:0/3503247449 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7faea00390b0 msgr2=0x7faea003b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:27.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.641+0000 7faeb9fdd640 1 --2- 172.21.15.6:0/3503247449 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7faea00390b0 0x7faea003b570 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0098c0 tx=0x7fae9c006db0 comp rx=0 tx=0).stop 2024-09-12T03:12:27.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.641+0000 7faeb9fdd640 1 -- 172.21.15.6:0/3503247449 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faeb407de00 msgr2=0x7faeb419db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:27.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.641+0000 7faeb9fdd640 1 --2- 172.21.15.6:0/3503247449 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faeb407de00 0x7faeb419db00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faea40097f0 tx=0x7faea40327b0 comp rx=0 tx=0).stop 2024-09-12T03:12:27.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.641+0000 7faeb9fdd640 1 -- 172.21.15.6:0/3503247449 shutdown_connections 2024-09-12T03:12:27.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.641+0000 7faeb9fdd640 1 --2- 172.21.15.6:0/3503247449 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7faea00390b0 0x7faea003b570 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:27.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.641+0000 7faeb9fdd640 1 --2- 172.21.15.6:0/3503247449 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faeb407de00 0x7faeb419db00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:27.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.641+0000 7faeb9fdd640 1 -- 172.21.15.6:0/3503247449 >> 172.21.15.6:0/3503247449 conn(0x7faeb41031b0 msgr2=0x7faeb4103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:27.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.641+0000 7faeb9fdd640 1 -- 172.21.15.6:0/3503247449 shutdown_connections 2024-09-12T03:12:27.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-09-12T03:12:26.641+0000 7faeb9fdd640 1 -- 172.21.15.6:0/3503247449 wait complete. 2024-09-12T03:12:27.209 INFO:teuthology.orchestra.run.smithi006.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-12T03:12:27.209 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:12:27.210 INFO:teuthology.orchestra.run.smithi006.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:12:27.210 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:12:27.210 INFO:teuthology.orchestra.run.smithi006.stdout:Or, if you are only running a single cluster on this host: 2024-09-12T03:12:27.210 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:12:27.210 INFO:teuthology.orchestra.run.smithi006.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-12T03:12:27.210 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:12:27.210 INFO:teuthology.orchestra.run.smithi006.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-12T03:12:27.210 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:12:27.210 INFO:teuthology.orchestra.run.smithi006.stdout: ceph telemetry on 2024-09-12T03:12:27.210 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:12:27.210 INFO:teuthology.orchestra.run.smithi006.stdout:For more information see: 2024-09-12T03:12:27.210 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:12:27.210 INFO:teuthology.orchestra.run.smithi006.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-12T03:12:27.210 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:12:27.211 INFO:teuthology.orchestra.run.smithi006.stdout:Bootstrap complete. 2024-09-12T03:12:27.244 INFO:tasks.cephadm:Fetching config... 2024-09-12T03:12:27.244 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-09-12T03:12:27.244 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-12T03:12:27.261 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-12T03:12:27.261 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-09-12T03:12:27.261 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-12T03:12:27.318 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-12T03:12:27.318 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-09-12T03:12:27.318 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-12T03:12:27.384 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-12T03:12:27.385 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-09-12T03:12:27.385 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-12T03:12:27.441 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-12T03:12:27.441 DEBUG:teuthology.orchestra.run.smithi006:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyyWudC1H8NyjplBuWJYyTGyCmOlQK9SLFIXzd+u4UcDVx3IqI5XWQqhwA9fx0ksZSLWYMp8mNXfFeVxGUupTk6aDeXYXsyOpOqRV7gGBZtgewf0xl2fBLdOxDHuP0denmuHNqJXXRpM+uJz9wT5mtuIU9jF1+EjgU5euAZPq/DEr+WrR7fOuwnrca/Rx+L37kgkSsw6eUE55qBMe/tAP8j4pT1B40FArlrcmmrnR13VZG6kIRsFQqhnmAbbi9Hsu02ZBWBSnnXcQyBE64BsX2/PiFF2kZEXe7pBfH0bLmUJKY7jUrqLTX2aGhOYOaBE0SI2fYbaMEuKAjuTDpEQjGJm7ANhRi9a43rPMllur6FOdWTnWj0bCMKiCjcx4vD136RT+oHNtqPTNz31rUi6GFMwbxMduPPnaST3F9vjAZhRYRCa5kjwUwSnXqTfNrVEd159zk4ZQd62bMNwgb/ciOPn8+ucq92c89M2kGmTi5BrH4Ezn+GMqOXB8m4vO2CRs= ceph-89d99938-70b4-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T03:12:27.516 INFO:teuthology.orchestra.run.smithi006.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyyWudC1H8NyjplBuWJYyTGyCmOlQK9SLFIXzd+u4UcDVx3IqI5XWQqhwA9fx0ksZSLWYMp8mNXfFeVxGUupTk6aDeXYXsyOpOqRV7gGBZtgewf0xl2fBLdOxDHuP0denmuHNqJXXRpM+uJz9wT5mtuIU9jF1+EjgU5euAZPq/DEr+WrR7fOuwnrca/Rx+L37kgkSsw6eUE55qBMe/tAP8j4pT1B40FArlrcmmrnR13VZG6kIRsFQqhnmAbbi9Hsu02ZBWBSnnXcQyBE64BsX2/PiFF2kZEXe7pBfH0bLmUJKY7jUrqLTX2aGhOYOaBE0SI2fYbaMEuKAjuTDpEQjGJm7ANhRi9a43rPMllur6FOdWTnWj0bCMKiCjcx4vD136RT+oHNtqPTNz31rUi6GFMwbxMduPPnaST3F9vjAZhRYRCa5kjwUwSnXqTfNrVEd159zk4ZQd62bMNwgb/ciOPn8+ucq92c89M2kGmTi5BrH4Ezn+GMqOXB8m4vO2CRs= ceph-89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:12:27.543 DEBUG:teuthology.orchestra.run.smithi118:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyyWudC1H8NyjplBuWJYyTGyCmOlQK9SLFIXzd+u4UcDVx3IqI5XWQqhwA9fx0ksZSLWYMp8mNXfFeVxGUupTk6aDeXYXsyOpOqRV7gGBZtgewf0xl2fBLdOxDHuP0denmuHNqJXXRpM+uJz9wT5mtuIU9jF1+EjgU5euAZPq/DEr+WrR7fOuwnrca/Rx+L37kgkSsw6eUE55qBMe/tAP8j4pT1B40FArlrcmmrnR13VZG6kIRsFQqhnmAbbi9Hsu02ZBWBSnnXcQyBE64BsX2/PiFF2kZEXe7pBfH0bLmUJKY7jUrqLTX2aGhOYOaBE0SI2fYbaMEuKAjuTDpEQjGJm7ANhRi9a43rPMllur6FOdWTnWj0bCMKiCjcx4vD136RT+oHNtqPTNz31rUi6GFMwbxMduPPnaST3F9vjAZhRYRCa5kjwUwSnXqTfNrVEd159zk4ZQd62bMNwgb/ciOPn8+ucq92c89M2kGmTi5BrH4Ezn+GMqOXB8m4vO2CRs= ceph-89d99938-70b4-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T03:12:27.587 INFO:teuthology.orchestra.run.smithi118.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyyWudC1H8NyjplBuWJYyTGyCmOlQK9SLFIXzd+u4UcDVx3IqI5XWQqhwA9fx0ksZSLWYMp8mNXfFeVxGUupTk6aDeXYXsyOpOqRV7gGBZtgewf0xl2fBLdOxDHuP0denmuHNqJXXRpM+uJz9wT5mtuIU9jF1+EjgU5euAZPq/DEr+WrR7fOuwnrca/Rx+L37kgkSsw6eUE55qBMe/tAP8j4pT1B40FArlrcmmrnR13VZG6kIRsFQqhnmAbbi9Hsu02ZBWBSnnXcQyBE64BsX2/PiFF2kZEXe7pBfH0bLmUJKY7jUrqLTX2aGhOYOaBE0SI2fYbaMEuKAjuTDpEQjGJm7ANhRi9a43rPMllur6FOdWTnWj0bCMKiCjcx4vD136RT+oHNtqPTNz31rUi6GFMwbxMduPPnaST3F9vjAZhRYRCa5kjwUwSnXqTfNrVEd159zk4ZQd62bMNwgb/ciOPn8+ucq92c89M2kGmTi5BrH4Ezn+GMqOXB8m4vO2CRs= ceph-89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:12:27.599 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyyWudC1H8NyjplBuWJYyTGyCmOlQK9SLFIXzd+u4UcDVx3IqI5XWQqhwA9fx0ksZSLWYMp8mNXfFeVxGUupTk6aDeXYXsyOpOqRV7gGBZtgewf0xl2fBLdOxDHuP0denmuHNqJXXRpM+uJz9wT5mtuIU9jF1+EjgU5euAZPq/DEr+WrR7fOuwnrca/Rx+L37kgkSsw6eUE55qBMe/tAP8j4pT1B40FArlrcmmrnR13VZG6kIRsFQqhnmAbbi9Hsu02ZBWBSnnXcQyBE64BsX2/PiFF2kZEXe7pBfH0bLmUJKY7jUrqLTX2aGhOYOaBE0SI2fYbaMEuKAjuTDpEQjGJm7ANhRi9a43rPMllur6FOdWTnWj0bCMKiCjcx4vD136RT+oHNtqPTNz31rUi6GFMwbxMduPPnaST3F9vjAZhRYRCa5kjwUwSnXqTfNrVEd159zk4ZQd62bMNwgb/ciOPn8+ucq92c89M2kGmTi5BrH4Ezn+GMqOXB8m4vO2CRs= ceph-89d99938-70b4-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T03:12:27.642 INFO:teuthology.orchestra.run.smithi161.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyyWudC1H8NyjplBuWJYyTGyCmOlQK9SLFIXzd+u4UcDVx3IqI5XWQqhwA9fx0ksZSLWYMp8mNXfFeVxGUupTk6aDeXYXsyOpOqRV7gGBZtgewf0xl2fBLdOxDHuP0denmuHNqJXXRpM+uJz9wT5mtuIU9jF1+EjgU5euAZPq/DEr+WrR7fOuwnrca/Rx+L37kgkSsw6eUE55qBMe/tAP8j4pT1B40FArlrcmmrnR13VZG6kIRsFQqhnmAbbi9Hsu02ZBWBSnnXcQyBE64BsX2/PiFF2kZEXe7pBfH0bLmUJKY7jUrqLTX2aGhOYOaBE0SI2fYbaMEuKAjuTDpEQjGJm7ANhRi9a43rPMllur6FOdWTnWj0bCMKiCjcx4vD136RT+oHNtqPTNz31rUi6GFMwbxMduPPnaST3F9vjAZhRYRCa5kjwUwSnXqTfNrVEd159zk4ZQd62bMNwgb/ciOPn8+ucq92c89M2kGmTi5BrH4Ezn+GMqOXB8m4vO2CRs= ceph-89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:12:27.655 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-12T03:12:27.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:27 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3503247449' entity='client.admin' 2024-09-12T03:12:27.908 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:28.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.386+0000 7f844a509640 1 -- 172.21.15.6:0/1668473007 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8444100540 msgr2=0x7f8444100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:28.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.386+0000 7f844a509640 1 --2- 172.21.15.6:0/1668473007 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8444100540 0x7f8444100940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8434009880 tx=0x7f843402a5d0 comp rx=0 tx=0).stop 2024-09-12T03:12:28.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.386+0000 7f844a509640 1 -- 172.21.15.6:0/1668473007 shutdown_connections 2024-09-12T03:12:28.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.386+0000 7f844a509640 1 --2- 172.21.15.6:0/1668473007 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8444100540 0x7f8444100940 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:28.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.386+0000 7f844a509640 1 -- 172.21.15.6:0/1668473007 >> 172.21.15.6:0/1668473007 conn(0x7f84440fbcd0 msgr2=0x7f84440fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:28.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.386+0000 7f844a509640 1 -- 172.21.15.6:0/1668473007 shutdown_connections 2024-09-12T03:12:28.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.386+0000 7f844a509640 1 -- 172.21.15.6:0/1668473007 wait complete. 2024-09-12T03:12:28.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.387+0000 7f844a509640 1 Processor -- start 2024-09-12T03:12:28.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.387+0000 7f844a509640 1 -- start start 2024-09-12T03:12:28.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.387+0000 7f844a509640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8444100540 0x7f8444196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:28.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.387+0000 7f844a509640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f84441065f0 con 0x7f8444100540 2024-09-12T03:12:28.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.387+0000 7f8443fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8444100540 0x7f8444196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:28.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.387+0000 7f8443fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8444100540 0x7f8444196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57854/0 (socket says 172.21.15.6:57854) 2024-09-12T03:12:28.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.387+0000 7f8443fff640 1 -- 172.21.15.6:0/3091434682 learned_addr learned my addr 172.21.15.6:0/3091434682 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:28.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.388+0000 7f8443fff640 1 -- 172.21.15.6:0/3091434682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84441972d0 con 0x7f8444100540 2024-09-12T03:12:28.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.388+0000 7f8443fff640 1 --2- 172.21.15.6:0/3091434682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8444100540 0x7f8444196d90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8434004f60 tx=0x7f84340048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:28.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.388+0000 7f84417fa640 1 -- 172.21.15.6:0/3091434682 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f843403f070 con 0x7f8444100540 2024-09-12T03:12:28.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.388+0000 7f84417fa640 1 -- 172.21.15.6:0/3091434682 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8434004cc0 con 0x7f8444100540 2024-09-12T03:12:28.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.388+0000 7f844a509640 1 -- 172.21.15.6:0/3091434682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8444197560 con 0x7f8444100540 2024-09-12T03:12:28.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.388+0000 7f84417fa640 1 -- 172.21.15.6:0/3091434682 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8434035030 con 0x7f8444100540 2024-09-12T03:12:28.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.388+0000 7f844a509640 1 -- 172.21.15.6:0/3091434682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f844419a210 con 0x7f8444100540 2024-09-12T03:12:28.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.389+0000 7f84417fa640 1 -- 172.21.15.6:0/3091434682 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f8434042070 con 0x7f8444100540 2024-09-12T03:12:28.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.389+0000 7f84417fa640 1 --2- 172.21.15.6:0/3091434682 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f842c0390c0 0x7f842c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:28.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.389+0000 7f84417fa640 1 -- 172.21.15.6:0/3091434682 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f843406dd60 con 0x7f8444100540 2024-09-12T03:12:28.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.389+0000 7f84437fe640 1 --2- 172.21.15.6:0/3091434682 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f842c0390c0 0x7f842c03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:28.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.390+0000 7f844a509640 1 -- 172.21.15.6:0/3091434682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8444106d70 con 0x7f8444100540 2024-09-12T03:12:28.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.390+0000 7f84437fe640 1 --2- 172.21.15.6:0/3091434682 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f842c0390c0 0x7f842c03b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f84380098c0 tx=0x7f8438006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:28.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.395+0000 7f84417fa640 1 -- 172.21.15.6:0/3091434682 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f84340433b0 con 0x7f8444100540 2024-09-12T03:12:28.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.571+0000 7f844a509640 1 -- 172.21.15.6:0/3091434682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f844402d7c0 con 0x7f8444100540 2024-09-12T03:12:28.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.574+0000 7f84417fa640 1 -- 172.21.15.6:0/3091434682 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f843402f300 con 0x7f8444100540 2024-09-12T03:12:28.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.576+0000 7f844a509640 1 -- 172.21.15.6:0/3091434682 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f842c0390c0 msgr2=0x7f842c03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:28.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.576+0000 7f844a509640 1 --2- 172.21.15.6:0/3091434682 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f842c0390c0 0x7f842c03b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f84380098c0 tx=0x7f8438006db0 comp rx=0 tx=0).stop 2024-09-12T03:12:28.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.576+0000 7f844a509640 1 -- 172.21.15.6:0/3091434682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8444100540 msgr2=0x7f8444196d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:28.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.576+0000 7f844a509640 1 --2- 172.21.15.6:0/3091434682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8444100540 0x7f8444196d90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8434004f60 tx=0x7f84340048f0 comp rx=0 tx=0).stop 2024-09-12T03:12:28.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.576+0000 7f844a509640 1 -- 172.21.15.6:0/3091434682 shutdown_connections 2024-09-12T03:12:28.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.576+0000 7f844a509640 1 --2- 172.21.15.6:0/3091434682 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f842c0390c0 0x7f842c03b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:28.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.576+0000 7f844a509640 1 --2- 172.21.15.6:0/3091434682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8444100540 0x7f8444196d90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:28.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.576+0000 7f844a509640 1 -- 172.21.15.6:0/3091434682 >> 172.21.15.6:0/3091434682 conn(0x7f84440fbcd0 msgr2=0x7f8444069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:28.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.576+0000 7f844a509640 1 -- 172.21.15.6:0/3091434682 shutdown_connections 2024-09-12T03:12:28.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:28.576+0000 7f844a509640 1 -- 172.21.15.6:0/3091434682 wait complete. 2024-09-12T03:12:29.540 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-12T03:12:29.540 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-12T03:12:29.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:29 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3091434682' entity='client.admin' 2024-09-12T03:12:29.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:29 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:29.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:29 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:29.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:29 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi006", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:12:29.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:29 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:29.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:29 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:29.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:29 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:29.998 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:30.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.440+0000 7f823fe61640 1 -- 172.21.15.6:0/3961870021 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8238100540 msgr2=0x7f8238100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:30.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.440+0000 7f823fe61640 1 --2- 172.21.15.6:0/3961870021 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8238100540 0x7f8238100940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8234009880 tx=0x7f823402a5d0 comp rx=0 tx=0).stop 2024-09-12T03:12:30.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.441+0000 7f823fe61640 1 -- 172.21.15.6:0/3961870021 shutdown_connections 2024-09-12T03:12:30.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.441+0000 7f823fe61640 1 --2- 172.21.15.6:0/3961870021 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8238100540 0x7f8238100940 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:30.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.441+0000 7f823fe61640 1 -- 172.21.15.6:0/3961870021 >> 172.21.15.6:0/3961870021 conn(0x7f82380fbcd0 msgr2=0x7f82380fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:30.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.441+0000 7f823fe61640 1 -- 172.21.15.6:0/3961870021 shutdown_connections 2024-09-12T03:12:30.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.441+0000 7f823fe61640 1 -- 172.21.15.6:0/3961870021 wait complete. 2024-09-12T03:12:30.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.441+0000 7f823fe61640 1 Processor -- start 2024-09-12T03:12:30.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.442+0000 7f823fe61640 1 -- start start 2024-09-12T03:12:30.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.442+0000 7f823fe61640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8238100540 0x7f82380fdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:30.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.442+0000 7f823fe61640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f82381065f0 con 0x7f8238100540 2024-09-12T03:12:30.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.442+0000 7f823dbfd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8238100540 0x7f82380fdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:30.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.442+0000 7f823dbfd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8238100540 0x7f82380fdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40182/0 (socket says 172.21.15.6:40182) 2024-09-12T03:12:30.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.442+0000 7f823dbfd640 1 -- 172.21.15.6:0/3008387324 learned_addr learned my addr 172.21.15.6:0/3008387324 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:30.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.442+0000 7f823dbfd640 1 -- 172.21.15.6:0/3008387324 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82380fe0d0 con 0x7f8238100540 2024-09-12T03:12:30.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.442+0000 7f823dbfd640 1 --2- 172.21.15.6:0/3008387324 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8238100540 0x7f82380fdb90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8234004f60 tx=0x7f82340048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:30.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.443+0000 7f822effd640 1 -- 172.21.15.6:0/3008387324 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f823403f070 con 0x7f8238100540 2024-09-12T03:12:30.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.443+0000 7f822effd640 1 -- 172.21.15.6:0/3008387324 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8234004cc0 con 0x7f8238100540 2024-09-12T03:12:30.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.443+0000 7f823fe61640 1 -- 172.21.15.6:0/3008387324 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82380fc2c0 con 0x7f8238100540 2024-09-12T03:12:30.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.443+0000 7f822effd640 1 -- 172.21.15.6:0/3008387324 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8234035030 con 0x7f8238100540 2024-09-12T03:12:30.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.443+0000 7f823fe61640 1 -- 172.21.15.6:0/3008387324 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82380fc800 con 0x7f8238100540 2024-09-12T03:12:30.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.444+0000 7f822effd640 1 -- 172.21.15.6:0/3008387324 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f8234042070 con 0x7f8238100540 2024-09-12T03:12:30.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.444+0000 7f823fe61640 1 -- 172.21.15.6:0/3008387324 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8238106d70 con 0x7f8238100540 2024-09-12T03:12:30.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.444+0000 7f822effd640 1 --2- 172.21.15.6:0/3008387324 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f82240390c0 0x7f822403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:30.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.444+0000 7f822effd640 1 -- 172.21.15.6:0/3008387324 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f823406cd20 con 0x7f8238100540 2024-09-12T03:12:30.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.444+0000 7f823d3fc640 1 --2- 172.21.15.6:0/3008387324 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f82240390c0 0x7f822403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:30.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.445+0000 7f823d3fc640 1 --2- 172.21.15.6:0/3008387324 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f82240390c0 0x7f822403b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f82280098c0 tx=0x7f8228006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:30.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.449+0000 7f822effd640 1 -- 172.21.15.6:0/3008387324 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82340433b0 con 0x7f8238100540 2024-09-12T03:12:30.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.631+0000 7f823fe61640 1 -- 172.21.15.6:0/3008387324 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f8238190320 con 0x7f82240390c0 2024-09-12T03:12:30.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.635+0000 7f822effd640 1 -- 172.21.15.6:0/3008387324 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8238190320 con 0x7f82240390c0 2024-09-12T03:12:30.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.637+0000 7f823fe61640 1 -- 172.21.15.6:0/3008387324 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f82240390c0 msgr2=0x7f822403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:30.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.637+0000 7f823fe61640 1 --2- 172.21.15.6:0/3008387324 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f82240390c0 0x7f822403b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f82280098c0 tx=0x7f8228006db0 comp rx=0 tx=0).stop 2024-09-12T03:12:30.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.637+0000 7f823fe61640 1 -- 172.21.15.6:0/3008387324 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8238100540 msgr2=0x7f82380fdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:30.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.637+0000 7f823fe61640 1 --2- 172.21.15.6:0/3008387324 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8238100540 0x7f82380fdb90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8234004f60 tx=0x7f82340048f0 comp rx=0 tx=0).stop 2024-09-12T03:12:30.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.637+0000 7f823fe61640 1 -- 172.21.15.6:0/3008387324 shutdown_connections 2024-09-12T03:12:30.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.637+0000 7f823fe61640 1 --2- 172.21.15.6:0/3008387324 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f82240390c0 0x7f822403b580 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:30.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.637+0000 7f823fe61640 1 --2- 172.21.15.6:0/3008387324 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8238100540 0x7f82380fdb90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:30.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.637+0000 7f823fe61640 1 -- 172.21.15.6:0/3008387324 >> 172.21.15.6:0/3008387324 conn(0x7f82380fbcd0 msgr2=0x7f8238069b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:30.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.637+0000 7f823fe61640 1 -- 172.21.15.6:0/3008387324 shutdown_connections 2024-09-12T03:12:30.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:30.637+0000 7f823fe61640 1 -- 172.21.15.6:0/3008387324 wait complete. 2024-09-12T03:12:31.542 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi118 2024-09-12T03:12:31.542 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-12T03:12:31.542 DEBUG:teuthology.orchestra.run.smithi118:> dd of=/etc/ceph/ceph.conf 2024-09-12T03:12:31.561 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-12T03:12:31.562 DEBUG:teuthology.orchestra.run.smithi118:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:12:31.621 INFO:tasks.cephadm:Adding host smithi118 to orchestrator... 2024-09-12T03:12:31.622 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph orch host add smithi118 2024-09-12T03:12:31.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:31 smithi006 ceph-mon[30991]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:31.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:31 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:31.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:31 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:31.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:31 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:31.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:31 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:31.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:31 smithi006 ceph-mon[30991]: Updating smithi006:/etc/ceph/ceph.conf 2024-09-12T03:12:31.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:31 smithi006 ceph-mon[30991]: Updating smithi006:/var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:12:31.863 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:31 smithi006 ceph-mon[30991]: Updating smithi006:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:12:31.863 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:31 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:31.863 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:31 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:31.863 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:31 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:32.034 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.433+0000 7f97d9345640 1 -- 172.21.15.6:0/2175019139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97d4102730 msgr2=0x7f97d4102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.433+0000 7f97d9345640 1 --2- 172.21.15.6:0/2175019139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97d4102730 0x7f97d4102b10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009810 tx=0x7f97c402a540 comp rx=0 tx=0).stop 2024-09-12T03:12:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.433+0000 7f97d9345640 1 -- 172.21.15.6:0/2175019139 shutdown_connections 2024-09-12T03:12:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.433+0000 7f97d9345640 1 --2- 172.21.15.6:0/2175019139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97d4102730 0x7f97d4102b10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.433+0000 7f97d9345640 1 -- 172.21.15.6:0/2175019139 >> 172.21.15.6:0/2175019139 conn(0x7f97d4077310 msgr2=0x7f97d40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.433+0000 7f97d9345640 1 -- 172.21.15.6:0/2175019139 shutdown_connections 2024-09-12T03:12:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.433+0000 7f97d9345640 1 -- 172.21.15.6:0/2175019139 wait complete. 2024-09-12T03:12:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.433+0000 7f97d9345640 1 Processor -- start 2024-09-12T03:12:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.434+0000 7f97d9345640 1 -- start start 2024-09-12T03:12:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.434+0000 7f97d9345640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97d4102730 0x7f97d4100390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.434+0000 7f97d9345640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f97d410bc80 con 0x7f97d4102730 2024-09-12T03:12:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.434+0000 7f97d2ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97d4102730 0x7f97d4100390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.434+0000 7f97d2ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97d4102730 0x7f97d4100390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40204/0 (socket says 172.21.15.6:40204) 2024-09-12T03:12:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.434+0000 7f97d2ffd640 1 -- 172.21.15.6:0/3599080617 learned_addr learned my addr 172.21.15.6:0/3599080617 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.434+0000 7f97d2ffd640 1 -- 172.21.15.6:0/3599080617 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d4102280 con 0x7f97d4102730 2024-09-12T03:12:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.435+0000 7f97d2ffd640 1 --2- 172.21.15.6:0/3599080617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97d4102730 0x7f97d4100390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f97c40045e0 tx=0x7f97c4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.435+0000 7f97bbfff640 1 -- 172.21.15.6:0/3599080617 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97c4030e70 con 0x7f97d4102730 2024-09-12T03:12:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.435+0000 7f97bbfff640 1 -- 172.21.15.6:0/3599080617 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97c40318e0 con 0x7f97d4102730 2024-09-12T03:12:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.435+0000 7f97d9345640 1 -- 172.21.15.6:0/3599080617 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d41008d0 con 0x7f97d4102730 2024-09-12T03:12:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.435+0000 7f97bbfff640 1 -- 172.21.15.6:0/3599080617 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97c40049c0 con 0x7f97d4102730 2024-09-12T03:12:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.435+0000 7f97d9345640 1 -- 172.21.15.6:0/3599080617 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d4100d90 con 0x7f97d4102730 2024-09-12T03:12:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.436+0000 7f97bbfff640 1 -- 172.21.15.6:0/3599080617 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f97c4039040 con 0x7f97d4102730 2024-09-12T03:12:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.436+0000 7f97bbfff640 1 --2- 172.21.15.6:0/3599080617 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f97bc039060 0x7f97bc03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.436+0000 7f97bbfff640 1 -- 172.21.15.6:0/3599080617 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f97c406d2e0 con 0x7f97d4102730 2024-09-12T03:12:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.436+0000 7f97d9345640 1 -- 172.21.15.6:0/3599080617 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97d4069f90 con 0x7f97d4102730 2024-09-12T03:12:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.436+0000 7f97d27fc640 1 --2- 172.21.15.6:0/3599080617 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f97bc039060 0x7f97bc03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:32.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.436+0000 7f97d27fc640 1 --2- 172.21.15.6:0/3599080617 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f97bc039060 0x7f97bc03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f97c800ad30 tx=0x7f97c80093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:32.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.441+0000 7f97bbfff640 1 -- 172.21.15.6:0/3599080617 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f97c4040030 con 0x7f97d4102730 2024-09-12T03:12:32.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:32.619+0000 7f97d9345640 1 -- 172.21.15.6:0/3599080617 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi118", "target": ["mon-mgr", ""]}) -- 0x7f97d41018c0 con 0x7f97bc039060 2024-09-12T03:12:33.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:32 smithi006 ceph-mon[30991]: Updating smithi006:/var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:12:34.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:33 smithi006 ceph-mon[30991]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi118", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:35.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:34 smithi006 ceph-mon[30991]: Deploying cephadm binary to smithi118 2024-09-12T03:12:36.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:36.565+0000 7f97bbfff640 1 -- 172.21.15.6:0/3599080617 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f97d41018c0 con 0x7f97bc039060 2024-09-12T03:12:36.565 INFO:teuthology.orchestra.run.smithi006.stdout:Added host 'smithi118' with addr '172.21.15.118' 2024-09-12T03:12:36.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:36.567+0000 7f97d9345640 1 -- 172.21.15.6:0/3599080617 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f97bc039060 msgr2=0x7f97bc03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:36.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:36.567+0000 7f97d9345640 1 --2- 172.21.15.6:0/3599080617 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f97bc039060 0x7f97bc03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f97c800ad30 tx=0x7f97c80093f0 comp rx=0 tx=0).stop 2024-09-12T03:12:36.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:36.567+0000 7f97d9345640 1 -- 172.21.15.6:0/3599080617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97d4102730 msgr2=0x7f97d4100390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:36.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:36.567+0000 7f97d9345640 1 --2- 172.21.15.6:0/3599080617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97d4102730 0x7f97d4100390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f97c40045e0 tx=0x7f97c4004610 comp rx=0 tx=0).stop 2024-09-12T03:12:36.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:36.567+0000 7f97d9345640 1 -- 172.21.15.6:0/3599080617 shutdown_connections 2024-09-12T03:12:36.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:36.567+0000 7f97d9345640 1 --2- 172.21.15.6:0/3599080617 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f97bc039060 0x7f97bc03b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:36.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:36.567+0000 7f97d9345640 1 --2- 172.21.15.6:0/3599080617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97d4102730 0x7f97d4100390 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:36.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:36.567+0000 7f97d9345640 1 -- 172.21.15.6:0/3599080617 >> 172.21.15.6:0/3599080617 conn(0x7f97d4077310 msgr2=0x7f97d41054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:36.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:36.567+0000 7f97d9345640 1 -- 172.21.15.6:0/3599080617 shutdown_connections 2024-09-12T03:12:36.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:36.567+0000 7f97d9345640 1 -- 172.21.15.6:0/3599080617 wait complete. 2024-09-12T03:12:37.204 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T03:12:37.457 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:37.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:37 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:37.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:37 smithi006 ceph-mon[30991]: Added host smithi118 2024-09-12T03:12:37.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:37 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:37.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:37 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:37.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:37 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:37.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.871+0000 7f484e56a640 1 -- 172.21.15.6:0/1801206752 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4848102730 msgr2=0x7f4848102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:37.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.871+0000 7f484e56a640 1 --2- 172.21.15.6:0/1801206752 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4848102730 0x7f4848102b10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4838009880 tx=0x7f483802a670 comp rx=0 tx=0).stop 2024-09-12T03:12:37.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.871+0000 7f484e56a640 1 -- 172.21.15.6:0/1801206752 shutdown_connections 2024-09-12T03:12:37.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.871+0000 7f484e56a640 1 --2- 172.21.15.6:0/1801206752 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4848102730 0x7f4848102b10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:37.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.871+0000 7f484e56a640 1 -- 172.21.15.6:0/1801206752 >> 172.21.15.6:0/1801206752 conn(0x7f48480fc130 msgr2=0x7f48480fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:37.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.871+0000 7f484e56a640 1 -- 172.21.15.6:0/1801206752 shutdown_connections 2024-09-12T03:12:37.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.871+0000 7f484e56a640 1 -- 172.21.15.6:0/1801206752 wait complete. 2024-09-12T03:12:37.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.872+0000 7f484e56a640 1 Processor -- start 2024-09-12T03:12:37.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.872+0000 7f484e56a640 1 -- start start 2024-09-12T03:12:37.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.873+0000 7f484e56a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4848102730 0x7f4848197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:37.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.873+0000 7f484e56a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f484810bca0 con 0x7f4848102730 2024-09-12T03:12:37.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.873+0000 7f4847fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4848102730 0x7f4848197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:37.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.873+0000 7f4847fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4848102730 0x7f4848197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40218/0 (socket says 172.21.15.6:40218) 2024-09-12T03:12:37.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.873+0000 7f4847fff640 1 -- 172.21.15.6:0/3449620670 learned_addr learned my addr 172.21.15.6:0/3449620670 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:37.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.873+0000 7f4847fff640 1 -- 172.21.15.6:0/3449620670 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4848197790 con 0x7f4848102730 2024-09-12T03:12:37.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.873+0000 7f4847fff640 1 --2- 172.21.15.6:0/3449620670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4848102730 0x7f4848197250 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4838009230 tx=0x7f4838003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:37.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.873+0000 7f48457fa640 1 -- 172.21.15.6:0/3449620670 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f483803f070 con 0x7f4848102730 2024-09-12T03:12:37.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.873+0000 7f48457fa640 1 -- 172.21.15.6:0/3449620670 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f483803bc80 con 0x7f4848102730 2024-09-12T03:12:37.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.873+0000 7f484e56a640 1 -- 172.21.15.6:0/3449620670 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4848197a20 con 0x7f4848102730 2024-09-12T03:12:37.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.873+0000 7f48457fa640 1 -- 172.21.15.6:0/3449620670 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4838035030 con 0x7f4848102730 2024-09-12T03:12:37.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.873+0000 7f484e56a640 1 -- 172.21.15.6:0/3449620670 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4848197e80 con 0x7f4848102730 2024-09-12T03:12:37.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.874+0000 7f48457fa640 1 -- 172.21.15.6:0/3449620670 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f4838042070 con 0x7f4848102730 2024-09-12T03:12:37.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.874+0000 7f48457fa640 1 --2- 172.21.15.6:0/3449620670 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f48300390c0 0x7f483003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:37.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.874+0000 7f484e56a640 1 -- 172.21.15.6:0/3449620670 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f484806a7b0 con 0x7f4848102730 2024-09-12T03:12:37.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.874+0000 7f48457fa640 1 -- 172.21.15.6:0/3449620670 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f483806cf00 con 0x7f4848102730 2024-09-12T03:12:37.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.875+0000 7f48477fe640 1 --2- 172.21.15.6:0/3449620670 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f48300390c0 0x7f483003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:37.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.875+0000 7f48477fe640 1 --2- 172.21.15.6:0/3449620670 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f48300390c0 0x7f483003b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f483c0098c0 tx=0x7f483c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:37.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:37.880+0000 7f48457fa640 1 -- 172.21.15.6:0/3449620670 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48380351d0 con 0x7f4848102730 2024-09-12T03:12:38.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:38.061+0000 7f484e56a640 1 -- 172.21.15.6:0/3449620670 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f48481985b0 con 0x7f48300390c0 2024-09-12T03:12:38.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:38.063+0000 7f48457fa640 1 -- 172.21.15.6:0/3449620670 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f48481985b0 con 0x7f48300390c0 2024-09-12T03:12:38.063 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:12:38.063 INFO:teuthology.orchestra.run.smithi006.stdout:[{"addr": "172.21.15.6", "hostname": "smithi006", "labels": [], "status": ""}, {"addr": "172.21.15.118", "hostname": "smithi118", "labels": [], "status": ""}] 2024-09-12T03:12:38.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:38.065+0000 7f484e56a640 1 -- 172.21.15.6:0/3449620670 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f48300390c0 msgr2=0x7f483003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:38.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:38.065+0000 7f484e56a640 1 --2- 172.21.15.6:0/3449620670 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f48300390c0 0x7f483003b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f483c0098c0 tx=0x7f483c006db0 comp rx=0 tx=0).stop 2024-09-12T03:12:38.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:38.065+0000 7f484e56a640 1 -- 172.21.15.6:0/3449620670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4848102730 msgr2=0x7f4848197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:38.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:38.065+0000 7f484e56a640 1 --2- 172.21.15.6:0/3449620670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4848102730 0x7f4848197250 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4838009230 tx=0x7f4838003d40 comp rx=0 tx=0).stop 2024-09-12T03:12:38.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:38.065+0000 7f484e56a640 1 -- 172.21.15.6:0/3449620670 shutdown_connections 2024-09-12T03:12:38.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:38.065+0000 7f484e56a640 1 --2- 172.21.15.6:0/3449620670 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f48300390c0 0x7f483003b580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:38.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:38.065+0000 7f484e56a640 1 --2- 172.21.15.6:0/3449620670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4848102730 0x7f4848197250 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:38.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:38.065+0000 7f484e56a640 1 -- 172.21.15.6:0/3449620670 >> 172.21.15.6:0/3449620670 conn(0x7f48480fc130 msgr2=0x7f48481054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:38.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:38.065+0000 7f484e56a640 1 -- 172.21.15.6:0/3449620670 shutdown_connections 2024-09-12T03:12:38.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:38.065+0000 7f484e56a640 1 -- 172.21.15.6:0/3449620670 wait complete. 2024-09-12T03:12:38.639 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi161 2024-09-12T03:12:38.639 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-12T03:12:38.639 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.conf 2024-09-12T03:12:38.658 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-12T03:12:38.658 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:12:38.715 INFO:tasks.cephadm:Adding host smithi161 to orchestrator... 2024-09-12T03:12:38.716 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph orch host add smithi161 2024-09-12T03:12:38.968 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:39.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:38 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:39.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:38 smithi006 ceph-mon[30991]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:12:39.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.402+0000 7f9acb7d6640 1 -- 172.21.15.6:0/1393951497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ac4106560 msgr2=0x7f9ac4106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:39.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.402+0000 7f9acb7d6640 1 --2- 172.21.15.6:0/1393951497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ac4106560 0x7f9ac4106940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009880 tx=0x7f9ac002a5d0 comp rx=0 tx=0).stop 2024-09-12T03:12:39.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.402+0000 7f9acb7d6640 1 -- 172.21.15.6:0/1393951497 shutdown_connections 2024-09-12T03:12:39.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.402+0000 7f9acb7d6640 1 --2- 172.21.15.6:0/1393951497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ac4106560 0x7f9ac4106940 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:39.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.402+0000 7f9acb7d6640 1 -- 172.21.15.6:0/1393951497 >> 172.21.15.6:0/1393951497 conn(0x7f9ac40fc1b0 msgr2=0x7f9ac40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.402+0000 7f9acb7d6640 1 -- 172.21.15.6:0/1393951497 shutdown_connections 2024-09-12T03:12:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.402+0000 7f9acb7d6640 1 -- 172.21.15.6:0/1393951497 wait complete. 2024-09-12T03:12:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.403+0000 7f9acb7d6640 1 Processor -- start 2024-09-12T03:12:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.403+0000 7f9acb7d6640 1 -- start start 2024-09-12T03:12:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.403+0000 7f9acb7d6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ac4106560 0x7f9ac41970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.403+0000 7f9acb7d6640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac410ab90 con 0x7f9ac4106560 2024-09-12T03:12:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.403+0000 7f9ac9572640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ac4106560 0x7f9ac41970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.403+0000 7f9ac9572640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ac4106560 0x7f9ac41970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37432/0 (socket says 172.21.15.6:37432) 2024-09-12T03:12:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.403+0000 7f9ac9572640 1 -- 172.21.15.6:0/2754809384 learned_addr learned my addr 172.21.15.6:0/2754809384 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.404+0000 7f9ac9572640 1 -- 172.21.15.6:0/2754809384 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac4197620 con 0x7f9ac4106560 2024-09-12T03:12:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.404+0000 7f9ac9572640 1 --2- 172.21.15.6:0/2754809384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ac4106560 0x7f9ac41970e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009230 tx=0x7f9ac0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.404+0000 7f9aba7fc640 1 -- 172.21.15.6:0/2754809384 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ac003f070 con 0x7f9ac4106560 2024-09-12T03:12:39.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.404+0000 7f9aba7fc640 1 -- 172.21.15.6:0/2754809384 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ac003bc80 con 0x7f9ac4106560 2024-09-12T03:12:39.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.404+0000 7f9acb7d6640 1 -- 172.21.15.6:0/2754809384 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac41978b0 con 0x7f9ac4106560 2024-09-12T03:12:39.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.404+0000 7f9aba7fc640 1 -- 172.21.15.6:0/2754809384 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ac0035030 con 0x7f9ac4106560 2024-09-12T03:12:39.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.404+0000 7f9acb7d6640 1 -- 172.21.15.6:0/2754809384 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac419b9b0 con 0x7f9ac4106560 2024-09-12T03:12:39.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.405+0000 7f9aba7fc640 1 -- 172.21.15.6:0/2754809384 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f9ac0042070 con 0x7f9ac4106560 2024-09-12T03:12:39.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.405+0000 7f9acb7d6640 1 -- 172.21.15.6:0/2754809384 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ac4101bd0 con 0x7f9ac4106560 2024-09-12T03:12:39.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.405+0000 7f9aba7fc640 1 --2- 172.21.15.6:0/2754809384 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9ab00390c0 0x7f9ab003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:39.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.405+0000 7f9aba7fc640 1 -- 172.21.15.6:0/2754809384 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9ac006cd10 con 0x7f9ac4106560 2024-09-12T03:12:39.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.405+0000 7f9ac8d71640 1 --2- 172.21.15.6:0/2754809384 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9ab00390c0 0x7f9ab003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:39.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.406+0000 7f9ac8d71640 1 --2- 172.21.15.6:0/2754809384 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9ab00390c0 0x7f9ab003b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40098c0 tx=0x7f9ab4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:39.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.410+0000 7f9aba7fc640 1 -- 172.21.15.6:0/2754809384 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ac00314c0 con 0x7f9ac4106560 2024-09-12T03:12:39.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:39.590+0000 7f9acb7d6640 1 -- 172.21.15.6:0/2754809384 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}) -- 0x7f9ac4198440 con 0x7f9ab00390c0 2024-09-12T03:12:40.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:40 smithi006 ceph-mon[30991]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:12:40.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:40 smithi006 ceph-mon[30991]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:41.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:41.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:41.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:41.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:41.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi118", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:12:41.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:41.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:41.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:41 smithi006 ceph-mon[30991]: Updating smithi118:/etc/ceph/ceph.conf 2024-09-12T03:12:41.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:41 smithi006 ceph-mon[30991]: Updating smithi118:/var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:12:41.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:41 smithi006 ceph-mon[30991]: Deploying cephadm binary to smithi161 2024-09-12T03:12:41.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:41 smithi006 ceph-mon[30991]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:12:41.863 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:41.863 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:41.863 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:42.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:42 smithi006 ceph-mon[30991]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:12:42.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:42 smithi006 ceph-mon[30991]: Updating smithi118:/var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:12:43.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:43.480+0000 7f9aba7fc640 1 -- 172.21.15.6:0/2754809384 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f9ac4198440 con 0x7f9ab00390c0 2024-09-12T03:12:43.481 INFO:teuthology.orchestra.run.smithi006.stdout:Added host 'smithi161' with addr '172.21.15.161' 2024-09-12T03:12:43.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:43.483+0000 7f9acb7d6640 1 -- 172.21.15.6:0/2754809384 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9ab00390c0 msgr2=0x7f9ab003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:43.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:43.483+0000 7f9acb7d6640 1 --2- 172.21.15.6:0/2754809384 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9ab00390c0 0x7f9ab003b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40098c0 tx=0x7f9ab4006db0 comp rx=0 tx=0).stop 2024-09-12T03:12:43.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:43.483+0000 7f9acb7d6640 1 -- 172.21.15.6:0/2754809384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ac4106560 msgr2=0x7f9ac41970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:43.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:43.483+0000 7f9acb7d6640 1 --2- 172.21.15.6:0/2754809384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ac4106560 0x7f9ac41970e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009230 tx=0x7f9ac0003d40 comp rx=0 tx=0).stop 2024-09-12T03:12:43.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:43.483+0000 7f9acb7d6640 1 -- 172.21.15.6:0/2754809384 shutdown_connections 2024-09-12T03:12:43.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:43.483+0000 7f9acb7d6640 1 --2- 172.21.15.6:0/2754809384 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9ab00390c0 0x7f9ab003b580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:43.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:43.483+0000 7f9acb7d6640 1 --2- 172.21.15.6:0/2754809384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ac4106560 0x7f9ac41970e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:43.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:43.483+0000 7f9acb7d6640 1 -- 172.21.15.6:0/2754809384 >> 172.21.15.6:0/2754809384 conn(0x7f9ac40fc1b0 msgr2=0x7f9ac4069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:43.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:43.483+0000 7f9acb7d6640 1 -- 172.21.15.6:0/2754809384 shutdown_connections 2024-09-12T03:12:43.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:43.483+0000 7f9acb7d6640 1 -- 172.21.15.6:0/2754809384 wait complete. 2024-09-12T03:12:44.045 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T03:12:44.295 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:44.677 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:44 smithi006 ceph-mon[30991]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:12:44.677 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:44 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:44.677 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:44 smithi006 ceph-mon[30991]: Added host smithi161 2024-09-12T03:12:44.677 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:44 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:44.677 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:44 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:44.677 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:44 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:44.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.760+0000 7fae795a8640 1 -- 172.21.15.6:0/2804169799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae74102730 msgr2=0x7fae74102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:44.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.760+0000 7fae795a8640 1 --2- 172.21.15.6:0/2804169799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae74102730 0x7fae74102b10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fae64009810 tx=0x7fae6402a540 comp rx=0 tx=0).stop 2024-09-12T03:12:44.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.760+0000 7fae795a8640 1 -- 172.21.15.6:0/2804169799 shutdown_connections 2024-09-12T03:12:44.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.760+0000 7fae795a8640 1 --2- 172.21.15.6:0/2804169799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae74102730 0x7fae74102b10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:44.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.760+0000 7fae795a8640 1 -- 172.21.15.6:0/2804169799 >> 172.21.15.6:0/2804169799 conn(0x7fae74077310 msgr2=0x7fae740fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:44.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.760+0000 7fae795a8640 1 -- 172.21.15.6:0/2804169799 shutdown_connections 2024-09-12T03:12:44.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.761+0000 7fae795a8640 1 -- 172.21.15.6:0/2804169799 wait complete. 2024-09-12T03:12:44.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.761+0000 7fae795a8640 1 Processor -- start 2024-09-12T03:12:44.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.761+0000 7fae795a8640 1 -- start start 2024-09-12T03:12:44.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.761+0000 7fae795a8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae74102730 0x7fae74100390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:44.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.761+0000 7fae795a8640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fae7410bc80 con 0x7fae74102730 2024-09-12T03:12:44.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.762+0000 7fae72ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae74102730 0x7fae74100390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:44.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.762+0000 7fae72ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae74102730 0x7fae74100390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37458/0 (socket says 172.21.15.6:37458) 2024-09-12T03:12:44.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.762+0000 7fae72ffd640 1 -- 172.21.15.6:0/324952982 learned_addr learned my addr 172.21.15.6:0/324952982 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:44.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.762+0000 7fae72ffd640 1 -- 172.21.15.6:0/324952982 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae74102280 con 0x7fae74102730 2024-09-12T03:12:44.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.762+0000 7fae72ffd640 1 --2- 172.21.15.6:0/324952982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae74102730 0x7fae74100390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fae64009930 tx=0x7fae64004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:44.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.762+0000 7fae5bfff640 1 -- 172.21.15.6:0/324952982 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae64031400 con 0x7fae74102730 2024-09-12T03:12:44.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.762+0000 7fae5bfff640 1 -- 172.21.15.6:0/324952982 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae640035f0 con 0x7fae74102730 2024-09-12T03:12:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.762+0000 7fae795a8640 1 -- 172.21.15.6:0/324952982 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae741008d0 con 0x7fae74102730 2024-09-12T03:12:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.762+0000 7fae5bfff640 1 -- 172.21.15.6:0/324952982 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae6403c440 con 0x7fae74102730 2024-09-12T03:12:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.762+0000 7fae795a8640 1 -- 172.21.15.6:0/324952982 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae74100d90 con 0x7fae74102730 2024-09-12T03:12:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.763+0000 7fae5bfff640 1 -- 172.21.15.6:0/324952982 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fae64039040 con 0x7fae74102730 2024-09-12T03:12:44.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.763+0000 7fae795a8640 1 -- 172.21.15.6:0/324952982 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae74069f90 con 0x7fae74102730 2024-09-12T03:12:44.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.763+0000 7fae5bfff640 1 --2- 172.21.15.6:0/324952982 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fae5c039060 0x7fae5c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:44.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.763+0000 7fae5bfff640 1 -- 172.21.15.6:0/324952982 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fae6406d830 con 0x7fae74102730 2024-09-12T03:12:44.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.763+0000 7fae727fc640 1 --2- 172.21.15.6:0/324952982 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fae5c039060 0x7fae5c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:44.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.764+0000 7fae727fc640 1 --2- 172.21.15.6:0/324952982 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fae5c039060 0x7fae5c03b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fae680098c0 tx=0x7fae68006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:44.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.768+0000 7fae5bfff640 1 -- 172.21.15.6:0/324952982 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae64040030 con 0x7fae74102730 2024-09-12T03:12:44.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.950+0000 7fae795a8640 1 -- 172.21.15.6:0/324952982 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fae741018c0 con 0x7fae5c039060 2024-09-12T03:12:44.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.951+0000 7fae5bfff640 1 -- 172.21.15.6:0/324952982 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fae741018c0 con 0x7fae5c039060 2024-09-12T03:12:44.951 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:12:44.951 INFO:teuthology.orchestra.run.smithi006.stdout:[{"addr": "172.21.15.6", "hostname": "smithi006", "labels": [], "status": ""}, {"addr": "172.21.15.118", "hostname": "smithi118", "labels": [], "status": ""}, {"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}] 2024-09-12T03:12:44.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.953+0000 7fae795a8640 1 -- 172.21.15.6:0/324952982 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fae5c039060 msgr2=0x7fae5c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:44.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.953+0000 7fae795a8640 1 --2- 172.21.15.6:0/324952982 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fae5c039060 0x7fae5c03b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fae680098c0 tx=0x7fae68006db0 comp rx=0 tx=0).stop 2024-09-12T03:12:44.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.953+0000 7fae795a8640 1 -- 172.21.15.6:0/324952982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae74102730 msgr2=0x7fae74100390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:44.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.953+0000 7fae795a8640 1 --2- 172.21.15.6:0/324952982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae74102730 0x7fae74100390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fae64009930 tx=0x7fae64004570 comp rx=0 tx=0).stop 2024-09-12T03:12:44.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.953+0000 7fae795a8640 1 -- 172.21.15.6:0/324952982 shutdown_connections 2024-09-12T03:12:44.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.953+0000 7fae795a8640 1 --2- 172.21.15.6:0/324952982 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fae5c039060 0x7fae5c03b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:44.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.953+0000 7fae795a8640 1 --2- 172.21.15.6:0/324952982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae74102730 0x7fae74100390 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:44.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.953+0000 7fae795a8640 1 -- 172.21.15.6:0/324952982 >> 172.21.15.6:0/324952982 conn(0x7fae74077310 msgr2=0x7fae741034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:44.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.953+0000 7fae795a8640 1 -- 172.21.15.6:0/324952982 shutdown_connections 2024-09-12T03:12:44.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:44.953+0000 7fae795a8640 1 -- 172.21.15.6:0/324952982 wait complete. 2024-09-12T03:12:45.517 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-12T03:12:45.518 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-12T03:12:45.765 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:46.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:45 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:46.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:45 smithi006 ceph-mon[30991]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:12:46.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.211+0000 7f3fb4aa8640 1 -- 172.21.15.6:0/1448300510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3fb0107660 msgr2=0x7f3fb0107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:46.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.211+0000 7f3fb4aa8640 1 --2- 172.21.15.6:0/1448300510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3fb0107660 0x7f3fb0107a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009880 tx=0x7f3fa002a670 comp rx=0 tx=0).stop 2024-09-12T03:12:46.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.211+0000 7f3fb4aa8640 1 -- 172.21.15.6:0/1448300510 shutdown_connections 2024-09-12T03:12:46.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.211+0000 7f3fb4aa8640 1 --2- 172.21.15.6:0/1448300510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3fb0107660 0x7f3fb0107a40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.211+0000 7f3fb4aa8640 1 -- 172.21.15.6:0/1448300510 >> 172.21.15.6:0/1448300510 conn(0x7f3fb00fb930 msgr2=0x7f3fb00fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:46.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.211+0000 7f3fb4aa8640 1 -- 172.21.15.6:0/1448300510 shutdown_connections 2024-09-12T03:12:46.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.211+0000 7f3fb4aa8640 1 -- 172.21.15.6:0/1448300510 wait complete. 2024-09-12T03:12:46.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.212+0000 7f3fb4aa8640 1 Processor -- start 2024-09-12T03:12:46.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.212+0000 7f3fb4aa8640 1 -- start start 2024-09-12T03:12:46.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.212+0000 7f3fb4aa8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3fb0107660 0x7f3fb0197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:46.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.212+0000 7f3fb4aa8640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb010bc50 con 0x7f3fb0107660 2024-09-12T03:12:46.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.212+0000 7f3fae59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3fb0107660 0x7f3fb0197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:46.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.212+0000 7f3fae59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3fb0107660 0x7f3fb0197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37478/0 (socket says 172.21.15.6:37478) 2024-09-12T03:12:46.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.212+0000 7f3fae59c640 1 -- 172.21.15.6:0/3732703727 learned_addr learned my addr 172.21.15.6:0/3732703727 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:12:46.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.212+0000 7f3fae59c640 1 -- 172.21.15.6:0/3732703727 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb0197650 con 0x7f3fb0107660 2024-09-12T03:12:46.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.212+0000 7f3fae59c640 1 --2- 172.21.15.6:0/3732703727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3fb0107660 0x7f3fb0197110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009230 tx=0x7f3fa0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:46.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.212+0000 7f3fa77fe640 1 -- 172.21.15.6:0/3732703727 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fa003f070 con 0x7f3fb0107660 2024-09-12T03:12:46.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.212+0000 7f3fa77fe640 1 -- 172.21.15.6:0/3732703727 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fa003bc80 con 0x7f3fb0107660 2024-09-12T03:12:46.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.213+0000 7f3fb4aa8640 1 -- 172.21.15.6:0/3732703727 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb0197940 con 0x7f3fb0107660 2024-09-12T03:12:46.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.213+0000 7f3fa77fe640 1 -- 172.21.15.6:0/3732703727 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fa0035030 con 0x7f3fb0107660 2024-09-12T03:12:46.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.213+0000 7f3fb4aa8640 1 -- 172.21.15.6:0/3732703727 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb019b9e0 con 0x7f3fb0107660 2024-09-12T03:12:46.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.213+0000 7f3fa77fe640 1 -- 172.21.15.6:0/3732703727 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f3fa0042070 con 0x7f3fb0107660 2024-09-12T03:12:46.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.214+0000 7f3fb4aa8640 1 -- 172.21.15.6:0/3732703727 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fb0102d00 con 0x7f3fb0107660 2024-09-12T03:12:46.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.214+0000 7f3fa77fe640 1 --2- 172.21.15.6:0/3732703727 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3f9c0390c0 0x7f3f9c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:46.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.214+0000 7f3fa77fe640 1 -- 172.21.15.6:0/3732703727 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3fa006cca0 con 0x7f3fb0107660 2024-09-12T03:12:46.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.214+0000 7f3fa7fff640 1 --2- 172.21.15.6:0/3732703727 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3f9c0390c0 0x7f3f9c03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:46.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.214+0000 7f3fa7fff640 1 --2- 172.21.15.6:0/3732703727 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3f9c0390c0 0x7f3f9c03b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3f980098c0 tx=0x7f3f98006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:46.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.219+0000 7f3fa77fe640 1 -- 172.21.15.6:0/3732703727 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3fa00372d0 con 0x7f3fb0107660 2024-09-12T03:12:46.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.397+0000 7f3fb4aa8640 1 -- 172.21.15.6:0/3732703727 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3fb0107ac0 con 0x7f3fb0107660 2024-09-12T03:12:46.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.828+0000 7f3fa77fe640 1 -- 172.21.15.6:0/3732703727 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f3fa0031e20 con 0x7f3fb0107660 2024-09-12T03:12:46.828 INFO:teuthology.orchestra.run.smithi006.stderr:adjusted tunables profile to default 2024-09-12T03:12:46.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.830+0000 7f3fb4aa8640 1 -- 172.21.15.6:0/3732703727 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3f9c0390c0 msgr2=0x7f3f9c03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:46.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.830+0000 7f3fb4aa8640 1 --2- 172.21.15.6:0/3732703727 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3f9c0390c0 0x7f3f9c03b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3f980098c0 tx=0x7f3f98006db0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.830+0000 7f3fb4aa8640 1 -- 172.21.15.6:0/3732703727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3fb0107660 msgr2=0x7f3fb0197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:46.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.830+0000 7f3fb4aa8640 1 --2- 172.21.15.6:0/3732703727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3fb0107660 0x7f3fb0197110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009230 tx=0x7f3fa0003d40 comp rx=0 tx=0).stop 2024-09-12T03:12:46.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.830+0000 7f3fb4aa8640 1 -- 172.21.15.6:0/3732703727 shutdown_connections 2024-09-12T03:12:46.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.830+0000 7f3fb4aa8640 1 --2- 172.21.15.6:0/3732703727 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3f9c0390c0 0x7f3f9c03b580 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.830+0000 7f3fb4aa8640 1 --2- 172.21.15.6:0/3732703727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3fb0107660 0x7f3fb0197110 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.830+0000 7f3fb4aa8640 1 -- 172.21.15.6:0/3732703727 >> 172.21.15.6:0/3732703727 conn(0x7f3fb00fb930 msgr2=0x7f3fb00fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:46.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.830+0000 7f3fb4aa8640 1 -- 172.21.15.6:0/3732703727 shutdown_connections 2024-09-12T03:12:46.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:12:46.830+0000 7f3fb4aa8640 1 -- 172.21.15.6:0/3732703727 wait complete. 2024-09-12T03:12:47.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:46 smithi006 ceph-mon[30991]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:12:47.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:46 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3732703727' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T03:12:47.464 INFO:tasks.cephadm:Adding mon.a on smithi006 2024-09-12T03:12:47.464 INFO:tasks.cephadm:Adding mon.b on smithi118 2024-09-12T03:12:47.464 INFO:tasks.cephadm:Adding mon.c on smithi161 2024-09-12T03:12:47.465 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi006:172.21.15.6=a;smithi118:172.21.15.118=b;smithi161:172.21.15.161=c' 2024-09-12T03:12:47.777 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-12T03:12:47.823 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-12T03:12:48.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:47 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3732703727' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T03:12:48.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:47 smithi006 ceph-mon[30991]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:12:48.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:47 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:48.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:47 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:48.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:47 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:48.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:47 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:48.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:47 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:12:48.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:47 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:48.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:47 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:48.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.333+0000 7f7efa222640 1 -- 172.21.15.161:0/3816117377 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ef40ff8b0 msgr2=0x7f7ef40ffcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:48.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.333+0000 7f7efa222640 1 --2- 172.21.15.161:0/3816117377 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ef40ff8b0 0x7f7ef40ffcb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009880 tx=0x7f7ee402a5d0 comp rx=0 tx=0).stop 2024-09-12T03:12:48.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.333+0000 7f7efa222640 1 -- 172.21.15.161:0/3816117377 shutdown_connections 2024-09-12T03:12:48.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.333+0000 7f7efa222640 1 --2- 172.21.15.161:0/3816117377 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ef40ff8b0 0x7f7ef40ffcb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:48.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.333+0000 7f7efa222640 1 -- 172.21.15.161:0/3816117377 >> 172.21.15.161:0/3816117377 conn(0x7f7ef40fb400 msgr2=0x7f7ef40fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:48.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.333+0000 7f7efa222640 1 -- 172.21.15.161:0/3816117377 shutdown_connections 2024-09-12T03:12:48.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.333+0000 7f7efa222640 1 -- 172.21.15.161:0/3816117377 wait complete. 2024-09-12T03:12:48.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.334+0000 7f7efa222640 1 Processor -- start 2024-09-12T03:12:48.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.334+0000 7f7efa222640 1 -- start start 2024-09-12T03:12:48.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.334+0000 7f7efa222640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ef40ff8b0 0x7f7ef4102110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:48.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.334+0000 7f7efa222640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef4107700 con 0x7f7ef40ff8b0 2024-09-12T03:12:48.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.334+0000 7f7ef37fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ef40ff8b0 0x7f7ef4102110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:48.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.335+0000 7f7ef37fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ef40ff8b0 0x7f7ef4102110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.161:36250/0 (socket says 172.21.15.161:36250) 2024-09-12T03:12:48.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.335+0000 7f7ef37fe640 1 -- 172.21.15.161:0/2559063129 learned_addr learned my addr 172.21.15.161:0/2559063129 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-12T03:12:48.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.335+0000 7f7ef37fe640 1 -- 172.21.15.161:0/2559063129 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef4102650 con 0x7f7ef40ff8b0 2024-09-12T03:12:48.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.335+0000 7f7ef37fe640 1 --2- 172.21.15.161:0/2559063129 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ef40ff8b0 0x7f7ef4102110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4004f60 tx=0x7f7ee40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:48.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.335+0000 7f7ef0ff9640 1 -- 172.21.15.161:0/2559063129 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ee403f070 con 0x7f7ef40ff8b0 2024-09-12T03:12:48.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.335+0000 7f7ef0ff9640 1 -- 172.21.15.161:0/2559063129 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ee4004d40 con 0x7f7ef40ff8b0 2024-09-12T03:12:48.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.335+0000 7f7efa222640 1 -- 172.21.15.161:0/2559063129 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef4100840 con 0x7f7ef40ff8b0 2024-09-12T03:12:48.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.335+0000 7f7ef0ff9640 1 -- 172.21.15.161:0/2559063129 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ee4035030 con 0x7f7ef40ff8b0 2024-09-12T03:12:48.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.335+0000 7f7efa222640 1 -- 172.21.15.161:0/2559063129 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef4100d80 con 0x7f7ef40ff8b0 2024-09-12T03:12:48.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.336+0000 7f7ef0ff9640 1 -- 172.21.15.161:0/2559063129 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f7ee4042070 con 0x7f7ef40ff8b0 2024-09-12T03:12:48.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.336+0000 7f7ef0ff9640 1 --2- 172.21.15.161:0/2559063129 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7edc0390c0 0x7f7edc03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:48.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.336+0000 7f7efa222640 1 -- 172.21.15.161:0/2559063129 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef4107e80 con 0x7f7ef40ff8b0 2024-09-12T03:12:48.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.337+0000 7f7ef0ff9640 1 -- 172.21.15.161:0/2559063129 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7ee406cdf0 con 0x7f7ef40ff8b0 2024-09-12T03:12:48.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.337+0000 7f7ef2ffd640 1 --2- 172.21.15.161:0/2559063129 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7edc0390c0 0x7f7edc03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:48.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.337+0000 7f7ef2ffd640 1 --2- 172.21.15.161:0/2559063129 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7edc0390c0 0x7f7edc03b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80098c0 tx=0x7f7ee8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:48.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.342+0000 7f7ef0ff9640 1 -- 172.21.15.161:0/2559063129 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7ee4039350 con 0x7f7ef40ff8b0 2024-09-12T03:12:48.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.528+0000 7f7efa222640 1 -- 172.21.15.161:0/2559063129 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi006:172.21.15.6=a;smithi118:172.21.15.118=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}) -- 0x7f7ef4190250 con 0x7f7edc0390c0 2024-09-12T03:12:48.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.536+0000 7f7ef0ff9640 1 -- 172.21.15.161:0/2559063129 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7ef4190250 con 0x7f7edc0390c0 2024-09-12T03:12:48.537 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mon update... 2024-09-12T03:12:48.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.538+0000 7f7efa222640 1 -- 172.21.15.161:0/2559063129 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7edc0390c0 msgr2=0x7f7edc03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:48.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.538+0000 7f7efa222640 1 --2- 172.21.15.161:0/2559063129 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7edc0390c0 0x7f7edc03b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80098c0 tx=0x7f7ee8006db0 comp rx=0 tx=0).stop 2024-09-12T03:12:48.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.538+0000 7f7efa222640 1 -- 172.21.15.161:0/2559063129 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ef40ff8b0 msgr2=0x7f7ef4102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:48.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.538+0000 7f7efa222640 1 --2- 172.21.15.161:0/2559063129 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ef40ff8b0 0x7f7ef4102110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4004f60 tx=0x7f7ee40048f0 comp rx=0 tx=0).stop 2024-09-12T03:12:48.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.538+0000 7f7efa222640 1 -- 172.21.15.161:0/2559063129 shutdown_connections 2024-09-12T03:12:48.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.538+0000 7f7efa222640 1 --2- 172.21.15.161:0/2559063129 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7edc0390c0 0x7f7edc03b580 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:48.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.538+0000 7f7efa222640 1 --2- 172.21.15.161:0/2559063129 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ef40ff8b0 0x7f7ef4102110 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:48.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.538+0000 7f7efa222640 1 -- 172.21.15.161:0/2559063129 >> 172.21.15.161:0/2559063129 conn(0x7f7ef40fb400 msgr2=0x7f7ef40fbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:48.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.539+0000 7f7efa222640 1 -- 172.21.15.161:0/2559063129 shutdown_connections 2024-09-12T03:12:48.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:48.539+0000 7f7efa222640 1 -- 172.21.15.161:0/2559063129 wait complete. 2024-09-12T03:12:49.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:48 smithi006 ceph-mon[30991]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:12:49.147 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:48 smithi006 ceph-mon[30991]: Updating smithi161:/etc/ceph/ceph.conf 2024-09-12T03:12:49.147 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:48 smithi006 ceph-mon[30991]: Updating smithi161:/var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:12:49.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:48 smithi006 ceph-mon[30991]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:12:49.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:48 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:49.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:48 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:49.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:48 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:49.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:48 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:49.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:48 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:49.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:48 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:49.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:48 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:49.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:48 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:49.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:48 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:12:49.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:48 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:12:49.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:48 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:49.150 DEBUG:teuthology.orchestra.run.smithi118:mon.b> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mon.b.service 2024-09-12T03:12:49.154 DEBUG:teuthology.orchestra.run.smithi161:mon.c> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mon.c.service 2024-09-12T03:12:49.157 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T03:12:49.157 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T03:12:49.430 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:12:49.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.871+0000 7f805d16e640 1 -- 172.21.15.161:0/3648003465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8058102790 msgr2=0x7f8058102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:49.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.871+0000 7f805d16e640 1 --2- 172.21.15.161:0/3648003465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8058102790 0x7f8058102b70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8048009810 tx=0x7f804802a540 comp rx=0 tx=0).stop 2024-09-12T03:12:49.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.871+0000 7f805d16e640 1 -- 172.21.15.161:0/3648003465 shutdown_connections 2024-09-12T03:12:49.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.871+0000 7f805d16e640 1 --2- 172.21.15.161:0/3648003465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8058102790 0x7f8058102b70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:49.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.871+0000 7f805d16e640 1 -- 172.21.15.161:0/3648003465 >> 172.21.15.161:0/3648003465 conn(0x7f8058077310 msgr2=0x7f80580fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:49.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.871+0000 7f805d16e640 1 -- 172.21.15.161:0/3648003465 shutdown_connections 2024-09-12T03:12:49.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.871+0000 7f805d16e640 1 -- 172.21.15.161:0/3648003465 wait complete. 2024-09-12T03:12:49.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.872+0000 7f805d16e640 1 Processor -- start 2024-09-12T03:12:49.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.872+0000 7f805d16e640 1 -- start start 2024-09-12T03:12:49.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.872+0000 7f805d16e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8058102790 0x7f80581971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:49.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.872+0000 7f805d16e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f805810bc80 con 0x7f8058102790 2024-09-12T03:12:49.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.872+0000 7f8056d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8058102790 0x7f80581971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:49.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.872+0000 7f8056d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8058102790 0x7f80581971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.161:36264/0 (socket says 172.21.15.161:36264) 2024-09-12T03:12:49.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.872+0000 7f8056d9d640 1 -- 172.21.15.161:0/3210659953 learned_addr learned my addr 172.21.15.161:0/3210659953 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-12T03:12:49.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.873+0000 7f8056d9d640 1 -- 172.21.15.161:0/3210659953 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8058197720 con 0x7f8058102790 2024-09-12T03:12:49.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.873+0000 7f8056d9d640 1 --2- 172.21.15.161:0/3210659953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8058102790 0x7f80581971e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f80480045e0 tx=0x7f8048004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:49.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.873+0000 7f8054d99640 1 -- 172.21.15.161:0/3210659953 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8048030e70 con 0x7f8058102790 2024-09-12T03:12:49.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.873+0000 7f805d16e640 1 -- 172.21.15.161:0/3210659953 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80581979b0 con 0x7f8058102790 2024-09-12T03:12:49.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.873+0000 7f8054d99640 1 -- 172.21.15.161:0/3210659953 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80480318e0 con 0x7f8058102790 2024-09-12T03:12:49.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.873+0000 7f8054d99640 1 -- 172.21.15.161:0/3210659953 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80480049c0 con 0x7f8058102790 2024-09-12T03:12:49.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.873+0000 7f805d16e640 1 -- 172.21.15.161:0/3210659953 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8058197e10 con 0x7f8058102790 2024-09-12T03:12:49.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.874+0000 7f8054d99640 1 -- 172.21.15.161:0/3210659953 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f8048039040 con 0x7f8058102790 2024-09-12T03:12:49.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.874+0000 7f805d16e640 1 -- 172.21.15.161:0/3210659953 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8058069f30 con 0x7f8058102790 2024-09-12T03:12:49.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.874+0000 7f8054d99640 1 --2- 172.21.15.161:0/3210659953 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8044039060 0x7f804403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:49.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.874+0000 7f8054d99640 1 -- 172.21.15.161:0/3210659953 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f804806d3c0 con 0x7f8058102790 2024-09-12T03:12:49.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.874+0000 7f804ffff640 1 --2- 172.21.15.161:0/3210659953 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8044039060 0x7f804403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:49.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.875+0000 7f804ffff640 1 --2- 172.21.15.161:0/3210659953 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8044039060 0x7f804403b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f80400098c0 tx=0x7f8040006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:49.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:49.879+0000 7f8054d99640 1 -- 172.21.15.161:0/3210659953 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8048040030 con 0x7f8058102790 2024-09-12T03:12:50.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:49 smithi006 ceph-mon[30991]: Updating smithi161:/var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:12:50.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:49 smithi006 ceph-mon[30991]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi006:172.21.15.6=a;smithi118:172.21.15.118=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:50.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:49 smithi006 ceph-mon[30991]: Saving service mon spec with placement smithi006:172.21.15.6=a;smithi118:172.21.15.118=b;smithi161:172.21.15.161=c;count:3 2024-09-12T03:12:50.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:49 smithi006 ceph-mon[30991]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T03:12:50.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:49 smithi006 ceph-mon[30991]: Reconfiguring daemon mon.a on smithi006 2024-09-12T03:12:50.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:49 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:50.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:49 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:50.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:49 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:50.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:49 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:50.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:49 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:50.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:49 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:50.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:49 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:12:50.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:49 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:50.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:50.119+0000 7f805d16e640 1 -- 172.21.15.161:0/3210659953 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8058198540 con 0x7f8058102790 2024-09-12T03:12:50.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:50.120+0000 7f8054d99640 1 -- 172.21.15.161:0/3210659953 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f8048037d70 con 0x7f8058102790 2024-09-12T03:12:50.120 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:12:50.121 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":1,"fsid":"89d99938-70b4-11ef-bcea-c7b262605968","modified":"2024-09-12T03:11:20.144939Z","created":"2024-09-12T03:11:20.144939Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-12T03:12:50.121 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 1 2024-09-12T03:12:50.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:50.122+0000 7f805d16e640 1 -- 172.21.15.161:0/3210659953 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8044039060 msgr2=0x7f804403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:50.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:50.122+0000 7f805d16e640 1 --2- 172.21.15.161:0/3210659953 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8044039060 0x7f804403b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f80400098c0 tx=0x7f8040006db0 comp rx=0 tx=0).stop 2024-09-12T03:12:50.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:50.122+0000 7f805d16e640 1 -- 172.21.15.161:0/3210659953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8058102790 msgr2=0x7f80581971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:50.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:50.122+0000 7f805d16e640 1 --2- 172.21.15.161:0/3210659953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8058102790 0x7f80581971e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f80480045e0 tx=0x7f8048004610 comp rx=0 tx=0).stop 2024-09-12T03:12:50.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:50.122+0000 7f805d16e640 1 -- 172.21.15.161:0/3210659953 shutdown_connections 2024-09-12T03:12:50.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:50.122+0000 7f805d16e640 1 --2- 172.21.15.161:0/3210659953 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8044039060 0x7f804403b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:50.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:50.122+0000 7f805d16e640 1 --2- 172.21.15.161:0/3210659953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8058102790 0x7f80581971e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:50.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:50.122+0000 7f805d16e640 1 -- 172.21.15.161:0/3210659953 >> 172.21.15.161:0/3210659953 conn(0x7f8058077310 msgr2=0x7f8058103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:50.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:50.122+0000 7f805d16e640 1 -- 172.21.15.161:0/3210659953 shutdown_connections 2024-09-12T03:12:50.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:50.122+0000 7f805d16e640 1 -- 172.21.15.161:0/3210659953 wait complete. 2024-09-12T03:12:51.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:50 smithi006 ceph-mon[30991]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:12:51.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:50 smithi006 ceph-mon[30991]: Deploying daemon mon.c on smithi161 2024-09-12T03:12:51.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:50 smithi006 ceph-mon[30991]: from='client.? 172.21.15.161:0/3210659953' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:12:52.192 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T03:12:52.193 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T03:12:52.559 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:12:53.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:52 smithi006 ceph-mon[30991]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:12:53.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.123+0000 7f727a115640 1 -- 172.21.15.161:0/672687897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7274102850 msgr2=0x7f7274102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:53.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.123+0000 7f727a115640 1 --2- 172.21.15.161:0/672687897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7274102850 0x7f7274102c30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7264009880 tx=0x7f726402a670 comp rx=0 tx=0).stop 2024-09-12T03:12:53.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.123+0000 7f727a115640 1 -- 172.21.15.161:0/672687897 shutdown_connections 2024-09-12T03:12:53.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.123+0000 7f727a115640 1 --2- 172.21.15.161:0/672687897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7274102850 0x7f7274102c30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:53.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.123+0000 7f727a115640 1 -- 172.21.15.161:0/672687897 >> 172.21.15.161:0/672687897 conn(0x7f72740fc250 msgr2=0x7f72740fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:53.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.123+0000 7f727a115640 1 -- 172.21.15.161:0/672687897 shutdown_connections 2024-09-12T03:12:53.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.123+0000 7f727a115640 1 -- 172.21.15.161:0/672687897 wait complete. 2024-09-12T03:12:53.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.124+0000 7f727a115640 1 Processor -- start 2024-09-12T03:12:53.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.124+0000 7f727a115640 1 -- start start 2024-09-12T03:12:53.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.124+0000 7f727a115640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7274102850 0x7f72741972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:53.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.124+0000 7f727a115640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f727410bdc0 con 0x7f7274102850 2024-09-12T03:12:53.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.125+0000 7f72737fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7274102850 0x7f72741972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:53.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.125+0000 7f72737fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7274102850 0x7f72741972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.161:36288/0 (socket says 172.21.15.161:36288) 2024-09-12T03:12:53.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.125+0000 7f72737fe640 1 -- 172.21.15.161:0/1226019501 learned_addr learned my addr 172.21.15.161:0/1226019501 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-12T03:12:53.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.125+0000 7f72737fe640 1 -- 172.21.15.161:0/1226019501 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7274197830 con 0x7f7274102850 2024-09-12T03:12:53.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.125+0000 7f72737fe640 1 --2- 172.21.15.161:0/1226019501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7274102850 0x7f72741972f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7264009230 tx=0x7f7264003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:53.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.125+0000 7f7270ff9640 1 -- 172.21.15.161:0/1226019501 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f726403f070 con 0x7f7274102850 2024-09-12T03:12:53.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.125+0000 7f727a115640 1 -- 172.21.15.161:0/1226019501 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7274197ac0 con 0x7f7274102850 2024-09-12T03:12:53.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.125+0000 7f7270ff9640 1 -- 172.21.15.161:0/1226019501 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f726403bc80 con 0x7f7274102850 2024-09-12T03:12:53.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.125+0000 7f7270ff9640 1 -- 172.21.15.161:0/1226019501 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7264035030 con 0x7f7274102850 2024-09-12T03:12:53.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.126+0000 7f727a115640 1 -- 172.21.15.161:0/1226019501 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7274197f20 con 0x7f7274102850 2024-09-12T03:12:53.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.126+0000 7f7270ff9640 1 -- 172.21.15.161:0/1226019501 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f7264042070 con 0x7f7274102850 2024-09-12T03:12:53.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.126+0000 7f727a115640 1 -- 172.21.15.161:0/1226019501 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f727406ad10 con 0x7f7274102850 2024-09-12T03:12:53.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.127+0000 7f7270ff9640 1 --2- 172.21.15.161:0/1226019501 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f725c039060 0x7f725c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:53.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.127+0000 7f7270ff9640 1 -- 172.21.15.161:0/1226019501 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f726406cd90 con 0x7f7274102850 2024-09-12T03:12:53.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.127+0000 7f7272ffd640 1 --2- 172.21.15.161:0/1226019501 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f725c039060 0x7f725c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:53.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.127+0000 7f7272ffd640 1 --2- 172.21.15.161:0/1226019501 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f725c039060 0x7f725c03b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f72680098c0 tx=0x7f7268006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:53.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.132+0000 7f7270ff9640 1 -- 172.21.15.161:0/1226019501 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f726404c350 con 0x7f7274102850 2024-09-12T03:12:53.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.378+0000 7f727a115640 1 -- 172.21.15.161:0/1226019501 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7274198650 con 0x7f7274102850 2024-09-12T03:12:53.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.379+0000 7f7270ff9640 1 -- 172.21.15.161:0/1226019501 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f726404ad50 con 0x7f7274102850 2024-09-12T03:12:53.379 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:12:53.380 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":1,"fsid":"89d99938-70b4-11ef-bcea-c7b262605968","modified":"2024-09-12T03:11:20.144939Z","created":"2024-09-12T03:11:20.144939Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-12T03:12:53.380 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 1 2024-09-12T03:12:53.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.381+0000 7f727a115640 1 -- 172.21.15.161:0/1226019501 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f725c039060 msgr2=0x7f725c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:53.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.381+0000 7f727a115640 1 --2- 172.21.15.161:0/1226019501 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f725c039060 0x7f725c03b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f72680098c0 tx=0x7f7268006db0 comp rx=0 tx=0).stop 2024-09-12T03:12:53.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.381+0000 7f727a115640 1 -- 172.21.15.161:0/1226019501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7274102850 msgr2=0x7f72741972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:53.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.381+0000 7f727a115640 1 --2- 172.21.15.161:0/1226019501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7274102850 0x7f72741972f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7264009230 tx=0x7f7264003d40 comp rx=0 tx=0).stop 2024-09-12T03:12:53.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.381+0000 7f727a115640 1 -- 172.21.15.161:0/1226019501 shutdown_connections 2024-09-12T03:12:53.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.381+0000 7f727a115640 1 --2- 172.21.15.161:0/1226019501 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f725c039060 0x7f725c03b520 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:53.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.381+0000 7f727a115640 1 --2- 172.21.15.161:0/1226019501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7274102850 0x7f72741972f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:53.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.381+0000 7f727a115640 1 -- 172.21.15.161:0/1226019501 >> 172.21.15.161:0/1226019501 conn(0x7f72740fc250 msgr2=0x7f7274105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:53.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.381+0000 7f727a115640 1 -- 172.21.15.161:0/1226019501 shutdown_connections 2024-09-12T03:12:53.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:12:53.381+0000 7f727a115640 1 -- 172.21.15.161:0/1226019501 wait complete. 2024-09-12T03:12:54.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:53 smithi006 ceph-mon[30991]: from='client.? 172.21.15.161:0/1226019501' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:12:54.607 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: mon.c@-1(synchronizing).mds e1 new map 2024-09-12T03:12:54.607 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-12T03:12:54.607 INFO:journalctl@ceph.mon.c.smithi161.stdout: e1 2024-09-12T03:12:54.608 INFO:journalctl@ceph.mon.c.smithi161.stdout: btime 2024-09-12T03:11:29:530327+0000 2024-09-12T03:12:54.608 INFO:journalctl@ceph.mon.c.smithi161.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T03:12:54.608 INFO:journalctl@ceph.mon.c.smithi161.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T03:12:54.608 INFO:journalctl@ceph.mon.c.smithi161.stdout: legacy client fscid: -1 2024-09-12T03:12:54.608 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2024-09-12T03:12:54.608 INFO:journalctl@ceph.mon.c.smithi161.stdout: No filesystems configured 2024-09-12T03:12:54.608 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-12T03:12:54.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-12T03:12:54.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-12T03:12:54.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-12T03:12:54.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-12T03:12:54.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-12T03:12:54.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T03:12:54.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:12:54.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:12:54.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:12:54.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:12:54.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:54.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: Added host smithi161 2024-09-12T03:12:54.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:54.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:54.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:54.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:54.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:12:54.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:12:54.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3732703727' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T03:12:54.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3732703727' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T03:12:54.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:12:54.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:54.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:54.866 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:54.866 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:54.866 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:12:54.866 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:54.866 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:54.866 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:12:54.866 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: Updating smithi161:/etc/ceph/ceph.conf 2024-09-12T03:12:54.866 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: Updating smithi161:/var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:12:54.866 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:12:54.866 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:54.867 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:54.867 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:54.867 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:54.867 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:54.867 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:54.867 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:54.867 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:54.867 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:12:54.867 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:12:54.867 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:54.867 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: Updating smithi161:/var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:12:54.867 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi006:172.21.15.6=a;smithi118:172.21.15.118=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:54.868 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: Saving service mon spec with placement smithi006:172.21.15.6=a;smithi118:172.21.15.118=b;smithi161:172.21.15.161=c;count:3 2024-09-12T03:12:54.868 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T03:12:54.868 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: Reconfiguring daemon mon.a on smithi006 2024-09-12T03:12:54.868 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:54.868 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:54.868 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:54.868 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:54.868 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:54.868 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:12:54.868 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:12:54.868 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:54.868 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:12:54.868 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: Deploying daemon mon.c on smithi161 2024-09-12T03:12:54.869 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='client.? 172.21.15.161:0/3210659953' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:12:54.869 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:12:54.869 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: from='client.? 172.21.15.161:0/1226019501' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:12:54.869 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 ceph-mon[35485]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-12T03:12:54.869 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 bash[35467]: 892bf1db843aa52b55ab06aa4ebd35b4b33aee932e2216aa054c52bcd1fcae17 2024-09-12T03:12:54.869 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:54 smithi161 systemd[1]: Started Ceph mon.c for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:12:55.098 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T03:12:55.099 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T03:12:55.346 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:13:00.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:13:00.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: mon.a calling monitor election 2024-09-12T03:13:00.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:13:00.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:00.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:13:00.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:13:00.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: mon.c calling monitor election 2024-09-12T03:13:00.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:00.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:13:00.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:13:00.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:13:00.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T03:13:00.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: monmap epoch 2 2024-09-12T03:13:00.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: fsid 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:13:00.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: last_changed 2024-09-12T03:12:54.613534+0000 2024-09-12T03:13:00.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: created 2024-09-12T03:11:20.144939+0000 2024-09-12T03:13:00.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: min_mon_release 19 (squid) 2024-09-12T03:13:00.114 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: election_strategy: 1 2024-09-12T03:13:00.114 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: 0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-09-12T03:13:00.114 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-12T03:13:00.114 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: fsmap 2024-09-12T03:13:00.114 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:13:00.114 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: mgrmap e13: x(active, since 40s) 2024-09-12T03:13:00.114 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: overall HEALTH_OK 2024-09-12T03:13:00.114 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:00.114 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:00.114 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:00.114 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:13:00.114 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:12:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:00.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:13:00.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: mon.a calling monitor election 2024-09-12T03:13:00.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:13:00.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:00.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:13:00.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:13:00.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: mon.c calling monitor election 2024-09-12T03:13:00.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:00.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:13:00.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:13:00.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:13:00.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T03:13:00.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: monmap epoch 2 2024-09-12T03:13:00.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: fsid 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:13:00.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: last_changed 2024-09-12T03:12:54.613534+0000 2024-09-12T03:13:00.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: created 2024-09-12T03:11:20.144939+0000 2024-09-12T03:13:00.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: min_mon_release 19 (squid) 2024-09-12T03:13:00.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: election_strategy: 1 2024-09-12T03:13:00.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: 0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-09-12T03:13:00.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-12T03:13:00.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: fsmap 2024-09-12T03:13:00.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:13:00.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: mgrmap e13: x(active, since 40s) 2024-09-12T03:13:00.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: overall HEALTH_OK 2024-09-12T03:13:00.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:00.117 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:00.117 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:00.117 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:13:00.117 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:12:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:00.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.219+0000 7f1ee6099640 1 -- 172.21.15.161:0/3691733156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ec8003530 msgr2=0x7f1ec80059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:00.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.219+0000 7f1ee6099640 1 --2- 172.21.15.161:0/3691733156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ec8003530 0x7f1ec80059c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1ee006aa60 tx=0x7f1ed002c490 comp rx=0 tx=0).stop 2024-09-12T03:13:00.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.219+0000 7f1ee6099640 1 -- 172.21.15.161:0/3691733156 shutdown_connections 2024-09-12T03:13:00.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.219+0000 7f1ee6099640 1 --2- 172.21.15.161:0/3691733156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ec8003530 0x7f1ec80059c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.219+0000 7f1ee6099640 1 -- 172.21.15.161:0/3691733156 >> 172.21.15.161:0/3691733156 conn(0x7f1ee00fc220 msgr2=0x7f1ee00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:00.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.219+0000 7f1ee6099640 1 -- 172.21.15.161:0/3691733156 shutdown_connections 2024-09-12T03:13:00.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.219+0000 7f1ee6099640 1 -- 172.21.15.161:0/3691733156 wait complete. 2024-09-12T03:13:00.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.220+0000 7f1ee6099640 1 Processor -- start 2024-09-12T03:13:00.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.220+0000 7f1ee6099640 1 -- start start 2024-09-12T03:13:00.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.221+0000 7f1ee6099640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ec8003530 0x7f1ee0197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:00.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.221+0000 7f1ee6099640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ee01978c0 0x7f1ee019bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:00.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.221+0000 7f1ee6099640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee0109780 con 0x7f1ec8003530 2024-09-12T03:13:00.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.221+0000 7f1ee6099640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee0109600 con 0x7f1ee01978c0 2024-09-12T03:13:00.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.221+0000 7f1edeffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ee01978c0 0x7f1ee019bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:00.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.221+0000 7f1edf7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ec8003530 0x7f1ee0197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:00.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.221+0000 7f1edf7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ec8003530 0x7f1ee0197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.161:57836/0 (socket says 172.21.15.161:57836) 2024-09-12T03:13:00.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.221+0000 7f1edeffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ee01978c0 0x7f1ee019bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43770/0 (socket says 172.21.15.161:43770) 2024-09-12T03:13:00.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.221+0000 7f1edf7fe640 1 -- 172.21.15.161:0/3715582650 learned_addr learned my addr 172.21.15.161:0/3715582650 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-12T03:13:00.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.221+0000 7f1edeffd640 1 -- 172.21.15.161:0/3715582650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ee01978c0 msgr2=0x7f1ee019bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T03:13:00.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.221+0000 7f1edeffd640 1 -- 172.21.15.161:0/3715582650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ee01978c0 msgr2=0x7f1ee019bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T03:13:00.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.221+0000 7f1edeffd640 1 --2- 172.21.15.161:0/3715582650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ee01978c0 0x7f1ee019bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T03:13:00.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.221+0000 7f1edeffd640 1 --2- 172.21.15.161:0/3715582650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ee01978c0 0x7f1ee019bc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:13:00.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.221+0000 7f1edf7fe640 1 -- 172.21.15.161:0/3715582650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ee01978c0 msgr2=0x7f1ee019bc00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:13:00.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.221+0000 7f1edf7fe640 1 --2- 172.21.15.161:0/3715582650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ee01978c0 0x7f1ee019bc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.221+0000 7f1edf7fe640 1 -- 172.21.15.161:0/3715582650 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee0197ee0 con 0x7f1ec8003530 2024-09-12T03:13:00.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.222+0000 7f1edf7fe640 1 --2- 172.21.15.161:0/3715582650 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ec8003530 0x7f1ee0197380 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0036040 tx=0x7f1ed0032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:00.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.222+0000 7f1edcff9640 1 -- 172.21.15.161:0/3715582650 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1ed0038040 con 0x7f1ec8003530 2024-09-12T03:13:00.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.222+0000 7f1edcff9640 1 -- 172.21.15.161:0/3715582650 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ed0005040 con 0x7f1ec8003530 2024-09-12T03:13:00.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.222+0000 7f1ee6099640 1 -- 172.21.15.161:0/3715582650 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee019c140 con 0x7f1ec8003530 2024-09-12T03:13:00.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.222+0000 7f1edcff9640 1 -- 172.21.15.161:0/3715582650 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1ed0004ac0 con 0x7f1ec8003530 2024-09-12T03:13:00.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.222+0000 7f1ee6099640 1 -- 172.21.15.161:0/3715582650 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee019c680 con 0x7f1ec8003530 2024-09-12T03:13:00.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.223+0000 7f1edcff9640 1 -- 172.21.15.161:0/3715582650 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f1ed0043070 con 0x7f1ec8003530 2024-09-12T03:13:00.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.223+0000 7f1edcff9640 1 --2- 172.21.15.161:0/3715582650 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1ecc038e50 0x7f1ecc03b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:00.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.223+0000 7f1edcff9640 1 -- 172.21.15.161:0/3715582650 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1ed0041070 con 0x7f1ec8003530 2024-09-12T03:13:00.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.223+0000 7f1ee6099640 1 -- 172.21.15.161:0/3715582650 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ee006ad10 con 0x7f1ec8003530 2024-09-12T03:13:00.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.224+0000 7f1edeffd640 1 --2- 172.21.15.161:0/3715582650 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1ecc038e50 0x7f1ecc03b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:00.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.224+0000 7f1edeffd640 1 --2- 172.21.15.161:0/3715582650 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1ecc038e50 0x7f1ecc03b310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4006fd0 tx=0x7f1ed4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:00.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.229+0000 7f1edcff9640 1 -- 172.21.15.161:0/3715582650 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ed0043c20 con 0x7f1ec8003530 2024-09-12T03:13:00.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.464+0000 7f1ee6099640 1 -- 172.21.15.161:0/3715582650 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1ee0190f70 con 0x7f1ec8003530 2024-09-12T03:13:00.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.465+0000 7f1edcff9640 1 -- 172.21.15.161:0/3715582650 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f1ed0034000 con 0x7f1ec8003530 2024-09-12T03:13:00.465 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:13:00.466 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":2,"fsid":"89d99938-70b4-11ef-bcea-c7b262605968","modified":"2024-09-12T03:12:54.613534Z","created":"2024-09-12T03:11:20.144939Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T03:13:00.466 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 2 2024-09-12T03:13:00.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.467+0000 7f1ee6099640 1 -- 172.21.15.161:0/3715582650 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1ecc038e50 msgr2=0x7f1ecc03b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:00.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.467+0000 7f1ee6099640 1 --2- 172.21.15.161:0/3715582650 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1ecc038e50 0x7f1ecc03b310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4006fd0 tx=0x7f1ed4008040 comp rx=0 tx=0).stop 2024-09-12T03:13:00.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.467+0000 7f1ee6099640 1 -- 172.21.15.161:0/3715582650 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ec8003530 msgr2=0x7f1ee0197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:00.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.467+0000 7f1ee6099640 1 --2- 172.21.15.161:0/3715582650 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ec8003530 0x7f1ee0197380 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0036040 tx=0x7f1ed0032c90 comp rx=0 tx=0).stop 2024-09-12T03:13:00.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.467+0000 7f1ee6099640 1 -- 172.21.15.161:0/3715582650 shutdown_connections 2024-09-12T03:13:00.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.467+0000 7f1ee6099640 1 --2- 172.21.15.161:0/3715582650 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1ecc038e50 0x7f1ecc03b310 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.467+0000 7f1ee6099640 1 --2- 172.21.15.161:0/3715582650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ee01978c0 0x7f1ee019bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.467+0000 7f1ee6099640 1 --2- 172.21.15.161:0/3715582650 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ec8003530 0x7f1ee0197380 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.467+0000 7f1ee6099640 1 -- 172.21.15.161:0/3715582650 >> 172.21.15.161:0/3715582650 conn(0x7f1ee00fc220 msgr2=0x7f1ee00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:00.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.467+0000 7f1ee6099640 1 -- 172.21.15.161:0/3715582650 shutdown_connections 2024-09-12T03:13:00.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:00.467+0000 7f1ee6099640 1 -- 172.21.15.161:0/3715582650 wait complete. 2024-09-12T03:13:00.770 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:00 smithi161 ceph-mon[35485]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:00.770 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:00 smithi161 ceph-mon[35485]: Deploying daemon mon.b on smithi118 2024-09-12T03:13:00.770 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:00 smithi161 ceph-mon[35485]: from='client.? 172.21.15.161:0/3715582650' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:13:00.770 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:00 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:13:01.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:00 smithi006 ceph-mon[30991]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:01.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:00 smithi006 ceph-mon[30991]: Deploying daemon mon.b on smithi118 2024-09-12T03:13:01.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:00 smithi006 ceph-mon[30991]: from='client.? 172.21.15.161:0/3715582650' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:13:01.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:00 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:13:02.057 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T03:13:02.057 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T03:13:02.311 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:13:02.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.678+0000 7f1083d86640 1 -- 172.21.15.161:0/3949832919 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f107c100610 msgr2=0x7f107c1009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:02.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.678+0000 7f1083d86640 1 --2- 172.21.15.161:0/3949832919 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f107c100610 0x7f107c1009f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1078009880 tx=0x7f107802a670 comp rx=0 tx=0).stop 2024-09-12T03:13:02.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.678+0000 7f1083d86640 1 -- 172.21.15.161:0/3949832919 shutdown_connections 2024-09-12T03:13:02.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.678+0000 7f1083d86640 1 --2- 172.21.15.161:0/3949832919 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f107c100610 0x7f107c1009f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.678+0000 7f1083d86640 1 -- 172.21.15.161:0/3949832919 >> 172.21.15.161:0/3949832919 conn(0x7f107c0fc220 msgr2=0x7f107c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:02.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.678+0000 7f1083d86640 1 -- 172.21.15.161:0/3949832919 shutdown_connections 2024-09-12T03:13:02.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.678+0000 7f1083d86640 1 -- 172.21.15.161:0/3949832919 wait complete. 2024-09-12T03:13:02.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.678+0000 7f1083d86640 1 Processor -- start 2024-09-12T03:13:02.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.679+0000 7f1083d86640 1 -- start start 2024-09-12T03:13:02.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.679+0000 7f1083d86640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f107c100610 0x7f107c1a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:02.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.679+0000 7f1083d86640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f107c1a3c00 0x7f107c19d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:02.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.679+0000 7f1083d86640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f107c112370 con 0x7f107c100610 2024-09-12T03:13:02.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.679+0000 7f1083d86640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f107c1121f0 con 0x7f107c1a3c00 2024-09-12T03:13:02.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.679+0000 7f1081321640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f107c1a3c00 0x7f107c19d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:02.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.679+0000 7f1081321640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f107c1a3c00 0x7f107c19d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43778/0 (socket says 172.21.15.161:43778) 2024-09-12T03:13:02.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.679+0000 7f1081321640 1 -- 172.21.15.161:0/2910102960 learned_addr learned my addr 172.21.15.161:0/2910102960 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-12T03:13:02.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.679+0000 7f1081321640 1 -- 172.21.15.161:0/2910102960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f107c1a3c00 msgr2=0x7f107c19d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T03:13:02.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.679+0000 7f1081321640 1 -- 172.21.15.161:0/2910102960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f107c1a3c00 msgr2=0x7f107c19d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T03:13:02.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.679+0000 7f1081321640 1 --2- 172.21.15.161:0/2910102960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f107c1a3c00 0x7f107c19d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T03:13:02.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.679+0000 7f1081b22640 1 --2- 172.21.15.161:0/2910102960 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f107c100610 0x7f107c1a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:02.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.679+0000 7f1081321640 1 --2- 172.21.15.161:0/2910102960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f107c1a3c00 0x7f107c19d840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:13:02.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.679+0000 7f1081b22640 1 -- 172.21.15.161:0/2910102960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f107c1a3c00 msgr2=0x7f107c19d840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:13:02.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.679+0000 7f1081b22640 1 --2- 172.21.15.161:0/2910102960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f107c1a3c00 0x7f107c19d840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.679+0000 7f1081b22640 1 -- 172.21.15.161:0/2910102960 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107c1a4210 con 0x7f107c100610 2024-09-12T03:13:02.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.680+0000 7f1081b22640 1 --2- 172.21.15.161:0/2910102960 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f107c100610 0x7f107c1a36c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1078005ab0 tx=0x7f1078032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:02.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.680+0000 7f1072ffd640 1 -- 172.21.15.161:0/2910102960 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1078035070 con 0x7f107c100610 2024-09-12T03:13:02.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.680+0000 7f1083d86640 1 -- 172.21.15.161:0/2910102960 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f107c19dd80 con 0x7f107c100610 2024-09-12T03:13:02.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.680+0000 7f1072ffd640 1 -- 172.21.15.161:0/2910102960 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1078032e40 con 0x7f107c100610 2024-09-12T03:13:02.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.680+0000 7f1072ffd640 1 -- 172.21.15.161:0/2910102960 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f107803bc10 con 0x7f107c100610 2024-09-12T03:13:02.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.680+0000 7f1083d86640 1 -- 172.21.15.161:0/2910102960 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f107c19e310 con 0x7f107c100610 2024-09-12T03:13:02.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.681+0000 7f1072ffd640 1 -- 172.21.15.161:0/2910102960 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f1078042030 con 0x7f107c100610 2024-09-12T03:13:02.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.681+0000 7f1083d86640 1 -- 172.21.15.161:0/2910102960 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f107c06ad10 con 0x7f107c100610 2024-09-12T03:13:02.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.681+0000 7f1072ffd640 1 --2- 172.21.15.161:0/2910102960 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1068038e00 0x7f106803b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:02.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.681+0000 7f1072ffd640 1 -- 172.21.15.161:0/2910102960 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1078040070 con 0x7f107c100610 2024-09-12T03:13:02.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.681+0000 7f1081321640 1 --2- 172.21.15.161:0/2910102960 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1068038e00 0x7f106803b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:02.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.682+0000 7f1081321640 1 --2- 172.21.15.161:0/2910102960 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1068038e00 0x7f106803b2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f106c009100 tx=0x7f106c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:02.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.686+0000 7f1072ffd640 1 -- 172.21.15.161:0/2910102960 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f107804a260 con 0x7f107c100610 2024-09-12T03:13:02.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.917+0000 7f1083d86640 1 -- 172.21.15.161:0/2910102960 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f107c19e5c0 con 0x7f107c100610 2024-09-12T03:13:02.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.917+0000 7f1072ffd640 1 -- 172.21.15.161:0/2910102960 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f107803bdb0 con 0x7f107c100610 2024-09-12T03:13:02.918 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:13:02.918 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":2,"fsid":"89d99938-70b4-11ef-bcea-c7b262605968","modified":"2024-09-12T03:12:54.613534Z","created":"2024-09-12T03:11:20.144939Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T03:13:02.918 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 2 2024-09-12T03:13:02.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.919+0000 7f1083d86640 1 -- 172.21.15.161:0/2910102960 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1068038e00 msgr2=0x7f106803b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:02.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.919+0000 7f1083d86640 1 --2- 172.21.15.161:0/2910102960 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1068038e00 0x7f106803b2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f106c009100 tx=0x7f106c008040 comp rx=0 tx=0).stop 2024-09-12T03:13:02.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.919+0000 7f1083d86640 1 -- 172.21.15.161:0/2910102960 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f107c100610 msgr2=0x7f107c1a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:02.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.919+0000 7f1083d86640 1 --2- 172.21.15.161:0/2910102960 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f107c100610 0x7f107c1a36c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1078005ab0 tx=0x7f1078032c50 comp rx=0 tx=0).stop 2024-09-12T03:13:02.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.919+0000 7f1083d86640 1 -- 172.21.15.161:0/2910102960 shutdown_connections 2024-09-12T03:13:02.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.919+0000 7f1083d86640 1 --2- 172.21.15.161:0/2910102960 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1068038e00 0x7f106803b2c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.919+0000 7f1083d86640 1 --2- 172.21.15.161:0/2910102960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f107c1a3c00 0x7f107c19d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.919+0000 7f1083d86640 1 --2- 172.21.15.161:0/2910102960 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f107c100610 0x7f107c1a36c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.919+0000 7f1083d86640 1 -- 172.21.15.161:0/2910102960 >> 172.21.15.161:0/2910102960 conn(0x7f107c0fc220 msgr2=0x7f107c0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:02.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.919+0000 7f1083d86640 1 -- 172.21.15.161:0/2910102960 shutdown_connections 2024-09-12T03:13:02.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:02.919+0000 7f1083d86640 1 -- 172.21.15.161:0/2910102960 wait complete. 2024-09-12T03:13:02.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:02 smithi161 ceph-mon[35485]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:03.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:02 smithi006 ceph-mon[30991]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:03.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:03 smithi118 systemd[1]: Started Ceph mon.b for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:13:04.529 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T03:13:04.529 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T03:13:04.780 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:13:08.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.401+0000 7f38c266a640 1 -- 172.21.15.161:0/1168617193 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38a4003530 msgr2=0x7f38a40059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:08.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.401+0000 7f38c266a640 1 --2- 172.21.15.161:0/1168617193 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38a4003530 0x7f38a40059c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f38bc1016c0 tx=0x7f38ac02c150 comp rx=0 tx=0).stop 2024-09-12T03:13:08.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.401+0000 7f38c266a640 1 -- 172.21.15.161:0/1168617193 shutdown_connections 2024-09-12T03:13:08.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.401+0000 7f38c266a640 1 --2- 172.21.15.161:0/1168617193 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38a4003530 0x7f38a40059c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.401+0000 7f38c266a640 1 -- 172.21.15.161:0/1168617193 >> 172.21.15.161:0/1168617193 conn(0x7f38bc0fc280 msgr2=0x7f38bc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:08.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.401+0000 7f38c266a640 1 -- 172.21.15.161:0/1168617193 shutdown_connections 2024-09-12T03:13:08.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.401+0000 7f38c266a640 1 -- 172.21.15.161:0/1168617193 wait complete. 2024-09-12T03:13:08.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.402+0000 7f38c266a640 1 Processor -- start 2024-09-12T03:13:08.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.402+0000 7f38c266a640 1 -- start start 2024-09-12T03:13:08.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.402+0000 7f38c266a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38a4003530 0x7f38bc102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.402+0000 7f38c266a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38bc102750 0x7f38bc102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.402+0000 7f38c266a640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38bc1a3510 0x7f38bc1a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.402+0000 7f38c266a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc108690 con 0x7f38a4003530 2024-09-12T03:13:08.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.402+0000 7f38c266a640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc108510 con 0x7f38bc1a3510 2024-09-12T03:13:08.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.402+0000 7f38c266a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc108810 con 0x7f38bc102750 2024-09-12T03:13:08.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.402+0000 7f38bb7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38bc102750 0x7f38bc102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.402+0000 7f38bbfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38a4003530 0x7f38bc102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.402+0000 7f38bbfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38a4003530 0x7f38bc102210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.161:57250/0 (socket says 172.21.15.161:57250) 2024-09-12T03:13:08.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.402+0000 7f38bb7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38bc102750 0x7f38bc102bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:52682/0 (socket says 172.21.15.161:52682) 2024-09-12T03:13:08.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.403+0000 7f38bbfff640 1 -- 172.21.15.161:0/752012426 learned_addr learned my addr 172.21.15.161:0/752012426 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-12T03:13:08.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.402+0000 7f38c0c07640 1 --2- 172.21.15.161:0/752012426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38bc1a3510 0x7f38bc1a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.403+0000 7f38bb7fe640 1 -- 172.21.15.161:0/752012426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38bc102750 msgr2=0x7f38bc102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T03:13:08.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.403+0000 7f38bb7fe640 1 -- 172.21.15.161:0/752012426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38bc102750 msgr2=0x7f38bc102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T03:13:08.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.403+0000 7f38c0c07640 1 -- 172.21.15.161:0/752012426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38bc1a3510 msgr2=0x7f38bc1a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-12T03:13:08.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.403+0000 7f38c0c07640 1 -- 172.21.15.161:0/752012426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38bc1a3510 msgr2=0x7f38bc1a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T03:13:08.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.403+0000 7f38bb7fe640 1 --2- 172.21.15.161:0/752012426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38bc102750 0x7f38bc102bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T03:13:08.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.403+0000 7f38c0c07640 1 --2- 172.21.15.161:0/752012426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38bc1a3510 0x7f38bc1a5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T03:13:08.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.403+0000 7f38c0c07640 1 --2- 172.21.15.161:0/752012426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38bc1a3510 0x7f38bc1a5900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:13:08.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.403+0000 7f38bb7fe640 1 --2- 172.21.15.161:0/752012426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38bc102750 0x7f38bc102bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:13:08.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.403+0000 7f38bbfff640 1 -- 172.21.15.161:0/752012426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38bc102750 msgr2=0x7f38bc102bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:13:08.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.403+0000 7f38bbfff640 1 --2- 172.21.15.161:0/752012426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38bc102750 0x7f38bc102bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.403+0000 7f38bbfff640 1 -- 172.21.15.161:0/752012426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38bc1a3510 msgr2=0x7f38bc1a5900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:13:08.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.403+0000 7f38bbfff640 1 --2- 172.21.15.161:0/752012426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38bc1a3510 0x7f38bc1a5900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.403+0000 7f38bbfff640 1 -- 172.21.15.161:0/752012426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc1a5fd0 con 0x7f38a4003530 2024-09-12T03:13:08.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.403+0000 7f38bbfff640 1 --2- 172.21.15.161:0/752012426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38a4003530 0x7f38bc102210 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0057c0 tx=0x7f38ac004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:08.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.403+0000 7f38b97fa640 1 -- 172.21.15.161:0/752012426 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac042020 con 0x7f38a4003530 2024-09-12T03:13:08.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.403+0000 7f38c266a640 1 -- 172.21.15.161:0/752012426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38bc1a6260 con 0x7f38a4003530 2024-09-12T03:13:08.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.404+0000 7f38c266a640 1 -- 172.21.15.161:0/752012426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38bc1a67f0 con 0x7f38a4003530 2024-09-12T03:13:08.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.404+0000 7f38b97fa640 1 -- 172.21.15.161:0/752012426 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38ac0050e0 con 0x7f38a4003530 2024-09-12T03:13:08.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.404+0000 7f38b97fa640 1 -- 172.21.15.161:0/752012426 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac032040 con 0x7f38a4003530 2024-09-12T03:13:08.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.404+0000 7f38b97fa640 1 -- 172.21.15.161:0/752012426 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f38ac045020 con 0x7f38a4003530 2024-09-12T03:13:08.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.405+0000 7f38b97fa640 1 --2- 172.21.15.161:0/752012426 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38a80394f0 0x7f38a803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.405+0000 7f38b97fa640 1 -- 172.21.15.161:0/752012426 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f38ac06edb0 con 0x7f38a4003530 2024-09-12T03:13:08.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.405+0000 7f38bb7fe640 1 --2- 172.21.15.161:0/752012426 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38a80394f0 0x7f38a803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.405+0000 7f38c266a640 1 -- 172.21.15.161:0/752012426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38bc1097c0 con 0x7f38a4003530 2024-09-12T03:13:08.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.405+0000 7f38bb7fe640 1 --2- 172.21.15.161:0/752012426 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38a80394f0 0x7f38a803b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f38b0006fd0 tx=0x7f38b0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:08.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.410+0000 7f38b97fa640 1 -- 172.21.15.161:0/752012426 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f38ac04c030 con 0x7f38a4003530 2024-09-12T03:13:08.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.657+0000 7f38c266a640 1 -- 172.21.15.161:0/752012426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f38bc1a6a70 con 0x7f38a4003530 2024-09-12T03:13:08.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:08.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:13:08.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: mon.c calling monitor election 2024-09-12T03:13:08.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: mon.a calling monitor election 2024-09-12T03:13:08.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:08.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:13:08.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:08.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:08.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:08.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:08.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:08.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:08.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:08.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T03:13:08.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: monmap epoch 3 2024-09-12T03:13:08.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: fsid 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:13:08.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: last_changed 2024-09-12T03:13:03.322589+0000 2024-09-12T03:13:08.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: created 2024-09-12T03:11:20.144939+0000 2024-09-12T03:13:08.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: min_mon_release 19 (squid) 2024-09-12T03:13:08.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: election_strategy: 1 2024-09-12T03:13:08.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: 0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-09-12T03:13:08.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-12T03:13:08.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: 2: [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] mon.b 2024-09-12T03:13:08.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: fsmap 2024-09-12T03:13:08.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:13:08.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: mgrmap e13: x(active, since 49s) 2024-09-12T03:13:08.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: overall HEALTH_OK 2024-09-12T03:13:08.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:08.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:08.662 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:08.662 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:08 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:08.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.658+0000 7f38b97fa640 1 -- 172.21.15.161:0/752012426 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f38ac045980 con 0x7f38a4003530 2024-09-12T03:13:08.662 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:13:08.662 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":3,"fsid":"89d99938-70b4-11ef-bcea-c7b262605968","modified":"2024-09-12T03:13:03.322589Z","created":"2024-09-12T03:11:20.144939Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:3300","nonce":0},{"type":"v1","addr":"172.21.15.118:6789","nonce":0}]},"addr":"172.21.15.118:6789/0","public_addr":"172.21.15.118:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T03:13:08.663 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 3 2024-09-12T03:13:08.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.661+0000 7f38c266a640 1 -- 172.21.15.161:0/752012426 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38a80394f0 msgr2=0x7f38a803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:08.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.661+0000 7f38c266a640 1 --2- 172.21.15.161:0/752012426 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38a80394f0 0x7f38a803b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f38b0006fd0 tx=0x7f38b0008040 comp rx=0 tx=0).stop 2024-09-12T03:13:08.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.661+0000 7f38c266a640 1 -- 172.21.15.161:0/752012426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38a4003530 msgr2=0x7f38bc102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:08.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.661+0000 7f38c266a640 1 --2- 172.21.15.161:0/752012426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38a4003530 0x7f38bc102210 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0057c0 tx=0x7f38ac004cb0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.662+0000 7f38c266a640 1 -- 172.21.15.161:0/752012426 shutdown_connections 2024-09-12T03:13:08.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.662+0000 7f38c266a640 1 --2- 172.21.15.161:0/752012426 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38a80394f0 0x7f38a803b9b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.662+0000 7f38c266a640 1 --2- 172.21.15.161:0/752012426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38bc1a3510 0x7f38bc1a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.662+0000 7f38c266a640 1 --2- 172.21.15.161:0/752012426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38bc102750 0x7f38bc102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.662+0000 7f38c266a640 1 --2- 172.21.15.161:0/752012426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38a4003530 0x7f38bc102210 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.662+0000 7f38c266a640 1 -- 172.21.15.161:0/752012426 >> 172.21.15.161:0/752012426 conn(0x7f38bc0fc280 msgr2=0x7f38bc06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:08.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.662+0000 7f38c266a640 1 -- 172.21.15.161:0/752012426 shutdown_connections 2024-09-12T03:13:08.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:08.662+0000 7f38c266a640 1 -- 172.21.15.161:0/752012426 wait complete. 2024-09-12T03:13:08.865 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:08.865 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:13:08.865 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: mon.c calling monitor election 2024-09-12T03:13:08.865 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: mon.a calling monitor election 2024-09-12T03:13:08.865 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:08.865 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:13:08.865 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:08.865 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:08.865 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:08.865 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:08.866 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:08.866 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:08.866 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:08.866 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T03:13:08.866 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: monmap epoch 3 2024-09-12T03:13:08.866 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: fsid 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:13:08.866 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: last_changed 2024-09-12T03:13:03.322589+0000 2024-09-12T03:13:08.866 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: created 2024-09-12T03:11:20.144939+0000 2024-09-12T03:13:08.866 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: min_mon_release 19 (squid) 2024-09-12T03:13:08.866 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: election_strategy: 1 2024-09-12T03:13:08.866 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: 0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-09-12T03:13:08.866 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-12T03:13:08.866 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: 2: [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] mon.b 2024-09-12T03:13:08.867 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: fsmap 2024-09-12T03:13:08.867 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:13:08.867 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: mgrmap e13: x(active, since 49s) 2024-09-12T03:13:08.867 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: overall HEALTH_OK 2024-09-12T03:13:08.867 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:08.867 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:08.867 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:08.867 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:08 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:09.294 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-12T03:13:09.294 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-12T03:13:09.561 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:09.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:09 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:09.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:09 smithi006 ceph-mon[30991]: from='client.? 172.21.15.161:0/752012426' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:13:09.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:09 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:09.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:09 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:09.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:09 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:09.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:09 smithi161 ceph-mon[35485]: from='client.? 172.21.15.161:0/752012426' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:13:09.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:09 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:09.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:09 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:10.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.002+0000 7f03f1d0d640 1 -- 172.21.15.6:0/2573421512 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03ec1028c0 msgr2=0x7f03ec102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:10.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.002+0000 7f03f1d0d640 1 --2- 172.21.15.6:0/2573421512 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03ec1028c0 0x7f03ec102ca0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009880 tx=0x7f03dc02a670 comp rx=0 tx=0).stop 2024-09-12T03:13:10.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.002+0000 7f03f1d0d640 1 -- 172.21.15.6:0/2573421512 shutdown_connections 2024-09-12T03:13:10.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.002+0000 7f03f1d0d640 1 --2- 172.21.15.6:0/2573421512 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03ec1028c0 0x7f03ec102ca0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.002+0000 7f03f1d0d640 1 -- 172.21.15.6:0/2573421512 >> 172.21.15.6:0/2573421512 conn(0x7f03ec0fc220 msgr2=0x7f03ec0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:10.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.002+0000 7f03f1d0d640 1 -- 172.21.15.6:0/2573421512 shutdown_connections 2024-09-12T03:13:10.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.002+0000 7f03f1d0d640 1 -- 172.21.15.6:0/2573421512 wait complete. 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.003+0000 7f03f1d0d640 1 Processor -- start 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.003+0000 7f03f1d0d640 1 -- start start 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.003+0000 7f03f1d0d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03ec1028c0 0x7f03ec1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.003+0000 7f03f1d0d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03ec197af0 0x7f03ec19be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.003+0000 7f03f1d0d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03ec19c3c0 0x7f03ec19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.003+0000 7f03f1d0d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f03ec10bd90 con 0x7f03ec197af0 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.003+0000 7f03f1d0d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f03ec10bc10 con 0x7f03ec19c3c0 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.003+0000 7f03f1d0d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f03ec10bf10 con 0x7f03ec1028c0 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.004+0000 7f03e3fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03ec197af0 0x7f03ec19be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.004+0000 7f03e3fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03ec197af0 0x7f03ec19be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35850/0 (socket says 172.21.15.6:35850) 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.004+0000 7f03e3fff640 1 -- 172.21.15.6:0/300465304 learned_addr learned my addr 172.21.15.6:0/300465304 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.004+0000 7f03eb7fe640 1 --2- 172.21.15.6:0/300465304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03ec1028c0 0x7f03ec1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.004+0000 7f03ebfff640 1 --2- 172.21.15.6:0/300465304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03ec19c3c0 0x7f03ec19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.004+0000 7f03e3fff640 1 -- 172.21.15.6:0/300465304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03ec1028c0 msgr2=0x7f03ec1975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.004+0000 7f03e3fff640 1 --2- 172.21.15.6:0/300465304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03ec1028c0 0x7f03ec1975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.004+0000 7f03e3fff640 1 -- 172.21.15.6:0/300465304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03ec19c3c0 msgr2=0x7f03ec19c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.004+0000 7f03e3fff640 1 --2- 172.21.15.6:0/300465304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03ec19c3c0 0x7f03ec19c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.004+0000 7f03e3fff640 1 -- 172.21.15.6:0/300465304 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ec1a4130 con 0x7f03ec197af0 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.004+0000 7f03e3fff640 1 --2- 172.21.15.6:0/300465304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03ec197af0 0x7f03ec19be80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f03d400b520 tx=0x7f03d400b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.004+0000 7f03e97fa640 1 -- 172.21.15.6:0/300465304 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4004230 con 0x7f03ec197af0 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.004+0000 7f03e97fa640 1 -- 172.21.15.6:0/300465304 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03d40043d0 con 0x7f03ec197af0 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.004+0000 7f03f1d0d640 1 -- 172.21.15.6:0/300465304 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ec1a4360 con 0x7f03ec197af0 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.004+0000 7f03e97fa640 1 -- 172.21.15.6:0/300465304 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4003d60 con 0x7f03ec197af0 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.004+0000 7f03f1d0d640 1 -- 172.21.15.6:0/300465304 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ec1a48c0 con 0x7f03ec197af0 2024-09-12T03:13:10.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.005+0000 7f03f1d0d640 1 -- 172.21.15.6:0/300465304 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03ec102d20 con 0x7f03ec197af0 2024-09-12T03:13:10.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.006+0000 7f03e97fa640 1 -- 172.21.15.6:0/300465304 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f03d400e070 con 0x7f03ec197af0 2024-09-12T03:13:10.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.006+0000 7f03e97fa640 1 --2- 172.21.15.6:0/300465304 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f03d80394f0 0x7f03d803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.006+0000 7f03e97fa640 1 -- 172.21.15.6:0/300465304 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f03d404b9f0 con 0x7f03ec197af0 2024-09-12T03:13:10.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.006+0000 7f03eb7fe640 1 --2- 172.21.15.6:0/300465304 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f03d80394f0 0x7f03d803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:10.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.006+0000 7f03eb7fe640 1 --2- 172.21.15.6:0/300465304 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f03d80394f0 0x7f03d803b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f03dc02abe0 tx=0x7f03dc02ab70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:10.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.010+0000 7f03e97fa640 1 -- 172.21.15.6:0/300465304 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f03d4014020 con 0x7f03ec197af0 2024-09-12T03:13:10.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.190+0000 7f03f1d0d640 1 -- 172.21.15.6:0/300465304 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f03ec1a4be0 con 0x7f03ec197af0 2024-09-12T03:13:10.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.191+0000 7f03e97fa640 1 -- 172.21.15.6:0/300465304 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f03d401f300 con 0x7f03ec197af0 2024-09-12T03:13:10.191 INFO:teuthology.orchestra.run.smithi006.stdout:# minimal ceph.conf for 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:13:10.191 INFO:teuthology.orchestra.run.smithi006.stdout:[global] 2024-09-12T03:13:10.192 INFO:teuthology.orchestra.run.smithi006.stdout: fsid = 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:13:10.192 INFO:teuthology.orchestra.run.smithi006.stdout: mon_host = [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] 2024-09-12T03:13:10.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.193+0000 7f03f1d0d640 1 -- 172.21.15.6:0/300465304 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f03d80394f0 msgr2=0x7f03d803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:10.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.193+0000 7f03f1d0d640 1 --2- 172.21.15.6:0/300465304 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f03d80394f0 0x7f03d803b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f03dc02abe0 tx=0x7f03dc02ab70 comp rx=0 tx=0).stop 2024-09-12T03:13:10.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.193+0000 7f03f1d0d640 1 -- 172.21.15.6:0/300465304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03ec197af0 msgr2=0x7f03ec19be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:10.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.193+0000 7f03f1d0d640 1 --2- 172.21.15.6:0/300465304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03ec197af0 0x7f03ec19be80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f03d400b520 tx=0x7f03d400b9e0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.193+0000 7f03f1d0d640 1 -- 172.21.15.6:0/300465304 shutdown_connections 2024-09-12T03:13:10.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.193+0000 7f03f1d0d640 1 --2- 172.21.15.6:0/300465304 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f03d80394f0 0x7f03d803b9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.193+0000 7f03f1d0d640 1 --2- 172.21.15.6:0/300465304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03ec19c3c0 0x7f03ec19c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.193+0000 7f03f1d0d640 1 --2- 172.21.15.6:0/300465304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03ec197af0 0x7f03ec19be80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.193+0000 7f03f1d0d640 1 --2- 172.21.15.6:0/300465304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03ec1028c0 0x7f03ec1975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.193+0000 7f03f1d0d640 1 -- 172.21.15.6:0/300465304 >> 172.21.15.6:0/300465304 conn(0x7f03ec0fc220 msgr2=0x7f03ec105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:10.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.193+0000 7f03f1d0d640 1 -- 172.21.15.6:0/300465304 shutdown_connections 2024-09-12T03:13:10.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:10.193+0000 7f03f1d0d640 1 -- 172.21.15.6:0/300465304 wait complete. 2024-09-12T03:13:10.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:10.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:13:10.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: mon.c calling monitor election 2024-09-12T03:13:10.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: mon.a calling monitor election 2024-09-12T03:13:10.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:10.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:13:10.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:10.714 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:10.714 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:10.714 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:10.714 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:10.714 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:10.714 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:10.714 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T03:13:10.714 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: monmap epoch 3 2024-09-12T03:13:10.715 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: fsid 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:13:10.715 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: last_changed 2024-09-12T03:13:03.322589+0000 2024-09-12T03:13:10.715 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: created 2024-09-12T03:11:20.144939+0000 2024-09-12T03:13:10.715 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: min_mon_release 19 (squid) 2024-09-12T03:13:10.715 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: election_strategy: 1 2024-09-12T03:13:10.715 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: 0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-09-12T03:13:10.716 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-12T03:13:10.716 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: 2: [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] mon.b 2024-09-12T03:13:10.716 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: fsmap 2024-09-12T03:13:10.716 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:13:10.716 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: mgrmap e13: x(active, since 49s) 2024-09-12T03:13:10.716 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: overall HEALTH_OK 2024-09-12T03:13:10.717 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:10.717 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:10.717 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:10.717 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:10.717 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:10.717 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: from='client.? 172.21.15.161:0/752012426' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:13:10.718 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:10.718 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:10 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:10.897 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-12T03:13:10.897 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-09-12T03:13:10.897 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T03:13:10.924 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-09-12T03:13:10.924 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:13:10.993 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-12T03:13:10.994 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T03:13:11.021 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-12T03:13:11.021 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:13:11.086 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-12T03:13:11.086 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T03:13:11.114 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-12T03:13:11.114 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:13:11.181 INFO:tasks.cephadm:Adding mgr.x on smithi006 2024-09-12T03:13:11.182 INFO:tasks.cephadm:Adding mgr.y on smithi118 2024-09-12T03:13:11.182 INFO:tasks.cephadm:Adding mgr.z on smithi161 2024-09-12T03:13:11.182 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi006=x;smithi118=y;smithi161=z' 2024-09-12T03:13:11.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: mon.b calling monitor election 2024-09-12T03:13:11.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: Updating smithi006:/etc/ceph/ceph.conf 2024-09-12T03:13:11.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: Updating smithi118:/etc/ceph/ceph.conf 2024-09-12T03:13:11.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: Updating smithi161:/etc/ceph/ceph.conf 2024-09-12T03:13:11.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: mon.b calling monitor election 2024-09-12T03:13:11.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: mon.c calling monitor election 2024-09-12T03:13:11.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: mon.a calling monitor election 2024-09-12T03:13:11.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T03:13:11.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: monmap epoch 3 2024-09-12T03:13:11.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: fsid 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:13:11.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: last_changed 2024-09-12T03:13:03.322589+0000 2024-09-12T03:13:11.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: created 2024-09-12T03:11:20.144939+0000 2024-09-12T03:13:11.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: min_mon_release 19 (squid) 2024-09-12T03:13:11.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: election_strategy: 1 2024-09-12T03:13:11.214 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: 0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-09-12T03:13:11.214 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-12T03:13:11.214 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: 2: [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] mon.b 2024-09-12T03:13:11.214 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: fsmap 2024-09-12T03:13:11.214 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:13:11.214 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: mgrmap e13: x(active, since 51s) 2024-09-12T03:13:11.214 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: overall HEALTH_OK 2024-09-12T03:13:11.214 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: Updating smithi006:/var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:13:11.214 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: Updating smithi161:/var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:13:11.214 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: Updating smithi118:/var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:13:11.214 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:11.214 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:11.214 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:11.214 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:11.214 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:11.215 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:11.215 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:11.215 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: Reconfiguring mon.a (monmap changed)... 2024-09-12T03:13:11.215 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:13:11.215 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:13:11.215 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:11.215 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:11 smithi118 ceph-mon[35048]: Reconfiguring daemon mon.a on smithi006 2024-09-12T03:13:11.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: mon.b calling monitor election 2024-09-12T03:13:11.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: Updating smithi006:/etc/ceph/ceph.conf 2024-09-12T03:13:11.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: Updating smithi118:/etc/ceph/ceph.conf 2024-09-12T03:13:11.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: Updating smithi161:/etc/ceph/ceph.conf 2024-09-12T03:13:11.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: mon.b calling monitor election 2024-09-12T03:13:11.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: mon.c calling monitor election 2024-09-12T03:13:11.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: mon.a calling monitor election 2024-09-12T03:13:11.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T03:13:11.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: monmap epoch 3 2024-09-12T03:13:11.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: fsid 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:13:11.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: last_changed 2024-09-12T03:13:03.322589+0000 2024-09-12T03:13:11.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: created 2024-09-12T03:11:20.144939+0000 2024-09-12T03:13:11.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: min_mon_release 19 (squid) 2024-09-12T03:13:11.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: election_strategy: 1 2024-09-12T03:13:11.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: 0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-09-12T03:13:11.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-12T03:13:11.366 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: 2: [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] mon.b 2024-09-12T03:13:11.366 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: fsmap 2024-09-12T03:13:11.366 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:13:11.366 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: mgrmap e13: x(active, since 51s) 2024-09-12T03:13:11.366 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: overall HEALTH_OK 2024-09-12T03:13:11.366 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: Updating smithi006:/var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:13:11.366 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: Updating smithi161:/var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:13:11.366 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: Updating smithi118:/var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:13:11.366 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:11.366 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:11.366 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:11.366 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:11.366 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:11.366 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:11.367 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:11.367 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: Reconfiguring mon.a (monmap changed)... 2024-09-12T03:13:11.367 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:13:11.367 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:13:11.367 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:11.367 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:11 smithi161 ceph-mon[35485]: Reconfiguring daemon mon.a on smithi006 2024-09-12T03:13:11.396 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: mon.b calling monitor election 2024-09-12T03:13:11.396 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: Updating smithi006:/etc/ceph/ceph.conf 2024-09-12T03:13:11.396 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: Updating smithi118:/etc/ceph/ceph.conf 2024-09-12T03:13:11.396 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: Updating smithi161:/etc/ceph/ceph.conf 2024-09-12T03:13:11.396 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: mon.b calling monitor election 2024-09-12T03:13:11.396 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: mon.c calling monitor election 2024-09-12T03:13:11.397 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: mon.a calling monitor election 2024-09-12T03:13:11.397 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T03:13:11.397 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: monmap epoch 3 2024-09-12T03:13:11.397 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: fsid 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:13:11.397 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: last_changed 2024-09-12T03:13:03.322589+0000 2024-09-12T03:13:11.397 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: created 2024-09-12T03:11:20.144939+0000 2024-09-12T03:13:11.397 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: min_mon_release 19 (squid) 2024-09-12T03:13:11.397 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: election_strategy: 1 2024-09-12T03:13:11.397 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: 0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-09-12T03:13:11.397 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-12T03:13:11.397 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: 2: [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] mon.b 2024-09-12T03:13:11.397 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: fsmap 2024-09-12T03:13:11.398 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:13:11.398 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: mgrmap e13: x(active, since 51s) 2024-09-12T03:13:11.398 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: overall HEALTH_OK 2024-09-12T03:13:11.398 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: Updating smithi006:/var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:13:11.398 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: Updating smithi161:/var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:13:11.398 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: Updating smithi118:/var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:13:11.398 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:11.398 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:11.398 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:11.398 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:11.398 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:11.398 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:11.398 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:11.398 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: Reconfiguring mon.a (monmap changed)... 2024-09-12T03:13:11.399 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:13:11.399 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:13:11.399 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:11.399 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:11 smithi006 ceph-mon[30991]: Reconfiguring daemon mon.a on smithi006 2024-09-12T03:13:11.461 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:13:11.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.863+0000 7f434c958640 1 -- 172.21.15.161:0/3430735344 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43481016e0 msgr2=0x7f4348101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:11.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.863+0000 7f434c958640 1 --2- 172.21.15.161:0/3430735344 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43481016e0 0x7f4348101ae0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4330009880 tx=0x7f433002a5d0 comp rx=0 tx=0).stop 2024-09-12T03:13:11.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.863+0000 7f434c958640 1 -- 172.21.15.161:0/3430735344 shutdown_connections 2024-09-12T03:13:11.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.863+0000 7f434c958640 1 --2- 172.21.15.161:0/3430735344 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43481016e0 0x7f4348101ae0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:11.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.863+0000 7f434c958640 1 -- 172.21.15.161:0/3430735344 >> 172.21.15.161:0/3430735344 conn(0x7f43480fce90 msgr2=0x7f43480ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:11.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.863+0000 7f434c958640 1 -- 172.21.15.161:0/3430735344 shutdown_connections 2024-09-12T03:13:11.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.863+0000 7f434c958640 1 -- 172.21.15.161:0/3430735344 wait complete. 2024-09-12T03:13:11.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.864+0000 7f434c958640 1 Processor -- start 2024-09-12T03:13:11.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.864+0000 7f434c958640 1 -- start start 2024-09-12T03:13:11.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.864+0000 7f434c958640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43481016e0 0x7f4348196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:11.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.864+0000 7f434c958640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4348197510 0x7f434819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:11.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.864+0000 7f434c958640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4348197990 0x7f434819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:11.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.864+0000 7f434c958640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f43481077d0 con 0x7f4348197510 2024-09-12T03:13:11.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.864+0000 7f434c958640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f4348107650 con 0x7f43481016e0 2024-09-12T03:13:11.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.864+0000 7f434c958640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4348107950 con 0x7f4348197990 2024-09-12T03:13:11.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.864+0000 7f4346d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4348197990 0x7f434819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:11.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.864+0000 7f4346d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4348197990 0x7f434819eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:52686/0 (socket says 172.21.15.161:52686) 2024-09-12T03:13:11.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.864+0000 7f4346d9d640 1 -- 172.21.15.161:0/1984630464 learned_addr learned my addr 172.21.15.161:0/1984630464 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-12T03:13:11.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.864+0000 7f434659c640 1 --2- 172.21.15.161:0/1984630464 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43481016e0 0x7f4348196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:11.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.864+0000 7f4345d9b640 1 --2- 172.21.15.161:0/1984630464 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4348197510 0x7f434819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:11.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.864+0000 7f4346d9d640 1 -- 172.21.15.161:0/1984630464 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43481016e0 msgr2=0x7f4348196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:11.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.864+0000 7f4346d9d640 1 --2- 172.21.15.161:0/1984630464 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43481016e0 0x7f4348196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:11.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.864+0000 7f4346d9d640 1 -- 172.21.15.161:0/1984630464 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4348197510 msgr2=0x7f434819c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:11.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.864+0000 7f4346d9d640 1 --2- 172.21.15.161:0/1984630464 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4348197510 0x7f434819c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:11.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.864+0000 7f4346d9d640 1 -- 172.21.15.161:0/1984630464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434819f3d0 con 0x7f4348197990 2024-09-12T03:13:11.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.865+0000 7f4346d9d640 1 --2- 172.21.15.161:0/1984630464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4348197990 0x7f434819eb10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f434000ba50 tx=0x7f434000bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:11.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.865+0000 7f433f7fe640 1 -- 172.21.15.161:0/1984630464 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4340019070 con 0x7f4348197990 2024-09-12T03:13:11.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.865+0000 7f433f7fe640 1 -- 172.21.15.161:0/1984630464 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f434000cbe0 con 0x7f4348197990 2024-09-12T03:13:11.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.865+0000 7f434c958640 1 -- 172.21.15.161:0/1984630464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f434819f6c0 con 0x7f4348197990 2024-09-12T03:13:11.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.865+0000 7f433f7fe640 1 -- 172.21.15.161:0/1984630464 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43400076a0 con 0x7f4348197990 2024-09-12T03:13:11.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.865+0000 7f434c958640 1 -- 172.21.15.161:0/1984630464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f434819fc30 con 0x7f4348197990 2024-09-12T03:13:11.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.866+0000 7f433f7fe640 1 -- 172.21.15.161:0/1984630464 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f434001c070 con 0x7f4348197990 2024-09-12T03:13:11.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.866+0000 7f434c958640 1 -- 172.21.15.161:0/1984630464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4348190590 con 0x7f4348197990 2024-09-12T03:13:11.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.866+0000 7f433f7fe640 1 --2- 172.21.15.161:0/1984630464 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f43340394f0 0x7f433403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:11.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.866+0000 7f433f7fe640 1 -- 172.21.15.161:0/1984630464 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f434004baa0 con 0x7f4348197990 2024-09-12T03:13:11.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.866+0000 7f434659c640 1 --2- 172.21.15.161:0/1984630464 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f43340394f0 0x7f433403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:11.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.867+0000 7f434659c640 1 --2- 172.21.15.161:0/1984630464 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f43340394f0 0x7f433403b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4330009230 tx=0x7f43300315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:11.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:11.871+0000 7f433f7fe640 1 -- 172.21.15.161:0/1984630464 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f43400112a0 con 0x7f4348197990 2024-09-12T03:13:12.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:12.046+0000 7f434c958640 1 -- 172.21.15.161:0/1984630464 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi006=x;smithi118=y;smithi161=z", "target": ["mon-mgr", ""]}) -- 0x7f43480626c0 con 0x7f43340394f0 2024-09-12T03:13:12.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:12.052+0000 7f433f7fe640 1 -- 172.21.15.161:0/1984630464 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f43480626c0 con 0x7f43340394f0 2024-09-12T03:13:12.053 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mgr update... 2024-09-12T03:13:12.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:12.054+0000 7f434c958640 1 -- 172.21.15.161:0/1984630464 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f43340394f0 msgr2=0x7f433403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:12.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:12.054+0000 7f434c958640 1 --2- 172.21.15.161:0/1984630464 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f43340394f0 0x7f433403b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4330009230 tx=0x7f43300315d0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:12.054+0000 7f434c958640 1 -- 172.21.15.161:0/1984630464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4348197990 msgr2=0x7f434819eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:12.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:12.054+0000 7f434c958640 1 --2- 172.21.15.161:0/1984630464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4348197990 0x7f434819eb10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f434000ba50 tx=0x7f434000bf10 comp rx=0 tx=0).stop 2024-09-12T03:13:12.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:12.055+0000 7f434c958640 1 -- 172.21.15.161:0/1984630464 shutdown_connections 2024-09-12T03:13:12.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:12.055+0000 7f434c958640 1 --2- 172.21.15.161:0/1984630464 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f43340394f0 0x7f433403b9b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:12.055+0000 7f434c958640 1 --2- 172.21.15.161:0/1984630464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4348197990 0x7f434819eb10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:12.055+0000 7f434c958640 1 --2- 172.21.15.161:0/1984630464 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4348197510 0x7f434819c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:12.055+0000 7f434c958640 1 --2- 172.21.15.161:0/1984630464 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43481016e0 0x7f4348196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:12.055+0000 7f434c958640 1 -- 172.21.15.161:0/1984630464 >> 172.21.15.161:0/1984630464 conn(0x7f43480fce90 msgr2=0x7f43480fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:12.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:12.055+0000 7f434c958640 1 -- 172.21.15.161:0/1984630464 shutdown_connections 2024-09-12T03:13:12.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:13:12.055+0000 7f434c958640 1 -- 172.21.15.161:0/1984630464 wait complete. 2024-09-12T03:13:12.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:12 smithi006 ceph-mon[30991]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:12.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:12 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:12.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:12 smithi006 ceph-mon[30991]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi006=x;smithi118=y;smithi161=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:13:12.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:12 smithi006 ceph-mon[30991]: Saving service mgr spec with placement smithi006=x;smithi118=y;smithi161=z;count:3 2024-09-12T03:13:12.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:12 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:12.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:12 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:12.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:12 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:12.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:12 smithi006 ceph-mon[30991]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T03:13:12.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:12 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:13:12.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:12 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:13:12.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:12 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:12.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:12 smithi006 ceph-mon[30991]: Reconfiguring daemon mgr.x on smithi006 2024-09-12T03:13:12.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:12 smithi118 ceph-mon[35048]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:12.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:12 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:12.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:12 smithi118 ceph-mon[35048]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi006=x;smithi118=y;smithi161=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:13:12.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:12 smithi118 ceph-mon[35048]: Saving service mgr spec with placement smithi006=x;smithi118=y;smithi161=z;count:3 2024-09-12T03:13:12.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:12 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:12.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:12 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:12.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:12 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:12.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:12 smithi118 ceph-mon[35048]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T03:13:12.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:12 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:13:12.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:12 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:13:12.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:12 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:12.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:12 smithi118 ceph-mon[35048]: Reconfiguring daemon mgr.x on smithi006 2024-09-12T03:13:12.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:12 smithi161 ceph-mon[35485]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:12.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:12 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:13:12.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:12 smithi161 ceph-mon[35485]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi006=x;smithi118=y;smithi161=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:13:12.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:12 smithi161 ceph-mon[35485]: Saving service mgr spec with placement smithi006=x;smithi118=y;smithi161=z;count:3 2024-09-12T03:13:12.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:12 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:12.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:12 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:12.590 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:12 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:12.590 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:12 smithi161 ceph-mon[35485]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T03:13:12.590 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:12 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:13:12.590 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:12 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:13:12.590 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:12 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:12.590 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:12 smithi161 ceph-mon[35485]: Reconfiguring daemon mgr.x on smithi006 2024-09-12T03:13:12.696 DEBUG:teuthology.orchestra.run.smithi118:mgr.y> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mgr.y.service 2024-09-12T03:13:12.699 DEBUG:teuthology.orchestra.run.smithi161:mgr.z> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mgr.z.service 2024-09-12T03:13:12.701 INFO:tasks.cephadm:Deploying OSDs... 2024-09-12T03:13:12.701 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-09-12T03:13:12.702 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T03:13:12.718 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T03:13:12.718 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_1 2024-09-12T03:13:12.774 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T03:13:12.774 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:13:12.774 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-12T03:13:12.774 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:13:12.774 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:13:12.774 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-09-12 03:12:30.369310177 +0000 2024-09-12T03:13:12.774 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-09-12 03:07:48.633496636 +0000 2024-09-12T03:13:12.774 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-09-12 03:07:48.633496636 +0000 2024-09-12T03:13:12.774 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-09-12 03:07:48.633496636 +0000 2024-09-12T03:13:12.775 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T03:13:12.838 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-09-12T03:13:12.838 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-09-12T03:13:12.838 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000206645 s, 2.5 MB/s 2024-09-12T03:13:12.840 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T03:13:12.898 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_2 2024-09-12T03:13:12.953 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T03:13:12.953 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:13:12.954 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-12T03:13:12.954 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:13:12.954 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:13:12.954 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-09-12 03:12:30.401309833 +0000 2024-09-12T03:13:12.954 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-09-12 03:07:48.961493118 +0000 2024-09-12T03:13:12.954 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-09-12 03:07:48.961493118 +0000 2024-09-12T03:13:12.954 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-09-12 03:07:48.961493118 +0000 2024-09-12T03:13:12.954 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T03:13:13.019 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-09-12T03:13:13.019 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-09-12T03:13:13.019 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000177443 s, 2.9 MB/s 2024-09-12T03:13:13.020 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T03:13:13.077 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_3 2024-09-12T03:13:13.134 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T03:13:13.134 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:13:13.134 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-12T03:13:13.134 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:13:13.134 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:13:13.134 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-09-12 03:12:30.435309467 +0000 2024-09-12T03:13:13.134 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-09-12 03:07:49.295489535 +0000 2024-09-12T03:13:13.134 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-09-12 03:07:49.295489535 +0000 2024-09-12T03:13:13.134 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-09-12 03:07:49.295489535 +0000 2024-09-12T03:13:13.135 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T03:13:13.198 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-09-12T03:13:13.198 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-09-12T03:13:13.198 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000209972 s, 2.4 MB/s 2024-09-12T03:13:13.199 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T03:13:13.256 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_4 2024-09-12T03:13:13.313 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T03:13:13.313 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:13:13.313 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-12T03:13:13.314 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:13:13.314 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:13:13.314 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-09-12 03:12:30.470309090 +0000 2024-09-12T03:13:13.314 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-09-12 03:07:49.633485909 +0000 2024-09-12T03:13:13.314 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-09-12 03:07:49.633485909 +0000 2024-09-12T03:13:13.314 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-09-12 03:07:49.633485909 +0000 2024-09-12T03:13:13.314 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T03:13:13.380 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-09-12T03:13:13.381 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-09-12T03:13:13.381 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.00023821 s, 2.1 MB/s 2024-09-12T03:13:13.382 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T03:13:13.438 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-12T03:13:13.438 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T03:13:13.457 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T03:13:13.457 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_1 2024-09-12T03:13:13.515 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T03:13:13.516 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:13:13.516 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-12T03:13:13.516 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:13:13.516 INFO:teuthology.orchestra.run.smithi118.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:13:13.516 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-09-12 03:13:02.751071307 +0000 2024-09-12T03:13:13.516 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-09-12 03:07:56.990884655 +0000 2024-09-12T03:13:13.516 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-09-12 03:07:56.990884655 +0000 2024-09-12T03:13:13.516 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: 2024-09-12 03:07:56.990884655 +0000 2024-09-12T03:13:13.516 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T03:13:13.581 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-09-12T03:13:13.581 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-09-12T03:13:13.581 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.00011766 s, 4.4 MB/s 2024-09-12T03:13:13.583 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T03:13:13.639 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_2 2024-09-12T03:13:13.696 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T03:13:13.696 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:13:13.697 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-12T03:13:13.697 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:13:13.697 INFO:teuthology.orchestra.run.smithi118.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:13:13.697 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-09-12 03:13:02.752071284 +0000 2024-09-12T03:13:13.697 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-09-12 03:07:57.304877558 +0000 2024-09-12T03:13:13.697 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-09-12 03:07:57.304877558 +0000 2024-09-12T03:13:13.697 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: 2024-09-12 03:07:57.304877558 +0000 2024-09-12T03:13:13.697 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T03:13:13.764 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-09-12T03:13:13.764 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-09-12T03:13:13.764 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000113339 s, 4.5 MB/s 2024-09-12T03:13:13.766 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T03:13:13.822 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_3 2024-09-12T03:13:13.877 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T03:13:13.877 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:13:13.877 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-12T03:13:13.877 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:13:13.877 INFO:teuthology.orchestra.run.smithi118.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:13:13.877 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-09-12 03:13:02.752071284 +0000 2024-09-12T03:13:13.878 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-09-12 03:07:57.611870618 +0000 2024-09-12T03:13:13.878 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-09-12 03:07:57.611870618 +0000 2024-09-12T03:13:13.878 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: 2024-09-12 03:07:57.611870618 +0000 2024-09-12T03:13:13.878 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T03:13:13.942 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-09-12T03:13:13.942 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-09-12T03:13:13.942 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 8.6024e-05 s, 6.0 MB/s 2024-09-12T03:13:13.943 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T03:13:14.001 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_4 2024-09-12T03:13:14.057 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T03:13:14.057 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:13:14.057 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-12T03:13:14.057 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:13:14.057 INFO:teuthology.orchestra.run.smithi118.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:13:14.057 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-09-12 03:13:02.752071284 +0000 2024-09-12T03:13:14.057 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-09-12 03:07:57.916863725 +0000 2024-09-12T03:13:14.057 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-09-12 03:07:57.916863725 +0000 2024-09-12T03:13:14.057 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: 2024-09-12 03:07:57.916863725 +0000 2024-09-12T03:13:14.058 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T03:13:14.121 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-09-12T03:13:14.121 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-09-12T03:13:14.121 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000114094 s, 4.5 MB/s 2024-09-12T03:13:14.123 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T03:13:14.179 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-12T03:13:14.179 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T03:13:14.199 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T03:13:14.199 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2024-09-12T03:13:14.258 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T03:13:14.258 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:13:14.258 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-12T03:13:14.258 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:13:14.258 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:13:14.258 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-09-12 03:12:53.513296315 +0000 2024-09-12T03:13:14.258 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-09-12 03:08:06.896143452 +0000 2024-09-12T03:13:14.258 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-09-12 03:08:06.896143452 +0000 2024-09-12T03:13:14.258 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-09-12 03:08:06.896143452 +0000 2024-09-12T03:13:14.259 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T03:13:14.326 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-09-12T03:13:14.326 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-09-12T03:13:14.326 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000206017 s, 2.5 MB/s 2024-09-12T03:13:14.327 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T03:13:14.387 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2024-09-12T03:13:14.445 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T03:13:14.445 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:13:14.445 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-12T03:13:14.445 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:13:14.445 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:13:14.445 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-09-12 03:12:53.513296315 +0000 2024-09-12T03:13:14.445 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-09-12 03:08:07.226136063 +0000 2024-09-12T03:13:14.445 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-09-12 03:08:07.226136063 +0000 2024-09-12T03:13:14.445 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-09-12 03:08:07.226136063 +0000 2024-09-12T03:13:14.446 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T03:13:14.511 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-09-12T03:13:14.511 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-09-12T03:13:14.511 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000212979 s, 2.4 MB/s 2024-09-12T03:13:14.512 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T03:13:14.575 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2024-09-12T03:13:14.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:14 smithi161 ceph-mon[35485]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:14.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:14 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:14.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:14 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:14.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:14 smithi161 ceph-mon[35485]: Reconfiguring mon.b (monmap changed)... 2024-09-12T03:13:14.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:14 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:13:14.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:14 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:13:14.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:14 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:14.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:14 smithi161 ceph-mon[35485]: Reconfiguring daemon mon.b on smithi118 2024-09-12T03:13:14.631 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T03:13:14.631 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:13:14.631 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-12T03:13:14.631 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:13:14.631 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:13:14.632 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-09-12 03:12:53.514296292 +0000 2024-09-12T03:13:14.632 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-09-12 03:08:07.602127644 +0000 2024-09-12T03:13:14.632 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-09-12 03:08:07.602127644 +0000 2024-09-12T03:13:14.632 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-09-12 03:08:07.602127644 +0000 2024-09-12T03:13:14.632 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T03:13:14.697 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-09-12T03:13:14.698 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-09-12T03:13:14.698 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000199564 s, 2.6 MB/s 2024-09-12T03:13:14.699 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T03:13:14.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:14 smithi118 ceph-mon[35048]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:14.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:14 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:14.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:14 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:14.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:14 smithi118 ceph-mon[35048]: Reconfiguring mon.b (monmap changed)... 2024-09-12T03:13:14.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:14 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:13:14.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:14 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:13:14.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:14 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:14.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:14 smithi118 ceph-mon[35048]: Reconfiguring daemon mon.b on smithi118 2024-09-12T03:13:14.757 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2024-09-12T03:13:14.813 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T03:13:14.814 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:13:14.814 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-12T03:13:14.814 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:13:14.814 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:13:14.814 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-09-12 03:12:53.514296292 +0000 2024-09-12T03:13:14.814 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-09-12 03:08:07.958119673 +0000 2024-09-12T03:13:14.814 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-09-12 03:08:07.958119673 +0000 2024-09-12T03:13:14.814 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-09-12 03:08:07.958119673 +0000 2024-09-12T03:13:14.814 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T03:13:14.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:14 smithi006 ceph-mon[30991]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:14.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:14 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:14.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:14 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:14.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:14 smithi006 ceph-mon[30991]: Reconfiguring mon.b (monmap changed)... 2024-09-12T03:13:14.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:14 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:13:14.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:14 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:13:14.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:14 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:14.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:14 smithi006 ceph-mon[30991]: Reconfiguring daemon mon.b on smithi118 2024-09-12T03:13:14.878 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-09-12T03:13:14.878 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-09-12T03:13:14.878 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000179557 s, 2.9 MB/s 2024-09-12T03:13:14.879 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T03:13:14.936 INFO:tasks.cephadm:Deploying osd.0 on smithi006 with /dev/vg_nvme/lv_4... 2024-09-12T03:13:14.936 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T03:13:15.199 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:16.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:15 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:16.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:15 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:16.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:15 smithi006 ceph-mon[30991]: Reconfiguring mon.c (monmap changed)... 2024-09-12T03:13:16.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:15 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:13:16.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:15 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:13:16.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:15 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:16.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:15 smithi006 ceph-mon[30991]: Reconfiguring daemon mon.c on smithi161 2024-09-12T03:13:16.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:15 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:16.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:15 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:16.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:15 smithi161 ceph-mon[35485]: Reconfiguring mon.c (monmap changed)... 2024-09-12T03:13:16.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:15 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:13:16.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:15 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:13:16.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:15 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:16.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:15 smithi161 ceph-mon[35485]: Reconfiguring daemon mon.c on smithi161 2024-09-12T03:13:16.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:15 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:16.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:15 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:16.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:15 smithi118 ceph-mon[35048]: Reconfiguring mon.c (monmap changed)... 2024-09-12T03:13:16.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:15 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:13:16.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:15 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:13:16.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:15 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:16.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:15 smithi118 ceph-mon[35048]: Reconfiguring daemon mon.c on smithi161 2024-09-12T03:13:17.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:17 smithi006 ceph-mon[30991]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:17.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:17.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:17.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:17.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:17.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:17.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:17.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:13:17.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:13:17.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:13:17.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:17.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:17 smithi006 ceph-mon[30991]: Deploying daemon mgr.z on smithi161 2024-09-12T03:13:17.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:17 smithi161 ceph-mon[35485]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:17.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:17.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:17.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:17.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:17.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:17.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:17.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:13:17.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:13:17.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:13:17.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:17.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:17 smithi161 ceph-mon[35485]: Deploying daemon mgr.z on smithi161 2024-09-12T03:13:17.454 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:13:17.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:17 smithi118 ceph-mon[35048]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:17.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:17.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:17.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:17.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:17.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:17.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:17.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:13:17.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:13:17.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:13:17.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:17.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:17 smithi118 ceph-mon[35048]: Deploying daemon mgr.z on smithi161 2024-09-12T03:13:17.481 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi006:vg_nvme/lv_4 2024-09-12T03:13:17.733 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:18.113 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:18 smithi161 podman[37274]: 2024-09-12 03:13:18.073686954 +0000 UTC m=+0.121411672 container create 27802b55bbb014b4d350737013d3cc92106d2f6924f73babadc91be61924bde2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:13:18.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.198+0000 7f59630e1640 1 -- 172.21.15.6:0/1638913922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f595c101c00 msgr2=0x7f595c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:18.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.198+0000 7f59630e1640 1 --2- 172.21.15.6:0/1638913922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f595c101c00 0x7f595c102000 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5950009900 tx=0x7f595002a410 comp rx=0 tx=0).stop 2024-09-12T03:13:18.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.198+0000 7f59630e1640 1 -- 172.21.15.6:0/1638913922 shutdown_connections 2024-09-12T03:13:18.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.198+0000 7f59630e1640 1 --2- 172.21.15.6:0/1638913922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f595c1037c0 0x7f595c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.198+0000 7f59630e1640 1 --2- 172.21.15.6:0/1638913922 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f595c102e00 0x7f595c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.198+0000 7f59630e1640 1 --2- 172.21.15.6:0/1638913922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f595c101c00 0x7f595c102000 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.198+0000 7f59630e1640 1 -- 172.21.15.6:0/1638913922 >> 172.21.15.6:0/1638913922 conn(0x7f595c0fd390 msgr2=0x7f595c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:18.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.198+0000 7f59630e1640 1 -- 172.21.15.6:0/1638913922 shutdown_connections 2024-09-12T03:13:18.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.198+0000 7f59630e1640 1 -- 172.21.15.6:0/1638913922 wait complete. 2024-09-12T03:13:18.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.199+0000 7f59630e1640 1 Processor -- start 2024-09-12T03:13:18.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.199+0000 7f59630e1640 1 -- start start 2024-09-12T03:13:18.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.199+0000 7f59630e1640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f595c101c00 0x7f595c197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.199+0000 7f59630e1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f595c102e00 0x7f595c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.199+0000 7f59630e1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f595c1037c0 0x7f595c19f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.199+0000 7f59630e1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f595c10d8e0 con 0x7f595c102e00 2024-09-12T03:13:18.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.199+0000 7f59630e1640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f595c10d760 con 0x7f595c101c00 2024-09-12T03:13:18.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.199+0000 7f59630e1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f595c10da60 con 0x7f595c1037c0 2024-09-12T03:13:18.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.199+0000 7f595bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f595c102e00 0x7f595c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:18.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.199+0000 7f595bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f595c102e00 0x7f595c1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35866/0 (socket says 172.21.15.6:35866) 2024-09-12T03:13:18.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.199+0000 7f595bfff640 1 -- 172.21.15.6:0/1752173975 learned_addr learned my addr 172.21.15.6:0/1752173975 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:13:18.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.199+0000 7f596167e640 1 --2- 172.21.15.6:0/1752173975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f595c1037c0 0x7f595c19f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:18.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.200+0000 7f5960e7d640 1 --2- 172.21.15.6:0/1752173975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f595c101c00 0x7f595c197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:18.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.200+0000 7f595bfff640 1 -- 172.21.15.6:0/1752173975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f595c1037c0 msgr2=0x7f595c19f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:18.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.200+0000 7f595bfff640 1 --2- 172.21.15.6:0/1752173975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f595c1037c0 0x7f595c19f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.200+0000 7f595bfff640 1 -- 172.21.15.6:0/1752173975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f595c101c00 msgr2=0x7f595c197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:18.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.200+0000 7f595bfff640 1 --2- 172.21.15.6:0/1752173975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f595c101c00 0x7f595c197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.200+0000 7f595bfff640 1 -- 172.21.15.6:0/1752173975 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595c19fbc0 con 0x7f595c102e00 2024-09-12T03:13:18.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.200+0000 7f5960e7d640 1 --2- 172.21.15.6:0/1752173975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f595c101c00 0x7f595c197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:18.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.200+0000 7f595bfff640 1 --2- 172.21.15.6:0/1752173975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f595c102e00 0x7f595c1984d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f594800b4a0 tx=0x7f594800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:18.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.200+0000 7f5959ffb640 1 -- 172.21.15.6:0/1752173975 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5948014070 con 0x7f595c102e00 2024-09-12T03:13:18.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.200+0000 7f5959ffb640 1 -- 172.21.15.6:0/1752173975 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5948010af0 con 0x7f595c102e00 2024-09-12T03:13:18.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.200+0000 7f59630e1640 1 -- 172.21.15.6:0/1752173975 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595c19feb0 con 0x7f595c102e00 2024-09-12T03:13:18.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.200+0000 7f5959ffb640 1 -- 172.21.15.6:0/1752173975 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5948017d10 con 0x7f595c102e00 2024-09-12T03:13:18.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.200+0000 7f59630e1640 1 -- 172.21.15.6:0/1752173975 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f595c1a03b0 con 0x7f595c102e00 2024-09-12T03:13:18.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.201+0000 7f5959ffb640 1 -- 172.21.15.6:0/1752173975 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f5948017070 con 0x7f595c102e00 2024-09-12T03:13:18.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.201+0000 7f5959ffb640 1 --2- 172.21.15.6:0/1752173975 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f594c0392c0 0x7f594c03b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.202+0000 7f5959ffb640 1 -- 172.21.15.6:0/1752173975 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f594804b6a0 con 0x7f595c102e00 2024-09-12T03:13:18.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.202+0000 7f59630e1640 1 -- 172.21.15.6:0/1752173975 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f595c1090a0 con 0x7f595c102e00 2024-09-12T03:13:18.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.202+0000 7f5960e7d640 1 --2- 172.21.15.6:0/1752173975 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f594c0392c0 0x7f594c03b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:18.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.202+0000 7f5960e7d640 1 --2- 172.21.15.6:0/1752173975 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f594c0392c0 0x7f594c03b780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5950006010 tx=0x7f5950032dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:18.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.206+0000 7f5959ffb640 1 -- 172.21.15.6:0/1752173975 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5948020db0 con 0x7f595c102e00 2024-09-12T03:13:18.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:18.390+0000 7f59630e1640 1 -- 172.21.15.6:0/1752173975 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f595c062630 con 0x7f594c0392c0 2024-09-12T03:13:18.432 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:18 smithi161 podman[37274]: 2024-09-12 03:13:18.298091551 +0000 UTC m=+0.345816272 container init 27802b55bbb014b4d350737013d3cc92106d2f6924f73babadc91be61924bde2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T03:13:18.432 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:18 smithi161 podman[37274]: 2024-09-12 03:13:18.301003233 +0000 UTC m=+0.348727954 container start 27802b55bbb014b4d350737013d3cc92106d2f6924f73babadc91be61924bde2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-12T03:13:18.432 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:18 smithi161 ceph-mgr[37293]: -- 172.21.15.161:0/1334924355 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564b530cd0e0 con 0x564b530ab400 2024-09-12T03:13:18.432 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:18 smithi161 bash[37274]: 27802b55bbb014b4d350737013d3cc92106d2f6924f73babadc91be61924bde2 2024-09-12T03:13:18.432 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:18 smithi161 systemd[1]: Started Ceph mgr.z for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:13:18.714 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:18 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:18.431+0000 7f38359e7040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:13:18.714 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:18 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:18.499+0000 7f38359e7040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:13:18.714 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:18 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:18.648+0000 7f38359e7040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:13:18.714 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:18 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:18.713+0000 7f38359e7040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:13:19.004 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:18 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:18.862+0000 7f38359e7040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:13:19.005 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:19 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:19.004+0000 7f38359e7040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:13:19.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:19 smithi118 ceph-mon[35048]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:19.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:13:19.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:13:19.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:19.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:19.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:19.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:19.214 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:13:19.214 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:13:19.214 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:13:19.214 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:19.324 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:19 smithi161 ceph-mon[35485]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:19.324 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:13:19.324 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:13:19.324 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:19.324 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:19.324 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:19.324 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:19.325 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:13:19.325 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:13:19.325 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:13:19.325 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:19 smithi006 ceph-mon[30991]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:13:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:13:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:13:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:13:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:13:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:19.613 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:19 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:19.323+0000 7f38359e7040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:13:19.613 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:19 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:19.384+0000 7f38359e7040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:13:19.613 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:19 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:19.569+0000 7f38359e7040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:13:20.158 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:20 smithi118 ceph-mon[35048]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:13:20.159 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:20 smithi118 ceph-mon[35048]: Deploying daemon mgr.y on smithi118 2024-09-12T03:13:20.186 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:20 smithi006 ceph-mon[30991]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:13:20.186 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:20 smithi006 ceph-mon[30991]: Deploying daemon mgr.y on smithi118 2024-09-12T03:13:20.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:20 smithi161 ceph-mon[35485]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:13:20.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:20 smithi161 ceph-mon[35485]: Deploying daemon mgr.y on smithi118 2024-09-12T03:13:20.462 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:20 smithi118 systemd[1]: Starting Ceph mgr.y for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:13:20.812 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:20 smithi118 podman[36399]: 2024-09-12 03:13:20.447239447 +0000 UTC m=+0.015790865 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:13:20.812 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:20 smithi118 podman[36399]: 2024-09-12 03:13:20.551798237 +0000 UTC m=+0.120349652 container create 19c6b21ba90fdbd48a95743c71bdd4c34e1f6e4dc2ba41bb45d6ca4ff72d2477 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:13:20.812 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:20 smithi118 podman[36399]: 2024-09-12 03:13:20.754346275 +0000 UTC m=+0.322897683 container init 19c6b21ba90fdbd48a95743c71bdd4c34e1f6e4dc2ba41bb45d6ca4ff72d2477 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-12T03:13:20.812 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:20 smithi118 podman[36399]: 2024-09-12 03:13:20.757059032 +0000 UTC m=+0.325610443 container start 19c6b21ba90fdbd48a95743c71bdd4c34e1f6e4dc2ba41bb45d6ca4ff72d2477 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:13:20.812 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:20 smithi118 ceph-mgr[36419]: -- 172.21.15.118:0/8079699 <== mon.2 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f855f230e0 con 0x55f855f01400 2024-09-12T03:13:20.813 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:20 smithi118 bash[36399]: 19c6b21ba90fdbd48a95743c71bdd4c34e1f6e4dc2ba41bb45d6ca4ff72d2477 2024-09-12T03:13:20.863 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:20 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:20.459+0000 7f38359e7040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:13:20.863 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:20 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:20.526+0000 7f38359e7040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:13:21.065 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:20 smithi118 systemd[1]: Started Ceph mgr.y for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:13:21.065 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:20 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:20.882+0000 7f107ef86040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:13:21.065 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:20 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:20.940+0000 7f107ef86040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:13:21.328 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:21 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:21.097+0000 7f107ef86040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:13:21.328 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:21 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:21.166+0000 7f107ef86040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:13:21.328 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:21 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:21.327+0000 7f107ef86040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:13:21.328 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:21 smithi118 ceph-mon[35048]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:21.329 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:21 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3788443929' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f55d68b-95a0-4ee2-a46f-bc422b38471e"}]: dispatch 2024-09-12T03:13:21.329 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:21 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3788443929' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f55d68b-95a0-4ee2-a46f-bc422b38471e"}]': finished 2024-09-12T03:13:21.329 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:21 smithi118 ceph-mon[35048]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T03:13:21.329 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:21 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:13:21.329 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:21 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:21.330 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:21 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:21.330 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:21 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:21.330 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:21 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:21.330 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:21 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:21.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:21 smithi006 ceph-mon[30991]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:21.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:21 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3788443929' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f55d68b-95a0-4ee2-a46f-bc422b38471e"}]: dispatch 2024-09-12T03:13:21.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:21 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3788443929' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f55d68b-95a0-4ee2-a46f-bc422b38471e"}]': finished 2024-09-12T03:13:21.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:21 smithi006 ceph-mon[30991]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T03:13:21.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:21 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:13:21.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:21 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:21.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:21 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:21.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:21 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:21.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:21 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:21.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:21 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:21.363 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:21 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:21.280+0000 7f38359e7040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:13:21.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:21 smithi161 ceph-mon[35485]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:21.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:21 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3788443929' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f55d68b-95a0-4ee2-a46f-bc422b38471e"}]: dispatch 2024-09-12T03:13:21.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:21 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3788443929' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f55d68b-95a0-4ee2-a46f-bc422b38471e"}]': finished 2024-09-12T03:13:21.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:21 smithi161 ceph-mon[35485]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T03:13:21.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:21 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:13:21.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:21 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:21.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:21 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:21.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:21 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:21.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:21 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:21.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:21 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:21.684 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:21 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:21.484+0000 7f107ef86040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:13:21.763 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:21 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:21.413+0000 7f38359e7040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:13:21.763 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:21 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:21.557+0000 7f38359e7040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:13:21.763 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:21 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:21.620+0000 7f38359e7040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:13:21.962 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:21 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:21.810+0000 7f107ef86040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:13:21.962 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:21 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:21.872+0000 7f107ef86040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:13:22.072 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:21 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:21.781+0000 7f38359e7040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:13:22.072 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:21 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:21.849+0000 7f38359e7040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:13:22.072 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:21 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:21.912+0000 7f38359e7040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:13:22.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:22 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/791438133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:13:22.362 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:22 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/791438133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:13:22.362 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:22 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:22.074+0000 7f38359e7040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:13:22.362 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:22 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:22.146+0000 7f38359e7040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:13:22.362 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:22 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:22.215+0000 7f38359e7040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:13:22.362 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:22 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:22.300+0000 7f38359e7040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:13:22.362 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:22 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:22.361+0000 7f38359e7040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:13:22.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:22 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/791438133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:13:22.462 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:22 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:22.059+0000 7f107ef86040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:13:22.613 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:22 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:22.421+0000 7f38359e7040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:13:23.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:23 smithi006 ceph-mon[30991]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:23.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:23 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:23.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:23 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:23.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:23 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:23.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:23 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:23.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:23 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:23.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:23 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:23.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:23 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:23.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:23 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:23.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:23 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:23.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:23 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:23.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:23 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:23.363 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:13:23 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z[37289]: 2024-09-12T03:13:23.216+0000 7f38359e7040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:13:23.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:23 smithi161 ceph-mon[35485]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:23.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:23 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:23.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:23 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:23.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:23 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:23.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:23 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:23.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:23 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:23.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:23 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:23.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:23 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:23.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:23 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:23.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:23 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:23.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:23 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:23.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:23 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:23.462 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:22 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:22.965+0000 7f107ef86040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:13:23.462 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:23 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:23.032+0000 7f107ef86040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:13:23.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:23 smithi118 ceph-mon[35048]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:23.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:23 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:23.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:23 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:23.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:23 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:23.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:23 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:23.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:23 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:23.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:23 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:23.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:23 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:23.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:23 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:23.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:23 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:23.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:23 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:23.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:23 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:23.997 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:23 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:23.743+0000 7f107ef86040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:13:23.998 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:23 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:23.857+0000 7f107ef86040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:13:23.998 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:23 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:23.997+0000 7f107ef86040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:13:24.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:24.082+0000 7f5959ffb640 1 -- 172.21.15.6:0/1752173975 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 14) ==== 92059+0+0 (secure 0 0 0) 0x7f5948010ca0 con 0x7f595c102e00 2024-09-12T03:13:24.276 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:24 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:24.058+0000 7f107ef86040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:13:24.276 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:24 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:24.208+0000 7f107ef86040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:13:24.276 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:24 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:24.276+0000 7f107ef86040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:13:24.276 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:24 smithi118 ceph-mon[35048]: Standby manager daemon z started 2024-09-12T03:13:24.276 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:24 smithi118 ceph-mon[35048]: from='mgr.? 172.21.15.161:0/1738857663' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T03:13:24.277 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:24 smithi118 ceph-mon[35048]: from='mgr.? 172.21.15.161:0/1738857663' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:13:24.277 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:24 smithi118 ceph-mon[35048]: from='mgr.? 172.21.15.161:0/1738857663' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T03:13:24.277 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:24 smithi118 ceph-mon[35048]: from='mgr.? 172.21.15.161:0/1738857663' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:13:24.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:24 smithi006 ceph-mon[30991]: Standby manager daemon z started 2024-09-12T03:13:24.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:24 smithi006 ceph-mon[30991]: from='mgr.? 172.21.15.161:0/1738857663' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T03:13:24.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:24 smithi006 ceph-mon[30991]: from='mgr.? 172.21.15.161:0/1738857663' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:13:24.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:24 smithi006 ceph-mon[30991]: from='mgr.? 172.21.15.161:0/1738857663' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T03:13:24.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:24 smithi006 ceph-mon[30991]: from='mgr.? 172.21.15.161:0/1738857663' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:13:24.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:24 smithi161 ceph-mon[35485]: Standby manager daemon z started 2024-09-12T03:13:24.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:24 smithi161 ceph-mon[35485]: from='mgr.? 172.21.15.161:0/1738857663' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T03:13:24.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:24 smithi161 ceph-mon[35485]: from='mgr.? 172.21.15.161:0/1738857663' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:13:24.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:24 smithi161 ceph-mon[35485]: from='mgr.? 172.21.15.161:0/1738857663' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T03:13:24.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:24 smithi161 ceph-mon[35485]: from='mgr.? 172.21.15.161:0/1738857663' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:13:24.558 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:24 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:24.336+0000 7f107ef86040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:13:24.558 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:24 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:24.493+0000 7f107ef86040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:13:24.559 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:24 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:24.558+0000 7f107ef86040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:13:24.826 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:24 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:24.625+0000 7f107ef86040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:13:24.826 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:24 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:24.707+0000 7f107ef86040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:13:24.826 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:24 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:24.766+0000 7f107ef86040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:13:24.826 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:24 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:24.825+0000 7f107ef86040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:13:25.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:25 smithi006 ceph-mon[30991]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:25.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:25 smithi006 ceph-mon[30991]: mgrmap e14: x(active, since 64s), standbys: z 2024-09-12T03:13:25.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:25 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T03:13:25.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:25 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:25.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:25 smithi161 ceph-mon[35485]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:25.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:25 smithi161 ceph-mon[35485]: mgrmap e14: x(active, since 64s), standbys: z 2024-09-12T03:13:25.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:25 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T03:13:25.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:25 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:25.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:25 smithi118 ceph-mon[35048]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:25.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:25 smithi118 ceph-mon[35048]: mgrmap e14: x(active, since 64s), standbys: z 2024-09-12T03:13:25.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:25 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T03:13:25.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:25 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:25.962 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:13:25 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y[36415]: 2024-09-12T03:13:25.610+0000 7f107ef86040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:13:26.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:26.091+0000 7f5959ffb640 1 -- 172.21.15.6:0/1752173975 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f594800fc00 con 0x7f595c102e00 2024-09-12T03:13:26.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:26 smithi006 ceph-mon[30991]: Standby manager daemon y started 2024-09-12T03:13:26.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:26 smithi006 ceph-mon[30991]: from='mgr.? 172.21.15.118:0/3690984180' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T03:13:26.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:26 smithi006 ceph-mon[30991]: from='mgr.? 172.21.15.118:0/3690984180' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:13:26.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:26 smithi006 ceph-mon[30991]: from='mgr.? 172.21.15.118:0/3690984180' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T03:13:26.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:26 smithi006 ceph-mon[30991]: from='mgr.? 172.21.15.118:0/3690984180' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:13:26.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:26 smithi161 ceph-mon[35485]: Standby manager daemon y started 2024-09-12T03:13:26.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:26 smithi161 ceph-mon[35485]: from='mgr.? 172.21.15.118:0/3690984180' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T03:13:26.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:26 smithi161 ceph-mon[35485]: from='mgr.? 172.21.15.118:0/3690984180' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:13:26.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:26 smithi161 ceph-mon[35485]: from='mgr.? 172.21.15.118:0/3690984180' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T03:13:26.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:26 smithi161 ceph-mon[35485]: from='mgr.? 172.21.15.118:0/3690984180' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:13:26.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:26 smithi118 ceph-mon[35048]: Standby manager daemon y started 2024-09-12T03:13:26.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:26 smithi118 ceph-mon[35048]: from='mgr.? 172.21.15.118:0/3690984180' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T03:13:26.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:26 smithi118 ceph-mon[35048]: from='mgr.? 172.21.15.118:0/3690984180' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:13:26.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:26 smithi118 ceph-mon[35048]: from='mgr.? 172.21.15.118:0/3690984180' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T03:13:26.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:26 smithi118 ceph-mon[35048]: from='mgr.? 172.21.15.118:0/3690984180' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:13:27.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:27 smithi006 ceph-mon[30991]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:27.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:27 smithi006 ceph-mon[30991]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-12T03:13:27.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:27 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T03:13:27.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:27 smithi161 ceph-mon[35485]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:27.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:27 smithi161 ceph-mon[35485]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-12T03:13:27.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:27 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T03:13:27.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:27 smithi118 ceph-mon[35048]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:27.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:27 smithi118 ceph-mon[35048]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-12T03:13:27.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:27 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T03:13:29.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:29 smithi006 ceph-mon[30991]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:29.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:29 smithi161 ceph-mon[35485]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:29.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:29 smithi118 ceph-mon[35048]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:30.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:30 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T03:13:30.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:30 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:30.363 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:30 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T03:13:30.363 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:30 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:30.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:30 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T03:13:30.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:30 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:31.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:31 smithi006 ceph-mon[30991]: Deploying daemon osd.0 on smithi006 2024-09-12T03:13:31.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:31 smithi006 ceph-mon[30991]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:31.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:31 smithi161 ceph-mon[35485]: Deploying daemon osd.0 on smithi006 2024-09-12T03:13:31.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:31 smithi161 ceph-mon[35485]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:31.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:31 smithi118 ceph-mon[35048]: Deploying daemon osd.0 on smithi006 2024-09-12T03:13:31.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:31 smithi118 ceph-mon[35048]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:33.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:33 smithi006 ceph-mon[30991]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:33.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:33 smithi161 ceph-mon[35485]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:33.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:33 smithi118 ceph-mon[35048]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:35.351 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:35 smithi006 ceph-mon[30991]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:35.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:35 smithi161 ceph-mon[35485]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:35.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:35 smithi118 ceph-mon[35048]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:36.317 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:36 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:36.317 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:36 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:36.317 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:36 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:36.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:36 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:36.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:36 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:36.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:36 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:36.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:36 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:36.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:36 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:36.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:36 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:37.241 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:37 smithi006 ceph-mon[30991]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:37.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:37 smithi161 ceph-mon[35485]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:37.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:37 smithi118 ceph-mon[35048]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:39.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:39 smithi006 ceph-mon[30991]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:39.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:39 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:39.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:39 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:39.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:39 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:39.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:39 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:39.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:39 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:39.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:39 smithi161 ceph-mon[35485]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:39.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:39 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:39.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:39 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:39.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:39 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:39.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:39 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:39.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:39 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:39.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:39 smithi118 ceph-mon[35048]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:39.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:39 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:39.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:39 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:39.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:39 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:39.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:39 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:39.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:39 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:39.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:39.720+0000 7f5959ffb640 1 -- 172.21.15.6:0/1752173975 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f595c062630 con 0x7f594c0392c0 2024-09-12T03:13:39.720 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 0 on host 'smithi006' 2024-09-12T03:13:39.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:39.724+0000 7f59630e1640 1 -- 172.21.15.6:0/1752173975 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f594c0392c0 msgr2=0x7f594c03b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:39.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:39.724+0000 7f59630e1640 1 --2- 172.21.15.6:0/1752173975 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f594c0392c0 0x7f594c03b780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5950006010 tx=0x7f5950032dd0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:39.724+0000 7f59630e1640 1 -- 172.21.15.6:0/1752173975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f595c102e00 msgr2=0x7f595c1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:39.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:39.724+0000 7f59630e1640 1 --2- 172.21.15.6:0/1752173975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f595c102e00 0x7f595c1984d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f594800b4a0 tx=0x7f594800b960 comp rx=0 tx=0).stop 2024-09-12T03:13:39.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:39.724+0000 7f59630e1640 1 -- 172.21.15.6:0/1752173975 shutdown_connections 2024-09-12T03:13:39.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:39.724+0000 7f59630e1640 1 --2- 172.21.15.6:0/1752173975 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f594c0392c0 0x7f594c03b780 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:39.724+0000 7f59630e1640 1 --2- 172.21.15.6:0/1752173975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f595c1037c0 0x7f595c19f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:39.724+0000 7f59630e1640 1 --2- 172.21.15.6:0/1752173975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f595c102e00 0x7f595c1984d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:39.724+0000 7f59630e1640 1 --2- 172.21.15.6:0/1752173975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f595c101c00 0x7f595c197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:39.724+0000 7f59630e1640 1 -- 172.21.15.6:0/1752173975 >> 172.21.15.6:0/1752173975 conn(0x7f595c0fd390 msgr2=0x7f595c106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:39.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:39.724+0000 7f59630e1640 1 -- 172.21.15.6:0/1752173975 shutdown_connections 2024-09-12T03:13:39.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:13:39.724+0000 7f59630e1640 1 -- 172.21.15.6:0/1752173975 wait complete. 2024-09-12T03:13:40.350 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:40 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:40.350 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:40 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:40.350 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:40 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:40.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:40 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:40.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:40 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:40.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:40 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:40.378 DEBUG:teuthology.orchestra.run.smithi006:osd.0> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.0.service 2024-09-12T03:13:40.380 INFO:tasks.cephadm:Deploying osd.1 on smithi118 with /dev/vg_nvme/lv_4... 2024-09-12T03:13:40.381 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T03:13:40.403 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:40 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:40.404 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:40 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:40.404 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:40 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:40.619 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:13:41.329 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:41 smithi118 ceph-mon[35048]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:41.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:41 smithi006 ceph-mon[30991]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:41.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:41 smithi161 ceph-mon[35485]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:42.049 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:13:41 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-0[45348]: 2024-09-12T03:13:41.913+0000 7faa1d3ca640 -1 osd.0 0 log_to_monitors true 2024-09-12T03:13:42.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:42 smithi118 ceph-mon[35048]: from='osd.0 [v2:172.21.15.6:6802/726729286,v1:172.21.15.6:6803/726729286]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T03:13:42.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:42 smithi118 ceph-mon[35048]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T03:13:42.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:42 smithi006 ceph-mon[30991]: from='osd.0 [v2:172.21.15.6:6802/726729286,v1:172.21.15.6:6803/726729286]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T03:13:42.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:42 smithi006 ceph-mon[30991]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T03:13:42.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:42 smithi161 ceph-mon[35485]: from='osd.0 [v2:172.21.15.6:6802/726729286,v1:172.21.15.6:6803/726729286]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T03:13:42.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:42 smithi161 ceph-mon[35485]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T03:13:43.007 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-12T03:13:43.040 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi118:vg_nvme/lv_4 2024-09-12T03:13:43.330 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:13:43.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:43 smithi006 ceph-mon[30991]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:43.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:43 smithi006 ceph-mon[30991]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T03:13:43.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:43 smithi006 ceph-mon[30991]: from='osd.0 [v2:172.21.15.6:6802/726729286,v1:172.21.15.6:6803/726729286]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-09-12T03:13:43.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:43 smithi006 ceph-mon[30991]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T03:13:43.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:43 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:13:43.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:43 smithi006 ceph-mon[30991]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-09-12T03:13:43.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:43 smithi161 ceph-mon[35485]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:43.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:43 smithi161 ceph-mon[35485]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T03:13:43.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:43 smithi161 ceph-mon[35485]: from='osd.0 [v2:172.21.15.6:6802/726729286,v1:172.21.15.6:6803/726729286]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-09-12T03:13:43.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:43 smithi161 ceph-mon[35485]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T03:13:43.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:43 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:13:43.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:43 smithi161 ceph-mon[35485]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-09-12T03:13:43.451 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:43 smithi118 ceph-mon[35048]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:43.451 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:43 smithi118 ceph-mon[35048]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T03:13:43.451 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:43 smithi118 ceph-mon[35048]: from='osd.0 [v2:172.21.15.6:6802/726729286,v1:172.21.15.6:6803/726729286]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-09-12T03:13:43.451 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:43 smithi118 ceph-mon[35048]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T03:13:43.451 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:43 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:13:43.451 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:43 smithi118 ceph-mon[35048]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-09-12T03:13:43.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.883+0000 7fb21165a640 1 -- 172.21.15.118:0/1895554716 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb20c100ad0 msgr2=0x7fb20c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:43.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.883+0000 7fb21165a640 1 --2- 172.21.15.118:0/1895554716 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb20c100ad0 0x7fb20c100ed0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009900 tx=0x7fb1fc02a410 comp rx=0 tx=0).stop 2024-09-12T03:13:43.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.883+0000 7fb21165a640 1 -- 172.21.15.118:0/1895554716 shutdown_connections 2024-09-12T03:13:43.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.883+0000 7fb21165a640 1 --2- 172.21.15.118:0/1895554716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb20c102690 0x7fb20c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:43.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.883+0000 7fb21165a640 1 --2- 172.21.15.118:0/1895554716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb20c101cd0 0x7fb20c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:43.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.883+0000 7fb21165a640 1 --2- 172.21.15.118:0/1895554716 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb20c100ad0 0x7fb20c100ed0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:43.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.883+0000 7fb21165a640 1 -- 172.21.15.118:0/1895554716 >> 172.21.15.118:0/1895554716 conn(0x7fb20c0fc280 msgr2=0x7fb20c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:43.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.883+0000 7fb21165a640 1 -- 172.21.15.118:0/1895554716 shutdown_connections 2024-09-12T03:13:43.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.883+0000 7fb21165a640 1 -- 172.21.15.118:0/1895554716 wait complete. 2024-09-12T03:13:43.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.883+0000 7fb21165a640 1 Processor -- start 2024-09-12T03:13:43.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.884+0000 7fb21165a640 1 -- start start 2024-09-12T03:13:43.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.884+0000 7fb21165a640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb20c100ad0 0x7fb20c196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:43.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.884+0000 7fb21165a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb20c101cd0 0x7fb20c197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:43.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.884+0000 7fb21165a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb20c102690 0x7fb20c19e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:43.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.884+0000 7fb21165a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c10c730 con 0x7fb20c102690 2024-09-12T03:13:43.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.884+0000 7fb21165a640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c10c5b0 con 0x7fb20c100ad0 2024-09-12T03:13:43.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.884+0000 7fb21165a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c10c8b0 con 0x7fb20c101cd0 2024-09-12T03:13:43.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.884+0000 7fb20affd640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb20c100ad0 0x7fb20c196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:43.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.884+0000 7fb20affd640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb20c100ad0 0x7fb20c196df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:60598/0 (socket says 172.21.15.118:60598) 2024-09-12T03:13:43.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.884+0000 7fb20affd640 1 -- 172.21.15.118:0/3553395536 learned_addr learned my addr 172.21.15.118:0/3553395536 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-09-12T03:13:43.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.884+0000 7fb20b7fe640 1 --2- 172.21.15.118:0/3553395536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb20c102690 0x7fb20c19e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:43.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.884+0000 7fb20a7fc640 1 --2- 172.21.15.118:0/3553395536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb20c101cd0 0x7fb20c197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:43.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.884+0000 7fb20affd640 1 -- 172.21.15.118:0/3553395536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb20c101cd0 msgr2=0x7fb20c197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:43.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.884+0000 7fb20affd640 1 --2- 172.21.15.118:0/3553395536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb20c101cd0 0x7fb20c197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:43.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.884+0000 7fb20affd640 1 -- 172.21.15.118:0/3553395536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb20c102690 msgr2=0x7fb20c19e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:43.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.884+0000 7fb20affd640 1 --2- 172.21.15.118:0/3553395536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb20c102690 0x7fb20c19e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:43.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.884+0000 7fb20affd640 1 -- 172.21.15.118:0/3553395536 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20c19ea20 con 0x7fb20c100ad0 2024-09-12T03:13:43.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.884+0000 7fb20affd640 1 --2- 172.21.15.118:0/3553395536 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb20c100ad0 0x7fb20c196df0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc031610 tx=0x7fb1fc004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:43.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.885+0000 7fb1f3fff640 1 -- 172.21.15.118:0/3553395536 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc03f030 con 0x7fb20c100ad0 2024-09-12T03:13:43.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.885+0000 7fb21165a640 1 -- 172.21.15.118:0/3553395536 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb20c19ecb0 con 0x7fb20c100ad0 2024-09-12T03:13:43.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.885+0000 7fb1f3fff640 1 -- 172.21.15.118:0/3553395536 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1fc003ad0 con 0x7fb20c100ad0 2024-09-12T03:13:43.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.885+0000 7fb1f3fff640 1 -- 172.21.15.118:0/3553395536 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc035070 con 0x7fb20c100ad0 2024-09-12T03:13:43.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.885+0000 7fb21165a640 1 -- 172.21.15.118:0/3553395536 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb20c19f190 con 0x7fb20c100ad0 2024-09-12T03:13:43.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.886+0000 7fb21165a640 1 -- 172.21.15.118:0/3553395536 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb20c107f20 con 0x7fb20c100ad0 2024-09-12T03:13:43.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.887+0000 7fb1f3fff640 1 -- 172.21.15.118:0/3553395536 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7fb1fc042030 con 0x7fb20c100ad0 2024-09-12T03:13:43.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.887+0000 7fb1f3fff640 1 --2- 172.21.15.118:0/3553395536 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb1f40a2b20 0x7fb1f40a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:43.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.888+0000 7fb1f3fff640 1 -- 172.21.15.118:0/3553395536 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7fb1fc0eda50 con 0x7fb20c100ad0 2024-09-12T03:13:43.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.888+0000 7fb20a7fc640 1 --2- 172.21.15.118:0/3553395536 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb1f40a2b20 0x7fb1f40a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:43.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.888+0000 7fb20a7fc640 1 --2- 172.21.15.118:0/3553395536 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb1f40a2b20 0x7fb1f40a4fe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb20c198310 tx=0x7fb20000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:43.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:43.892+0000 7fb1f3fff640 1 -- 172.21.15.118:0/3553395536 <== mon.2 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb1fc0bbef0 con 0x7fb20c100ad0 2024-09-12T03:13:44.066 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:13:44.066+0000 7fb21165a640 1 -- 172.21.15.118:0/3553395536 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb20c062630 con 0x7fb1f40a2b20 2024-09-12T03:13:44.361 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:13:43 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-0[45348]: 2024-09-12T03:13:43.905+0000 7faa18b2a640 -1 osd.0 0 waiting for initial osdmap 2024-09-12T03:13:44.362 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:13:43 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-0[45348]: 2024-09-12T03:13:43.910+0000 7faa1115c640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:13:44.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:44 smithi006 ceph-mon[30991]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-09-12T03:13:44.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:44 smithi006 ceph-mon[30991]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T03:13:44.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:44 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:13:44.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:44 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:13:44.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:44 smithi006 ceph-mon[30991]: Detected new or changed devices on smithi006 2024-09-12T03:13:44.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:44 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:44.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:44 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:44.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:44 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:13:44.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:44 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:44.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:44 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:44.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:44 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:44.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:44 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:44.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:44 smithi006 ceph-mon[30991]: from='osd.0 ' entity='osd.0' 2024-09-12T03:13:44.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:44 smithi161 ceph-mon[35485]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-09-12T03:13:44.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:44 smithi161 ceph-mon[35485]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T03:13:44.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:44 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:13:44.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:44 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:13:44.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:44 smithi161 ceph-mon[35485]: Detected new or changed devices on smithi006 2024-09-12T03:13:44.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:44 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:44.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:44 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:44.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:44 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:13:44.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:44 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:44.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:44 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:44.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:44 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:44.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:44 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:44.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:44 smithi161 ceph-mon[35485]: from='osd.0 ' entity='osd.0' 2024-09-12T03:13:44.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:44 smithi118 ceph-mon[35048]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-09-12T03:13:44.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:44 smithi118 ceph-mon[35048]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T03:13:44.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:44 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:13:44.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:44 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:13:44.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:44 smithi118 ceph-mon[35048]: Detected new or changed devices on smithi006 2024-09-12T03:13:44.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:44 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:44.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:44 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:44.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:44 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:13:44.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:44 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:44.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:44 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:44.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:44 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:44.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:44 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:13:44.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:44 smithi118 ceph-mon[35048]: from='osd.0 ' entity='osd.0' 2024-09-12T03:13:45.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:45 smithi006 ceph-mon[30991]: purged_snaps scrub starts 2024-09-12T03:13:45.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:45 smithi006 ceph-mon[30991]: purged_snaps scrub ok 2024-09-12T03:13:45.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:45 smithi006 ceph-mon[30991]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:45.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:45 smithi006 ceph-mon[30991]: Adjusting osd_memory_target on smithi006 to 17092M 2024-09-12T03:13:45.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:45 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:13:45.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:45 smithi006 ceph-mon[30991]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:13:45.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:45 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:13:45.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:45 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:13:45.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:45 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:45.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:45 smithi006 ceph-mon[30991]: osd.0 [v2:172.21.15.6:6802/726729286,v1:172.21.15.6:6803/726729286] boot 2024-09-12T03:13:45.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:45 smithi006 ceph-mon[30991]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T03:13:45.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:45 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:13:45.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:45 smithi161 ceph-mon[35485]: purged_snaps scrub starts 2024-09-12T03:13:45.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:45 smithi161 ceph-mon[35485]: purged_snaps scrub ok 2024-09-12T03:13:45.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:45 smithi161 ceph-mon[35485]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:45.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:45 smithi161 ceph-mon[35485]: Adjusting osd_memory_target on smithi006 to 17092M 2024-09-12T03:13:45.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:45 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:13:45.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:45 smithi161 ceph-mon[35485]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:13:45.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:45 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:13:45.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:45 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:13:45.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:45 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:45.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:45 smithi161 ceph-mon[35485]: osd.0 [v2:172.21.15.6:6802/726729286,v1:172.21.15.6:6803/726729286] boot 2024-09-12T03:13:45.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:45 smithi161 ceph-mon[35485]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T03:13:45.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:45 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:13:45.389 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:45 smithi118 ceph-mon[35048]: purged_snaps scrub starts 2024-09-12T03:13:45.389 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:45 smithi118 ceph-mon[35048]: purged_snaps scrub ok 2024-09-12T03:13:45.389 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:45 smithi118 ceph-mon[35048]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:13:45.390 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:45 smithi118 ceph-mon[35048]: Adjusting osd_memory_target on smithi006 to 17092M 2024-09-12T03:13:45.390 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:45 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:13:45.390 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:45 smithi118 ceph-mon[35048]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:13:45.390 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:45 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:13:45.390 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:45 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:13:45.391 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:45 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:45.391 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:45 smithi118 ceph-mon[35048]: osd.0 [v2:172.21.15.6:6802/726729286,v1:172.21.15.6:6803/726729286] boot 2024-09-12T03:13:45.391 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:45 smithi118 ceph-mon[35048]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T03:13:45.391 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:45 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:13:47.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:46 smithi118 ceph-mon[35048]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:13:47.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:46 smithi118 ceph-mon[35048]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T03:13:47.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:46 smithi118 ceph-mon[35048]: from='client.? 172.21.15.118:0/3770691774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83afdb01-c731-4de1-96c6-3f798549b1eb"}]: dispatch 2024-09-12T03:13:47.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:46 smithi118 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83afdb01-c731-4de1-96c6-3f798549b1eb"}]: dispatch 2024-09-12T03:13:47.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:46 smithi118 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83afdb01-c731-4de1-96c6-3f798549b1eb"}]': finished 2024-09-12T03:13:47.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:46 smithi118 ceph-mon[35048]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T03:13:47.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:46 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:13:47.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:46 smithi118 ceph-mon[35048]: from='client.? 172.21.15.118:0/1976641759' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:13:47.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:46 smithi006 ceph-mon[30991]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:13:47.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:46 smithi006 ceph-mon[30991]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T03:13:47.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:46 smithi006 ceph-mon[30991]: from='client.? 172.21.15.118:0/3770691774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83afdb01-c731-4de1-96c6-3f798549b1eb"}]: dispatch 2024-09-12T03:13:47.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:46 smithi006 ceph-mon[30991]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83afdb01-c731-4de1-96c6-3f798549b1eb"}]: dispatch 2024-09-12T03:13:47.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:46 smithi006 ceph-mon[30991]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83afdb01-c731-4de1-96c6-3f798549b1eb"}]': finished 2024-09-12T03:13:47.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:46 smithi006 ceph-mon[30991]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T03:13:47.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:46 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:13:47.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:46 smithi006 ceph-mon[30991]: from='client.? 172.21.15.118:0/1976641759' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:13:47.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:46 smithi161 ceph-mon[35485]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:13:47.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:46 smithi161 ceph-mon[35485]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T03:13:47.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:46 smithi161 ceph-mon[35485]: from='client.? 172.21.15.118:0/3770691774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83afdb01-c731-4de1-96c6-3f798549b1eb"}]: dispatch 2024-09-12T03:13:47.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:46 smithi161 ceph-mon[35485]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83afdb01-c731-4de1-96c6-3f798549b1eb"}]: dispatch 2024-09-12T03:13:47.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:46 smithi161 ceph-mon[35485]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83afdb01-c731-4de1-96c6-3f798549b1eb"}]': finished 2024-09-12T03:13:47.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:46 smithi161 ceph-mon[35485]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T03:13:47.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:46 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:13:47.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:46 smithi161 ceph-mon[35485]: from='client.? 172.21.15.118:0/1976641759' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:13:49.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:48 smithi118 ceph-mon[35048]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:13:49.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:48 smithi006 ceph-mon[30991]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:13:49.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:48 smithi161 ceph-mon[35485]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:13:51.192 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:50 smithi118 ceph-mon[35048]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:13:51.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:50 smithi006 ceph-mon[30991]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:13:51.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:50 smithi161 ceph-mon[35485]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:13:53.158 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:52 smithi118 ceph-mon[35048]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:13:53.273 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:52 smithi006 ceph-mon[30991]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:13:53.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:52 smithi161 ceph-mon[35485]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:13:55.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:54 smithi118 ceph-mon[35048]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:13:55.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:54 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T03:13:55.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:54 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:55.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:54 smithi006 ceph-mon[30991]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:13:55.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:54 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T03:13:55.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:54 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:55.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:54 smithi161 ceph-mon[35485]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:13:55.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T03:13:55.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:56.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:55 smithi118 ceph-mon[35048]: Deploying daemon osd.1 on smithi118 2024-09-12T03:13:56.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:55 smithi006 ceph-mon[30991]: Deploying daemon osd.1 on smithi118 2024-09-12T03:13:56.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:55 smithi161 ceph-mon[35485]: Deploying daemon osd.1 on smithi118 2024-09-12T03:13:57.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:56 smithi118 ceph-mon[35048]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:13:57.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:56 smithi006 ceph-mon[30991]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:13:57.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:56 smithi161 ceph-mon[35485]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:13:59.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:13:58 smithi118 ceph-mon[35048]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:13:59.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:13:58 smithi006 ceph-mon[30991]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:13:59.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:13:58 smithi161 ceph-mon[35485]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:14:01.172 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:00 smithi118 ceph-mon[35048]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:14:01.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:00 smithi006 ceph-mon[30991]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:14:01.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:00 smithi161 ceph-mon[35485]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:14:02.178 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:01 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:02.178 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:01 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:02.178 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:01 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:02.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:01 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:02.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:01 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:02.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:01 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:02.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:01 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:02.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:01 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:02.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:01 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:03.088 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:02 smithi118 ceph-mon[35048]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:14:03.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:02 smithi006 ceph-mon[30991]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:14:03.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:02 smithi161 ceph-mon[35485]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:14:05.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:04 smithi006 ceph-mon[30991]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:14:05.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:04 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:05.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:04 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:05.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:04 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:05.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:04 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:05.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:04 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:05.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:04 smithi161 ceph-mon[35485]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:14:05.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:04 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:05.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:04 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:05.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:04 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:05.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:04 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:05.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:04 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:05.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:04 smithi118 ceph-mon[35048]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:14:05.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:04 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:05.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:04 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:05.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:04 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:05.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:04 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:05.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:04 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:05.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:14:05.639+0000 7fb1f3fff640 1 -- 172.21.15.118:0/3553395536 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb20c062630 con 0x7fb1f40a2b20 2024-09-12T03:14:05.640 INFO:teuthology.orchestra.run.smithi118.stdout:Created osd(s) 1 on host 'smithi118' 2024-09-12T03:14:05.642 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:14:05.642+0000 7fb21165a640 1 -- 172.21.15.118:0/3553395536 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb1f40a2b20 msgr2=0x7fb1f40a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:05.643 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:14:05.642+0000 7fb21165a640 1 --2- 172.21.15.118:0/3553395536 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb1f40a2b20 0x7fb1f40a4fe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb20c198310 tx=0x7fb20000a2d0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.643 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:14:05.642+0000 7fb21165a640 1 -- 172.21.15.118:0/3553395536 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb20c100ad0 msgr2=0x7fb20c196df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:05.643 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:14:05.642+0000 7fb21165a640 1 --2- 172.21.15.118:0/3553395536 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb20c100ad0 0x7fb20c196df0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc031610 tx=0x7fb1fc004270 comp rx=0 tx=0).stop 2024-09-12T03:14:05.643 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:14:05.642+0000 7fb21165a640 1 -- 172.21.15.118:0/3553395536 shutdown_connections 2024-09-12T03:14:05.643 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:14:05.642+0000 7fb21165a640 1 --2- 172.21.15.118:0/3553395536 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb1f40a2b20 0x7fb1f40a4fe0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.643 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:14:05.642+0000 7fb21165a640 1 --2- 172.21.15.118:0/3553395536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb20c102690 0x7fb20c19e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.643 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:14:05.642+0000 7fb21165a640 1 --2- 172.21.15.118:0/3553395536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb20c101cd0 0x7fb20c197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.643 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:14:05.642+0000 7fb21165a640 1 --2- 172.21.15.118:0/3553395536 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb20c100ad0 0x7fb20c196df0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.643 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:14:05.642+0000 7fb21165a640 1 -- 172.21.15.118:0/3553395536 >> 172.21.15.118:0/3553395536 conn(0x7fb20c0fc280 msgr2=0x7fb20c1050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:05.643 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:14:05.643+0000 7fb21165a640 1 -- 172.21.15.118:0/3553395536 shutdown_connections 2024-09-12T03:14:05.643 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:14:05.643+0000 7fb21165a640 1 -- 172.21.15.118:0/3553395536 wait complete. 2024-09-12T03:14:05.943 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:05 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:05.943 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:05 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:05.943 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:05 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:06.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:05 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:06.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:05 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:06.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:05 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:06.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:05 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:06.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:05 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:06.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:05 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:06.286 DEBUG:teuthology.orchestra.run.smithi118:osd.1> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.1.service 2024-09-12T03:14:06.289 INFO:tasks.cephadm:Deploying osd.2 on smithi161 with /dev/vg_nvme/lv_4... 2024-09-12T03:14:06.289 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T03:14:06.539 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:14:06.913 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:06 smithi161 ceph-mon[35485]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:14:07.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:06 smithi006 ceph-mon[30991]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:14:07.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:06 smithi118 ceph-mon[35048]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:14:07.712 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:14:07 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-1[39554]: 2024-09-12T03:14:07.447+0000 7fc29fa64640 -1 osd.1 0 log_to_monitors true 2024-09-12T03:14:07.885 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:07 smithi161 ceph-mon[35485]: from='osd.1 [v2:172.21.15.118:6800/3377555415,v1:172.21.15.118:6801/3377555415]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:14:07.885 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:07 smithi161 ceph-mon[35485]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:14:08.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:07 smithi006 ceph-mon[30991]: from='osd.1 [v2:172.21.15.118:6800/3377555415,v1:172.21.15.118:6801/3377555415]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:14:08.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:07 smithi006 ceph-mon[30991]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:14:08.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:07 smithi118 ceph-mon[35048]: from='osd.1 [v2:172.21.15.118:6800/3377555415,v1:172.21.15.118:6801/3377555415]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:14:08.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:07 smithi118 ceph-mon[35048]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:14:08.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:08 smithi118 ceph-mon[35048]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:14:08.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:08 smithi118 ceph-mon[35048]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T03:14:08.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:08 smithi118 ceph-mon[35048]: from='osd.1 [v2:172.21.15.118:6800/3377555415,v1:172.21.15.118:6801/3377555415]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:14:08.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:08 smithi118 ceph-mon[35048]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T03:14:08.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:08 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:14:08.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:08 smithi118 ceph-mon[35048]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:14:08.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:08 smithi161 ceph-mon[35485]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:14:08.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:08 smithi161 ceph-mon[35485]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T03:14:08.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:08 smithi161 ceph-mon[35485]: from='osd.1 [v2:172.21.15.118:6800/3377555415,v1:172.21.15.118:6801/3377555415]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:14:08.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:08 smithi161 ceph-mon[35485]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T03:14:08.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:08 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:14:08.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:08 smithi161 ceph-mon[35485]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:14:08.977 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:14:09.006 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_4 2024-09-12T03:14:09.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:08 smithi006 ceph-mon[30991]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:14:09.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:08 smithi006 ceph-mon[30991]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T03:14:09.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:08 smithi006 ceph-mon[30991]: from='osd.1 [v2:172.21.15.118:6800/3377555415,v1:172.21.15.118:6801/3377555415]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:14:09.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:08 smithi006 ceph-mon[30991]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T03:14:09.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:08 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:14:09.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:08 smithi006 ceph-mon[30991]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:14:09.262 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:14:09.797 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:14:09 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-1[39554]: 2024-09-12T03:14:09.484+0000 7fc29b1c4640 -1 osd.1 0 waiting for initial osdmap 2024-09-12T03:14:09.798 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:14:09 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-1[39554]: 2024-09-12T03:14:09.487+0000 7fc2937f6640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:14:09.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.812+0000 7f80c8a8e640 1 -- 172.21.15.161:0/1080473758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80c4102df0 msgr2=0x7f80c4103270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:09.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.812+0000 7f80c8a8e640 1 --2- 172.21.15.161:0/1080473758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80c4102df0 0x7f80c4103270 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009900 tx=0x7f80b802a410 comp rx=0 tx=0).stop 2024-09-12T03:14:09.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.812+0000 7f80c8a8e640 1 -- 172.21.15.161:0/1080473758 shutdown_connections 2024-09-12T03:14:09.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.812+0000 7f80c8a8e640 1 --2- 172.21.15.161:0/1080473758 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80c41037b0 0x7f80c410a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.812+0000 7f80c8a8e640 1 --2- 172.21.15.161:0/1080473758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80c4102df0 0x7f80c4103270 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.812+0000 7f80c8a8e640 1 --2- 172.21.15.161:0/1080473758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80c4101bf0 0x7f80c4101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.812+0000 7f80c8a8e640 1 -- 172.21.15.161:0/1080473758 >> 172.21.15.161:0/1080473758 conn(0x7f80c40fd3c0 msgr2=0x7f80c40ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:09.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.812+0000 7f80c8a8e640 1 -- 172.21.15.161:0/1080473758 shutdown_connections 2024-09-12T03:14:09.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.812+0000 7f80c8a8e640 1 -- 172.21.15.161:0/1080473758 wait complete. 2024-09-12T03:14:09.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.813+0000 7f80c8a8e640 1 Processor -- start 2024-09-12T03:14:09.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.813+0000 7f80c8a8e640 1 -- start start 2024-09-12T03:14:09.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.813+0000 7f80c8a8e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80c4101bf0 0x7f80c4075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.813+0000 7f80c8a8e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80c4102df0 0x7f80c40745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.813+0000 7f80c8a8e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80c41037b0 0x7f80c4074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.813+0000 7f80c8a8e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f80c410d970 con 0x7f80c4101bf0 2024-09-12T03:14:09.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.813+0000 7f80c8a8e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f80c410d7f0 con 0x7f80c4102df0 2024-09-12T03:14:09.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.813+0000 7f80c8a8e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f80c410daf0 con 0x7f80c41037b0 2024-09-12T03:14:09.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.813+0000 7f80c2d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80c41037b0 0x7f80c4074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:09.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.813+0000 7f80c2d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80c41037b0 0x7f80c4074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55310/0 (socket says 172.21.15.161:55310) 2024-09-12T03:14:09.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.813+0000 7f80c2d9d640 1 -- 172.21.15.161:0/3228175650 learned_addr learned my addr 172.21.15.161:0/3228175650 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-12T03:14:09.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.813+0000 7f80c259c640 1 --2- 172.21.15.161:0/3228175650 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80c4101bf0 0x7f80c4075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:09.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.813+0000 7f80c1d9b640 1 --2- 172.21.15.161:0/3228175650 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80c4102df0 0x7f80c40745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:09.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.813+0000 7f80c2d9d640 1 -- 172.21.15.161:0/3228175650 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80c4102df0 msgr2=0x7f80c40745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:09.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.813+0000 7f80c2d9d640 1 --2- 172.21.15.161:0/3228175650 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80c4102df0 0x7f80c40745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.813+0000 7f80c2d9d640 1 -- 172.21.15.161:0/3228175650 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80c4101bf0 msgr2=0x7f80c4075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:09.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.813+0000 7f80c2d9d640 1 --2- 172.21.15.161:0/3228175650 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80c4101bf0 0x7f80c4075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.813+0000 7f80c2d9d640 1 -- 172.21.15.161:0/3228175650 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c40753e0 con 0x7f80c41037b0 2024-09-12T03:14:09.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.813+0000 7f80c259c640 1 --2- 172.21.15.161:0/3228175650 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80c4101bf0 0x7f80c4075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:09.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.814+0000 7f80c2d9d640 1 --2- 172.21.15.161:0/3228175650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80c41037b0 0x7f80c4074b20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00b970 tx=0x7f80bc00be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:09.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.814+0000 7f80b37fe640 1 -- 172.21.15.161:0/3228175650 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc019070 con 0x7f80c41037b0 2024-09-12T03:14:09.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.814+0000 7f80b37fe640 1 -- 172.21.15.161:0/3228175650 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80bc00cb40 con 0x7f80c41037b0 2024-09-12T03:14:09.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.814+0000 7f80c8a8e640 1 -- 172.21.15.161:0/3228175650 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c41a21c0 con 0x7f80c41037b0 2024-09-12T03:14:09.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.814+0000 7f80b37fe640 1 -- 172.21.15.161:0/3228175650 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc007650 con 0x7f80c41037b0 2024-09-12T03:14:09.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.814+0000 7f80c8a8e640 1 -- 172.21.15.161:0/3228175650 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c41a27a0 con 0x7f80c41037b0 2024-09-12T03:14:09.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.815+0000 7f80c8a8e640 1 -- 172.21.15.161:0/3228175650 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c4102070 con 0x7f80c41037b0 2024-09-12T03:14:09.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.816+0000 7f80b37fe640 1 -- 172.21.15.161:0/3228175650 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f80bc01c070 con 0x7f80c41037b0 2024-09-12T03:14:09.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.816+0000 7f80b37fe640 1 --2- 172.21.15.161:0/3228175650 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f80ac0a2b70 0x7f80ac0a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.817+0000 7f80b37fe640 1 -- 172.21.15.161:0/3228175650 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f80bc0ccbf0 con 0x7f80c41037b0 2024-09-12T03:14:09.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.817+0000 7f80c259c640 1 --2- 172.21.15.161:0/3228175650 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f80ac0a2b70 0x7f80ac0a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:09.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.817+0000 7f80c259c640 1 --2- 172.21.15.161:0/3228175650 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f80ac0a2b70 0x7f80ac0a5030 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f80c4102c50 tx=0x7f80b4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:09.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.821+0000 7f80b37fe640 1 -- 172.21.15.161:0/3228175650 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f80bc09b4c0 con 0x7f80c41037b0 2024-09-12T03:14:09.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:09.994+0000 7f80c8a8e640 1 -- 172.21.15.161:0/3228175650 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f80c4062630 con 0x7f80ac0a2b70 2024-09-12T03:14:09.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:09 smithi161 ceph-mon[35485]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-12T03:14:09.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:09 smithi161 ceph-mon[35485]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T03:14:09.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:09 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:14:09.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:09 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:14:09.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:09 smithi161 ceph-mon[35485]: Detected new or changed devices on smithi118 2024-09-12T03:14:09.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:09 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:09.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:09 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:09.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:09 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:09.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:09 smithi161 ceph-mon[35485]: Adjusting osd_memory_target on smithi118 to 17088M 2024-09-12T03:14:09.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:09 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:09.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:09 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:09.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:09 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:09.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:09 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:09.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:09 smithi161 ceph-mon[35485]: from='osd.1 ' entity='osd.1' 2024-09-12T03:14:09.996 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:09 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:14:10.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:09 smithi006 ceph-mon[30991]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-12T03:14:10.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:09 smithi006 ceph-mon[30991]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T03:14:10.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:09 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:14:10.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:09 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:14:10.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:09 smithi006 ceph-mon[30991]: Detected new or changed devices on smithi118 2024-09-12T03:14:10.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:09 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:10.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:09 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:10.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:09 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:10.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:09 smithi006 ceph-mon[30991]: Adjusting osd_memory_target on smithi118 to 17088M 2024-09-12T03:14:10.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:09 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:10.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:09 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:10.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:09 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:10.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:09 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:10.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:09 smithi006 ceph-mon[30991]: from='osd.1 ' entity='osd.1' 2024-09-12T03:14:10.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:09 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:14:10.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:09 smithi118 ceph-mon[35048]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-12T03:14:10.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:09 smithi118 ceph-mon[35048]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T03:14:10.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:09 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:14:10.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:09 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:14:10.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:09 smithi118 ceph-mon[35048]: Detected new or changed devices on smithi118 2024-09-12T03:14:10.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:09 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:10.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:09 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:10.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:09 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:10.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:09 smithi118 ceph-mon[35048]: Adjusting osd_memory_target on smithi118 to 17088M 2024-09-12T03:14:10.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:09 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:10.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:09 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:10.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:09 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:10.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:09 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:10.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:09 smithi118 ceph-mon[35048]: from='osd.1 ' entity='osd.1' 2024-09-12T03:14:10.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:09 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:14:11.033 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:10 smithi161 ceph-mon[35485]: purged_snaps scrub starts 2024-09-12T03:14:11.033 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:10 smithi161 ceph-mon[35485]: purged_snaps scrub ok 2024-09-12T03:14:11.034 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:10 smithi161 ceph-mon[35485]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:14:11.034 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:10 smithi161 ceph-mon[35485]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:14:11.034 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:10 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:14:11.034 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:10 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:14:11.034 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:10 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:11.034 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:10 smithi161 ceph-mon[35485]: osd.1 [v2:172.21.15.118:6800/3377555415,v1:172.21.15.118:6801/3377555415] boot 2024-09-12T03:14:11.034 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:10 smithi161 ceph-mon[35485]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T03:14:11.034 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:10 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:14:11.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:10 smithi006 ceph-mon[30991]: purged_snaps scrub starts 2024-09-12T03:14:11.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:10 smithi006 ceph-mon[30991]: purged_snaps scrub ok 2024-09-12T03:14:11.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:10 smithi006 ceph-mon[30991]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:14:11.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:10 smithi006 ceph-mon[30991]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:14:11.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:10 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:14:11.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:10 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:14:11.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:10 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:11.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:10 smithi006 ceph-mon[30991]: osd.1 [v2:172.21.15.118:6800/3377555415,v1:172.21.15.118:6801/3377555415] boot 2024-09-12T03:14:11.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:10 smithi006 ceph-mon[30991]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T03:14:11.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:10 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:14:11.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:10 smithi118 ceph-mon[35048]: purged_snaps scrub starts 2024-09-12T03:14:11.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:10 smithi118 ceph-mon[35048]: purged_snaps scrub ok 2024-09-12T03:14:11.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:10 smithi118 ceph-mon[35048]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:14:11.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:10 smithi118 ceph-mon[35048]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:14:11.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:10 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:14:11.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:10 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:14:11.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:10 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:11.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:10 smithi118 ceph-mon[35048]: osd.1 [v2:172.21.15.118:6800/3377555415,v1:172.21.15.118:6801/3377555415] boot 2024-09-12T03:14:11.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:10 smithi118 ceph-mon[35048]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T03:14:11.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:10 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:14:13.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:12 smithi006 ceph-mon[30991]: pgmap v59: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:13.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:12 smithi006 ceph-mon[30991]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T03:14:13.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:12 smithi006 ceph-mon[30991]: from='client.? 172.21.15.161:0/1662707924' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aed718b4-b849-403a-b81c-3a140b5c380c"}]: dispatch 2024-09-12T03:14:13.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:12 smithi006 ceph-mon[30991]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aed718b4-b849-403a-b81c-3a140b5c380c"}]: dispatch 2024-09-12T03:14:13.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:12 smithi006 ceph-mon[30991]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aed718b4-b849-403a-b81c-3a140b5c380c"}]': finished 2024-09-12T03:14:13.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:12 smithi006 ceph-mon[30991]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T03:14:13.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:12 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:14:13.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:12 smithi161 ceph-mon[35485]: pgmap v59: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:13.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:12 smithi161 ceph-mon[35485]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T03:14:13.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:12 smithi161 ceph-mon[35485]: from='client.? 172.21.15.161:0/1662707924' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aed718b4-b849-403a-b81c-3a140b5c380c"}]: dispatch 2024-09-12T03:14:13.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:12 smithi161 ceph-mon[35485]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aed718b4-b849-403a-b81c-3a140b5c380c"}]: dispatch 2024-09-12T03:14:13.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:12 smithi161 ceph-mon[35485]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aed718b4-b849-403a-b81c-3a140b5c380c"}]': finished 2024-09-12T03:14:13.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:12 smithi161 ceph-mon[35485]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T03:14:13.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:12 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:14:13.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:12 smithi118 ceph-mon[35048]: pgmap v59: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:13.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:12 smithi118 ceph-mon[35048]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T03:14:13.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:12 smithi118 ceph-mon[35048]: from='client.? 172.21.15.161:0/1662707924' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aed718b4-b849-403a-b81c-3a140b5c380c"}]: dispatch 2024-09-12T03:14:13.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:12 smithi118 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aed718b4-b849-403a-b81c-3a140b5c380c"}]: dispatch 2024-09-12T03:14:13.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:12 smithi118 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aed718b4-b849-403a-b81c-3a140b5c380c"}]': finished 2024-09-12T03:14:13.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:12 smithi118 ceph-mon[35048]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T03:14:13.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:12 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:14:14.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:13 smithi006 ceph-mon[30991]: from='client.? 172.21.15.161:0/4027745866' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:14:14.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:13 smithi161 ceph-mon[35485]: from='client.? 172.21.15.161:0/4027745866' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:14:14.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:13 smithi118 ceph-mon[35048]: from='client.? 172.21.15.161:0/4027745866' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:14:15.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:14 smithi006 ceph-mon[30991]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:15.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:14 smithi161 ceph-mon[35485]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:15.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:14 smithi118 ceph-mon[35048]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:17.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:16 smithi006 ceph-mon[30991]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:17.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:16 smithi161 ceph-mon[35485]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:17.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:16 smithi118 ceph-mon[35048]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:18.853 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:18 smithi161 ceph-mon[35485]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:19.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:18 smithi006 ceph-mon[30991]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:19.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:18 smithi118 ceph-mon[35048]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:20.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:20 smithi161 ceph-mon[35485]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:20.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T03:14:20.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:21.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:20 smithi006 ceph-mon[30991]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:21.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T03:14:21.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:21.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:20 smithi118 ceph-mon[35048]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:21.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T03:14:21.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:21.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:21 smithi161 ceph-mon[35485]: Deploying daemon osd.2 on smithi161 2024-09-12T03:14:22.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:21 smithi006 ceph-mon[30991]: Deploying daemon osd.2 on smithi161 2024-09-12T03:14:22.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:21 smithi118 ceph-mon[35048]: Deploying daemon osd.2 on smithi161 2024-09-12T03:14:23.083 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:22 smithi161 ceph-mon[35485]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:23.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:22 smithi006 ceph-mon[30991]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:23.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:22 smithi118 ceph-mon[35048]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:25.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:24 smithi006 ceph-mon[30991]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:25.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:24 smithi161 ceph-mon[35485]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:25.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:24 smithi118 ceph-mon[35048]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:26.936 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:26 smithi161 ceph-mon[35485]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:27.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:26 smithi006 ceph-mon[30991]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:27.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:26 smithi118 ceph-mon[35048]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:28.084 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:27 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:28.084 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:27 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:28.084 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:27 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:28.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:27 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:28.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:27 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:28.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:27 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:28.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:27 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:28.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:27 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:28.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:27 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:29.248 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:28 smithi161 ceph-mon[35485]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:29.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:28 smithi006 ceph-mon[30991]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:29.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:28 smithi118 ceph-mon[35048]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:30.426 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:30.426+0000 7f80b37fe640 1 -- 172.21.15.161:0/3228175650 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f80c4062630 con 0x7f80ac0a2b70 2024-09-12T03:14:30.426 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 2 on host 'smithi161' 2024-09-12T03:14:30.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:30.428+0000 7f80c8a8e640 1 -- 172.21.15.161:0/3228175650 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f80ac0a2b70 msgr2=0x7f80ac0a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:30.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:30.428+0000 7f80c8a8e640 1 --2- 172.21.15.161:0/3228175650 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f80ac0a2b70 0x7f80ac0a5030 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f80c4102c50 tx=0x7f80b4008040 comp rx=0 tx=0).stop 2024-09-12T03:14:30.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:30.428+0000 7f80c8a8e640 1 -- 172.21.15.161:0/3228175650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80c41037b0 msgr2=0x7f80c4074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:30.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:30.428+0000 7f80c8a8e640 1 --2- 172.21.15.161:0/3228175650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80c41037b0 0x7f80c4074b20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00b970 tx=0x7f80bc00be30 comp rx=0 tx=0).stop 2024-09-12T03:14:30.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:30.428+0000 7f80c8a8e640 1 -- 172.21.15.161:0/3228175650 shutdown_connections 2024-09-12T03:14:30.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:30.428+0000 7f80c8a8e640 1 --2- 172.21.15.161:0/3228175650 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f80ac0a2b70 0x7f80ac0a5030 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:30.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:30.428+0000 7f80c8a8e640 1 --2- 172.21.15.161:0/3228175650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80c41037b0 0x7f80c4074b20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:30.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:30.428+0000 7f80c8a8e640 1 --2- 172.21.15.161:0/3228175650 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80c4102df0 0x7f80c40745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:30.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:30.428+0000 7f80c8a8e640 1 --2- 172.21.15.161:0/3228175650 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80c4101bf0 0x7f80c4075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:30.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:30.428+0000 7f80c8a8e640 1 -- 172.21.15.161:0/3228175650 >> 172.21.15.161:0/3228175650 conn(0x7f80c40fd3c0 msgr2=0x7f80c4106210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:30.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:30.428+0000 7f80c8a8e640 1 -- 172.21.15.161:0/3228175650 shutdown_connections 2024-09-12T03:14:30.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:14:30.428+0000 7f80c8a8e640 1 -- 172.21.15.161:0/3228175650 wait complete. 2024-09-12T03:14:30.730 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:30 smithi161 ceph-mon[35485]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:30.731 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:30 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:30.731 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:30 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:30.731 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:30 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:30.731 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:30 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:30.731 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:30 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:30.731 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:30 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:30.731 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:30 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:30.731 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:30 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:30.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:30 smithi006 ceph-mon[30991]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:30.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:30 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:30.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:30 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:30.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:30 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:30.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:30 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:30.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:30 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:30.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:30 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:30.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:30 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:30.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:30 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:30.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:30 smithi118 ceph-mon[35048]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:30.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:30 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:30.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:30 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:30.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:30 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:30.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:30 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:30.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:30 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:30.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:30 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:30.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:30 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:30.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:30 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:31.033 DEBUG:teuthology.orchestra.run.smithi161:osd.2> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.2.service 2024-09-12T03:14:31.036 INFO:tasks.cephadm:Deploying osd.3 on smithi006 with /dev/vg_nvme/lv_3... 2024-09-12T03:14:31.036 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T03:14:31.301 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:14:32.637 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:32 smithi006 ceph-mon[30991]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:32.729 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:32 smithi161 ceph-mon[35485]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:32.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:32 smithi118 ceph-mon[35048]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:33.568 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:14:33 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-2[40470]: 2024-09-12T03:14:33.304+0000 7f001822a640 -1 osd.2 0 log_to_monitors true 2024-09-12T03:14:33.760 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:33 smithi006 ceph-mon[30991]: from='osd.2 [v2:172.21.15.161:6800/392541009,v1:172.21.15.161:6801/392541009]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:14:33.760 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:33 smithi006 ceph-mon[30991]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:14:33.762 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:14:33.789 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi006:vg_nvme/lv_3 2024-09-12T03:14:33.861 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:33 smithi161 ceph-mon[35485]: from='osd.2 [v2:172.21.15.161:6800/392541009,v1:172.21.15.161:6801/392541009]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:14:33.861 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:33 smithi161 ceph-mon[35485]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:14:33.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:33 smithi118 ceph-mon[35048]: from='osd.2 [v2:172.21.15.161:6800/392541009,v1:172.21.15.161:6801/392541009]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:14:33.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:33 smithi118 ceph-mon[35048]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:14:34.055 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:14:34.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.507+0000 7fe231edc640 1 -- 172.21.15.6:0/1641292047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe22c101c00 msgr2=0x7fe22c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:34.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.507+0000 7fe231edc640 1 --2- 172.21.15.6:0/1641292047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe22c101c00 0x7fe22c102000 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009900 tx=0x7fe21c02a410 comp rx=0 tx=0).stop 2024-09-12T03:14:34.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.507+0000 7fe231edc640 1 -- 172.21.15.6:0/1641292047 shutdown_connections 2024-09-12T03:14:34.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.507+0000 7fe231edc640 1 --2- 172.21.15.6:0/1641292047 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe22c1037c0 0x7fe22c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:34.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.507+0000 7fe231edc640 1 --2- 172.21.15.6:0/1641292047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe22c102e00 0x7fe22c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:34.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.507+0000 7fe231edc640 1 --2- 172.21.15.6:0/1641292047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe22c101c00 0x7fe22c102000 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:34.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.507+0000 7fe231edc640 1 -- 172.21.15.6:0/1641292047 >> 172.21.15.6:0/1641292047 conn(0x7fe22c0fd390 msgr2=0x7fe22c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:34.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.507+0000 7fe231edc640 1 -- 172.21.15.6:0/1641292047 shutdown_connections 2024-09-12T03:14:34.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.507+0000 7fe231edc640 1 -- 172.21.15.6:0/1641292047 wait complete. 2024-09-12T03:14:34.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.507+0000 7fe231edc640 1 Processor -- start 2024-09-12T03:14:34.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.507+0000 7fe231edc640 1 -- start start 2024-09-12T03:14:34.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.508+0000 7fe231edc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe22c101c00 0x7fe22c19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:34.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.508+0000 7fe231edc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe22c102e00 0x7fe22c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:34.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.508+0000 7fe231edc640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe22c1037c0 0x7fe22c1a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:34.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.508+0000 7fe231edc640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c10d860 con 0x7fe22c101c00 2024-09-12T03:14:34.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.508+0000 7fe231edc640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c10d6e0 con 0x7fe22c1037c0 2024-09-12T03:14:34.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.508+0000 7fe231edc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c10d9e0 con 0x7fe22c102e00 2024-09-12T03:14:34.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.508+0000 7fe22b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe22c101c00 0x7fe22c19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:34.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.508+0000 7fe22b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe22c101c00 0x7fe22c19a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51376/0 (socket says 172.21.15.6:51376) 2024-09-12T03:14:34.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.508+0000 7fe22b7fe640 1 -- 172.21.15.6:0/2601792426 learned_addr learned my addr 172.21.15.6:0/2601792426 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:14:34.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.508+0000 7fe22affd640 1 --2- 172.21.15.6:0/2601792426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe22c102e00 0x7fe22c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:34.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.508+0000 7fe22bfff640 1 --2- 172.21.15.6:0/2601792426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe22c1037c0 0x7fe22c1a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:34.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.508+0000 7fe22b7fe640 1 -- 172.21.15.6:0/2601792426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe22c102e00 msgr2=0x7fe22c19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:34.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.508+0000 7fe22b7fe640 1 --2- 172.21.15.6:0/2601792426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe22c102e00 0x7fe22c19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:34.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.508+0000 7fe22b7fe640 1 -- 172.21.15.6:0/2601792426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe22c1037c0 msgr2=0x7fe22c1a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:34.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.508+0000 7fe22b7fe640 1 --2- 172.21.15.6:0/2601792426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe22c1037c0 0x7fe22c1a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:34.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.508+0000 7fe22b7fe640 1 -- 172.21.15.6:0/2601792426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22c1a1ca0 con 0x7fe22c101c00 2024-09-12T03:14:34.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.508+0000 7fe22bfff640 1 --2- 172.21.15.6:0/2601792426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe22c1037c0 0x7fe22c1a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:34.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.508+0000 7fe22b7fe640 1 --2- 172.21.15.6:0/2601792426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe22c101c00 0x7fe22c19a070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe21c031610 tx=0x7fe21c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:34.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.508+0000 7fe228ff9640 1 -- 172.21.15.6:0/2601792426 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c03f030 con 0x7fe22c101c00 2024-09-12T03:14:34.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.509+0000 7fe228ff9640 1 -- 172.21.15.6:0/2601792426 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe21c003ce0 con 0x7fe22c101c00 2024-09-12T03:14:34.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.509+0000 7fe231edc640 1 -- 172.21.15.6:0/2601792426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22c1a1f30 con 0x7fe22c101c00 2024-09-12T03:14:34.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.509+0000 7fe228ff9640 1 -- 172.21.15.6:0/2601792426 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c035070 con 0x7fe22c101c00 2024-09-12T03:14:34.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.509+0000 7fe231edc640 1 -- 172.21.15.6:0/2601792426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe22c1a2330 con 0x7fe22c101c00 2024-09-12T03:14:34.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.510+0000 7fe231edc640 1 -- 172.21.15.6:0/2601792426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe22c1a2560 con 0x7fe22c101c00 2024-09-12T03:14:34.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.511+0000 7fe228ff9640 1 -- 172.21.15.6:0/2601792426 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7fe21c042030 con 0x7fe22c101c00 2024-09-12T03:14:34.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.512+0000 7fe228ff9640 1 --2- 172.21.15.6:0/2601792426 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe2140a2b20 0x7fe2140a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:34.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.512+0000 7fe228ff9640 1 -- 172.21.15.6:0/2601792426 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 2445+0+0 (secure 0 0 0) 0x7fe21c0ede20 con 0x7fe22c101c00 2024-09-12T03:14:34.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.512+0000 7fe22affd640 1 --2- 172.21.15.6:0/2601792426 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe2140a2b20 0x7fe2140a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:34.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.512+0000 7fe22affd640 1 --2- 172.21.15.6:0/2601792426 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe2140a2b20 0x7fe2140a4fe0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe22c19b590 tx=0x7fe22000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:34.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.517+0000 7fe228ff9640 1 -- 172.21.15.6:0/2601792426 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe21c0bc4b0 con 0x7fe22c101c00 2024-09-12T03:14:34.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:34.704+0000 7fe231edc640 1 -- 172.21.15.6:0/2601792426 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe22c075b40 con 0x7fe2140a2b20 2024-09-12T03:14:34.705 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:34 smithi006 ceph-mon[30991]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:34.705 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:34 smithi006 ceph-mon[30991]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T03:14:34.705 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:34 smithi006 ceph-mon[30991]: from='osd.2 [v2:172.21.15.161:6800/392541009,v1:172.21.15.161:6801/392541009]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:14:34.705 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:34 smithi006 ceph-mon[30991]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T03:14:34.705 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:34 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:14:34.706 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:34 smithi006 ceph-mon[30991]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:14:34.706 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:34 smithi006 ceph-mon[30991]: Detected new or changed devices on smithi161 2024-09-12T03:14:34.706 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:34 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:34.706 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:34 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:34.706 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:34 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:34.706 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:34 smithi006 ceph-mon[30991]: Adjusting osd_memory_target on smithi161 to 17088M 2024-09-12T03:14:34.706 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:34 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:34.706 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:34 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:34.706 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:34 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:34.706 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:34 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:34.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:34 smithi161 ceph-mon[35485]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:34.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:34 smithi161 ceph-mon[35485]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T03:14:34.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:34 smithi161 ceph-mon[35485]: from='osd.2 [v2:172.21.15.161:6800/392541009,v1:172.21.15.161:6801/392541009]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:14:34.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:34 smithi161 ceph-mon[35485]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T03:14:34.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:34 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:14:34.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:34 smithi161 ceph-mon[35485]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:14:34.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:34 smithi161 ceph-mon[35485]: Detected new or changed devices on smithi161 2024-09-12T03:14:34.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:34 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:34.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:34 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:34.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:34 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:34.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:34 smithi161 ceph-mon[35485]: Adjusting osd_memory_target on smithi161 to 17088M 2024-09-12T03:14:34.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:34 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:34.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:34 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:34.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:34 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:34.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:34 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:34.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:34 smithi118 ceph-mon[35048]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:34.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:34 smithi118 ceph-mon[35048]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T03:14:34.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:34 smithi118 ceph-mon[35048]: from='osd.2 [v2:172.21.15.161:6800/392541009,v1:172.21.15.161:6801/392541009]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:14:34.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:34 smithi118 ceph-mon[35048]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T03:14:34.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:34 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:14:34.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:34 smithi118 ceph-mon[35048]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:14:34.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:34 smithi118 ceph-mon[35048]: Detected new or changed devices on smithi161 2024-09-12T03:14:34.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:34 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:34.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:34 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:34.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:34 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:34.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:34 smithi118 ceph-mon[35048]: Adjusting osd_memory_target on smithi161 to 17088M 2024-09-12T03:14:34.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:34 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:34.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:34 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:34.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:34 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:34.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:34 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:35.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:35 smithi006 ceph-mon[30991]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-12T03:14:35.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:35 smithi006 ceph-mon[30991]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T03:14:35.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:35 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:14:35.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:35 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:14:35.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:35 smithi006 ceph-mon[30991]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:14:35.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:35 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:14:35.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:35 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:14:35.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:35 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:35.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:35 smithi006 ceph-mon[30991]: from='osd.2 ' entity='osd.2' 2024-09-12T03:14:35.863 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:14:35 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-2[40470]: 2024-09-12T03:14:35.363+0000 7f00149c1640 -1 osd.2 0 waiting for initial osdmap 2024-09-12T03:14:35.863 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:14:35 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-2[40470]: 2024-09-12T03:14:35.369+0000 7f000bfbc640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:14:35.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:35 smithi161 ceph-mon[35485]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-12T03:14:35.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:35 smithi161 ceph-mon[35485]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T03:14:35.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:35 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:14:35.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:35 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:14:35.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:35 smithi161 ceph-mon[35485]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:14:35.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:35 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:14:35.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:35 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:14:35.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:35 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:35.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:35 smithi161 ceph-mon[35485]: from='osd.2 ' entity='osd.2' 2024-09-12T03:14:35.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:35 smithi118 ceph-mon[35048]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-12T03:14:35.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:35 smithi118 ceph-mon[35048]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T03:14:35.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:35 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:14:35.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:35 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:14:35.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:35 smithi118 ceph-mon[35048]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:14:35.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:35 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:14:35.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:35 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:14:35.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:35 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:35.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:35 smithi118 ceph-mon[35048]: from='osd.2 ' entity='osd.2' 2024-09-12T03:14:36.759 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:36 smithi006 ceph-mon[30991]: purged_snaps scrub starts 2024-09-12T03:14:36.759 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:36 smithi006 ceph-mon[30991]: purged_snaps scrub ok 2024-09-12T03:14:36.759 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:36 smithi006 ceph-mon[30991]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:36.759 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:36 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:14:36.759 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:36 smithi006 ceph-mon[30991]: osd.2 [v2:172.21.15.161:6800/392541009,v1:172.21.15.161:6801/392541009] boot 2024-09-12T03:14:36.759 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:36 smithi006 ceph-mon[30991]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T03:14:36.759 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:36 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:14:36.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:36 smithi161 ceph-mon[35485]: purged_snaps scrub starts 2024-09-12T03:14:36.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:36 smithi161 ceph-mon[35485]: purged_snaps scrub ok 2024-09-12T03:14:36.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:36 smithi161 ceph-mon[35485]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:36.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:36 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:14:36.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:36 smithi161 ceph-mon[35485]: osd.2 [v2:172.21.15.161:6800/392541009,v1:172.21.15.161:6801/392541009] boot 2024-09-12T03:14:36.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:36 smithi161 ceph-mon[35485]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T03:14:36.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:36 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:14:36.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:36 smithi118 ceph-mon[35048]: purged_snaps scrub starts 2024-09-12T03:14:36.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:36 smithi118 ceph-mon[35048]: purged_snaps scrub ok 2024-09-12T03:14:36.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:36 smithi118 ceph-mon[35048]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:14:36.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:36 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:14:36.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:36 smithi118 ceph-mon[35048]: osd.2 [v2:172.21.15.161:6800/392541009,v1:172.21.15.161:6801/392541009] boot 2024-09-12T03:14:36.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:36 smithi118 ceph-mon[35048]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T03:14:36.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:36 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:14:37.915 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:37 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1715485087' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25ced4c1-bf31-402d-8397-d42fd57f615f"}]: dispatch 2024-09-12T03:14:37.915 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:37 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1715485087' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25ced4c1-bf31-402d-8397-d42fd57f615f"}]': finished 2024-09-12T03:14:37.915 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:37 smithi006 ceph-mon[30991]: osdmap e19: 4 total, 3 up, 4 in 2024-09-12T03:14:37.915 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:37 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:14:37.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:37 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1715485087' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25ced4c1-bf31-402d-8397-d42fd57f615f"}]: dispatch 2024-09-12T03:14:37.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:37 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1715485087' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25ced4c1-bf31-402d-8397-d42fd57f615f"}]': finished 2024-09-12T03:14:37.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:37 smithi118 ceph-mon[35048]: osdmap e19: 4 total, 3 up, 4 in 2024-09-12T03:14:37.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:37 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:14:38.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:37 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1715485087' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25ced4c1-bf31-402d-8397-d42fd57f615f"}]: dispatch 2024-09-12T03:14:38.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:37 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1715485087' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25ced4c1-bf31-402d-8397-d42fd57f615f"}]': finished 2024-09-12T03:14:38.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:37 smithi161 ceph-mon[35485]: osdmap e19: 4 total, 3 up, 4 in 2024-09-12T03:14:38.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:37 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:14:38.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:38 smithi118 ceph-mon[35048]: pgmap v78: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:14:38.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:38 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:14:38.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:38 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2221760124' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:14:38.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:38 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T03:14:38.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:38 smithi118 ceph-mon[35048]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T03:14:38.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:38 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:14:38.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:38 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:14:39.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:38 smithi006 ceph-mon[30991]: pgmap v78: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:14:39.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:38 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:14:39.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:38 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2221760124' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:14:39.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:38 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T03:14:39.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:38 smithi006 ceph-mon[30991]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T03:14:39.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:38 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:14:39.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:38 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:14:39.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:38 smithi161 ceph-mon[35485]: pgmap v78: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:14:39.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:38 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:14:39.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:38 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2221760124' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:14:39.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:38 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T03:14:39.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:38 smithi161 ceph-mon[35485]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T03:14:39.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:38 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:14:39.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:38 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:14:40.110 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:14:39 smithi006 sudo[50241]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:14:40.111 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:14:39 smithi006 sudo[50241]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:14:40.111 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:14:39 smithi006 sudo[50241]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:14:40.111 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:14:39 smithi006 sudo[50241]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:14:40.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:40 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:14:40.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:40 smithi006 ceph-mon[30991]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T03:14:40.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:40 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:14:40.362 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:14:40 smithi006 sudo[50249]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:14:40.362 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:14:40 smithi006 sudo[50249]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:14:40.362 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:14:40 smithi006 sudo[50249]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:14:40.362 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:14:40 smithi006 sudo[50249]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:14:40.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:40 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:14:40.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:40 smithi161 ceph-mon[35485]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T03:14:40.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:40 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:14:40.402 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:40 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:14:40.402 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:40 smithi118 ceph-mon[35048]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T03:14:40.402 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:40 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:14:40.656 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:14:40 smithi118 sudo[43310]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:14:40.656 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:14:40 smithi118 sudo[43310]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:14:40.657 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:14:40 smithi118 sudo[43310]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:14:40.657 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:14:40 smithi118 sudo[43310]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:14:40.962 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:14:40 smithi118 sudo[43313]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:14:40.962 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:14:40 smithi118 sudo[43313]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:14:40.962 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:14:40 smithi118 sudo[43313]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:14:40.962 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:14:40 smithi118 sudo[43313]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:14:41.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:41 smithi161 ceph-mon[35485]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:14:41.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:41 smithi161 ceph-mon[35485]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T03:14:41.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:41 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:14:41.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:41 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:41.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:41 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:41.182 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:14:40 smithi161 sudo[44576]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:14:41.182 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:14:40 smithi161 sudo[44576]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:14:41.182 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:14:40 smithi161 sudo[44576]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:14:41.182 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:14:40 smithi161 sudo[44576]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:14:41.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:41 smithi118 ceph-mon[35048]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:14:41.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:41 smithi118 ceph-mon[35048]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T03:14:41.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:41 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:14:41.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:41 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:41.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:41 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:41.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:41 smithi006 ceph-mon[30991]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:14:41.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:41 smithi006 ceph-mon[30991]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T03:14:41.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:14:41.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:41.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:41.613 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:14:41 smithi161 sudo[44579]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:14:41.613 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:14:41 smithi161 sudo[44579]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:14:41.613 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:14:41 smithi161 sudo[44579]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:14:41.613 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:14:41 smithi161 sudo[44579]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:14:42.109 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:41 smithi006 sudo[50814]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:14:42.110 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:41 smithi006 sudo[50814]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:14:42.110 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:41 smithi006 sudo[50814]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:14:42.110 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:41 smithi006 sudo[50814]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:14:42.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:42.125+0000 7fe228ff9640 1 -- 172.21.15.6:0/2601792426 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe21c0f4080 con 0x7fe22c101c00 2024-09-12T03:14:42.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:42 smithi006 sudo[51005]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:14:42.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:42 smithi006 sudo[51005]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:14:42.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:42 smithi006 sudo[51005]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:14:42.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:42 smithi006 sudo[51005]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:14:42.608 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:42 smithi161 sudo[44582]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:14:42.608 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:42 smithi161 sudo[44582]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:14:42.608 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:42 smithi161 sudo[44582]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:14:42.608 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:42 smithi161 sudo[44582]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:14:42.752 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:42 smithi006 ceph-mon[30991]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 101 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:14:42.752 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:42.752 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:42 smithi006 ceph-mon[30991]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:14:42.753 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:42 smithi006 ceph-mon[30991]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:14:42.753 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:14:42.753 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:14:42.753 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:14:42.753 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:42 smithi006 ceph-mon[30991]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T03:14:42.753 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:14:42.753 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:14:42.753 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:14:42.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:42 smithi161 sudo[44585]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:14:42.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:42 smithi161 sudo[44585]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:14:42.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:42 smithi161 sudo[44585]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:14:42.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:42 smithi161 sudo[44585]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:14:42.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:42 smithi161 ceph-mon[35485]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 101 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:14:42.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:42.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:42 smithi161 ceph-mon[35485]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:14:42.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:42 smithi161 ceph-mon[35485]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:14:42.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:14:42.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:14:42.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:14:42.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:42 smithi161 ceph-mon[35485]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T03:14:42.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:14:42.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:14:42.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:14:43.106 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:42 smithi118 sudo[43319]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:14:43.107 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:42 smithi118 sudo[43319]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:14:43.107 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:42 smithi118 sudo[43319]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:14:43.107 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:42 smithi118 sudo[43319]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:14:43.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 sudo[43322]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:14:43.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 sudo[43322]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:14:43.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 sudo[43322]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:14:43.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 sudo[43322]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:14:43.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 ceph-mon[35048]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 101 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:14:43.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:43.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 ceph-mon[35048]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:14:43.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 ceph-mon[35048]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:14:43.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:14:43.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:14:43.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:14:43.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 ceph-mon[35048]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T03:14:43.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:14:43.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:14:43.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:14:43.849 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:43 smithi006 ceph-mon[30991]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:14:43.850 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:43 smithi006 ceph-mon[30991]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:14:43.850 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:43 smithi006 ceph-mon[30991]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:14:43.850 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:43 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:14:43.850 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:43 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:14:43.850 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:43 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:14:43.850 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:43 smithi006 ceph-mon[30991]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:14:43.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 ceph-mon[35048]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:14:43.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 ceph-mon[35048]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:14:43.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 ceph-mon[35048]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:14:43.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:14:43.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:14:43.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:14:43.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:43 smithi118 ceph-mon[35048]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:14:44.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:43 smithi161 ceph-mon[35485]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:14:44.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:43 smithi161 ceph-mon[35485]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:14:44.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:43 smithi161 ceph-mon[35485]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:14:44.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:43 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:14:44.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:43 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:14:44.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:43 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:14:44.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:43 smithi161 ceph-mon[35485]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:14:44.766 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:44 smithi006 ceph-mon[30991]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:14:44.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:44 smithi118 ceph-mon[35048]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:14:45.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:44 smithi161 ceph-mon[35485]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:14:46.734 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:46 smithi006 ceph-mon[30991]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:14:46.735 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:46 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T03:14:46.735 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:46 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:46.735 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:46 smithi006 ceph-mon[30991]: Deploying daemon osd.3 on smithi006 2024-09-12T03:14:46.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:46 smithi118 ceph-mon[35048]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:14:46.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:46 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T03:14:46.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:46 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:46.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:46 smithi118 ceph-mon[35048]: Deploying daemon osd.3 on smithi006 2024-09-12T03:14:47.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:46 smithi161 ceph-mon[35485]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:14:47.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:46 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T03:14:47.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:46 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:47.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:46 smithi161 ceph-mon[35485]: Deploying daemon osd.3 on smithi006 2024-09-12T03:14:48.792 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:48 smithi006 ceph-mon[30991]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-12T03:14:48.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:48 smithi118 ceph-mon[35048]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-12T03:14:49.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:48 smithi161 ceph-mon[35485]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-12T03:14:50.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:50 smithi006 ceph-mon[30991]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-12T03:14:50.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:50 smithi118 ceph-mon[35048]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-12T03:14:51.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:50 smithi161 ceph-mon[35485]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-12T03:14:52.817 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:52 smithi006 ceph-mon[30991]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-12T03:14:52.818 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:52 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:52.818 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:52 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:52.818 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:52 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:52.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:52 smithi118 ceph-mon[35048]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-12T03:14:52.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:52 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:52.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:52 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:52.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:52 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:53.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:52 smithi161 ceph-mon[35485]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-12T03:14:53.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:52 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:53.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:52 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:53.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:52 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:54.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:54 smithi006 ceph-mon[30991]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-12T03:14:54.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:54 smithi118 ceph-mon[35048]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-12T03:14:55.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:54 smithi161 ceph-mon[35485]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-12T03:14:55.955 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:55 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:55.955 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:55 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:55.955 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:55 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:55.955 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:55 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:55.955 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:55 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:55.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:55.995+0000 7fe228ff9640 1 -- 172.21.15.6:0/2601792426 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe22c075b40 con 0x7fe2140a2b20 2024-09-12T03:14:55.996 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 3 on host 'smithi006' 2024-09-12T03:14:55.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:55.997+0000 7fe231edc640 1 -- 172.21.15.6:0/2601792426 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe2140a2b20 msgr2=0x7fe2140a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:55.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:55.997+0000 7fe231edc640 1 --2- 172.21.15.6:0/2601792426 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe2140a2b20 0x7fe2140a4fe0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe22c19b590 tx=0x7fe22000b410 comp rx=0 tx=0).stop 2024-09-12T03:14:55.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:55.997+0000 7fe231edc640 1 -- 172.21.15.6:0/2601792426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe22c101c00 msgr2=0x7fe22c19a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:55.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:55.997+0000 7fe231edc640 1 --2- 172.21.15.6:0/2601792426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe22c101c00 0x7fe22c19a070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe21c031610 tx=0x7fe21c0043b0 comp rx=0 tx=0).stop 2024-09-12T03:14:55.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:55.997+0000 7fe231edc640 1 -- 172.21.15.6:0/2601792426 shutdown_connections 2024-09-12T03:14:55.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:55.997+0000 7fe231edc640 1 --2- 172.21.15.6:0/2601792426 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe2140a2b20 0x7fe2140a4fe0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:55.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:55.997+0000 7fe231edc640 1 --2- 172.21.15.6:0/2601792426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe22c1037c0 0x7fe22c1a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:55.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:55.997+0000 7fe231edc640 1 --2- 172.21.15.6:0/2601792426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe22c102e00 0x7fe22c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:55.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:55.997+0000 7fe231edc640 1 --2- 172.21.15.6:0/2601792426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe22c101c00 0x7fe22c19a070 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:55.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:55.997+0000 7fe231edc640 1 -- 172.21.15.6:0/2601792426 >> 172.21.15.6:0/2601792426 conn(0x7fe22c0fd390 msgr2=0x7fe22c074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:55.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:55.998+0000 7fe231edc640 1 -- 172.21.15.6:0/2601792426 shutdown_connections 2024-09-12T03:14:55.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:14:55.998+0000 7fe231edc640 1 -- 172.21.15.6:0/2601792426 wait complete. 2024-09-12T03:14:56.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:55 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:56.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:55 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:56.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:55 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:56.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:55 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:56.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:55 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:56.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:55 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:56.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:55 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:56.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:55 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:56.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:55 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:56.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:55 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:56.611 DEBUG:teuthology.orchestra.run.smithi006:osd.3> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.3.service 2024-09-12T03:14:56.613 INFO:tasks.cephadm:Deploying osd.4 on smithi118 with /dev/vg_nvme/lv_3... 2024-09-12T03:14:56.614 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T03:14:56.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:56 smithi006 ceph-mon[30991]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-12T03:14:56.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:56 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:56.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:56 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:56.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:56 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:56.868 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:14:56.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:56 smithi118 ceph-mon[35048]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-12T03:14:57.006 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:56 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:57.006 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:56 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:57.006 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:56 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:57.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:56 smithi161 ceph-mon[35485]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-12T03:14:57.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:56 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:57.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:56 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:57.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:56 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:58.361 INFO:journalctl@ceph.osd.3.smithi006.stdout:Sep 12 03:14:58 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-3[51847]: 2024-09-12T03:14:58.107+0000 7f8250162640 -1 osd.3 0 log_to_monitors true 2024-09-12T03:14:59.086 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:58 smithi118 ceph-mon[35048]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-12T03:14:59.086 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:58 smithi118 ceph-mon[35048]: from='osd.3 [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T03:14:59.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:58 smithi006 ceph-mon[30991]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-12T03:14:59.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:58 smithi006 ceph-mon[30991]: from='osd.3 [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T03:14:59.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:58 smithi161 ceph-mon[35485]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-12T03:14:59.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:58 smithi161 ceph-mon[35485]: from='osd.3 [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T03:14:59.292 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-12T03:14:59.319 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi118:vg_nvme/lv_3 2024-09-12T03:14:59.571 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:14:59.935 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:59 smithi118 ceph-mon[35048]: from='osd.3 [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T03:14:59.935 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:59 smithi118 ceph-mon[35048]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T03:14:59.935 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:59 smithi118 ceph-mon[35048]: from='osd.3 [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-09-12T03:14:59.935 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:59 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:14:59.935 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:59 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:59.935 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:59 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:59.935 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:59 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:59.935 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:59 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:59.935 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:59 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:14:59.935 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:59 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:59.935 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:59 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:59.936 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:14:59 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:00.020 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.020+0000 7ff656528640 1 -- 172.21.15.118:0/3339262408 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff650101cd0 msgr2=0x7ff650102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:00.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.020+0000 7ff656528640 1 --2- 172.21.15.118:0/3339262408 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff650101cd0 0x7ff650102150 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff644009900 tx=0x7ff64402a410 comp rx=0 tx=0).stop 2024-09-12T03:15:00.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.020+0000 7ff656528640 1 -- 172.21.15.118:0/3339262408 shutdown_connections 2024-09-12T03:15:00.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.020+0000 7ff656528640 1 --2- 172.21.15.118:0/3339262408 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff650102690 0x7ff650108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:00.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.020+0000 7ff656528640 1 --2- 172.21.15.118:0/3339262408 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff650101cd0 0x7ff650102150 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:00.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.020+0000 7ff656528640 1 --2- 172.21.15.118:0/3339262408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff650100ad0 0x7ff650100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:00.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.020+0000 7ff656528640 1 -- 172.21.15.118:0/3339262408 >> 172.21.15.118:0/3339262408 conn(0x7ff6500fc280 msgr2=0x7ff6500fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:00.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.020+0000 7ff656528640 1 -- 172.21.15.118:0/3339262408 shutdown_connections 2024-09-12T03:15:00.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.020+0000 7ff656528640 1 -- 172.21.15.118:0/3339262408 wait complete. 2024-09-12T03:15:00.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.021+0000 7ff656528640 1 Processor -- start 2024-09-12T03:15:00.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.021+0000 7ff656528640 1 -- start start 2024-09-12T03:15:00.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.021+0000 7ff656528640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff650100ad0 0x7ff650104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:00.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.021+0000 7ff656528640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff650101cd0 0x7ff6501030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:00.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.021+0000 7ff656528640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff650102690 0x7ff6501035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:00.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.021+0000 7ff656528640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff65010c7b0 con 0x7ff650101cd0 2024-09-12T03:15:00.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.021+0000 7ff656528640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7ff65010c630 con 0x7ff650100ad0 2024-09-12T03:15:00.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.021+0000 7ff656528640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff65010c930 con 0x7ff650102690 2024-09-12T03:15:00.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.021+0000 7ff64ffff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff650100ad0 0x7ff650104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:00.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.021+0000 7ff64ffff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff650100ad0 0x7ff650104a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:56782/0 (socket says 172.21.15.118:56782) 2024-09-12T03:15:00.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.021+0000 7ff64ffff640 1 -- 172.21.15.118:0/1949624079 learned_addr learned my addr 172.21.15.118:0/1949624079 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-09-12T03:15:00.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.021+0000 7ff64f7fe640 1 --2- 172.21.15.118:0/1949624079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff650101cd0 0x7ff6501030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:00.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.021+0000 7ff654ac5640 1 --2- 172.21.15.118:0/1949624079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff650102690 0x7ff6501035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:00.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.022+0000 7ff64ffff640 1 -- 172.21.15.118:0/1949624079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff650102690 msgr2=0x7ff6501035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:00.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.022+0000 7ff64ffff640 1 --2- 172.21.15.118:0/1949624079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff650102690 0x7ff6501035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:00.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.022+0000 7ff64ffff640 1 -- 172.21.15.118:0/1949624079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff650101cd0 msgr2=0x7ff6501030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:00.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.022+0000 7ff64ffff640 1 --2- 172.21.15.118:0/1949624079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff650101cd0 0x7ff6501030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:00.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.022+0000 7ff64ffff640 1 -- 172.21.15.118:0/1949624079 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff650103eb0 con 0x7ff650100ad0 2024-09-12T03:15:00.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.022+0000 7ff654ac5640 1 --2- 172.21.15.118:0/1949624079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff650102690 0x7ff6501035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:15:00.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.022+0000 7ff64ffff640 1 --2- 172.21.15.118:0/1949624079 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff650100ad0 0x7ff650104a60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff64000d770 tx=0x7ff64000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:00.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.022+0000 7ff64d7fa640 1 -- 172.21.15.118:0/1949624079 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640017070 con 0x7ff650100ad0 2024-09-12T03:15:00.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.022+0000 7ff656528640 1 -- 172.21.15.118:0/1949624079 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6501a2100 con 0x7ff650100ad0 2024-09-12T03:15:00.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.022+0000 7ff64d7fa640 1 -- 172.21.15.118:0/1949624079 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff640009c30 con 0x7ff650100ad0 2024-09-12T03:15:00.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.022+0000 7ff64d7fa640 1 -- 172.21.15.118:0/1949624079 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640003dd0 con 0x7ff650100ad0 2024-09-12T03:15:00.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.022+0000 7ff656528640 1 -- 172.21.15.118:0/1949624079 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6501a26e0 con 0x7ff650100ad0 2024-09-12T03:15:00.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.023+0000 7ff656528640 1 -- 172.21.15.118:0/1949624079 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6501a2910 con 0x7ff650100ad0 2024-09-12T03:15:00.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.024+0000 7ff64d7fa640 1 -- 172.21.15.118:0/1949624079 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ff640003f70 con 0x7ff650100ad0 2024-09-12T03:15:00.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.025+0000 7ff64d7fa640 1 --2- 172.21.15.118:0/1949624079 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff6380a2b80 0x7ff6380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:00.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.025+0000 7ff64d7fa640 1 -- 172.21.15.118:0/1949624079 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3389+0+0 (secure 0 0 0) 0x7ff6400cda60 con 0x7ff650100ad0 2024-09-12T03:15:00.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.025+0000 7ff64f7fe640 1 --2- 172.21.15.118:0/1949624079 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff6380a2b80 0x7ff6380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:00.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.026+0000 7ff64f7fe640 1 --2- 172.21.15.118:0/1949624079 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff6380a2b80 0x7ff6380a5040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff6440314e0 tx=0x7ff644000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:00.033 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.033+0000 7ff64d7fa640 1 -- 172.21.15.118:0/1949624079 <== mon.2 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff640009de0 con 0x7ff650100ad0 2024-09-12T03:15:00.112 INFO:journalctl@ceph.osd.3.smithi006.stdout:Sep 12 03:14:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-3[51847]: 2024-09-12T03:14:59.811+0000 7f824b8c2640 -1 osd.3 0 waiting for initial osdmap 2024-09-12T03:15:00.112 INFO:journalctl@ceph.osd.3.smithi006.stdout:Sep 12 03:14:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-3[51847]: 2024-09-12T03:14:59.817+0000 7f8247efc640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:15:00.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:59 smithi006 ceph-mon[30991]: from='osd.3 [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T03:15:00.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:59 smithi006 ceph-mon[30991]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T03:15:00.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:59 smithi006 ceph-mon[30991]: from='osd.3 [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-09-12T03:15:00.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:15:00.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:00.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:00.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:00.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:00.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:00.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:00.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:00.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:14:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:00.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:59 smithi161 ceph-mon[35485]: from='osd.3 [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T03:15:00.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:59 smithi161 ceph-mon[35485]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T03:15:00.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:59 smithi161 ceph-mon[35485]: from='osd.3 [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-09-12T03:15:00.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:15:00.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:00.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:00.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:00.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:00.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:00.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:00.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:00.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:14:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:00.215 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:00.215+0000 7ff656528640 1 -- 172.21.15.118:0/1949624079 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff65006a770 con 0x7ff6380a2b80 2024-09-12T03:15:00.887 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:00 smithi118 ceph-mon[35048]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:15:00.887 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:00 smithi118 ceph-mon[35048]: Detected new or changed devices on smithi006 2024-09-12T03:15:00.887 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:00 smithi118 ceph-mon[35048]: Adjusting osd_memory_target on smithi006 to 8546M 2024-09-12T03:15:00.887 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:00 smithi118 ceph-mon[35048]: from='osd.3 [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-09-12T03:15:00.887 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:00 smithi118 ceph-mon[35048]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T03:15:00.887 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:00 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:15:00.887 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:00 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:15:00.887 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:00 smithi118 ceph-mon[35048]: osd.3 [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151] boot 2024-09-12T03:15:00.888 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:00 smithi118 ceph-mon[35048]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T03:15:00.888 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:00 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:15:00.888 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:00 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:15:00.888 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:00 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:15:00.888 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:00 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:01.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:00 smithi006 ceph-mon[30991]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:15:01.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:00 smithi006 ceph-mon[30991]: Detected new or changed devices on smithi006 2024-09-12T03:15:01.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:00 smithi006 ceph-mon[30991]: Adjusting osd_memory_target on smithi006 to 8546M 2024-09-12T03:15:01.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:00 smithi006 ceph-mon[30991]: from='osd.3 [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-09-12T03:15:01.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:00 smithi006 ceph-mon[30991]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T03:15:01.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:00 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:15:01.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:00 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:15:01.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:00 smithi006 ceph-mon[30991]: osd.3 [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151] boot 2024-09-12T03:15:01.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:00 smithi006 ceph-mon[30991]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T03:15:01.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:00 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:15:01.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:00 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:15:01.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:00 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:15:01.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:00 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:01.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:00 smithi161 ceph-mon[35485]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:15:01.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:00 smithi161 ceph-mon[35485]: Detected new or changed devices on smithi006 2024-09-12T03:15:01.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:00 smithi161 ceph-mon[35485]: Adjusting osd_memory_target on smithi006 to 8546M 2024-09-12T03:15:01.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:00 smithi161 ceph-mon[35485]: from='osd.3 [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-09-12T03:15:01.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:00 smithi161 ceph-mon[35485]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T03:15:01.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:00 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:15:01.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:00 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:15:01.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:00 smithi161 ceph-mon[35485]: osd.3 [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151] boot 2024-09-12T03:15:01.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:00 smithi161 ceph-mon[35485]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T03:15:01.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:00 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:15:01.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:00 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:15:01.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:00 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:15:01.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:00 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:02.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:01 smithi006 ceph-mon[30991]: purged_snaps scrub starts 2024-09-12T03:15:02.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:01 smithi006 ceph-mon[30991]: purged_snaps scrub ok 2024-09-12T03:15:02.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:01 smithi006 ceph-mon[30991]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:15:02.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:01 smithi006 ceph-mon[30991]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T03:15:02.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:01 smithi161 ceph-mon[35485]: purged_snaps scrub starts 2024-09-12T03:15:02.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:01 smithi161 ceph-mon[35485]: purged_snaps scrub ok 2024-09-12T03:15:02.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:01 smithi161 ceph-mon[35485]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:15:02.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:01 smithi161 ceph-mon[35485]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T03:15:02.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:01 smithi118 ceph-mon[35048]: purged_snaps scrub starts 2024-09-12T03:15:02.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:01 smithi118 ceph-mon[35048]: purged_snaps scrub ok 2024-09-12T03:15:02.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:01 smithi118 ceph-mon[35048]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:15:02.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:01 smithi118 ceph-mon[35048]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T03:15:03.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:03 smithi118 ceph-mon[35048]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:15:03.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:03 smithi118 ceph-mon[35048]: osdmap e27: 4 total, 4 up, 4 in 2024-09-12T03:15:03.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:03 smithi118 ceph-mon[35048]: from='client.? 172.21.15.118:0/1528257264' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d884604-920b-4c35-b41e-d629ad9f9279"}]: dispatch 2024-09-12T03:15:03.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:03 smithi118 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d884604-920b-4c35-b41e-d629ad9f9279"}]: dispatch 2024-09-12T03:15:03.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:03 smithi118 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d884604-920b-4c35-b41e-d629ad9f9279"}]': finished 2024-09-12T03:15:03.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:03 smithi118 ceph-mon[35048]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T03:15:03.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:03 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:15:03.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:03 smithi118 ceph-mon[35048]: from='client.? 172.21.15.118:0/1595808879' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:15:03.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:03 smithi006 ceph-mon[30991]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:15:03.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:03 smithi006 ceph-mon[30991]: osdmap e27: 4 total, 4 up, 4 in 2024-09-12T03:15:03.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:03 smithi006 ceph-mon[30991]: from='client.? 172.21.15.118:0/1528257264' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d884604-920b-4c35-b41e-d629ad9f9279"}]: dispatch 2024-09-12T03:15:03.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:03 smithi006 ceph-mon[30991]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d884604-920b-4c35-b41e-d629ad9f9279"}]: dispatch 2024-09-12T03:15:03.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:03 smithi006 ceph-mon[30991]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d884604-920b-4c35-b41e-d629ad9f9279"}]': finished 2024-09-12T03:15:03.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:03 smithi006 ceph-mon[30991]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T03:15:03.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:03 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:15:03.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:03 smithi006 ceph-mon[30991]: from='client.? 172.21.15.118:0/1595808879' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:15:03.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:03 smithi161 ceph-mon[35485]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:15:03.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:03 smithi161 ceph-mon[35485]: osdmap e27: 4 total, 4 up, 4 in 2024-09-12T03:15:03.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:03 smithi161 ceph-mon[35485]: from='client.? 172.21.15.118:0/1528257264' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d884604-920b-4c35-b41e-d629ad9f9279"}]: dispatch 2024-09-12T03:15:03.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:03 smithi161 ceph-mon[35485]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d884604-920b-4c35-b41e-d629ad9f9279"}]: dispatch 2024-09-12T03:15:03.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:03 smithi161 ceph-mon[35485]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d884604-920b-4c35-b41e-d629ad9f9279"}]': finished 2024-09-12T03:15:03.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:03 smithi161 ceph-mon[35485]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T03:15:03.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:03 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:15:03.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:03 smithi161 ceph-mon[35485]: from='client.? 172.21.15.118:0/1595808879' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:15:04.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:04 smithi006 ceph-mon[30991]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:15:04.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:04 smithi006 ceph-mon[30991]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T03:15:04.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:04 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:15:04.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:04 smithi161 ceph-mon[35485]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:15:04.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:04 smithi161 ceph-mon[35485]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T03:15:04.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:04 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:15:04.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:04 smithi118 ceph-mon[35048]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:15:04.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:04 smithi118 ceph-mon[35048]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T03:15:04.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:04 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:15:06.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:06 smithi006 ceph-mon[30991]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:15:06.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:06 smithi161 ceph-mon[35485]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:15:06.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:06 smithi118 ceph-mon[35048]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:15:08.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:08 smithi006 ceph-mon[30991]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:15:08.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:08 smithi161 ceph-mon[35485]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:15:08.871 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:08 smithi118 ceph-mon[35048]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:15:10.800 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:10 smithi118 ceph-mon[35048]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-12T03:15:10.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:10 smithi006 ceph-mon[30991]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-12T03:15:10.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:10 smithi161 ceph-mon[35485]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-12T03:15:11.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:11 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T03:15:11.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:11 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:11.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:11 smithi006 ceph-mon[30991]: Deploying daemon osd.4 on smithi118 2024-09-12T03:15:11.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:11 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T03:15:11.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:11 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:11.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:11 smithi161 ceph-mon[35485]: Deploying daemon osd.4 on smithi118 2024-09-12T03:15:11.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:11 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T03:15:11.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:11 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:11.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:11 smithi118 ceph-mon[35048]: Deploying daemon osd.4 on smithi118 2024-09-12T03:15:12.730 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:12 smithi118 ceph-mon[35048]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-12T03:15:12.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:12 smithi006 ceph-mon[30991]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-12T03:15:12.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:12 smithi161 ceph-mon[35485]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-12T03:15:14.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:14 smithi118 ceph-mon[35048]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:15:14.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:14 smithi006 ceph-mon[30991]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:15:14.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:14 smithi161 ceph-mon[35485]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:15:16.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:16 smithi118 ceph-mon[35048]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T03:15:16.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:16 smithi006 ceph-mon[30991]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T03:15:16.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:16 smithi161 ceph-mon[35485]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T03:15:17.734 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:17.734 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:17.734 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:17.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:17.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:17.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:17.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:17.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:17.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:19.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:19 smithi006 ceph-mon[30991]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:15:19.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:19 smithi161 ceph-mon[35485]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:15:19.427 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:19 smithi118 ceph-mon[35048]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:15:20.880 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:20 smithi118 ceph-mon[35048]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:15:20.880 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:20.880 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:20.880 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:20.880 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:20.880 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:20.922 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:20.921+0000 7ff64d7fa640 1 -- 172.21.15.118:0/1949624079 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff65006a770 con 0x7ff6380a2b80 2024-09-12T03:15:20.923 INFO:teuthology.orchestra.run.smithi118.stdout:Created osd(s) 4 on host 'smithi118' 2024-09-12T03:15:20.925 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:20.924+0000 7ff656528640 1 -- 172.21.15.118:0/1949624079 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff6380a2b80 msgr2=0x7ff6380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:20.925 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:20.924+0000 7ff656528640 1 --2- 172.21.15.118:0/1949624079 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff6380a2b80 0x7ff6380a5040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff6440314e0 tx=0x7ff644000bc0 comp rx=0 tx=0).stop 2024-09-12T03:15:20.925 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:20.924+0000 7ff656528640 1 -- 172.21.15.118:0/1949624079 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff650100ad0 msgr2=0x7ff650104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:20.925 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:20.924+0000 7ff656528640 1 --2- 172.21.15.118:0/1949624079 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff650100ad0 0x7ff650104a60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff64000d770 tx=0x7ff64000dc30 comp rx=0 tx=0).stop 2024-09-12T03:15:20.925 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:20.924+0000 7ff656528640 1 -- 172.21.15.118:0/1949624079 shutdown_connections 2024-09-12T03:15:20.925 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:20.924+0000 7ff656528640 1 --2- 172.21.15.118:0/1949624079 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff6380a2b80 0x7ff6380a5040 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:20.925 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:20.924+0000 7ff656528640 1 --2- 172.21.15.118:0/1949624079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff650102690 0x7ff6501035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:20.925 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:20.924+0000 7ff656528640 1 --2- 172.21.15.118:0/1949624079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff650101cd0 0x7ff6501030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:20.926 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:20.924+0000 7ff656528640 1 --2- 172.21.15.118:0/1949624079 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff650100ad0 0x7ff650104a60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:20.926 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:20.924+0000 7ff656528640 1 -- 172.21.15.118:0/1949624079 >> 172.21.15.118:0/1949624079 conn(0x7ff6500fc280 msgr2=0x7ff65006a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:20.926 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:20.924+0000 7ff656528640 1 -- 172.21.15.118:0/1949624079 shutdown_connections 2024-09-12T03:15:20.926 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:15:20.924+0000 7ff656528640 1 -- 172.21.15.118:0/1949624079 wait complete. 2024-09-12T03:15:21.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:20 smithi006 ceph-mon[30991]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:15:21.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:21.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:21.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:21.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:21.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:21.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:20 smithi161 ceph-mon[35485]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:15:21.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:21.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:21.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:21.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:21.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:21.576 DEBUG:teuthology.orchestra.run.smithi118:osd.4> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.4.service 2024-09-12T03:15:21.579 INFO:tasks.cephadm:Deploying osd.5 on smithi161 with /dev/vg_nvme/lv_3... 2024-09-12T03:15:21.579 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T03:15:21.833 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:15:21.852 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:21 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:21.852 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:21 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:21.852 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:21 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:21.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:21 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:21.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:21 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:21.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:21 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:22.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:21 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:22.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:21 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:22.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:21 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:23.145 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:22 smithi161 ceph-mon[35485]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:15:23.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:22 smithi118 ceph-mon[35048]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:15:23.213 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:15:23 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4[46088]: 2024-09-12T03:15:23.114+0000 7fde4a449640 -1 osd.4 0 log_to_monitors true 2024-09-12T03:15:23.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:22 smithi006 ceph-mon[30991]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:15:24.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:23 smithi118 ceph-mon[35048]: from='osd.4 [v2:172.21.15.118:6808/150177024,v1:172.21.15.118:6809/150177024]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:15:24.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:23 smithi118 ceph-mon[35048]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:15:24.213 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:23 smithi161 ceph-mon[35485]: from='osd.4 [v2:172.21.15.118:6808/150177024,v1:172.21.15.118:6809/150177024]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:15:24.213 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:23 smithi161 ceph-mon[35485]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:15:24.215 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:15:24.240 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_3 2024-09-12T03:15:24.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:23 smithi006 ceph-mon[30991]: from='osd.4 [v2:172.21.15.118:6808/150177024,v1:172.21.15.118:6809/150177024]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:15:24.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:23 smithi006 ceph-mon[30991]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:15:24.500 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:15:24.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.951+0000 7fe890b27640 1 -- 172.21.15.161:0/230777584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe88c107940 msgr2=0x7fe88c109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:24.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.951+0000 7fe890b27640 1 --2- 172.21.15.161:0/230777584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe88c107940 0x7fe88c109d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe884009ca0 tx=0x7fe88402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.951+0000 7fe890b27640 1 -- 172.21.15.161:0/230777584 shutdown_connections 2024-09-12T03:15:24.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.951+0000 7fe890b27640 1 --2- 172.21.15.161:0/230777584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe88c107940 0x7fe88c109d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.951+0000 7fe890b27640 1 --2- 172.21.15.161:0/230777584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe88c105010 0x7fe88c107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.951+0000 7fe890b27640 1 --2- 172.21.15.161:0/230777584 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe88c1026e0 0x7fe88c104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.951+0000 7fe890b27640 1 -- 172.21.15.161:0/230777584 >> 172.21.15.161:0/230777584 conn(0x7fe88c0fc280 msgr2=0x7fe88c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:24.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.951+0000 7fe890b27640 1 -- 172.21.15.161:0/230777584 shutdown_connections 2024-09-12T03:15:24.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.951+0000 7fe890b27640 1 -- 172.21.15.161:0/230777584 wait complete. 2024-09-12T03:15:24.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.952+0000 7fe890b27640 1 Processor -- start 2024-09-12T03:15:24.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.952+0000 7fe890b27640 1 -- start start 2024-09-12T03:15:24.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.952+0000 7fe890b27640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe88c1026e0 0x7fe88c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.952+0000 7fe890b27640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe88c105010 0x7fe88c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.952+0000 7fe890b27640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe88c107940 0x7fe88c19f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.952+0000 7fe890b27640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c10d880 con 0x7fe88c105010 2024-09-12T03:15:24.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.952+0000 7fe890b27640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c10d700 con 0x7fe88c107940 2024-09-12T03:15:24.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.952+0000 7fe890b27640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c10da00 con 0x7fe88c1026e0 2024-09-12T03:15:24.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.952+0000 7fe88a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe88c1026e0 0x7fe88c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.952+0000 7fe88a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe88c1026e0 0x7fe88c197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38748/0 (socket says 172.21.15.161:38748) 2024-09-12T03:15:24.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.952+0000 7fe88a59c640 1 -- 172.21.15.161:0/4060552625 learned_addr learned my addr 172.21.15.161:0/4060552625 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-12T03:15:24.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.953+0000 7fe88ad9d640 1 --2- 172.21.15.161:0/4060552625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe88c107940 0x7fe88c19f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.953+0000 7fe889d9b640 1 --2- 172.21.15.161:0/4060552625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe88c105010 0x7fe88c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.953+0000 7fe88a59c640 1 -- 172.21.15.161:0/4060552625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe88c107940 msgr2=0x7fe88c19f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:24.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.953+0000 7fe88a59c640 1 --2- 172.21.15.161:0/4060552625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe88c107940 0x7fe88c19f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.953+0000 7fe88a59c640 1 -- 172.21.15.161:0/4060552625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe88c105010 msgr2=0x7fe88c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:24.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.953+0000 7fe88a59c640 1 --2- 172.21.15.161:0/4060552625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe88c105010 0x7fe88c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.953+0000 7fe88a59c640 1 -- 172.21.15.161:0/4060552625 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88c19fb00 con 0x7fe88c1026e0 2024-09-12T03:15:24.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.953+0000 7fe889d9b640 1 --2- 172.21.15.161:0/4060552625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe88c105010 0x7fe88c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:15:24.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.953+0000 7fe88a59c640 1 --2- 172.21.15.161:0/4060552625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe88c1026e0 0x7fe88c197ed0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00e9f0 tx=0x7fe87c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:24.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.953+0000 7fe87b7fe640 1 -- 172.21.15.161:0/4060552625 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c00cd80 con 0x7fe88c1026e0 2024-09-12T03:15:24.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.953+0000 7fe87b7fe640 1 -- 172.21.15.161:0/4060552625 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe87c005500 con 0x7fe88c1026e0 2024-09-12T03:15:24.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.953+0000 7fe890b27640 1 -- 172.21.15.161:0/4060552625 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88c19fdf0 con 0x7fe88c1026e0 2024-09-12T03:15:24.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.953+0000 7fe87b7fe640 1 -- 172.21.15.161:0/4060552625 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c010690 con 0x7fe88c1026e0 2024-09-12T03:15:24.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.953+0000 7fe890b27640 1 -- 172.21.15.161:0/4060552625 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe88c1a0240 con 0x7fe88c1026e0 2024-09-12T03:15:24.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.954+0000 7fe890b27640 1 -- 172.21.15.161:0/4060552625 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe88c0fdd70 con 0x7fe88c1026e0 2024-09-12T03:15:24.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.956+0000 7fe87b7fe640 1 -- 172.21.15.161:0/4060552625 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe87c0046c0 con 0x7fe88c1026e0 2024-09-12T03:15:24.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.956+0000 7fe87b7fe640 1 --2- 172.21.15.161:0/4060552625 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe8740a2c30 0x7fe8740a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.956+0000 7fe87b7fe640 1 -- 172.21.15.161:0/4060552625 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7fe87c0ce2c0 con 0x7fe88c1026e0 2024-09-12T03:15:24.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.957+0000 7fe889d9b640 1 --2- 172.21.15.161:0/4060552625 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe8740a2c30 0x7fe8740a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.957+0000 7fe889d9b640 1 --2- 172.21.15.161:0/4060552625 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe8740a2c30 0x7fe8740a50f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe88c1993f0 tx=0x7fe880005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:24.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:24.961+0000 7fe87b7fe640 1 -- 172.21.15.161:0/4060552625 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe87c09c3b0 con 0x7fe88c1026e0 2024-09-12T03:15:25.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:25.143+0000 7fe890b27640 1 -- 172.21.15.161:0/4060552625 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe88c062630 con 0x7fe8740a2c30 2024-09-12T03:15:25.144 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:24 smithi161 ceph-mon[35485]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:15:25.144 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:24 smithi161 ceph-mon[35485]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T03:15:25.144 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:24 smithi161 ceph-mon[35485]: from='osd.4 [v2:172.21.15.118:6808/150177024,v1:172.21.15.118:6809/150177024]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:15:25.145 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:24 smithi161 ceph-mon[35485]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T03:15:25.145 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:15:25.145 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:24 smithi161 ceph-mon[35485]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:15:25.145 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:25.145 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:25.145 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:25.145 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:25.145 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:25.145 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:25.145 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:25.145 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:25.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:24 smithi118 ceph-mon[35048]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:15:25.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:24 smithi118 ceph-mon[35048]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T03:15:25.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:24 smithi118 ceph-mon[35048]: from='osd.4 [v2:172.21.15.118:6808/150177024,v1:172.21.15.118:6809/150177024]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:15:25.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:24 smithi118 ceph-mon[35048]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T03:15:25.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:15:25.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:24 smithi118 ceph-mon[35048]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:15:25.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:25.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:25.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:25.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:25.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:25.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:25.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:25.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:25.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:24 smithi006 ceph-mon[30991]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:15:25.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:24 smithi006 ceph-mon[30991]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T03:15:25.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:24 smithi006 ceph-mon[30991]: from='osd.4 [v2:172.21.15.118:6808/150177024,v1:172.21.15.118:6809/150177024]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:15:25.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:24 smithi006 ceph-mon[30991]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T03:15:25.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:15:25.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:24 smithi006 ceph-mon[30991]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:15:25.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:25.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:25.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:25.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:25.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:25.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:25.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:25.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:25.943 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:15:25 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4[46088]: 2024-09-12T03:15:25.555+0000 7fde45ba9640 -1 osd.4 0 waiting for initial osdmap 2024-09-12T03:15:25.943 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:15:25 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4[46088]: 2024-09-12T03:15:25.560+0000 7fde3e1db640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:15:26.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:25 smithi161 ceph-mon[35485]: Detected new or changed devices on smithi118 2024-09-12T03:15:26.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:25 smithi161 ceph-mon[35485]: Adjusting osd_memory_target on smithi118 to 8544M 2024-09-12T03:15:26.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:25 smithi161 ceph-mon[35485]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-12T03:15:26.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:25 smithi161 ceph-mon[35485]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T03:15:26.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:25 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:15:26.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:25 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:15:26.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:25 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:15:26.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:25 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:15:26.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:25 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:26.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:25 smithi161 ceph-mon[35485]: from='osd.4 ' entity='osd.4' 2024-09-12T03:15:26.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:25 smithi118 ceph-mon[35048]: Detected new or changed devices on smithi118 2024-09-12T03:15:26.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:25 smithi118 ceph-mon[35048]: Adjusting osd_memory_target on smithi118 to 8544M 2024-09-12T03:15:26.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:25 smithi118 ceph-mon[35048]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-12T03:15:26.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:25 smithi118 ceph-mon[35048]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T03:15:26.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:25 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:15:26.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:25 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:15:26.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:25 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:15:26.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:25 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:15:26.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:25 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:26.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:25 smithi118 ceph-mon[35048]: from='osd.4 ' entity='osd.4' 2024-09-12T03:15:26.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:25 smithi006 ceph-mon[30991]: Detected new or changed devices on smithi118 2024-09-12T03:15:26.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:25 smithi006 ceph-mon[30991]: Adjusting osd_memory_target on smithi118 to 8544M 2024-09-12T03:15:26.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:25 smithi006 ceph-mon[30991]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-12T03:15:26.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:25 smithi006 ceph-mon[30991]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T03:15:26.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:25 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:15:26.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:25 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:15:26.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:25 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:15:26.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:25 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:15:26.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:25 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:26.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:25 smithi006 ceph-mon[30991]: from='osd.4 ' entity='osd.4' 2024-09-12T03:15:27.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:26 smithi161 ceph-mon[35485]: purged_snaps scrub starts 2024-09-12T03:15:27.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:26 smithi161 ceph-mon[35485]: purged_snaps scrub ok 2024-09-12T03:15:27.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:26 smithi161 ceph-mon[35485]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:15:27.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:26 smithi161 ceph-mon[35485]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:15:27.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:26 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:15:27.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:26 smithi161 ceph-mon[35485]: osd.4 [v2:172.21.15.118:6808/150177024,v1:172.21.15.118:6809/150177024] boot 2024-09-12T03:15:27.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:26 smithi161 ceph-mon[35485]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T03:15:27.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:26 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:15:27.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:26 smithi118 ceph-mon[35048]: purged_snaps scrub starts 2024-09-12T03:15:27.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:26 smithi118 ceph-mon[35048]: purged_snaps scrub ok 2024-09-12T03:15:27.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:26 smithi118 ceph-mon[35048]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:15:27.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:26 smithi118 ceph-mon[35048]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:15:27.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:26 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:15:27.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:26 smithi118 ceph-mon[35048]: osd.4 [v2:172.21.15.118:6808/150177024,v1:172.21.15.118:6809/150177024] boot 2024-09-12T03:15:27.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:26 smithi118 ceph-mon[35048]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T03:15:27.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:26 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:15:27.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:26 smithi006 ceph-mon[30991]: purged_snaps scrub starts 2024-09-12T03:15:27.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:26 smithi006 ceph-mon[30991]: purged_snaps scrub ok 2024-09-12T03:15:27.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:26 smithi006 ceph-mon[30991]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:15:27.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:26 smithi006 ceph-mon[30991]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:15:27.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:26 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:15:27.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:26 smithi006 ceph-mon[30991]: osd.4 [v2:172.21.15.118:6808/150177024,v1:172.21.15.118:6809/150177024] boot 2024-09-12T03:15:27.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:26 smithi006 ceph-mon[30991]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T03:15:27.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:26 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:15:28.043 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:27 smithi161 ceph-mon[35485]: from='client.? 172.21.15.161:0/478524498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d40c0ef-99f6-426f-90c7-27579edc66af"}]: dispatch 2024-09-12T03:15:28.043 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:27 smithi161 ceph-mon[35485]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d40c0ef-99f6-426f-90c7-27579edc66af"}]: dispatch 2024-09-12T03:15:28.043 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:27 smithi161 ceph-mon[35485]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d40c0ef-99f6-426f-90c7-27579edc66af"}]': finished 2024-09-12T03:15:28.043 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:27 smithi161 ceph-mon[35485]: osdmap e33: 6 total, 5 up, 6 in 2024-09-12T03:15:28.043 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:27 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:15:28.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:27 smithi118 ceph-mon[35048]: from='client.? 172.21.15.161:0/478524498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d40c0ef-99f6-426f-90c7-27579edc66af"}]: dispatch 2024-09-12T03:15:28.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:27 smithi118 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d40c0ef-99f6-426f-90c7-27579edc66af"}]: dispatch 2024-09-12T03:15:28.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:27 smithi118 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d40c0ef-99f6-426f-90c7-27579edc66af"}]': finished 2024-09-12T03:15:28.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:27 smithi118 ceph-mon[35048]: osdmap e33: 6 total, 5 up, 6 in 2024-09-12T03:15:28.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:27 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:15:28.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:27 smithi006 ceph-mon[30991]: from='client.? 172.21.15.161:0/478524498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d40c0ef-99f6-426f-90c7-27579edc66af"}]: dispatch 2024-09-12T03:15:28.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:27 smithi006 ceph-mon[30991]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d40c0ef-99f6-426f-90c7-27579edc66af"}]: dispatch 2024-09-12T03:15:28.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:27 smithi006 ceph-mon[30991]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d40c0ef-99f6-426f-90c7-27579edc66af"}]': finished 2024-09-12T03:15:28.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:27 smithi006 ceph-mon[30991]: osdmap e33: 6 total, 5 up, 6 in 2024-09-12T03:15:28.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:27 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:15:29.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:28 smithi118 ceph-mon[35048]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:29.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:28 smithi118 ceph-mon[35048]: from='client.? 172.21.15.161:0/1743482451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:15:29.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:28 smithi006 ceph-mon[30991]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:29.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:28 smithi006 ceph-mon[30991]: from='client.? 172.21.15.161:0/1743482451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:15:29.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:28 smithi161 ceph-mon[35485]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:29.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:28 smithi161 ceph-mon[35485]: from='client.? 172.21.15.161:0/1743482451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:15:31.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:30 smithi118 ceph-mon[35048]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:31.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:30 smithi006 ceph-mon[30991]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:31.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:30 smithi161 ceph-mon[35485]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:33.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:32 smithi161 ceph-mon[35485]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:33.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:32 smithi118 ceph-mon[35048]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:33.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:32 smithi006 ceph-mon[30991]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:35.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:34 smithi118 ceph-mon[35048]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:35.240 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:34 smithi161 ceph-mon[35485]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:35.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:34 smithi006 ceph-mon[30991]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:37.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:36 smithi161 ceph-mon[35485]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:37.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:36 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T03:15:37.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:36 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:37.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:36 smithi161 ceph-mon[35485]: Deploying daemon osd.5 on smithi161 2024-09-12T03:15:37.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:36 smithi006 ceph-mon[30991]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:37.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:36 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T03:15:37.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:36 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:37.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:36 smithi006 ceph-mon[30991]: Deploying daemon osd.5 on smithi161 2024-09-12T03:15:37.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:36 smithi118 ceph-mon[35048]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:37.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:36 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T03:15:37.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:36 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:37.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:36 smithi118 ceph-mon[35048]: Deploying daemon osd.5 on smithi161 2024-09-12T03:15:39.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:38 smithi006 ceph-mon[30991]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:39.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:38 smithi161 ceph-mon[35485]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:39.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:38 smithi118 ceph-mon[35048]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:41.011 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:40 smithi161 ceph-mon[35485]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:41.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:40 smithi006 ceph-mon[30991]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:41.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:40 smithi118 ceph-mon[35048]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:43.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:42 smithi118 ceph-mon[35048]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:43.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:42 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:43.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:42 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:43.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:42 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:43.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:42 smithi006 ceph-mon[30991]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:43.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:43.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:43.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:43.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:42 smithi161 ceph-mon[35485]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:43.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:43.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:43.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:45.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:44 smithi006 ceph-mon[30991]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:45.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:44 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:45.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:44 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:45.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:44 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:45.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:44 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:45.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:44 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:45.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:44 smithi161 ceph-mon[35485]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:45.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:44 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:45.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:44 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:45.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:44 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:45.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:44 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:45.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:44 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:45.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:44 smithi118 ceph-mon[35048]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:45.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:44 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:45.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:44 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:45.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:44 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:45.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:44 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:45.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:44 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:46.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:46.145+0000 7fe87b7fe640 1 -- 172.21.15.161:0/4060552625 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe88c062630 con 0x7fe8740a2c30 2024-09-12T03:15:46.146 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 5 on host 'smithi161' 2024-09-12T03:15:46.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:46.148+0000 7fe890b27640 1 -- 172.21.15.161:0/4060552625 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe8740a2c30 msgr2=0x7fe8740a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:46.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:46.148+0000 7fe890b27640 1 --2- 172.21.15.161:0/4060552625 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe8740a2c30 0x7fe8740a50f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe88c1993f0 tx=0x7fe880005ec0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:46.148+0000 7fe890b27640 1 -- 172.21.15.161:0/4060552625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe88c1026e0 msgr2=0x7fe88c197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:46.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:46.148+0000 7fe890b27640 1 --2- 172.21.15.161:0/4060552625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe88c1026e0 0x7fe88c197ed0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00e9f0 tx=0x7fe87c00edf0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:46.148+0000 7fe890b27640 1 -- 172.21.15.161:0/4060552625 shutdown_connections 2024-09-12T03:15:46.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:46.148+0000 7fe890b27640 1 --2- 172.21.15.161:0/4060552625 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe8740a2c30 0x7fe8740a50f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:46.148+0000 7fe890b27640 1 --2- 172.21.15.161:0/4060552625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe88c107940 0x7fe88c19f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:46.148+0000 7fe890b27640 1 --2- 172.21.15.161:0/4060552625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe88c105010 0x7fe88c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:46.148+0000 7fe890b27640 1 --2- 172.21.15.161:0/4060552625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe88c1026e0 0x7fe88c197ed0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:46.148+0000 7fe890b27640 1 -- 172.21.15.161:0/4060552625 >> 172.21.15.161:0/4060552625 conn(0x7fe88c0fc280 msgr2=0x7fe88c1030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:46.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:46.148+0000 7fe890b27640 1 -- 172.21.15.161:0/4060552625 shutdown_connections 2024-09-12T03:15:46.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:15:46.148+0000 7fe890b27640 1 -- 172.21.15.161:0/4060552625 wait complete. 2024-09-12T03:15:46.756 DEBUG:teuthology.orchestra.run.smithi161:osd.5> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.5.service 2024-09-12T03:15:46.758 INFO:tasks.cephadm:Deploying osd.6 on smithi006 with /dev/vg_nvme/lv_2... 2024-09-12T03:15:46.759 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T03:15:46.984 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:46 smithi161 ceph-mon[35485]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:46.985 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:46 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:46.985 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:46 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:46.985 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:46 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:47.024 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:15:47.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:46 smithi006 ceph-mon[30991]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:47.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:46 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:47.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:46 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:47.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:46 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:47.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:46 smithi118 ceph-mon[35048]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:47.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:46 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:47.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:46 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:47.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:46 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:48.863 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:15:48 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:15:48.519+0000 7fee0d91e640 -1 osd.5 0 log_to_monitors true 2024-09-12T03:15:49.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:48 smithi006 ceph-mon[30991]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:49.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:48 smithi006 ceph-mon[30991]: from='osd.5 [v2:172.21.15.161:6808/3758167854,v1:172.21.15.161:6809/3758167854]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:15:49.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:48 smithi006 ceph-mon[30991]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:15:49.201 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:48 smithi161 ceph-mon[35485]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:49.201 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:48 smithi161 ceph-mon[35485]: from='osd.5 [v2:172.21.15.161:6808/3758167854,v1:172.21.15.161:6809/3758167854]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:15:49.201 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:48 smithi161 ceph-mon[35485]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:15:49.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:48 smithi118 ceph-mon[35048]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:49.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:48 smithi118 ceph-mon[35048]: from='osd.5 [v2:172.21.15.161:6808/3758167854,v1:172.21.15.161:6809/3758167854]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:15:49.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:48 smithi118 ceph-mon[35048]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:15:49.646 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:15:49.674 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi006:vg_nvme/lv_2 2024-09-12T03:15:49.935 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:15:50.314 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:49 smithi006 ceph-mon[30991]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T03:15:50.314 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:49 smithi006 ceph-mon[30991]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T03:15:50.314 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:49 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:15:50.314 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:49 smithi006 ceph-mon[30991]: from='osd.5 [v2:172.21.15.161:6808/3758167854,v1:172.21.15.161:6809/3758167854]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:15:50.314 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:49 smithi006 ceph-mon[30991]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:15:50.315 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:49 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:50.315 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:49 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:50.315 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:49 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:50.315 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:49 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:50.315 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:49 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:50.315 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:49 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:50.315 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:49 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:50.315 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:49 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:50.363 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:15:49 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:15:49.994+0000 7fee0907e640 -1 osd.5 0 waiting for initial osdmap 2024-09-12T03:15:50.363 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:15:50 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:15:50.002+0000 7fee056b8640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:15:50.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:49 smithi161 ceph-mon[35485]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T03:15:50.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:49 smithi161 ceph-mon[35485]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T03:15:50.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:49 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:15:50.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:49 smithi161 ceph-mon[35485]: from='osd.5 [v2:172.21.15.161:6808/3758167854,v1:172.21.15.161:6809/3758167854]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:15:50.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:49 smithi161 ceph-mon[35485]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:15:50.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:49 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:50.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:49 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:50.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:49 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:50.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:49 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:50.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:49 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:50.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:49 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:50.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:49 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:50.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:49 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:50.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.401+0000 7fd6fda9c640 1 -- 172.21.15.6:0/2459281781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6f8102690 msgr2=0x7fd6f8108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:50.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.401+0000 7fd6fda9c640 1 --2- 172.21.15.6:0/2459281781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6f8102690 0x7fd6f8108f20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0009ca0 tx=0x7fd6f002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:15:50.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.401+0000 7fd6fda9c640 1 -- 172.21.15.6:0/2459281781 shutdown_connections 2024-09-12T03:15:50.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.401+0000 7fd6fda9c640 1 --2- 172.21.15.6:0/2459281781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6f8102690 0x7fd6f8108f20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:50.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.401+0000 7fd6fda9c640 1 --2- 172.21.15.6:0/2459281781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6f8101cd0 0x7fd6f8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:50.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.401+0000 7fd6fda9c640 1 --2- 172.21.15.6:0/2459281781 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6f8100ad0 0x7fd6f8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:50.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.401+0000 7fd6fda9c640 1 -- 172.21.15.6:0/2459281781 >> 172.21.15.6:0/2459281781 conn(0x7fd6f80fc280 msgr2=0x7fd6f80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:50.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.401+0000 7fd6fda9c640 1 -- 172.21.15.6:0/2459281781 shutdown_connections 2024-09-12T03:15:50.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.401+0000 7fd6fda9c640 1 -- 172.21.15.6:0/2459281781 wait complete. 2024-09-12T03:15:50.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.402+0000 7fd6fda9c640 1 Processor -- start 2024-09-12T03:15:50.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.402+0000 7fd6fda9c640 1 -- start start 2024-09-12T03:15:50.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.402+0000 7fd6fda9c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6f8100ad0 0x7fd6f8197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:50.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.402+0000 7fd6fda9c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6f8101cd0 0x7fd6f8198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:50.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.402+0000 7fd6fda9c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6f8102690 0x7fd6f819f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:50.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.402+0000 7fd6fda9c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f810d880 con 0x7fd6f8100ad0 2024-09-12T03:15:50.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.402+0000 7fd6fda9c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f810d700 con 0x7fd6f8101cd0 2024-09-12T03:15:50.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.402+0000 7fd6fda9c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f810da00 con 0x7fd6f8102690 2024-09-12T03:15:50.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.402+0000 7fd6f77fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6f8100ad0 0x7fd6f8197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:50.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.402+0000 7fd6f77fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6f8100ad0 0x7fd6f8197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39586/0 (socket says 172.21.15.6:39586) 2024-09-12T03:15:50.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.402+0000 7fd6f77fe640 1 -- 172.21.15.6:0/2505684590 learned_addr learned my addr 172.21.15.6:0/2505684590 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:15:50.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.403+0000 7fd6eeffd640 1 --2- 172.21.15.6:0/2505684590 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6f8101cd0 0x7fd6f8198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:50.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.403+0000 7fd6f7fff640 1 --2- 172.21.15.6:0/2505684590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6f8102690 0x7fd6f819f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:50.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.403+0000 7fd6f77fe640 1 -- 172.21.15.6:0/2505684590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6f8102690 msgr2=0x7fd6f819f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:50.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.403+0000 7fd6f77fe640 1 --2- 172.21.15.6:0/2505684590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6f8102690 0x7fd6f819f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:50.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.403+0000 7fd6f77fe640 1 -- 172.21.15.6:0/2505684590 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6f8101cd0 msgr2=0x7fd6f8198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:50.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.403+0000 7fd6f77fe640 1 --2- 172.21.15.6:0/2505684590 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6f8101cd0 0x7fd6f8198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:50.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.403+0000 7fd6f77fe640 1 -- 172.21.15.6:0/2505684590 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f819fa10 con 0x7fd6f8100ad0 2024-09-12T03:15:50.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.403+0000 7fd6f77fe640 1 --2- 172.21.15.6:0/2505684590 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6f8100ad0 0x7fd6f8197de0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800d770 tx=0x7fd6e800db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:50.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.403+0000 7fd6f57fa640 1 -- 172.21.15.6:0/2505684590 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8017070 con 0x7fd6f8100ad0 2024-09-12T03:15:50.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.403+0000 7fd6f57fa640 1 -- 172.21.15.6:0/2505684590 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6e8009d00 con 0x7fd6f8100ad0 2024-09-12T03:15:50.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.403+0000 7fd6fda9c640 1 -- 172.21.15.6:0/2505684590 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f819fd00 con 0x7fd6f8100ad0 2024-09-12T03:15:50.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.403+0000 7fd6f57fa640 1 -- 172.21.15.6:0/2505684590 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8003d30 con 0x7fd6f8100ad0 2024-09-12T03:15:50.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.403+0000 7fd6fda9c640 1 -- 172.21.15.6:0/2505684590 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f81a01d0 con 0x7fd6f8100ad0 2024-09-12T03:15:50.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.405+0000 7fd6fda9c640 1 -- 172.21.15.6:0/2505684590 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f8107e00 con 0x7fd6f8100ad0 2024-09-12T03:15:50.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.406+0000 7fd6f57fa640 1 -- 172.21.15.6:0/2505684590 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd6e8009820 con 0x7fd6f8100ad0 2024-09-12T03:15:50.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.407+0000 7fd6f57fa640 1 --2- 172.21.15.6:0/2505684590 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd6e40a2c30 0x7fd6e40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:50.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.407+0000 7fd6eeffd640 1 --2- 172.21.15.6:0/2505684590 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd6e40a2c30 0x7fd6e40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:50.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.407+0000 7fd6f57fa640 1 -- 172.21.15.6:0/2505684590 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4566+0+0 (secure 0 0 0) 0x7fd6e80cd9e0 con 0x7fd6f8100ad0 2024-09-12T03:15:50.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.407+0000 7fd6eeffd640 1 --2- 172.21.15.6:0/2505684590 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd6e40a2c30 0x7fd6e40a50f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8199300 tx=0x7fd6e000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:50.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.411+0000 7fd6f57fa640 1 -- 172.21.15.6:0/2505684590 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd6e809b7c0 con 0x7fd6f8100ad0 2024-09-12T03:15:50.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:49 smithi118 ceph-mon[35048]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T03:15:50.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:49 smithi118 ceph-mon[35048]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T03:15:50.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:49 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:15:50.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:49 smithi118 ceph-mon[35048]: from='osd.5 [v2:172.21.15.161:6808/3758167854,v1:172.21.15.161:6809/3758167854]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:15:50.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:49 smithi118 ceph-mon[35048]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:15:50.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:49 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:50.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:49 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:50.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:49 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:50.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:49 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:50.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:49 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:50.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:49 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:50.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:49 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:50.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:49 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:15:50.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:15:50.599+0000 7fd6fda9c640 1 -- 172.21.15.6:0/2505684590 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd6f8062630 con 0x7fd6e40a2c30 2024-09-12T03:15:51.139 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:50 smithi006 ceph-mon[30991]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:51.139 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:50 smithi006 ceph-mon[30991]: Detected new or changed devices on smithi161 2024-09-12T03:15:51.139 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:50 smithi006 ceph-mon[30991]: Adjusting osd_memory_target on smithi161 to 8544M 2024-09-12T03:15:51.139 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:50 smithi006 ceph-mon[30991]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-12T03:15:51.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:50 smithi006 ceph-mon[30991]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T03:15:51.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:50 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:15:51.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:50 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:15:51.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:50 smithi006 ceph-mon[30991]: osd.5 [v2:172.21.15.161:6808/3758167854,v1:172.21.15.161:6809/3758167854] boot 2024-09-12T03:15:51.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:50 smithi006 ceph-mon[30991]: osdmap e36: 6 total, 6 up, 6 in 2024-09-12T03:15:51.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:50 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:15:51.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:50 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:15:51.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:50 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:15:51.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:50 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:51.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:50 smithi161 ceph-mon[35485]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:51.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:50 smithi161 ceph-mon[35485]: Detected new or changed devices on smithi161 2024-09-12T03:15:51.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:50 smithi161 ceph-mon[35485]: Adjusting osd_memory_target on smithi161 to 8544M 2024-09-12T03:15:51.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:50 smithi161 ceph-mon[35485]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-12T03:15:51.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:50 smithi161 ceph-mon[35485]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T03:15:51.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:50 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:15:51.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:50 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:15:51.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:50 smithi161 ceph-mon[35485]: osd.5 [v2:172.21.15.161:6808/3758167854,v1:172.21.15.161:6809/3758167854] boot 2024-09-12T03:15:51.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:50 smithi161 ceph-mon[35485]: osdmap e36: 6 total, 6 up, 6 in 2024-09-12T03:15:51.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:50 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:15:51.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:50 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:15:51.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:50 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:15:51.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:50 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:51.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:50 smithi118 ceph-mon[35048]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:15:51.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:50 smithi118 ceph-mon[35048]: Detected new or changed devices on smithi161 2024-09-12T03:15:51.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:50 smithi118 ceph-mon[35048]: Adjusting osd_memory_target on smithi161 to 8544M 2024-09-12T03:15:51.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:50 smithi118 ceph-mon[35048]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-12T03:15:51.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:50 smithi118 ceph-mon[35048]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T03:15:51.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:50 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:15:51.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:50 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:15:51.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:50 smithi118 ceph-mon[35048]: osd.5 [v2:172.21.15.161:6808/3758167854,v1:172.21.15.161:6809/3758167854] boot 2024-09-12T03:15:51.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:50 smithi118 ceph-mon[35048]: osdmap e36: 6 total, 6 up, 6 in 2024-09-12T03:15:51.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:50 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:15:51.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:50 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:15:51.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:50 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:15:51.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:50 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:52.340 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:52 smithi006 ceph-mon[30991]: purged_snaps scrub starts 2024-09-12T03:15:52.340 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:52 smithi006 ceph-mon[30991]: purged_snaps scrub ok 2024-09-12T03:15:52.340 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:52 smithi006 ceph-mon[30991]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:15:52.340 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:52 smithi006 ceph-mon[30991]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T03:15:52.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:52 smithi118 ceph-mon[35048]: purged_snaps scrub starts 2024-09-12T03:15:52.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:52 smithi118 ceph-mon[35048]: purged_snaps scrub ok 2024-09-12T03:15:52.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:52 smithi118 ceph-mon[35048]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:15:52.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:52 smithi118 ceph-mon[35048]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T03:15:52.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:52 smithi161 ceph-mon[35485]: purged_snaps scrub starts 2024-09-12T03:15:52.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:52 smithi161 ceph-mon[35485]: purged_snaps scrub ok 2024-09-12T03:15:52.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:52 smithi161 ceph-mon[35485]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:15:52.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:52 smithi161 ceph-mon[35485]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T03:15:53.426 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:53 smithi006 ceph-mon[30991]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:15:53.428 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:53 smithi006 ceph-mon[30991]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T03:15:53.429 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:53 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/214824203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d9f9510-b107-40c2-9a98-9012715944b3"}]: dispatch 2024-09-12T03:15:53.429 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:53 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/214824203' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d9f9510-b107-40c2-9a98-9012715944b3"}]': finished 2024-09-12T03:15:53.429 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:53 smithi006 ceph-mon[30991]: osdmap e39: 7 total, 6 up, 7 in 2024-09-12T03:15:53.429 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:53 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:15:53.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:53 smithi118 ceph-mon[35048]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:15:53.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:53 smithi118 ceph-mon[35048]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T03:15:53.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:53 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/214824203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d9f9510-b107-40c2-9a98-9012715944b3"}]: dispatch 2024-09-12T03:15:53.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:53 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/214824203' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d9f9510-b107-40c2-9a98-9012715944b3"}]': finished 2024-09-12T03:15:53.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:53 smithi118 ceph-mon[35048]: osdmap e39: 7 total, 6 up, 7 in 2024-09-12T03:15:53.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:53 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:15:53.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:53 smithi161 ceph-mon[35485]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:15:53.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:53 smithi161 ceph-mon[35485]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T03:15:53.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:53 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/214824203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d9f9510-b107-40c2-9a98-9012715944b3"}]: dispatch 2024-09-12T03:15:53.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:53 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/214824203' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d9f9510-b107-40c2-9a98-9012715944b3"}]': finished 2024-09-12T03:15:53.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:53 smithi161 ceph-mon[35485]: osdmap e39: 7 total, 6 up, 7 in 2024-09-12T03:15:53.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:53 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:15:54.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:54 smithi006 ceph-mon[30991]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:15:54.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:54 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1501515049' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:15:54.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:54 smithi006 ceph-mon[30991]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T03:15:54.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:54 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:15:54.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:54 smithi161 ceph-mon[35485]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:15:54.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:54 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1501515049' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:15:54.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:54 smithi161 ceph-mon[35485]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T03:15:54.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:54 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:15:54.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:54 smithi118 ceph-mon[35048]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:15:54.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:54 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1501515049' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:15:54.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:54 smithi118 ceph-mon[35048]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T03:15:54.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:54 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:15:56.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:56 smithi006 ceph-mon[30991]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:15:56.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:56 smithi161 ceph-mon[35485]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:15:56.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:56 smithi118 ceph-mon[35048]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:15:58.519 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:15:58 smithi006 ceph-mon[30991]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:15:58.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:15:58 smithi161 ceph-mon[35485]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:15:58.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:15:58 smithi118 ceph-mon[35048]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:16:00.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:00 smithi006 ceph-mon[30991]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-12T03:16:00.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:00 smithi161 ceph-mon[35485]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-12T03:16:00.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:00 smithi118 ceph-mon[35048]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-12T03:16:02.514 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:02 smithi118 ceph-mon[35048]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-12T03:16:02.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:02 smithi006 ceph-mon[30991]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-12T03:16:02.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:02 smithi161 ceph-mon[35485]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-12T03:16:03.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:03 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T03:16:03.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:03 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:03.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:03 smithi006 ceph-mon[30991]: Deploying daemon osd.6 on smithi006 2024-09-12T03:16:03.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:03 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T03:16:03.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:03 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:03.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:03 smithi161 ceph-mon[35485]: Deploying daemon osd.6 on smithi006 2024-09-12T03:16:03.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:03 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T03:16:03.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:03 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:03.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:03 smithi118 ceph-mon[35048]: Deploying daemon osd.6 on smithi006 2024-09-12T03:16:04.518 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:04 smithi006 ceph-mon[30991]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:16:04.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:04 smithi161 ceph-mon[35485]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:16:04.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:04 smithi118 ceph-mon[35048]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:16:06.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:06 smithi006 ceph-mon[30991]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-12T03:16:06.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:06 smithi161 ceph-mon[35485]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-12T03:16:06.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:06 smithi118 ceph-mon[35048]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-12T03:16:08.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:08 smithi006 ceph-mon[30991]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:16:08.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:08 smithi161 ceph-mon[35485]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:16:08.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:08 smithi118 ceph-mon[35048]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:16:09.515 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:09 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:09.515 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:09 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:09.515 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:09 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:09.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:09 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:09.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:09 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:09.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:09 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:09.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:09 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:09.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:09 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:09.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:09 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:10.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:10 smithi118 ceph-mon[35048]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:16:10.730 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:10 smithi006 ceph-mon[30991]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:16:10.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:10 smithi161 ceph-mon[35485]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:16:11.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:16:11.940+0000 7fd6f57fa640 1 -- 172.21.15.6:0/2505684590 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd6f8062630 con 0x7fd6e40a2c30 2024-09-12T03:16:11.943 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 6 on host 'smithi006' 2024-09-12T03:16:11.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:16:11.944+0000 7fd6fda9c640 1 -- 172.21.15.6:0/2505684590 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd6e40a2c30 msgr2=0x7fd6e40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:11.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:16:11.944+0000 7fd6fda9c640 1 --2- 172.21.15.6:0/2505684590 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd6e40a2c30 0x7fd6e40a50f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8199300 tx=0x7fd6e000a2d0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:16:11.944+0000 7fd6fda9c640 1 -- 172.21.15.6:0/2505684590 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6f8100ad0 msgr2=0x7fd6f8197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:11.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:16:11.944+0000 7fd6fda9c640 1 --2- 172.21.15.6:0/2505684590 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6f8100ad0 0x7fd6f8197de0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800d770 tx=0x7fd6e800db70 comp rx=0 tx=0).stop 2024-09-12T03:16:11.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:16:11.944+0000 7fd6fda9c640 1 -- 172.21.15.6:0/2505684590 shutdown_connections 2024-09-12T03:16:11.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:16:11.944+0000 7fd6fda9c640 1 --2- 172.21.15.6:0/2505684590 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd6e40a2c30 0x7fd6e40a50f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:16:11.944+0000 7fd6fda9c640 1 --2- 172.21.15.6:0/2505684590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6f8102690 0x7fd6f819f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:16:11.944+0000 7fd6fda9c640 1 --2- 172.21.15.6:0/2505684590 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6f8101cd0 0x7fd6f8198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:16:11.944+0000 7fd6fda9c640 1 --2- 172.21.15.6:0/2505684590 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6f8100ad0 0x7fd6f8197de0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:16:11.944+0000 7fd6fda9c640 1 -- 172.21.15.6:0/2505684590 >> 172.21.15.6:0/2505684590 conn(0x7fd6f80fc280 msgr2=0x7fd6f8104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:11.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:16:11.944+0000 7fd6fda9c640 1 -- 172.21.15.6:0/2505684590 shutdown_connections 2024-09-12T03:16:11.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:16:11.944+0000 7fd6fda9c640 1 -- 172.21.15.6:0/2505684590 wait complete. 2024-09-12T03:16:12.187 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:12 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:12.188 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:12 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:12.188 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:12 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:12.188 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:12 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:12.188 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:12 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:12.188 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:12 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:12.188 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:12 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:12.188 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:12 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:12.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:12 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:12.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:12 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:12.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:12 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:12.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:12 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:12.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:12 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:12.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:12 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:12.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:12 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:12.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:12 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:12.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:12 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:12.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:12 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:12.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:12 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:12.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:12 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:12.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:12 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:12.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:12 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:12.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:12 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:12.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:12 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:12.633 DEBUG:teuthology.orchestra.run.smithi006:osd.6> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.6.service 2024-09-12T03:16:12.636 INFO:tasks.cephadm:Deploying osd.7 on smithi118 with /dev/vg_nvme/lv_2... 2024-09-12T03:16:12.636 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T03:16:12.889 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:16:13.259 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:13 smithi118 ceph-mon[35048]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:16:13.335 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:13 smithi006 ceph-mon[30991]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:16:13.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:13 smithi161 ceph-mon[35485]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:16:14.951 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:16:14 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:16:14.734+0000 7ffb3f170640 -1 osd.6 0 log_to_monitors true 2024-09-12T03:16:15.302 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:15 smithi118 ceph-mon[35048]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:16:15.302 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:15 smithi118 ceph-mon[35048]: from='osd.6 [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T03:16:15.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:15 smithi006 ceph-mon[30991]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:16:15.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:15 smithi006 ceph-mon[30991]: from='osd.6 [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T03:16:15.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:15 smithi161 ceph-mon[35485]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:16:15.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:15 smithi161 ceph-mon[35485]: from='osd.6 [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T03:16:15.450 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-12T03:16:15.475 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi118:vg_nvme/lv_2 2024-09-12T03:16:15.732 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:16:16.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.159+0000 7fb8a0860640 1 -- 172.21.15.118:0/1375855326 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb89c102e00 msgr2=0x7fb89c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:16.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.159+0000 7fb8a0860640 1 --2- 172.21.15.118:0/1375855326 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb89c102e00 0x7fb89c103280 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb890009900 tx=0x7fb89002a410 comp rx=0 tx=0).stop 2024-09-12T03:16:16.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.160+0000 7fb8a0860640 1 -- 172.21.15.118:0/1375855326 shutdown_connections 2024-09-12T03:16:16.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.160+0000 7fb8a0860640 1 --2- 172.21.15.118:0/1375855326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb89c1037c0 0x7fb89c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:16.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.160+0000 7fb8a0860640 1 --2- 172.21.15.118:0/1375855326 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb89c102e00 0x7fb89c103280 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:16.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.160+0000 7fb8a0860640 1 --2- 172.21.15.118:0/1375855326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb89c101c00 0x7fb89c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:16.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.160+0000 7fb8a0860640 1 -- 172.21.15.118:0/1375855326 >> 172.21.15.118:0/1375855326 conn(0x7fb89c0fd3b0 msgr2=0x7fb89c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:16.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.160+0000 7fb8a0860640 1 -- 172.21.15.118:0/1375855326 shutdown_connections 2024-09-12T03:16:16.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.160+0000 7fb8a0860640 1 -- 172.21.15.118:0/1375855326 wait complete. 2024-09-12T03:16:16.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.160+0000 7fb8a0860640 1 Processor -- start 2024-09-12T03:16:16.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.160+0000 7fb8a0860640 1 -- start start 2024-09-12T03:16:16.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.160+0000 7fb8a0860640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb89c101c00 0x7fb89c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:16.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.160+0000 7fb8a0860640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb89c102e00 0x7fb89c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:16.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.160+0000 7fb8a0860640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb89c1037c0 0x7fb89c19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:16.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.161+0000 7fb8a0860640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c10d860 con 0x7fb89c102e00 2024-09-12T03:16:16.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.161+0000 7fb8a0860640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c10d6e0 con 0x7fb89c1037c0 2024-09-12T03:16:16.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.161+0000 7fb8a0860640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c10d9e0 con 0x7fb89c101c00 2024-09-12T03:16:16.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.161+0000 7fb89ad9d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb89c1037c0 0x7fb89c19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:16.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.161+0000 7fb89ad9d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb89c1037c0 0x7fb89c19f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:41426/0 (socket says 172.21.15.118:41426) 2024-09-12T03:16:16.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.161+0000 7fb89ad9d640 1 -- 172.21.15.118:0/826291226 learned_addr learned my addr 172.21.15.118:0/826291226 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-09-12T03:16:16.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.161+0000 7fb899d9b640 1 --2- 172.21.15.118:0/826291226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb89c102e00 0x7fb89c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:16.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.161+0000 7fb89a59c640 1 --2- 172.21.15.118:0/826291226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb89c101c00 0x7fb89c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:16.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.161+0000 7fb89ad9d640 1 -- 172.21.15.118:0/826291226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb89c101c00 msgr2=0x7fb89c197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:16.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.161+0000 7fb89ad9d640 1 --2- 172.21.15.118:0/826291226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb89c101c00 0x7fb89c197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:16.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.161+0000 7fb89ad9d640 1 -- 172.21.15.118:0/826291226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb89c102e00 msgr2=0x7fb89c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:16.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.161+0000 7fb89ad9d640 1 --2- 172.21.15.118:0/826291226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb89c102e00 0x7fb89c198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:16.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.161+0000 7fb89ad9d640 1 -- 172.21.15.118:0/826291226 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb89c19faf0 con 0x7fb89c1037c0 2024-09-12T03:16:16.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.161+0000 7fb89ad9d640 1 --2- 172.21.15.118:0/826291226 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb89c1037c0 0x7fb89c19f480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb89400cce0 tx=0x7fb89400a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:16.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.161+0000 7fb88b7fe640 1 -- 172.21.15.118:0/826291226 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89400f450 con 0x7fb89c1037c0 2024-09-12T03:16:16.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.161+0000 7fb8a0860640 1 -- 172.21.15.118:0/826291226 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb89c19fde0 con 0x7fb89c1037c0 2024-09-12T03:16:16.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.161+0000 7fb88b7fe640 1 -- 172.21.15.118:0/826291226 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb89400fad0 con 0x7fb89c1037c0 2024-09-12T03:16:16.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.161+0000 7fb88b7fe640 1 -- 172.21.15.118:0/826291226 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894018c10 con 0x7fb89c1037c0 2024-09-12T03:16:16.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.161+0000 7fb8a0860640 1 -- 172.21.15.118:0/826291226 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb89c1a0240 con 0x7fb89c1037c0 2024-09-12T03:16:16.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.162+0000 7fb8a0860640 1 -- 172.21.15.118:0/826291226 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb89c108f60 con 0x7fb89c1037c0 2024-09-12T03:16:16.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.165+0000 7fb88b7fe640 1 -- 172.21.15.118:0/826291226 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb89400f5f0 con 0x7fb89c1037c0 2024-09-12T03:16:16.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.165+0000 7fb88b7fe640 1 --2- 172.21.15.118:0/826291226 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb8840a2bb0 0x7fb8840a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:16.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.165+0000 7fb88b7fe640 1 -- 172.21.15.118:0/826291226 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4685+0+0 (secure 0 0 0) 0x7fb8940ce900 con 0x7fb89c1037c0 2024-09-12T03:16:16.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.165+0000 7fb89a59c640 1 --2- 172.21.15.118:0/826291226 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb8840a2bb0 0x7fb8840a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:16.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.166+0000 7fb89a59c640 1 --2- 172.21.15.118:0/826291226 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb8840a2bb0 0x7fb8840a5070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb89c102c60 tx=0x7fb88c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:16.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.169+0000 7fb88b7fe640 1 -- 172.21.15.118:0/826291226 <== mon.2 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb894014030 con 0x7fb89c1037c0 2024-09-12T03:16:16.346 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:16 smithi118 ceph-mon[35048]: from='osd.6 [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T03:16:16.347 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:16 smithi118 ceph-mon[35048]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T03:16:16.347 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:16 smithi118 ceph-mon[35048]: from='osd.6 [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-09-12T03:16:16.347 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:16 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:16:16.347 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:16 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:16.347 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:16 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:16.347 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:16 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:16.347 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:16 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:16.347 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:16 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:16.347 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:16 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:16.347 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:16 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:16.347 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:16 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:16.347 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:16 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:16.348 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:16 smithi118 ceph-mon[35048]: from='osd.6 [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-09-12T03:16:16.348 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:16 smithi118 ceph-mon[35048]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T03:16:16.348 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:16.346+0000 7fb8a0860640 1 -- 172.21.15.118:0/826291226 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb89c062630 con 0x7fb8840a2bb0 2024-09-12T03:16:16.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:16 smithi006 ceph-mon[30991]: from='osd.6 [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T03:16:16.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:16 smithi006 ceph-mon[30991]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T03:16:16.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:16 smithi006 ceph-mon[30991]: from='osd.6 [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-09-12T03:16:16.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:16 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:16:16.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:16 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:16.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:16 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:16.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:16 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:16.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:16 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:16.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:16 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:16.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:16 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:16.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:16 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:16.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:16 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:16.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:16 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:16.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:16 smithi006 ceph-mon[30991]: from='osd.6 [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-09-12T03:16:16.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:16 smithi006 ceph-mon[30991]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T03:16:16.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:16 smithi161 ceph-mon[35485]: from='osd.6 [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T03:16:16.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:16 smithi161 ceph-mon[35485]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T03:16:16.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:16 smithi161 ceph-mon[35485]: from='osd.6 [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-09-12T03:16:16.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:16 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:16:16.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:16 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:16.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:16 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:16.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:16 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:16.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:16 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:16.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:16 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:16.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:16 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:16.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:16 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:16.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:16 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:16.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:16 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:16.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:16 smithi161 ceph-mon[35485]: from='osd.6 [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-09-12T03:16:16.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:16 smithi161 ceph-mon[35485]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T03:16:17.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:16:16 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:16:16.873+0000 7ffb3a8d0640 -1 osd.6 0 waiting for initial osdmap 2024-09-12T03:16:17.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:16:16 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:16:16.886+0000 7ffb32f02640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:16:17.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:17 smithi006 ceph-mon[30991]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:16:17.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:17 smithi006 ceph-mon[30991]: Detected new or changed devices on smithi006 2024-09-12T03:16:17.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:17 smithi006 ceph-mon[30991]: Adjusting osd_memory_target on smithi006 to 5697M 2024-09-12T03:16:17.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:16:17.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:16:17.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:16:17.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:16:17.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:17.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:17 smithi006 ceph-mon[30991]: from='osd.6 [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874]' entity='osd.6' 2024-09-12T03:16:17.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:16:17.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:17 smithi161 ceph-mon[35485]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:16:17.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:17 smithi161 ceph-mon[35485]: Detected new or changed devices on smithi006 2024-09-12T03:16:17.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:17 smithi161 ceph-mon[35485]: Adjusting osd_memory_target on smithi006 to 5697M 2024-09-12T03:16:17.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:16:17.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:16:17.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:16:17.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:16:17.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:17.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:17 smithi161 ceph-mon[35485]: from='osd.6 [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874]' entity='osd.6' 2024-09-12T03:16:17.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:16:17.465 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:17 smithi118 ceph-mon[35048]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:16:17.465 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:17 smithi118 ceph-mon[35048]: Detected new or changed devices on smithi006 2024-09-12T03:16:17.465 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:17 smithi118 ceph-mon[35048]: Adjusting osd_memory_target on smithi006 to 5697M 2024-09-12T03:16:17.465 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:16:17.465 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:16:17.465 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:16:17.465 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:16:17.465 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:17.465 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:17 smithi118 ceph-mon[35048]: from='osd.6 [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874]' entity='osd.6' 2024-09-12T03:16:17.465 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:16:18.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:18 smithi006 ceph-mon[30991]: purged_snaps scrub starts 2024-09-12T03:16:18.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:18 smithi006 ceph-mon[30991]: purged_snaps scrub ok 2024-09-12T03:16:18.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:18 smithi006 ceph-mon[30991]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:16:18.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:18 smithi006 ceph-mon[30991]: osd.6 [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874] boot 2024-09-12T03:16:18.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:18 smithi006 ceph-mon[30991]: osdmap e43: 7 total, 7 up, 7 in 2024-09-12T03:16:18.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:18 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:16:18.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:18 smithi161 ceph-mon[35485]: purged_snaps scrub starts 2024-09-12T03:16:18.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:18 smithi161 ceph-mon[35485]: purged_snaps scrub ok 2024-09-12T03:16:18.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:18 smithi161 ceph-mon[35485]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:16:18.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:18 smithi161 ceph-mon[35485]: osd.6 [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874] boot 2024-09-12T03:16:18.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:18 smithi161 ceph-mon[35485]: osdmap e43: 7 total, 7 up, 7 in 2024-09-12T03:16:18.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:18 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:16:18.383 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:18 smithi118 ceph-mon[35048]: purged_snaps scrub starts 2024-09-12T03:16:18.383 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:18 smithi118 ceph-mon[35048]: purged_snaps scrub ok 2024-09-12T03:16:18.383 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:18 smithi118 ceph-mon[35048]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:16:18.383 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:18 smithi118 ceph-mon[35048]: osd.6 [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874] boot 2024-09-12T03:16:18.383 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:18 smithi118 ceph-mon[35048]: osdmap e43: 7 total, 7 up, 7 in 2024-09-12T03:16:18.383 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:18 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:16:19.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:19 smithi118 ceph-mon[35048]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:16:19.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:19 smithi118 ceph-mon[35048]: from='client.? 172.21.15.118:0/4176976276' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84635ae1-3240-4954-af14-76bb3aa162e8"}]: dispatch 2024-09-12T03:16:19.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:19 smithi118 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84635ae1-3240-4954-af14-76bb3aa162e8"}]: dispatch 2024-09-12T03:16:19.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:19 smithi118 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84635ae1-3240-4954-af14-76bb3aa162e8"}]': finished 2024-09-12T03:16:19.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:19 smithi118 ceph-mon[35048]: osdmap e44: 8 total, 7 up, 8 in 2024-09-12T03:16:19.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:16:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:19 smithi006 ceph-mon[30991]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:16:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:19 smithi006 ceph-mon[30991]: from='client.? 172.21.15.118:0/4176976276' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84635ae1-3240-4954-af14-76bb3aa162e8"}]: dispatch 2024-09-12T03:16:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:19 smithi006 ceph-mon[30991]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84635ae1-3240-4954-af14-76bb3aa162e8"}]: dispatch 2024-09-12T03:16:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:19 smithi006 ceph-mon[30991]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84635ae1-3240-4954-af14-76bb3aa162e8"}]': finished 2024-09-12T03:16:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:19 smithi006 ceph-mon[30991]: osdmap e44: 8 total, 7 up, 8 in 2024-09-12T03:16:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:16:19.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:19 smithi161 ceph-mon[35485]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:16:19.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:19 smithi161 ceph-mon[35485]: from='client.? 172.21.15.118:0/4176976276' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84635ae1-3240-4954-af14-76bb3aa162e8"}]: dispatch 2024-09-12T03:16:19.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:19 smithi161 ceph-mon[35485]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84635ae1-3240-4954-af14-76bb3aa162e8"}]: dispatch 2024-09-12T03:16:19.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:19 smithi161 ceph-mon[35485]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84635ae1-3240-4954-af14-76bb3aa162e8"}]': finished 2024-09-12T03:16:19.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:19 smithi161 ceph-mon[35485]: osdmap e44: 8 total, 7 up, 8 in 2024-09-12T03:16:19.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:16:20.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:20 smithi006 ceph-mon[30991]: from='client.? 172.21.15.118:0/1843204288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:16:20.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:20 smithi006 ceph-mon[30991]: osdmap e45: 8 total, 7 up, 8 in 2024-09-12T03:16:20.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:16:20.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:20 smithi161 ceph-mon[35485]: from='client.? 172.21.15.118:0/1843204288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:16:20.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:20 smithi161 ceph-mon[35485]: osdmap e45: 8 total, 7 up, 8 in 2024-09-12T03:16:20.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:16:20.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:20 smithi118 ceph-mon[35048]: from='client.? 172.21.15.118:0/1843204288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:16:20.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:20 smithi118 ceph-mon[35048]: osdmap e45: 8 total, 7 up, 8 in 2024-09-12T03:16:20.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:16:21.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:21 smithi006 ceph-mon[30991]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:21.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:21 smithi161 ceph-mon[35485]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:21.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:21 smithi118 ceph-mon[35048]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:23.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:23 smithi006 ceph-mon[30991]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:23.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:23 smithi161 ceph-mon[35485]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:23.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:23 smithi118 ceph-mon[35048]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:25.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:25 smithi006 ceph-mon[30991]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:25.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:25 smithi161 ceph-mon[35485]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:25.384 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:25 smithi118 ceph-mon[35048]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:27.360 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:27 smithi118 ceph-mon[35048]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:27.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:27 smithi006 ceph-mon[30991]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:27.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:27 smithi161 ceph-mon[35485]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:28.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:28 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T03:16:28.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:28 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:28.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:28 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T03:16:28.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:28 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:28.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:28 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T03:16:28.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:28 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:29.309 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:29 smithi118 ceph-mon[35048]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:29.309 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:29 smithi118 ceph-mon[35048]: Deploying daemon osd.7 on smithi118 2024-09-12T03:16:29.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:29 smithi006 ceph-mon[30991]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:29.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:29 smithi006 ceph-mon[30991]: Deploying daemon osd.7 on smithi118 2024-09-12T03:16:29.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:29 smithi161 ceph-mon[35485]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:29.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:29 smithi161 ceph-mon[35485]: Deploying daemon osd.7 on smithi118 2024-09-12T03:16:31.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:31 smithi118 ceph-mon[35048]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:31.451 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:31 smithi006 ceph-mon[30991]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:31.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:31 smithi161 ceph-mon[35485]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:33.415 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:33 smithi161 ceph-mon[35485]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:33.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:33 smithi118 ceph-mon[35048]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:33.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:33 smithi006 ceph-mon[30991]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:34.315 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:34 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:34.315 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:34 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:34.315 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:34 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:34.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:34 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:34.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:34 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:34.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:34 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:34.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:34 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:34.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:34 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:34.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:34 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:35.360 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:35 smithi118 ceph-mon[35048]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:35.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:35 smithi006 ceph-mon[30991]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:35.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:35 smithi161 ceph-mon[35485]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:36.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:36 smithi118 ceph-mon[35048]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:36.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:36 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:36.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:36 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:36.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:36 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:36.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:36 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:36.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:36 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:37.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:37.029+0000 7fb88b7fe640 1 -- 172.21.15.118:0/826291226 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb89c062630 con 0x7fb8840a2bb0 2024-09-12T03:16:37.030 INFO:teuthology.orchestra.run.smithi118.stdout:Created osd(s) 7 on host 'smithi118' 2024-09-12T03:16:37.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:37.031+0000 7fb8a0860640 1 -- 172.21.15.118:0/826291226 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb8840a2bb0 msgr2=0x7fb8840a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:37.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:37.031+0000 7fb8a0860640 1 --2- 172.21.15.118:0/826291226 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb8840a2bb0 0x7fb8840a5070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb89c102c60 tx=0x7fb88c005f90 comp rx=0 tx=0).stop 2024-09-12T03:16:37.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:37.031+0000 7fb8a0860640 1 -- 172.21.15.118:0/826291226 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb89c1037c0 msgr2=0x7fb89c19f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:37.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:37.031+0000 7fb8a0860640 1 --2- 172.21.15.118:0/826291226 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb89c1037c0 0x7fb89c19f480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb89400cce0 tx=0x7fb89400a620 comp rx=0 tx=0).stop 2024-09-12T03:16:37.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:37.031+0000 7fb8a0860640 1 -- 172.21.15.118:0/826291226 shutdown_connections 2024-09-12T03:16:37.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:37.031+0000 7fb8a0860640 1 --2- 172.21.15.118:0/826291226 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb8840a2bb0 0x7fb8840a5070 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:37.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:37.031+0000 7fb8a0860640 1 --2- 172.21.15.118:0/826291226 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb89c1037c0 0x7fb89c19f480 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:37.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:37.031+0000 7fb8a0860640 1 --2- 172.21.15.118:0/826291226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb89c102e00 0x7fb89c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:37.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:37.031+0000 7fb8a0860640 1 --2- 172.21.15.118:0/826291226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb89c101c00 0x7fb89c197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:37.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:37.031+0000 7fb8a0860640 1 -- 172.21.15.118:0/826291226 >> 172.21.15.118:0/826291226 conn(0x7fb89c0fd3b0 msgr2=0x7fb89c106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:37.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:37.031+0000 7fb8a0860640 1 -- 172.21.15.118:0/826291226 shutdown_connections 2024-09-12T03:16:37.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:16:37.031+0000 7fb8a0860640 1 -- 172.21.15.118:0/826291226 wait complete. 2024-09-12T03:16:37.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:36 smithi006 ceph-mon[30991]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:37.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:36 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:37.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:36 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:37.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:36 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:37.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:36 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:37.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:36 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:37.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:36 smithi161 ceph-mon[35485]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:37.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:36 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:37.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:36 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:37.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:36 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:37.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:36 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:37.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:36 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:37.672 DEBUG:teuthology.orchestra.run.smithi118:osd.7> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.7.service 2024-09-12T03:16:37.674 INFO:tasks.cephadm:Deploying osd.8 on smithi161 with /dev/vg_nvme/lv_2... 2024-09-12T03:16:37.675 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T03:16:37.932 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:16:37.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:37 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:37.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:37 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:37.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:37 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:38.053 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:37 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:38.053 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:37 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:38.053 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:37 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:38.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:37 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:38.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:37 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:38.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:37 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:39.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:39 smithi006 ceph-mon[30991]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:39.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:39 smithi161 ceph-mon[35485]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:39.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:39 smithi118 ceph-mon[35048]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:39.965 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:16:39 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:16:39.567+0000 7f24219ed640 -1 osd.7 0 log_to_monitors true 2024-09-12T03:16:40.335 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:40 smithi118 ceph-mon[35048]: from='osd.7 [v2:172.21.15.118:6816/269980353,v1:172.21.15.118:6817/269980353]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:16:40.335 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:40 smithi118 ceph-mon[35048]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:16:40.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:40 smithi006 ceph-mon[30991]: from='osd.7 [v2:172.21.15.118:6816/269980353,v1:172.21.15.118:6817/269980353]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:16:40.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:40 smithi006 ceph-mon[30991]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:16:40.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:40 smithi161 ceph-mon[35485]: from='osd.7 [v2:172.21.15.118:6816/269980353,v1:172.21.15.118:6817/269980353]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:16:40.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:40 smithi161 ceph-mon[35485]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:16:40.450 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:16:40.475 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_2 2024-09-12T03:16:40.728 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:16:41.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.211+0000 7f9addd56640 1 -- 172.21.15.161:0/2372485865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ad8104ff0 msgr2=0x7f9ad81073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:41.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.211+0000 7f9addd56640 1 --2- 172.21.15.161:0/2372485865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ad8104ff0 0x7f9ad81073e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009900 tx=0x7f9acc02a410 comp rx=0 tx=0).stop 2024-09-12T03:16:41.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.211+0000 7f9addd56640 1 -- 172.21.15.161:0/2372485865 shutdown_connections 2024-09-12T03:16:41.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.211+0000 7f9addd56640 1 --2- 172.21.15.161:0/2372485865 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ad8107920 0x7f9ad8109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.211+0000 7f9addd56640 1 --2- 172.21.15.161:0/2372485865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ad8104ff0 0x7f9ad81073e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.211+0000 7f9addd56640 1 --2- 172.21.15.161:0/2372485865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ad81026c0 0x7f9ad8104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.211+0000 7f9addd56640 1 -- 172.21.15.161:0/2372485865 >> 172.21.15.161:0/2372485865 conn(0x7f9ad80fc280 msgr2=0x7f9ad80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:41.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.211+0000 7f9addd56640 1 -- 172.21.15.161:0/2372485865 shutdown_connections 2024-09-12T03:16:41.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.211+0000 7f9addd56640 1 -- 172.21.15.161:0/2372485865 wait complete. 2024-09-12T03:16:41.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.212+0000 7f9addd56640 1 Processor -- start 2024-09-12T03:16:41.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.212+0000 7f9addd56640 1 -- start start 2024-09-12T03:16:41.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.212+0000 7f9addd56640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ad81026c0 0x7f9ad8197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:41.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.212+0000 7f9addd56640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ad8104ff0 0x7f9ad8198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:41.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.212+0000 7f9addd56640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ad8107920 0x7f9ad819f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:41.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.212+0000 7f9addd56640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad810d830 con 0x7f9ad8107920 2024-09-12T03:16:41.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.212+0000 7f9addd56640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad810d6b0 con 0x7f9ad8104ff0 2024-09-12T03:16:41.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.212+0000 7f9addd56640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad810d9b0 con 0x7f9ad81026c0 2024-09-12T03:16:41.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.212+0000 7f9ad77fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ad81026c0 0x7f9ad8197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:41.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.212+0000 7f9ad77fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ad81026c0 0x7f9ad8197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54082/0 (socket says 172.21.15.161:54082) 2024-09-12T03:16:41.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.212+0000 7f9ad77fe640 1 -- 172.21.15.161:0/3968576836 learned_addr learned my addr 172.21.15.161:0/3968576836 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-12T03:16:41.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.212+0000 7f9ad6ffd640 1 --2- 172.21.15.161:0/3968576836 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ad8104ff0 0x7f9ad8198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:41.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.213+0000 7f9ad7fff640 1 --2- 172.21.15.161:0/3968576836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ad8107920 0x7f9ad819f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:41.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.213+0000 7f9ad77fe640 1 -- 172.21.15.161:0/3968576836 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ad8104ff0 msgr2=0x7f9ad8198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:41.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.213+0000 7f9ad77fe640 1 --2- 172.21.15.161:0/3968576836 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ad8104ff0 0x7f9ad8198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.213+0000 7f9ad77fe640 1 -- 172.21.15.161:0/3968576836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ad8107920 msgr2=0x7f9ad819f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:41.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.213+0000 7f9ad77fe640 1 --2- 172.21.15.161:0/3968576836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ad8107920 0x7f9ad819f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.213+0000 7f9ad77fe640 1 -- 172.21.15.161:0/3968576836 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad819fb70 con 0x7f9ad81026c0 2024-09-12T03:16:41.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.213+0000 7f9ad77fe640 1 --2- 172.21.15.161:0/3968576836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ad81026c0 0x7f9ad8197f40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9ac800d950 tx=0x7f9ac800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:41.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.213+0000 7f9ad4ff9640 1 -- 172.21.15.161:0/3968576836 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8017070 con 0x7f9ad81026c0 2024-09-12T03:16:41.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.213+0000 7f9ad4ff9640 1 -- 172.21.15.161:0/3968576836 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ac8009c30 con 0x7f9ad81026c0 2024-09-12T03:16:41.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.213+0000 7f9addd56640 1 -- 172.21.15.161:0/3968576836 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad819fe60 con 0x7f9ad81026c0 2024-09-12T03:16:41.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.213+0000 7f9ad4ff9640 1 -- 172.21.15.161:0/3968576836 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac800f6b0 con 0x7f9ad81026c0 2024-09-12T03:16:41.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.213+0000 7f9addd56640 1 -- 172.21.15.161:0/3968576836 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad81a02b0 con 0x7f9ad81026c0 2024-09-12T03:16:41.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.214+0000 7f9addd56640 1 -- 172.21.15.161:0/3968576836 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad80fdd70 con 0x7f9ad81026c0 2024-09-12T03:16:41.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.216+0000 7f9ad4ff9640 1 -- 172.21.15.161:0/3968576836 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f9ac80109d0 con 0x7f9ad81026c0 2024-09-12T03:16:41.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.216+0000 7f9ad4ff9640 1 --2- 172.21.15.161:0/3968576836 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9ac00a2bb0 0x7f9ac00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:41.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.216+0000 7f9ad4ff9640 1 -- 172.21.15.161:0/3968576836 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5117+0+0 (secure 0 0 0) 0x7f9ac80cdf80 con 0x7f9ad81026c0 2024-09-12T03:16:41.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.216+0000 7f9ad6ffd640 1 --2- 172.21.15.161:0/3968576836 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9ac00a2bb0 0x7f9ac00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:41.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.217+0000 7f9ad6ffd640 1 --2- 172.21.15.161:0/3968576836 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9ac00a2bb0 0x7f9ac00a5070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9acc005ab0 tx=0x7f9acc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:41.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.220+0000 7f9ad4ff9640 1 -- 172.21.15.161:0/3968576836 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ac809bb60 con 0x7f9ad81026c0 2024-09-12T03:16:41.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:41 smithi006 ceph-mon[30991]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:41.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:41 smithi006 ceph-mon[30991]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T03:16:41.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:41 smithi006 ceph-mon[30991]: from='osd.7 [v2:172.21.15.118:6816/269980353,v1:172.21.15.118:6817/269980353]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:16:41.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:41 smithi006 ceph-mon[30991]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T03:16:41.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:16:41.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:41 smithi006 ceph-mon[30991]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:16:41.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:41.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:41.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:41.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:41.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:41.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:41.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:41.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:41.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:41 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:41.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:41 smithi161 ceph-mon[35485]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:41.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:41 smithi161 ceph-mon[35485]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T03:16:41.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:41 smithi161 ceph-mon[35485]: from='osd.7 [v2:172.21.15.118:6816/269980353,v1:172.21.15.118:6817/269980353]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:16:41.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:41 smithi161 ceph-mon[35485]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T03:16:41.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:41 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:16:41.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:41 smithi161 ceph-mon[35485]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:16:41.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:41 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:41.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:41 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:41.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:41 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:41.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:41 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:41.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:41 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:41.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:41 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:41.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:41 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:41.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:41 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:41.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:41 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:41.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:16:41.403+0000 7f9addd56640 1 -- 172.21.15.161:0/3968576836 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9ad8062630 con 0x7f9ac00a2bb0 2024-09-12T03:16:41.462 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:16:41 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:16:41.040+0000 7f241d14d640 -1 osd.7 0 waiting for initial osdmap 2024-09-12T03:16:41.462 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:16:41 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:16:41.050+0000 7f2419787640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:16:41.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:41 smithi118 ceph-mon[35048]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:41.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:41 smithi118 ceph-mon[35048]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T03:16:41.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:41 smithi118 ceph-mon[35048]: from='osd.7 [v2:172.21.15.118:6816/269980353,v1:172.21.15.118:6817/269980353]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:16:41.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:41 smithi118 ceph-mon[35048]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T03:16:41.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:41 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:16:41.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:41 smithi118 ceph-mon[35048]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:16:41.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:41 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:41.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:41 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:41.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:41 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:41.464 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:41 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:41.464 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:41 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:41.464 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:41 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:41.464 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:41 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:41.464 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:41 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:41.464 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:41 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:42.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:42 smithi006 ceph-mon[30991]: Detected new or changed devices on smithi118 2024-09-12T03:16:42.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:42 smithi006 ceph-mon[30991]: Adjusting osd_memory_target on smithi118 to 5696M 2024-09-12T03:16:42.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:42 smithi006 ceph-mon[30991]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-12T03:16:42.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:42 smithi006 ceph-mon[30991]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T03:16:42.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:16:42.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:16:42.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:16:42.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:16:42.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:42.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:16:42.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:42 smithi161 ceph-mon[35485]: Detected new or changed devices on smithi118 2024-09-12T03:16:42.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:42 smithi161 ceph-mon[35485]: Adjusting osd_memory_target on smithi118 to 5696M 2024-09-12T03:16:42.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:42 smithi161 ceph-mon[35485]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-12T03:16:42.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:42 smithi161 ceph-mon[35485]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T03:16:42.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:16:42.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:16:42.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:16:42.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:16:42.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:42.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:16:42.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:42 smithi118 ceph-mon[35048]: Detected new or changed devices on smithi118 2024-09-12T03:16:42.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:42 smithi118 ceph-mon[35048]: Adjusting osd_memory_target on smithi118 to 5696M 2024-09-12T03:16:42.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:42 smithi118 ceph-mon[35048]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-12T03:16:42.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:42 smithi118 ceph-mon[35048]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T03:16:42.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:42 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:16:42.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:42 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:16:42.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:42 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:16:42.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:42 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:16:42.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:42 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:42.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:42 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:16:43.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:43 smithi006 ceph-mon[30991]: purged_snaps scrub starts 2024-09-12T03:16:43.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:43 smithi006 ceph-mon[30991]: purged_snaps scrub ok 2024-09-12T03:16:43.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:43 smithi006 ceph-mon[30991]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:43.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:43 smithi006 ceph-mon[30991]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:16:43.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:43 smithi006 ceph-mon[30991]: osd.7 [v2:172.21.15.118:6816/269980353,v1:172.21.15.118:6817/269980353] boot 2024-09-12T03:16:43.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:43 smithi006 ceph-mon[30991]: osdmap e48: 8 total, 8 up, 8 in 2024-09-12T03:16:43.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:43 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:16:43.363 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:43 smithi118 ceph-mon[35048]: purged_snaps scrub starts 2024-09-12T03:16:43.363 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:43 smithi118 ceph-mon[35048]: purged_snaps scrub ok 2024-09-12T03:16:43.363 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:43 smithi118 ceph-mon[35048]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:43.363 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:43 smithi118 ceph-mon[35048]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:16:43.363 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:43 smithi118 ceph-mon[35048]: osd.7 [v2:172.21.15.118:6816/269980353,v1:172.21.15.118:6817/269980353] boot 2024-09-12T03:16:43.364 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:43 smithi118 ceph-mon[35048]: osdmap e48: 8 total, 8 up, 8 in 2024-09-12T03:16:43.364 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:43 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:16:43.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:43 smithi161 ceph-mon[35485]: purged_snaps scrub starts 2024-09-12T03:16:43.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:43 smithi161 ceph-mon[35485]: purged_snaps scrub ok 2024-09-12T03:16:43.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:43 smithi161 ceph-mon[35485]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:16:43.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:43 smithi161 ceph-mon[35485]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:16:43.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:43 smithi161 ceph-mon[35485]: osd.7 [v2:172.21.15.118:6816/269980353,v1:172.21.15.118:6817/269980353] boot 2024-09-12T03:16:43.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:43 smithi161 ceph-mon[35485]: osdmap e48: 8 total, 8 up, 8 in 2024-09-12T03:16:43.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:43 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:16:44.320 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:44 smithi161 ceph-mon[35485]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T03:16:44.320 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:44 smithi161 ceph-mon[35485]: from='client.? 172.21.15.161:0/601916894' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fd58e55-1492-4595-8647-823ea3e3c5d7"}]: dispatch 2024-09-12T03:16:44.320 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:44 smithi161 ceph-mon[35485]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fd58e55-1492-4595-8647-823ea3e3c5d7"}]: dispatch 2024-09-12T03:16:44.320 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:44 smithi161 ceph-mon[35485]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fd58e55-1492-4595-8647-823ea3e3c5d7"}]': finished 2024-09-12T03:16:44.321 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:44 smithi161 ceph-mon[35485]: osdmap e50: 9 total, 8 up, 9 in 2024-09-12T03:16:44.321 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:44 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:16:44.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:44 smithi006 ceph-mon[30991]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T03:16:44.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:44 smithi006 ceph-mon[30991]: from='client.? 172.21.15.161:0/601916894' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fd58e55-1492-4595-8647-823ea3e3c5d7"}]: dispatch 2024-09-12T03:16:44.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:44 smithi006 ceph-mon[30991]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fd58e55-1492-4595-8647-823ea3e3c5d7"}]: dispatch 2024-09-12T03:16:44.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:44 smithi006 ceph-mon[30991]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fd58e55-1492-4595-8647-823ea3e3c5d7"}]': finished 2024-09-12T03:16:44.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:44 smithi006 ceph-mon[30991]: osdmap e50: 9 total, 8 up, 9 in 2024-09-12T03:16:44.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:44 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:16:44.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:44 smithi118 ceph-mon[35048]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T03:16:44.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:44 smithi118 ceph-mon[35048]: from='client.? 172.21.15.161:0/601916894' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fd58e55-1492-4595-8647-823ea3e3c5d7"}]: dispatch 2024-09-12T03:16:44.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:44 smithi118 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fd58e55-1492-4595-8647-823ea3e3c5d7"}]: dispatch 2024-09-12T03:16:44.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:44 smithi118 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fd58e55-1492-4595-8647-823ea3e3c5d7"}]': finished 2024-09-12T03:16:44.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:44 smithi118 ceph-mon[35048]: osdmap e50: 9 total, 8 up, 9 in 2024-09-12T03:16:44.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:44 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:16:45.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:45 smithi006 ceph-mon[30991]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:45.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:45 smithi006 ceph-mon[30991]: from='client.? 172.21.15.161:0/3347717286' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:16:45.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:45 smithi161 ceph-mon[35485]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:45.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:45 smithi161 ceph-mon[35485]: from='client.? 172.21.15.161:0/3347717286' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:16:45.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:45 smithi118 ceph-mon[35048]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:45.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:45 smithi118 ceph-mon[35048]: from='client.? 172.21.15.161:0/3347717286' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:16:46.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:46 smithi006 ceph-mon[30991]: osdmap e51: 9 total, 8 up, 9 in 2024-09-12T03:16:46.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:46 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:16:46.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:46 smithi161 ceph-mon[35485]: osdmap e51: 9 total, 8 up, 9 in 2024-09-12T03:16:46.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:46 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:16:46.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:46 smithi118 ceph-mon[35048]: osdmap e51: 9 total, 8 up, 9 in 2024-09-12T03:16:46.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:46 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:16:47.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:47 smithi006 ceph-mon[30991]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:47.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:47 smithi006 ceph-mon[30991]: osdmap e52: 9 total, 8 up, 9 in 2024-09-12T03:16:47.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:47 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:16:47.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:47 smithi161 ceph-mon[35485]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:47.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:47 smithi161 ceph-mon[35485]: osdmap e52: 9 total, 8 up, 9 in 2024-09-12T03:16:47.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:47 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:16:47.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:47 smithi118 ceph-mon[35048]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:47.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:47 smithi118 ceph-mon[35048]: osdmap e52: 9 total, 8 up, 9 in 2024-09-12T03:16:47.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:47 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:16:49.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:49 smithi006 ceph-mon[30991]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:49.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:49 smithi161 ceph-mon[35485]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:49.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:49 smithi118 ceph-mon[35048]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:51.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:51 smithi161 ceph-mon[35485]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:51.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:51 smithi006 ceph-mon[30991]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:51.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:51 smithi118 ceph-mon[35048]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:53.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:53 smithi006 ceph-mon[30991]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:53.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:53 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T03:16:53.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:53 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:53.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:53 smithi161 ceph-mon[35485]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:53.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:53 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T03:16:53.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:53 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:53.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:53 smithi118 ceph-mon[35048]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:53.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:53 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T03:16:53.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:53 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:54.261 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:54 smithi161 ceph-mon[35485]: Deploying daemon osd.8 on smithi161 2024-09-12T03:16:54.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:54 smithi118 ceph-mon[35048]: Deploying daemon osd.8 on smithi161 2024-09-12T03:16:54.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:54 smithi006 ceph-mon[30991]: Deploying daemon osd.8 on smithi161 2024-09-12T03:16:55.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:55 smithi161 ceph-mon[35485]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:55.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:55 smithi118 ceph-mon[35048]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:55.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:55 smithi006 ceph-mon[30991]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:57.269 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:57 smithi161 ceph-mon[35485]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:57.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:57 smithi118 ceph-mon[35048]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:57.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:57 smithi006 ceph-mon[30991]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:59.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:59 smithi161 ceph-mon[35485]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:59.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:59.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:59.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:16:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:59.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:59 smithi118 ceph-mon[35048]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:59.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:59 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:59.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:59 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:59.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:16:59 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:59.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:59 smithi006 ceph-mon[30991]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:16:59.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:59.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:16:59.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:16:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:01.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:01 smithi161 ceph-mon[35485]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:17:01.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:01 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:01.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:01 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:01.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:01 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:01.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:01 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:01.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:01 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:01.414 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:01 smithi006 ceph-mon[30991]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:17:01.414 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:01 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:01.414 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:01 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:01.414 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:01 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:01.414 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:01 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:01.414 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:01 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:01.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:01 smithi118 ceph-mon[35048]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:17:01.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:01 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:01.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:01 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:01.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:01 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:01.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:01 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:01.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:01 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:01.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:01.588+0000 7f9ad4ff9640 1 -- 172.21.15.161:0/3968576836 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9ad8062630 con 0x7f9ac00a2bb0 2024-09-12T03:17:01.589 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 8 on host 'smithi161' 2024-09-12T03:17:01.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:01.591+0000 7f9addd56640 1 -- 172.21.15.161:0/3968576836 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9ac00a2bb0 msgr2=0x7f9ac00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:01.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:01.591+0000 7f9addd56640 1 --2- 172.21.15.161:0/3968576836 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9ac00a2bb0 0x7f9ac00a5070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9acc005ab0 tx=0x7f9acc000bc0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:01.591+0000 7f9addd56640 1 -- 172.21.15.161:0/3968576836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ad81026c0 msgr2=0x7f9ad8197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:01.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:01.591+0000 7f9addd56640 1 --2- 172.21.15.161:0/3968576836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ad81026c0 0x7f9ad8197f40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9ac800d950 tx=0x7f9ac800de10 comp rx=0 tx=0).stop 2024-09-12T03:17:01.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:01.591+0000 7f9addd56640 1 -- 172.21.15.161:0/3968576836 shutdown_connections 2024-09-12T03:17:01.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:01.591+0000 7f9addd56640 1 --2- 172.21.15.161:0/3968576836 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9ac00a2bb0 0x7f9ac00a5070 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:01.591+0000 7f9addd56640 1 --2- 172.21.15.161:0/3968576836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ad8107920 0x7f9ad819f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:01.591+0000 7f9addd56640 1 --2- 172.21.15.161:0/3968576836 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ad8104ff0 0x7f9ad8198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:01.591+0000 7f9addd56640 1 --2- 172.21.15.161:0/3968576836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ad81026c0 0x7f9ad8197f40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:01.591+0000 7f9addd56640 1 -- 172.21.15.161:0/3968576836 >> 172.21.15.161:0/3968576836 conn(0x7f9ad80fc280 msgr2=0x7f9ad8103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:01.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:01.591+0000 7f9addd56640 1 -- 172.21.15.161:0/3968576836 shutdown_connections 2024-09-12T03:17:01.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:01.591+0000 7f9addd56640 1 -- 172.21.15.161:0/3968576836 wait complete. 2024-09-12T03:17:02.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:02 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:02.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:02 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:02.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:02 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:02.180 DEBUG:teuthology.orchestra.run.smithi161:osd.8> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.8.service 2024-09-12T03:17:02.183 INFO:tasks.cephadm:Deploying osd.9 on smithi006 with /dev/vg_nvme/lv_1... 2024-09-12T03:17:02.183 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T03:17:02.378 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:02 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:02.378 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:02 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:02.378 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:02 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:02.454 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:02.486 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:02 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:02.486 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:02 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:02.486 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:02 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:03.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:03 smithi161 ceph-mon[35485]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:17:03.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:03 smithi118 ceph-mon[35048]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:17:03.505 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:03 smithi006 ceph-mon[30991]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:17:04.441 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:17:04 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:17:04.209+0000 7fd9093c9640 -1 osd.8 0 log_to_monitors true 2024-09-12T03:17:05.051 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:17:05.078 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi006:vg_nvme/lv_1 2024-09-12T03:17:05.331 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:05.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:05 smithi006 ceph-mon[30991]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:17:05.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:05 smithi006 ceph-mon[30991]: from='osd.8 [v2:172.21.15.161:6816/1783416134,v1:172.21.15.161:6817/1783416134]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:17:05.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:05 smithi006 ceph-mon[30991]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:17:05.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:05 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:05.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:05 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:05.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:05 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:05.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:05 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:05.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:05 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:05.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:05 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:05.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:05 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:05.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:05 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:05.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:05 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:05.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:05 smithi161 ceph-mon[35485]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:17:05.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:05 smithi161 ceph-mon[35485]: from='osd.8 [v2:172.21.15.161:6816/1783416134,v1:172.21.15.161:6817/1783416134]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:17:05.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:05 smithi161 ceph-mon[35485]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:17:05.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:05 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:05.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:05 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:05.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:05 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:05.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:05 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:05.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:05 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:05.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:05 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:05.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:05 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:05.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:05 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:05.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:05 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:05.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:05 smithi118 ceph-mon[35048]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:17:05.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:05 smithi118 ceph-mon[35048]: from='osd.8 [v2:172.21.15.161:6816/1783416134,v1:172.21.15.161:6817/1783416134]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:17:05.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:05 smithi118 ceph-mon[35048]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:17:05.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:05 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:05.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:05 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:05.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:05 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:05.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:05 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:05.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:05 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:05.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:05 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:05.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:05 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:05.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:05 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:05.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:05 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:05.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.746+0000 7fa656e2c640 1 -- 172.21.15.6:0/492363444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa650103870 msgr2=0x7fa65010a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:05.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.746+0000 7fa656e2c640 1 --2- 172.21.15.6:0/492363444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa650103870 0x7fa65010a100 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa648009cd0 tx=0x7fa64802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.746+0000 7fa656e2c640 1 -- 172.21.15.6:0/492363444 shutdown_connections 2024-09-12T03:17:05.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.746+0000 7fa656e2c640 1 --2- 172.21.15.6:0/492363444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa650103870 0x7fa65010a100 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.746+0000 7fa656e2c640 1 --2- 172.21.15.6:0/492363444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa650102eb0 0x7fa650103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.746+0000 7fa656e2c640 1 --2- 172.21.15.6:0/492363444 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6500ffed0 0x7fa6501002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.746+0000 7fa656e2c640 1 -- 172.21.15.6:0/492363444 >> 172.21.15.6:0/492363444 conn(0x7fa6500fba40 msgr2=0x7fa6500fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:05.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.746+0000 7fa656e2c640 1 -- 172.21.15.6:0/492363444 shutdown_connections 2024-09-12T03:17:05.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.746+0000 7fa656e2c640 1 -- 172.21.15.6:0/492363444 wait complete. 2024-09-12T03:17:05.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.746+0000 7fa656e2c640 1 Processor -- start 2024-09-12T03:17:05.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.746+0000 7fa656e2c640 1 -- start start 2024-09-12T03:17:05.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.746+0000 7fa656e2c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6500ffed0 0x7fa650102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:05.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.746+0000 7fa656e2c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa650102eb0 0x7fa650100e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:05.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.746+0000 7fa656e2c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa650103870 0x7fa6501013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:05.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.746+0000 7fa656e2c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa65010d8b0 con 0x7fa650102eb0 2024-09-12T03:17:05.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.746+0000 7fa656e2c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa65010d730 con 0x7fa6500ffed0 2024-09-12T03:17:05.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.746+0000 7fa656e2c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa65010da30 con 0x7fa650103870 2024-09-12T03:17:05.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.747+0000 7fa64ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa650102eb0 0x7fa650100e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:05.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.747+0000 7fa654bc8640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6500ffed0 0x7fa650102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:05.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.747+0000 7fa64ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa650102eb0 0x7fa650100e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54746/0 (socket says 172.21.15.6:54746) 2024-09-12T03:17:05.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.747+0000 7fa64ffff640 1 -- 172.21.15.6:0/4018183991 learned_addr learned my addr 172.21.15.6:0/4018183991 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:17:05.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.747+0000 7fa6553c9640 1 --2- 172.21.15.6:0/4018183991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa650103870 0x7fa6501013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:05.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.747+0000 7fa64ffff640 1 -- 172.21.15.6:0/4018183991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa650103870 msgr2=0x7fa6501013a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:05.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.747+0000 7fa64ffff640 1 --2- 172.21.15.6:0/4018183991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa650103870 0x7fa6501013a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.747+0000 7fa64ffff640 1 -- 172.21.15.6:0/4018183991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6500ffed0 msgr2=0x7fa650102810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:05.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.747+0000 7fa64ffff640 1 --2- 172.21.15.6:0/4018183991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6500ffed0 0x7fa650102810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.747+0000 7fa64ffff640 1 -- 172.21.15.6:0/4018183991 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa650101c60 con 0x7fa650102eb0 2024-09-12T03:17:05.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.747+0000 7fa6553c9640 1 --2- 172.21.15.6:0/4018183991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa650103870 0x7fa6501013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:05.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.747+0000 7fa64ffff640 1 --2- 172.21.15.6:0/4018183991 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa650102eb0 0x7fa650100e60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa64400d770 tx=0x7fa64400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:05.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.747+0000 7fa64dffb640 1 -- 172.21.15.6:0/4018183991 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644017070 con 0x7fa650102eb0 2024-09-12T03:17:05.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.747+0000 7fa656e2c640 1 -- 172.21.15.6:0/4018183991 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6501a2100 con 0x7fa650102eb0 2024-09-12T03:17:05.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.747+0000 7fa64dffb640 1 -- 172.21.15.6:0/4018183991 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa644005500 con 0x7fa650102eb0 2024-09-12T03:17:05.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.747+0000 7fa64dffb640 1 -- 172.21.15.6:0/4018183991 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644004e00 con 0x7fa650102eb0 2024-09-12T03:17:05.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.748+0000 7fa656e2c640 1 -- 172.21.15.6:0/4018183991 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6501a2610 con 0x7fa650102eb0 2024-09-12T03:17:05.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.749+0000 7fa656e2c640 1 -- 172.21.15.6:0/4018183991 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa650109120 con 0x7fa650102eb0 2024-09-12T03:17:05.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.750+0000 7fa64dffb640 1 -- 172.21.15.6:0/4018183991 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fa644004fa0 con 0x7fa650102eb0 2024-09-12T03:17:05.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.750+0000 7fa64dffb640 1 --2- 172.21.15.6:0/4018183991 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa6380a2c00 0x7fa6380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:05.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.750+0000 7fa654bc8640 1 --2- 172.21.15.6:0/4018183991 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa6380a2c00 0x7fa6380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:05.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.750+0000 7fa64dffb640 1 -- 172.21.15.6:0/4018183991 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5533+0+0 (secure 0 0 0) 0x7fa6440cdea0 con 0x7fa650102eb0 2024-09-12T03:17:05.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.751+0000 7fa654bc8640 1 --2- 172.21.15.6:0/4018183991 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa6380a2c00 0x7fa6380a50c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa640005fd0 tx=0x7fa64000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:05.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.755+0000 7fa64dffb640 1 -- 172.21.15.6:0/4018183991 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa64409b8e0 con 0x7fa650102eb0 2024-09-12T03:17:05.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:05.939+0000 7fa656e2c640 1 -- 172.21.15.6:0/4018183991 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa650062630 con 0x7fa6380a2c00 2024-09-12T03:17:06.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:06 smithi006 ceph-mon[30991]: Detected new or changed devices on smithi161 2024-09-12T03:17:06.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:06 smithi006 ceph-mon[30991]: Adjusting osd_memory_target on smithi161 to 5696M 2024-09-12T03:17:06.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:06 smithi006 ceph-mon[30991]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T03:17:06.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:06 smithi006 ceph-mon[30991]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T03:17:06.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:06 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:17:06.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:06 smithi006 ceph-mon[30991]: from='osd.8 [v2:172.21.15.161:6816/1783416134,v1:172.21.15.161:6817/1783416134]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:17:06.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:06 smithi006 ceph-mon[30991]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:17:06.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:06 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:17:06.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:06 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:17:06.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:06 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:06.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:06 smithi118 ceph-mon[35048]: Detected new or changed devices on smithi161 2024-09-12T03:17:06.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:06 smithi118 ceph-mon[35048]: Adjusting osd_memory_target on smithi161 to 5696M 2024-09-12T03:17:06.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:06 smithi118 ceph-mon[35048]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T03:17:06.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:06 smithi118 ceph-mon[35048]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T03:17:06.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:06 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:17:06.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:06 smithi118 ceph-mon[35048]: from='osd.8 [v2:172.21.15.161:6816/1783416134,v1:172.21.15.161:6817/1783416134]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:17:06.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:06 smithi118 ceph-mon[35048]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:17:06.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:06 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:17:06.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:06 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:17:06.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:06 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:06.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:06 smithi161 ceph-mon[35485]: Detected new or changed devices on smithi161 2024-09-12T03:17:06.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:06 smithi161 ceph-mon[35485]: Adjusting osd_memory_target on smithi161 to 5696M 2024-09-12T03:17:06.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:06 smithi161 ceph-mon[35485]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T03:17:06.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:06 smithi161 ceph-mon[35485]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T03:17:06.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:06 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:17:06.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:06 smithi161 ceph-mon[35485]: from='osd.8 [v2:172.21.15.161:6816/1783416134,v1:172.21.15.161:6817/1783416134]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:17:06.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:06 smithi161 ceph-mon[35485]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:17:06.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:06 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:17:06.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:06 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:17:06.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:06 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:07.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:07 smithi006 ceph-mon[30991]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:17:07.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:07 smithi006 ceph-mon[30991]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:07.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:07 smithi006 ceph-mon[30991]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-12T03:17:07.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:07 smithi006 ceph-mon[30991]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T03:17:07.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:07 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:17:07.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:07 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:17:07.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:07 smithi006 ceph-mon[30991]: from='osd.8 ' entity='osd.8' 2024-09-12T03:17:07.363 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:17:06 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:17:06.948+0000 7fd905b60640 -1 osd.8 0 waiting for initial osdmap 2024-09-12T03:17:07.363 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:17:06 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:17:06.963+0000 7fd8fd15b640 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:17:07.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:07 smithi161 ceph-mon[35485]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:17:07.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:07 smithi161 ceph-mon[35485]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:07.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:07 smithi161 ceph-mon[35485]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-12T03:17:07.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:07 smithi161 ceph-mon[35485]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T03:17:07.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:07 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:17:07.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:07 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:17:07.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:07 smithi161 ceph-mon[35485]: from='osd.8 ' entity='osd.8' 2024-09-12T03:17:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:07 smithi118 ceph-mon[35048]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:17:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:07 smithi118 ceph-mon[35048]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:07 smithi118 ceph-mon[35048]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-12T03:17:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:07 smithi118 ceph-mon[35048]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T03:17:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:07 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:17:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:07 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:17:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:07 smithi118 ceph-mon[35048]: from='osd.8 ' entity='osd.8' 2024-09-12T03:17:08.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:08 smithi118 ceph-mon[35048]: purged_snaps scrub starts 2024-09-12T03:17:08.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:08 smithi118 ceph-mon[35048]: purged_snaps scrub ok 2024-09-12T03:17:08.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:08 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:17:08.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:08 smithi118 ceph-mon[35048]: osd.8 [v2:172.21.15.161:6816/1783416134,v1:172.21.15.161:6817/1783416134] boot 2024-09-12T03:17:08.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:08 smithi118 ceph-mon[35048]: osdmap e55: 9 total, 9 up, 9 in 2024-09-12T03:17:08.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:08 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:17:08.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:08 smithi006 ceph-mon[30991]: purged_snaps scrub starts 2024-09-12T03:17:08.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:08 smithi006 ceph-mon[30991]: purged_snaps scrub ok 2024-09-12T03:17:08.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:08 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:17:08.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:08 smithi006 ceph-mon[30991]: osd.8 [v2:172.21.15.161:6816/1783416134,v1:172.21.15.161:6817/1783416134] boot 2024-09-12T03:17:08.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:08 smithi006 ceph-mon[30991]: osdmap e55: 9 total, 9 up, 9 in 2024-09-12T03:17:08.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:08 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:17:08.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:08 smithi161 ceph-mon[35485]: purged_snaps scrub starts 2024-09-12T03:17:08.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:08 smithi161 ceph-mon[35485]: purged_snaps scrub ok 2024-09-12T03:17:08.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:08 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:17:08.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:08 smithi161 ceph-mon[35485]: osd.8 [v2:172.21.15.161:6816/1783416134,v1:172.21.15.161:6817/1783416134] boot 2024-09-12T03:17:08.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:08 smithi161 ceph-mon[35485]: osdmap e55: 9 total, 9 up, 9 in 2024-09-12T03:17:08.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:08 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:17:09.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:09 smithi118 ceph-mon[35048]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:17:09.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:09 smithi118 ceph-mon[35048]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T03:17:09.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:09 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/345489012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90508a6d-a49c-43c6-a600-5bbccd8702ea"}]: dispatch 2024-09-12T03:17:09.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:09 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/345489012' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90508a6d-a49c-43c6-a600-5bbccd8702ea"}]': finished 2024-09-12T03:17:09.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:09 smithi118 ceph-mon[35048]: osdmap e57: 10 total, 9 up, 10 in 2024-09-12T03:17:09.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:09 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:09.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:09 smithi006 ceph-mon[30991]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:17:09.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:09 smithi006 ceph-mon[30991]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T03:17:09.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:09 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/345489012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90508a6d-a49c-43c6-a600-5bbccd8702ea"}]: dispatch 2024-09-12T03:17:09.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:09 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/345489012' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90508a6d-a49c-43c6-a600-5bbccd8702ea"}]': finished 2024-09-12T03:17:09.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:09 smithi006 ceph-mon[30991]: osdmap e57: 10 total, 9 up, 10 in 2024-09-12T03:17:09.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:09 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:09.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:09 smithi161 ceph-mon[35485]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:17:09.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:09 smithi161 ceph-mon[35485]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T03:17:09.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:09 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/345489012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90508a6d-a49c-43c6-a600-5bbccd8702ea"}]: dispatch 2024-09-12T03:17:09.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:09 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/345489012' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90508a6d-a49c-43c6-a600-5bbccd8702ea"}]': finished 2024-09-12T03:17:09.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:09 smithi161 ceph-mon[35485]: osdmap e57: 10 total, 9 up, 10 in 2024-09-12T03:17:09.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:09 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:10.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:10 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/29813019' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:17:10.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:10 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/29813019' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:17:10.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:10 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/29813019' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:17:11.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:11 smithi118 ceph-mon[35048]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:17:11.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:11 smithi118 ceph-mon[35048]: osdmap e58: 10 total, 9 up, 10 in 2024-09-12T03:17:11.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:11 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:11.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:11 smithi006 ceph-mon[30991]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:17:11.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:11 smithi006 ceph-mon[30991]: osdmap e58: 10 total, 9 up, 10 in 2024-09-12T03:17:11.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:11 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:11.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:11 smithi161 ceph-mon[35485]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:17:11.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:11 smithi161 ceph-mon[35485]: osdmap e58: 10 total, 9 up, 10 in 2024-09-12T03:17:11.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:11 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:12.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:12 smithi118 ceph-mon[35048]: osdmap e59: 10 total, 9 up, 10 in 2024-09-12T03:17:12.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:12 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:12.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:12 smithi118 ceph-mon[35048]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:17:12.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:12 smithi006 ceph-mon[30991]: osdmap e59: 10 total, 9 up, 10 in 2024-09-12T03:17:12.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:12 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:12.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:12 smithi006 ceph-mon[30991]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:17:12.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:12 smithi161 ceph-mon[35485]: osdmap e59: 10 total, 9 up, 10 in 2024-09-12T03:17:12.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:12 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:12.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:12 smithi161 ceph-mon[35485]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:17:14.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:14 smithi006 ceph-mon[30991]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-12T03:17:14.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:14 smithi161 ceph-mon[35485]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-12T03:17:14.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:14 smithi118 ceph-mon[35048]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-12T03:17:16.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:16 smithi006 ceph-mon[30991]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-12T03:17:16.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:16 smithi161 ceph-mon[35485]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-12T03:17:16.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:16 smithi118 ceph-mon[35048]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-12T03:17:18.577 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:18 smithi006 ceph-mon[30991]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T03:17:18.577 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:18 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T03:17:18.578 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:18 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:18.578 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:18 smithi006 ceph-mon[30991]: Deploying daemon osd.9 on smithi006 2024-09-12T03:17:18.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:18 smithi161 ceph-mon[35485]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T03:17:18.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:18 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T03:17:18.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:18 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:18.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:18 smithi161 ceph-mon[35485]: Deploying daemon osd.9 on smithi006 2024-09-12T03:17:18.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:18 smithi118 ceph-mon[35048]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T03:17:18.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:18 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T03:17:18.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:18 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:18.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:18 smithi118 ceph-mon[35048]: Deploying daemon osd.9 on smithi006 2024-09-12T03:17:20.555 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:20 smithi006 ceph-mon[30991]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-12T03:17:20.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:20 smithi161 ceph-mon[35485]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-12T03:17:20.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:20 smithi118 ceph-mon[35048]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-12T03:17:22.413 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:22 smithi006 ceph-mon[30991]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-12T03:17:22.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:22 smithi161 ceph-mon[35485]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-12T03:17:22.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:22 smithi118 ceph-mon[35048]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-12T03:17:24.570 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:24 smithi006 ceph-mon[30991]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:17:24.570 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:24.570 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:24.570 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:24.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:24 smithi161 ceph-mon[35485]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:17:24.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:24.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:24.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:24.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:24 smithi118 ceph-mon[35048]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:17:24.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:24.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:24.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:26.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:26 smithi006 ceph-mon[30991]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:17:26.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:26 smithi161 ceph-mon[35485]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:17:26.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:26 smithi118 ceph-mon[35048]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:17:27.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:27.443+0000 7fa64dffb640 1 -- 172.21.15.6:0/4018183991 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa650062630 con 0x7fa6380a2c00 2024-09-12T03:17:27.444 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 9 on host 'smithi006' 2024-09-12T03:17:27.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:27.446+0000 7fa656e2c640 1 -- 172.21.15.6:0/4018183991 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa6380a2c00 msgr2=0x7fa6380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:27.446+0000 7fa656e2c640 1 --2- 172.21.15.6:0/4018183991 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa6380a2c00 0x7fa6380a50c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa640005fd0 tx=0x7fa64000b410 comp rx=0 tx=0).stop 2024-09-12T03:17:27.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:27.446+0000 7fa656e2c640 1 -- 172.21.15.6:0/4018183991 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa650102eb0 msgr2=0x7fa650100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:27.446+0000 7fa656e2c640 1 --2- 172.21.15.6:0/4018183991 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa650102eb0 0x7fa650100e60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa64400d770 tx=0x7fa64400dc30 comp rx=0 tx=0).stop 2024-09-12T03:17:27.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:27.446+0000 7fa656e2c640 1 -- 172.21.15.6:0/4018183991 shutdown_connections 2024-09-12T03:17:27.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:27.446+0000 7fa656e2c640 1 --2- 172.21.15.6:0/4018183991 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa6380a2c00 0x7fa6380a50c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:27.446+0000 7fa656e2c640 1 --2- 172.21.15.6:0/4018183991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa650103870 0x7fa6501013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:27.446+0000 7fa656e2c640 1 --2- 172.21.15.6:0/4018183991 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa650102eb0 0x7fa650100e60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:27.446+0000 7fa656e2c640 1 --2- 172.21.15.6:0/4018183991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6500ffed0 0x7fa650102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:27.446+0000 7fa656e2c640 1 -- 172.21.15.6:0/4018183991 >> 172.21.15.6:0/4018183991 conn(0x7fa6500fba40 msgr2=0x7fa6501062f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:27.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:27.446+0000 7fa656e2c640 1 -- 172.21.15.6:0/4018183991 shutdown_connections 2024-09-12T03:17:27.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:17:27.446+0000 7fa656e2c640 1 -- 172.21.15.6:0/4018183991 wait complete. 2024-09-12T03:17:27.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:27 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:27.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:27 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:27.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:27 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:27.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:27 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:27.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:27 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:27.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:27 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:27.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:27 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:27.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:27 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:28.071 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:27 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:28.071 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:27 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:28.071 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:27 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:28.072 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:27 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:28.072 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:27 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:28.072 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:27 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:28.072 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:27 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:28.072 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:27 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:28.099 DEBUG:teuthology.orchestra.run.smithi006:osd.9> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.9.service 2024-09-12T03:17:28.102 INFO:tasks.cephadm:Deploying osd.10 on smithi118 with /dev/vg_nvme/lv_1... 2024-09-12T03:17:28.102 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T03:17:28.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:27 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:28.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:27 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:28.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:27 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:28.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:27 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:28.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:27 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:28.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:27 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:28.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:27 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:28.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:27 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:28.359 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:17:28.809 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:28 smithi118 ceph-mon[35048]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:17:29.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:28 smithi006 ceph-mon[30991]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:17:29.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:28 smithi161 ceph-mon[35485]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:17:30.576 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:17:30 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:17:30.115+0000 7fcec6f92640 -1 osd.9 0 log_to_monitors true 2024-09-12T03:17:30.820 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:30 smithi118 ceph-mon[35048]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:17:30.820 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:30 smithi118 ceph-mon[35048]: from='osd.9 [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T03:17:30.823 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-12T03:17:30.849 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi118:vg_nvme/lv_1 2024-09-12T03:17:30.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:30 smithi006 ceph-mon[30991]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:17:30.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:30 smithi006 ceph-mon[30991]: from='osd.9 [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T03:17:31.102 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:17:31.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:30 smithi161 ceph-mon[35485]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:17:31.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:30 smithi161 ceph-mon[35485]: from='osd.9 [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T03:17:31.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.582+0000 7fb638211640 1 -- 172.21.15.118:0/3189283553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb630101d10 msgr2=0x7fb630102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:31.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.582+0000 7fb638211640 1 --2- 172.21.15.118:0/3189283553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb630101d10 0x7fb630102190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009900 tx=0x7fb62c02a410 comp rx=0 tx=0).stop 2024-09-12T03:17:31.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.582+0000 7fb638211640 1 -- 172.21.15.118:0/3189283553 shutdown_connections 2024-09-12T03:17:31.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.582+0000 7fb638211640 1 --2- 172.21.15.118:0/3189283553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6301026d0 0x7fb630108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.582+0000 7fb638211640 1 --2- 172.21.15.118:0/3189283553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb630101d10 0x7fb630102190 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.582+0000 7fb638211640 1 --2- 172.21.15.118:0/3189283553 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb630100b10 0x7fb630100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.582+0000 7fb638211640 1 -- 172.21.15.118:0/3189283553 >> 172.21.15.118:0/3189283553 conn(0x7fb6300fc2a0 msgr2=0x7fb6300fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:31.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.582+0000 7fb638211640 1 -- 172.21.15.118:0/3189283553 shutdown_connections 2024-09-12T03:17:31.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.582+0000 7fb638211640 1 -- 172.21.15.118:0/3189283553 wait complete. 2024-09-12T03:17:31.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.583+0000 7fb638211640 1 Processor -- start 2024-09-12T03:17:31.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.583+0000 7fb638211640 1 -- start start 2024-09-12T03:17:31.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.583+0000 7fb638211640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb630100b10 0x7fb63010b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:31.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.583+0000 7fb638211640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb630101d10 0x7fb630109ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:31.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.583+0000 7fb638211640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6301026d0 0x7fb63010a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:31.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.583+0000 7fb638211640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb63010d8c0 con 0x7fb630101d10 2024-09-12T03:17:31.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.583+0000 7fb638211640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fb63010d740 con 0x7fb630100b10 2024-09-12T03:17:31.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.583+0000 7fb638211640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb63010da40 con 0x7fb6301026d0 2024-09-12T03:17:31.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.583+0000 7fb635fad640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb630100b10 0x7fb63010b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:31.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.583+0000 7fb635fad640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb630100b10 0x7fb63010b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:46712/0 (socket says 172.21.15.118:46712) 2024-09-12T03:17:31.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.583+0000 7fb635fad640 1 -- 172.21.15.118:0/3556165254 learned_addr learned my addr 172.21.15.118:0/3556165254 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-09-12T03:17:31.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.583+0000 7fb6367ae640 1 --2- 172.21.15.118:0/3556165254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6301026d0 0x7fb63010a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:31.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.583+0000 7fb6357ac640 1 --2- 172.21.15.118:0/3556165254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb630101d10 0x7fb630109ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:31.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.584+0000 7fb635fad640 1 -- 172.21.15.118:0/3556165254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6301026d0 msgr2=0x7fb63010a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:31.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.584+0000 7fb635fad640 1 --2- 172.21.15.118:0/3556165254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6301026d0 0x7fb63010a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.584+0000 7fb635fad640 1 -- 172.21.15.118:0/3556165254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb630101d10 msgr2=0x7fb630109ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:31.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.584+0000 7fb635fad640 1 --2- 172.21.15.118:0/3556165254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb630101d10 0x7fb630109ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.584+0000 7fb635fad640 1 -- 172.21.15.118:0/3556165254 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63010a8e0 con 0x7fb630100b10 2024-09-12T03:17:31.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.584+0000 7fb635fad640 1 --2- 172.21.15.118:0/3556165254 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb630100b10 0x7fb63010b490 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb62000e9c0 tx=0x7fb62000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:31.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.584+0000 7fb626ffd640 1 -- 172.21.15.118:0/3556165254 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62000cd80 con 0x7fb630100b10 2024-09-12T03:17:31.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.584+0000 7fb626ffd640 1 -- 172.21.15.118:0/3556165254 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6200035f0 con 0x7fb630100b10 2024-09-12T03:17:31.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.584+0000 7fb638211640 1 -- 172.21.15.118:0/3556165254 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6301a2070 con 0x7fb630100b10 2024-09-12T03:17:31.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.584+0000 7fb626ffd640 1 -- 172.21.15.118:0/3556165254 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620010690 con 0x7fb630100b10 2024-09-12T03:17:31.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.584+0000 7fb638211640 1 -- 172.21.15.118:0/3556165254 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6301a2580 con 0x7fb630100b10 2024-09-12T03:17:31.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.585+0000 7fb638211640 1 -- 172.21.15.118:0/3556165254 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb630107eb0 con 0x7fb630100b10 2024-09-12T03:17:31.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.587+0000 7fb626ffd640 1 -- 172.21.15.118:0/3556165254 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb6200037a0 con 0x7fb630100b10 2024-09-12T03:17:31.602 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.587+0000 7fb626ffd640 1 --2- 172.21.15.118:0/3556165254 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb61c0a2b80 0x7fb61c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:31.602 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.588+0000 7fb626ffd640 1 -- 172.21.15.118:0/3556165254 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5965+0+0 (secure 0 0 0) 0x7fb6200ceb80 con 0x7fb630100b10 2024-09-12T03:17:31.602 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.588+0000 7fb6357ac640 1 --2- 172.21.15.118:0/3556165254 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb61c0a2b80 0x7fb61c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:31.602 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.588+0000 7fb6357ac640 1 --2- 172.21.15.118:0/3556165254 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb61c0a2b80 0x7fb61c0a5040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0314e0 tx=0x7fb62c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:31.602 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.591+0000 7fb626ffd640 1 -- 172.21.15.118:0/3556165254 <== mon.2 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb62009c4a0 con 0x7fb630100b10 2024-09-12T03:17:31.767 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:31.767+0000 7fb638211640 1 -- 172.21.15.118:0/3556165254 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb630062630 con 0x7fb61c0a2b80 2024-09-12T03:17:31.768 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:31 smithi118 ceph-mon[35048]: from='osd.9 [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T03:17:31.768 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:31 smithi118 ceph-mon[35048]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T03:17:31.768 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:31 smithi118 ceph-mon[35048]: from='osd.9 [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-09-12T03:17:31.768 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:31 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:31.768 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:31 smithi118 ceph-mon[35048]: Detected new or changed devices on smithi006 2024-09-12T03:17:31.768 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:31 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:31.768 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:31 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:31.768 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:31 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:31.769 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:31 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:31.769 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:31 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:31.769 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:31 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:31.769 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:31 smithi118 ceph-mon[35048]: Adjusting osd_memory_target on smithi006 to 4273M 2024-09-12T03:17:31.769 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:31 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:31.769 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:31 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:31.769 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:31 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:31.769 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:31 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:32.112 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:17:31 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:17:31.729+0000 7fcec26f2640 -1 osd.9 0 waiting for initial osdmap 2024-09-12T03:17:32.112 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:17:31 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:17:31.745+0000 7fcebed2c640 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:17:32.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:31 smithi006 ceph-mon[30991]: from='osd.9 [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T03:17:32.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:31 smithi006 ceph-mon[30991]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T03:17:32.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:31 smithi006 ceph-mon[30991]: from='osd.9 [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-09-12T03:17:32.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:31 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:32.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:31 smithi006 ceph-mon[30991]: Detected new or changed devices on smithi006 2024-09-12T03:17:32.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:31 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:32.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:31 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:32.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:31 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:32.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:31 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:32.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:31 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:32.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:31 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:32.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:31 smithi006 ceph-mon[30991]: Adjusting osd_memory_target on smithi006 to 4273M 2024-09-12T03:17:32.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:31 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:32.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:31 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:32.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:31 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:32.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:31 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:32.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:31 smithi161 ceph-mon[35485]: from='osd.9 [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T03:17:32.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:31 smithi161 ceph-mon[35485]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T03:17:32.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:31 smithi161 ceph-mon[35485]: from='osd.9 [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-09-12T03:17:32.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:31 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:32.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:31 smithi161 ceph-mon[35485]: Detected new or changed devices on smithi006 2024-09-12T03:17:32.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:31 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:32.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:31 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:32.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:31 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:32.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:31 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:32.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:31 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:32.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:31 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:32.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:31 smithi161 ceph-mon[35485]: Adjusting osd_memory_target on smithi006 to 4273M 2024-09-12T03:17:32.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:31 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:32.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:31 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:32.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:31 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:32.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:31 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:33.015 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:32 smithi161 ceph-mon[35485]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:17:33.015 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:32 smithi161 ceph-mon[35485]: from='osd.9 [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-09-12T03:17:33.015 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:32 smithi161 ceph-mon[35485]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T03:17:33.016 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:32 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:33.016 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:32 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:33.016 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:32 smithi161 ceph-mon[35485]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:33.016 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:32 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:17:33.016 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:32 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:17:33.016 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:32 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:33.017 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:32 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:33.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:32 smithi006 ceph-mon[30991]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:17:33.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:32 smithi006 ceph-mon[30991]: from='osd.9 [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-09-12T03:17:33.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:32 smithi006 ceph-mon[30991]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T03:17:33.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:32 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:33.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:32 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:33.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:32 smithi006 ceph-mon[30991]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:33.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:32 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:17:33.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:32 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:17:33.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:32 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:33.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:32 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:33.146 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:32 smithi118 ceph-mon[35048]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:17:33.146 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:32 smithi118 ceph-mon[35048]: from='osd.9 [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-09-12T03:17:33.146 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:32 smithi118 ceph-mon[35048]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T03:17:33.146 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:32 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:33.146 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:32 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:33.146 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:32 smithi118 ceph-mon[35048]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:33.146 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:32 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:17:33.146 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:32 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:17:33.146 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:32 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:33.146 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:32 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:34.031 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:33 smithi161 ceph-mon[35485]: purged_snaps scrub starts 2024-09-12T03:17:34.031 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:33 smithi161 ceph-mon[35485]: purged_snaps scrub ok 2024-09-12T03:17:34.031 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:33 smithi161 ceph-mon[35485]: osd.9 [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365] boot 2024-09-12T03:17:34.031 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:33 smithi161 ceph-mon[35485]: osdmap e62: 10 total, 10 up, 10 in 2024-09-12T03:17:34.031 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:33 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:34.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:33 smithi006 ceph-mon[30991]: purged_snaps scrub starts 2024-09-12T03:17:34.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:33 smithi006 ceph-mon[30991]: purged_snaps scrub ok 2024-09-12T03:17:34.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:33 smithi006 ceph-mon[30991]: osd.9 [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365] boot 2024-09-12T03:17:34.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:33 smithi006 ceph-mon[30991]: osdmap e62: 10 total, 10 up, 10 in 2024-09-12T03:17:34.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:33 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:34.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:33 smithi118 ceph-mon[35048]: purged_snaps scrub starts 2024-09-12T03:17:34.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:33 smithi118 ceph-mon[35048]: purged_snaps scrub ok 2024-09-12T03:17:34.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:33 smithi118 ceph-mon[35048]: osd.9 [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365] boot 2024-09-12T03:17:34.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:33 smithi118 ceph-mon[35048]: osdmap e62: 10 total, 10 up, 10 in 2024-09-12T03:17:34.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:33 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:17:34.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:34 smithi118 ceph-mon[35048]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:17:34.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:34 smithi118 ceph-mon[35048]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T03:17:34.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:34 smithi118 ceph-mon[35048]: from='client.? 172.21.15.118:0/1087855234' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "911e86d1-669a-4935-82c0-7da51d141df8"}]: dispatch 2024-09-12T03:17:34.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:34 smithi118 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "911e86d1-669a-4935-82c0-7da51d141df8"}]: dispatch 2024-09-12T03:17:34.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:34 smithi118 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "911e86d1-669a-4935-82c0-7da51d141df8"}]': finished 2024-09-12T03:17:34.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:34 smithi118 ceph-mon[35048]: osdmap e64: 11 total, 10 up, 11 in 2024-09-12T03:17:34.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:34 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:17:34.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:34 smithi118 ceph-mon[35048]: from='client.? 172.21.15.118:0/3096398088' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:17:35.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:34 smithi006 ceph-mon[30991]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:17:35.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:34 smithi006 ceph-mon[30991]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T03:17:35.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:34 smithi006 ceph-mon[30991]: from='client.? 172.21.15.118:0/1087855234' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "911e86d1-669a-4935-82c0-7da51d141df8"}]: dispatch 2024-09-12T03:17:35.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:34 smithi006 ceph-mon[30991]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "911e86d1-669a-4935-82c0-7da51d141df8"}]: dispatch 2024-09-12T03:17:35.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:34 smithi006 ceph-mon[30991]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "911e86d1-669a-4935-82c0-7da51d141df8"}]': finished 2024-09-12T03:17:35.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:34 smithi006 ceph-mon[30991]: osdmap e64: 11 total, 10 up, 11 in 2024-09-12T03:17:35.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:34 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:17:35.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:34 smithi006 ceph-mon[30991]: from='client.? 172.21.15.118:0/3096398088' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:17:35.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:34 smithi161 ceph-mon[35485]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:17:35.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:34 smithi161 ceph-mon[35485]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T03:17:35.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:34 smithi161 ceph-mon[35485]: from='client.? 172.21.15.118:0/1087855234' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "911e86d1-669a-4935-82c0-7da51d141df8"}]: dispatch 2024-09-12T03:17:35.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:34 smithi161 ceph-mon[35485]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "911e86d1-669a-4935-82c0-7da51d141df8"}]: dispatch 2024-09-12T03:17:35.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:34 smithi161 ceph-mon[35485]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "911e86d1-669a-4935-82c0-7da51d141df8"}]': finished 2024-09-12T03:17:35.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:34 smithi161 ceph-mon[35485]: osdmap e64: 11 total, 10 up, 11 in 2024-09-12T03:17:35.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:34 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:17:35.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:34 smithi161 ceph-mon[35485]: from='client.? 172.21.15.118:0/3096398088' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:17:37.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:36 smithi006 ceph-mon[30991]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:37.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:36 smithi161 ceph-mon[35485]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:37.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:36 smithi118 ceph-mon[35048]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:38.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:38 smithi118 ceph-mon[35048]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:39.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:38 smithi006 ceph-mon[30991]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:39.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:38 smithi161 ceph-mon[35485]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:41.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:40 smithi006 ceph-mon[30991]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:41.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:40 smithi161 ceph-mon[35485]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:41.136 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:40 smithi118 ceph-mon[35048]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:43.040 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:42 smithi118 ceph-mon[35048]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:43.040 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:42 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T03:17:43.040 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:42 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:43.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:42 smithi006 ceph-mon[30991]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:43.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T03:17:43.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:43.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:42 smithi161 ceph-mon[35485]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:43.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T03:17:43.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:44.068 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:43 smithi118 ceph-mon[35048]: Deploying daemon osd.10 on smithi118 2024-09-12T03:17:44.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:43 smithi006 ceph-mon[30991]: Deploying daemon osd.10 on smithi118 2024-09-12T03:17:44.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:43 smithi161 ceph-mon[35485]: Deploying daemon osd.10 on smithi118 2024-09-12T03:17:45.031 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:44 smithi118 ceph-mon[35048]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:45.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:44 smithi006 ceph-mon[30991]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:45.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:44 smithi161 ceph-mon[35485]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:46.856 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:46 smithi118 ceph-mon[35048]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:47.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:46 smithi006 ceph-mon[30991]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:47.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:46 smithi161 ceph-mon[35485]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:48.853 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:48 smithi118 ceph-mon[35048]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:48.853 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:48 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:48.853 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:48 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:48.853 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:48 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:49.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:48 smithi006 ceph-mon[30991]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:49.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:48 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:49.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:48 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:49.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:48 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:49.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:48 smithi161 ceph-mon[35485]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:49.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:48 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:49.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:48 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:49.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:48 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:51.109 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:50 smithi118 ceph-mon[35048]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:51.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:50 smithi006 ceph-mon[30991]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:51.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:50 smithi161 ceph-mon[35485]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:51.154 INFO:teuthology.orchestra.run.smithi118.stdout:Created osd(s) 10 on host 'smithi118' 2024-09-12T03:17:51.154 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:51.154+0000 7fb626ffd640 1 -- 172.21.15.118:0/3556165254 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb630062630 con 0x7fb61c0a2b80 2024-09-12T03:17:51.156 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:51.156+0000 7fb638211640 1 -- 172.21.15.118:0/3556165254 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb61c0a2b80 msgr2=0x7fb61c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:51.156 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:51.156+0000 7fb638211640 1 --2- 172.21.15.118:0/3556165254 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb61c0a2b80 0x7fb61c0a5040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0314e0 tx=0x7fb62c000bc0 comp rx=0 tx=0).stop 2024-09-12T03:17:51.156 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:51.156+0000 7fb638211640 1 -- 172.21.15.118:0/3556165254 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb630100b10 msgr2=0x7fb63010b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:51.156 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:51.156+0000 7fb638211640 1 --2- 172.21.15.118:0/3556165254 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb630100b10 0x7fb63010b490 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb62000e9c0 tx=0x7fb62000ee80 comp rx=0 tx=0).stop 2024-09-12T03:17:51.156 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:51.156+0000 7fb638211640 1 -- 172.21.15.118:0/3556165254 shutdown_connections 2024-09-12T03:17:51.156 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:51.156+0000 7fb638211640 1 --2- 172.21.15.118:0/3556165254 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb61c0a2b80 0x7fb61c0a5040 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:51.157 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:51.156+0000 7fb638211640 1 --2- 172.21.15.118:0/3556165254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6301026d0 0x7fb63010a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:51.157 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:51.156+0000 7fb638211640 1 --2- 172.21.15.118:0/3556165254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb630101d10 0x7fb630109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:51.157 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:51.156+0000 7fb638211640 1 --2- 172.21.15.118:0/3556165254 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb630100b10 0x7fb63010b490 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:51.157 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:51.156+0000 7fb638211640 1 -- 172.21.15.118:0/3556165254 >> 172.21.15.118:0/3556165254 conn(0x7fb6300fc2a0 msgr2=0x7fb630105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:51.157 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:51.156+0000 7fb638211640 1 -- 172.21.15.118:0/3556165254 shutdown_connections 2024-09-12T03:17:51.157 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-12T03:17:51.156+0000 7fb638211640 1 -- 172.21.15.118:0/3556165254 wait complete. 2024-09-12T03:17:51.855 DEBUG:teuthology.orchestra.run.smithi118:osd.10> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.10.service 2024-09-12T03:17:51.858 INFO:tasks.cephadm:Deploying osd.11 on smithi161 with /dev/vg_nvme/lv_1... 2024-09-12T03:17:51.858 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T03:17:52.117 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:17:52.447 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:52 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:52.447 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:52 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:52.447 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:52 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:52.447 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:52 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:52.457 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:52 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:52.457 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:52 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:52.457 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:52 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:52.457 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:52 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:52.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:52 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:52.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:52 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:52.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:52 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:52.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:52 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:53.407 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:53 smithi161 ceph-mon[35485]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:53.427 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:53 smithi118 ceph-mon[35048]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:53.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:53 smithi006 ceph-mon[30991]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:54.713 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:17:54 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:17:54.414+0000 7fd3df1a2640 -1 osd.10 0 log_to_monitors true 2024-09-12T03:17:54.714 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-12T03:17:54.740 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_1 2024-09-12T03:17:54.995 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:17:55.398 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:55 smithi161 ceph-mon[35485]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:55.398 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:55 smithi161 ceph-mon[35485]: from='osd.10 [v2:172.21.15.118:6824/2167146341,v1:172.21.15.118:6825/2167146341]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:17:55.398 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:55 smithi161 ceph-mon[35485]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:17:55.398 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:55 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:55.398 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:55 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:55.398 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:55 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:55.398 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:55 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:55.398 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:55 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:55.398 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:55 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:55.399 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:55 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:55.399 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:55 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:55.399 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:55 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:55.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:55 smithi118 ceph-mon[35048]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:55.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:55 smithi118 ceph-mon[35048]: from='osd.10 [v2:172.21.15.118:6824/2167146341,v1:172.21.15.118:6825/2167146341]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:17:55.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:55 smithi118 ceph-mon[35048]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:17:55.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:55 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:55.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:55 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:55.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:55 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:55.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:55 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:55.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:55 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:55.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:55 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:55.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:55 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:55.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:55 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:55.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:55 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:55.514 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.481+0000 7f7966071640 1 -- 172.21.15.161:0/3326081961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79601026a0 msgr2=0x7f7960104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:55.514 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.481+0000 7f7966071640 1 --2- 172.21.15.161:0/3326081961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79601026a0 0x7f7960104a90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7954009900 tx=0x7f795402a3f0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.514 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.481+0000 7f7966071640 1 -- 172.21.15.161:0/3326081961 shutdown_connections 2024-09-12T03:17:55.514 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.481+0000 7f7966071640 1 --2- 172.21.15.161:0/3326081961 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7960107900 0x7f7960109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.514 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.481+0000 7f7966071640 1 --2- 172.21.15.161:0/3326081961 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7960104fd0 0x7f79601073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.514 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.481+0000 7f7966071640 1 --2- 172.21.15.161:0/3326081961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79601026a0 0x7f7960104a90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.514 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.481+0000 7f7966071640 1 -- 172.21.15.161:0/3326081961 >> 172.21.15.161:0/3326081961 conn(0x7f79600fc2a0 msgr2=0x7f79600fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:55.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.481+0000 7f7966071640 1 -- 172.21.15.161:0/3326081961 shutdown_connections 2024-09-12T03:17:55.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.481+0000 7f7966071640 1 -- 172.21.15.161:0/3326081961 wait complete. 2024-09-12T03:17:55.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.482+0000 7f7966071640 1 Processor -- start 2024-09-12T03:17:55.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.482+0000 7f7966071640 1 -- start start 2024-09-12T03:17:55.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.482+0000 7f7966071640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79601026a0 0x7f7960197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:55.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.482+0000 7f7966071640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7960104fd0 0x7f7960198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:55.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.482+0000 7f7966071640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7960107900 0x7f796019f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:55.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.482+0000 7f7966071640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f796010d8b0 con 0x7f79601026a0 2024-09-12T03:17:55.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.482+0000 7f7966071640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f796010d730 con 0x7f7960107900 2024-09-12T03:17:55.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.482+0000 7f7966071640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f796010da30 con 0x7f7960104fd0 2024-09-12T03:17:55.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.482+0000 7f795effd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7960104fd0 0x7f7960198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:55.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.482+0000 7f795effd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7960104fd0 0x7f7960198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33676/0 (socket says 172.21.15.161:33676) 2024-09-12T03:17:55.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.482+0000 7f795effd640 1 -- 172.21.15.161:0/3219627073 learned_addr learned my addr 172.21.15.161:0/3219627073 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-12T03:17:55.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.483+0000 7f795ffff640 1 --2- 172.21.15.161:0/3219627073 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7960107900 0x7f796019f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:55.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.483+0000 7f795f7fe640 1 --2- 172.21.15.161:0/3219627073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79601026a0 0x7f7960197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:55.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.483+0000 7f795effd640 1 -- 172.21.15.161:0/3219627073 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7960107900 msgr2=0x7f796019f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:55.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.483+0000 7f795effd640 1 --2- 172.21.15.161:0/3219627073 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7960107900 0x7f796019f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.483+0000 7f795effd640 1 -- 172.21.15.161:0/3219627073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79601026a0 msgr2=0x7f7960197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:55.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.483+0000 7f795effd640 1 --2- 172.21.15.161:0/3219627073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79601026a0 0x7f7960197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.483+0000 7f795effd640 1 -- 172.21.15.161:0/3219627073 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796019fa70 con 0x7f7960104fd0 2024-09-12T03:17:55.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.483+0000 7f795ffff640 1 --2- 172.21.15.161:0/3219627073 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7960107900 0x7f796019f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:55.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.483+0000 7f795f7fe640 1 --2- 172.21.15.161:0/3219627073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79601026a0 0x7f7960197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:55.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.483+0000 7f795effd640 1 --2- 172.21.15.161:0/3219627073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7960104fd0 0x7f7960198380 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f795000c970 tx=0x7f795000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:55.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.483+0000 7f795cff9640 1 -- 172.21.15.161:0/3219627073 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950014070 con 0x7f7960104fd0 2024-09-12T03:17:55.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.483+0000 7f795cff9640 1 -- 172.21.15.161:0/3219627073 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7950004440 con 0x7f7960104fd0 2024-09-12T03:17:55.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.483+0000 7f795cff9640 1 -- 172.21.15.161:0/3219627073 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795000f450 con 0x7f7960104fd0 2024-09-12T03:17:55.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.483+0000 7f7966071640 1 -- 172.21.15.161:0/3219627073 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f796019fd60 con 0x7f7960104fd0 2024-09-12T03:17:55.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.483+0000 7f7966071640 1 -- 172.21.15.161:0/3219627073 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79601a0240 con 0x7f7960104fd0 2024-09-12T03:17:55.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.484+0000 7f7966071640 1 -- 172.21.15.161:0/3219627073 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79600fdd90 con 0x7f7960104fd0 2024-09-12T03:17:55.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.485+0000 7f795cff9640 1 -- 172.21.15.161:0/3219627073 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f7950017070 con 0x7f7960104fd0 2024-09-12T03:17:55.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.486+0000 7f795cff9640 1 --2- 172.21.15.161:0/3219627073 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f79480a2bb0 0x7f79480a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:55.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.486+0000 7f795cff9640 1 -- 172.21.15.161:0/3219627073 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6398+0+0 (secure 0 0 0) 0x7f79500ce4c0 con 0x7f7960104fd0 2024-09-12T03:17:55.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.486+0000 7f795f7fe640 1 --2- 172.21.15.161:0/3219627073 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f79480a2bb0 0x7f79480a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:55.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.486+0000 7f795f7fe640 1 --2- 172.21.15.161:0/3219627073 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f79480a2bb0 0x7f79480a5070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7954000c00 tx=0x7f7954031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:55.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.490+0000 7f795cff9640 1 -- 172.21.15.161:0/3219627073 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f795001e030 con 0x7f7960104fd0 2024-09-12T03:17:55.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:55 smithi006 ceph-mon[30991]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:55 smithi006 ceph-mon[30991]: from='osd.10 [v2:172.21.15.118:6824/2167146341,v1:172.21.15.118:6825/2167146341]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:17:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:55 smithi006 ceph-mon[30991]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:17:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:55 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:55 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:55 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:55 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:55 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:55 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:55 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:55 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:55.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:55 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:55.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:17:55.665+0000 7f7966071640 1 -- 172.21.15.161:0/3219627073 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7960062630 con 0x7f79480a2bb0 2024-09-12T03:17:56.326 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:56 smithi161 ceph-mon[35485]: Detected new or changed devices on smithi118 2024-09-12T03:17:56.326 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:56 smithi161 ceph-mon[35485]: Adjusting osd_memory_target on smithi118 to 4272M 2024-09-12T03:17:56.326 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:56 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:56.326 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:56 smithi161 ceph-mon[35485]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T03:17:56.326 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:56 smithi161 ceph-mon[35485]: from='osd.10 [v2:172.21.15.118:6824/2167146341,v1:172.21.15.118:6825/2167146341]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:17:56.326 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:56 smithi161 ceph-mon[35485]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T03:17:56.326 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:56 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:17:56.326 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:56 smithi161 ceph-mon[35485]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:17:56.326 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:56 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:56.327 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:56 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:56.327 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:56 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:56.327 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:56 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:56.327 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:56 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:17:56.327 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:56 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:17:56.327 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:56 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:56.327 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:56 smithi161 ceph-mon[35485]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-12T03:17:56.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:56 smithi118 ceph-mon[35048]: Detected new or changed devices on smithi118 2024-09-12T03:17:56.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:56 smithi118 ceph-mon[35048]: Adjusting osd_memory_target on smithi118 to 4272M 2024-09-12T03:17:56.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:56 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:56.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:56 smithi118 ceph-mon[35048]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T03:17:56.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:56 smithi118 ceph-mon[35048]: from='osd.10 [v2:172.21.15.118:6824/2167146341,v1:172.21.15.118:6825/2167146341]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:17:56.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:56 smithi118 ceph-mon[35048]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T03:17:56.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:56 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:17:56.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:56 smithi118 ceph-mon[35048]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:17:56.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:56 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:56.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:56 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:56.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:56 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:56.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:56 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:56.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:56 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:17:56.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:56 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:17:56.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:56 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:56.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:56 smithi118 ceph-mon[35048]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-12T03:17:56.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:56 smithi006 ceph-mon[30991]: Detected new or changed devices on smithi118 2024-09-12T03:17:56.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:56 smithi006 ceph-mon[30991]: Adjusting osd_memory_target on smithi118 to 4272M 2024-09-12T03:17:56.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:56 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:56.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:56 smithi006 ceph-mon[30991]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T03:17:56.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:56 smithi006 ceph-mon[30991]: from='osd.10 [v2:172.21.15.118:6824/2167146341,v1:172.21.15.118:6825/2167146341]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:17:56.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:56 smithi006 ceph-mon[30991]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T03:17:56.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:56 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:17:56.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:56 smithi006 ceph-mon[30991]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-12T03:17:56.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:56 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:56.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:56 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:56.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:56 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:56.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:56 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:17:56.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:56 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:17:56.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:56 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:17:56.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:56 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:56.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:56 smithi006 ceph-mon[30991]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-12T03:17:57.172 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:17:56 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:17:56.782+0000 7fd3da902640 -1 osd.10 0 waiting for initial osdmap 2024-09-12T03:17:57.173 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:17:56 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:17:56.794+0000 7fd3d2f34640 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:17:57.282 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:57 smithi161 ceph-mon[35485]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:57.282 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:57 smithi161 ceph-mon[35485]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:57.282 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:57 smithi161 ceph-mon[35485]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T03:17:57.282 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:57 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:17:57.283 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:57 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:17:57.283 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:57 smithi161 ceph-mon[35485]: from='osd.10 ' entity='osd.10' 2024-09-12T03:17:57.283 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:57 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:17:57.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:57 smithi118 ceph-mon[35048]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:57.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:57 smithi118 ceph-mon[35048]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:57.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:57 smithi118 ceph-mon[35048]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T03:17:57.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:57 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:17:57.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:57 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:17:57.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:57 smithi118 ceph-mon[35048]: from='osd.10 ' entity='osd.10' 2024-09-12T03:17:57.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:57 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:17:57.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:57 smithi006 ceph-mon[30991]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:57.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:57 smithi006 ceph-mon[30991]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:57.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:57 smithi006 ceph-mon[30991]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T03:17:57.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:57 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:17:57.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:57 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:17:57.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:57 smithi006 ceph-mon[30991]: from='osd.10 ' entity='osd.10' 2024-09-12T03:17:57.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:57 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:17:58.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:58 smithi118 ceph-mon[35048]: purged_snaps scrub starts 2024-09-12T03:17:58.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:58 smithi118 ceph-mon[35048]: purged_snaps scrub ok 2024-09-12T03:17:58.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:58 smithi118 ceph-mon[35048]: osd.10 [v2:172.21.15.118:6824/2167146341,v1:172.21.15.118:6825/2167146341] boot 2024-09-12T03:17:58.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:58 smithi118 ceph-mon[35048]: osdmap e67: 11 total, 11 up, 11 in 2024-09-12T03:17:58.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:58 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:17:58.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:58 smithi118 ceph-mon[35048]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:58.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:58 smithi118 ceph-mon[35048]: from='client.? 172.21.15.161:0/3138965669' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a15c6a7-ff43-473c-bb48-0c129c316df5"}]: dispatch 2024-09-12T03:17:58.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:58 smithi118 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a15c6a7-ff43-473c-bb48-0c129c316df5"}]: dispatch 2024-09-12T03:17:58.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:58 smithi118 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a15c6a7-ff43-473c-bb48-0c129c316df5"}]': finished 2024-09-12T03:17:58.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:58 smithi118 ceph-mon[35048]: osdmap e68: 12 total, 11 up, 12 in 2024-09-12T03:17:58.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:58 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:17:58.588 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:58 smithi161 ceph-mon[35485]: purged_snaps scrub starts 2024-09-12T03:17:58.588 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:58 smithi161 ceph-mon[35485]: purged_snaps scrub ok 2024-09-12T03:17:58.588 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:58 smithi161 ceph-mon[35485]: osd.10 [v2:172.21.15.118:6824/2167146341,v1:172.21.15.118:6825/2167146341] boot 2024-09-12T03:17:58.588 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:58 smithi161 ceph-mon[35485]: osdmap e67: 11 total, 11 up, 11 in 2024-09-12T03:17:58.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:58 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:17:58.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:58 smithi161 ceph-mon[35485]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:58.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:58 smithi161 ceph-mon[35485]: from='client.? 172.21.15.161:0/3138965669' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a15c6a7-ff43-473c-bb48-0c129c316df5"}]: dispatch 2024-09-12T03:17:58.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:58 smithi161 ceph-mon[35485]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a15c6a7-ff43-473c-bb48-0c129c316df5"}]: dispatch 2024-09-12T03:17:58.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:58 smithi161 ceph-mon[35485]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a15c6a7-ff43-473c-bb48-0c129c316df5"}]': finished 2024-09-12T03:17:58.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:58 smithi161 ceph-mon[35485]: osdmap e68: 12 total, 11 up, 12 in 2024-09-12T03:17:58.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:58 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:17:58.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:58 smithi006 ceph-mon[30991]: purged_snaps scrub starts 2024-09-12T03:17:58.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:58 smithi006 ceph-mon[30991]: purged_snaps scrub ok 2024-09-12T03:17:58.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:58 smithi006 ceph-mon[30991]: osd.10 [v2:172.21.15.118:6824/2167146341,v1:172.21.15.118:6825/2167146341] boot 2024-09-12T03:17:58.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:58 smithi006 ceph-mon[30991]: osdmap e67: 11 total, 11 up, 11 in 2024-09-12T03:17:58.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:58 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:17:58.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:58 smithi006 ceph-mon[30991]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:17:58.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:58 smithi006 ceph-mon[30991]: from='client.? 172.21.15.161:0/3138965669' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a15c6a7-ff43-473c-bb48-0c129c316df5"}]: dispatch 2024-09-12T03:17:58.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:58 smithi006 ceph-mon[30991]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a15c6a7-ff43-473c-bb48-0c129c316df5"}]: dispatch 2024-09-12T03:17:58.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:58 smithi006 ceph-mon[30991]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a15c6a7-ff43-473c-bb48-0c129c316df5"}]': finished 2024-09-12T03:17:58.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:58 smithi006 ceph-mon[30991]: osdmap e68: 12 total, 11 up, 12 in 2024-09-12T03:17:58.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:58 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:17:59.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:59 smithi006 ceph-mon[30991]: from='client.? 172.21.15.161:0/3232716365' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:17:59.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:59 smithi006 ceph-mon[30991]: osdmap e69: 12 total, 11 up, 12 in 2024-09-12T03:17:59.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:17:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:17:59.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:59 smithi161 ceph-mon[35485]: from='client.? 172.21.15.161:0/3232716365' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:17:59.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:59 smithi161 ceph-mon[35485]: osdmap e69: 12 total, 11 up, 12 in 2024-09-12T03:17:59.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:17:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:17:59.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:59 smithi118 ceph-mon[35048]: from='client.? 172.21.15.161:0/3232716365' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:17:59.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:59 smithi118 ceph-mon[35048]: osdmap e69: 12 total, 11 up, 12 in 2024-09-12T03:17:59.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:17:59 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:01.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:01 smithi006 ceph-mon[30991]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:18:01.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:01 smithi006 ceph-mon[30991]: osdmap e70: 12 total, 11 up, 12 in 2024-09-12T03:18:01.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:01 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:01.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:01 smithi161 ceph-mon[35485]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:18:01.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:01 smithi161 ceph-mon[35485]: osdmap e70: 12 total, 11 up, 12 in 2024-09-12T03:18:01.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:01 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:01.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:01 smithi118 ceph-mon[35048]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:18:01.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:01 smithi118 ceph-mon[35048]: osdmap e70: 12 total, 11 up, 12 in 2024-09-12T03:18:01.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:01 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:02.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:02 smithi006 ceph-mon[30991]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T03:18:02.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:02 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:02.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:02 smithi161 ceph-mon[35485]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T03:18:02.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:02 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:02.363 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:02 smithi118 ceph-mon[35048]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T03:18:02.364 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:02 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:03.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:03 smithi006 ceph-mon[30991]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:18:03.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:03 smithi161 ceph-mon[35485]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:18:03.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:03 smithi118 ceph-mon[35048]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:18:05.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:05 smithi161 ceph-mon[35485]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-12T03:18:05.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:05 smithi006 ceph-mon[30991]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-12T03:18:05.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:05 smithi118 ceph-mon[35048]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-12T03:18:07.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:07 smithi006 ceph-mon[30991]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-12T03:18:07.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:07 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T03:18:07.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:07 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:07.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:07 smithi161 ceph-mon[35485]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-12T03:18:07.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:07 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T03:18:07.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:07 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:07 smithi118 ceph-mon[35048]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-12T03:18:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:07 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T03:18:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:07 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:08.103 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:08 smithi161 ceph-mon[35485]: Deploying daemon osd.11 on smithi161 2024-09-12T03:18:08.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:08 smithi006 ceph-mon[30991]: Deploying daemon osd.11 on smithi161 2024-09-12T03:18:08.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:08 smithi118 ceph-mon[35048]: Deploying daemon osd.11 on smithi161 2024-09-12T03:18:09.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:09 smithi006 ceph-mon[30991]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T03:18:09.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:09 smithi161 ceph-mon[35485]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T03:18:09.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:09 smithi118 ceph-mon[35048]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T03:18:11.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:11 smithi161 ceph-mon[35485]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-12T03:18:11.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:11 smithi006 ceph-mon[30991]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-12T03:18:11.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:11 smithi118 ceph-mon[35048]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-12T03:18:13.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:13 smithi006 ceph-mon[30991]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-12T03:18:13.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:13 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:13.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:13 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:13.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:13 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:13.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:13 smithi161 ceph-mon[35485]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-12T03:18:13.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:13 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:13.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:13 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:13.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:13 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:13.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:13 smithi118 ceph-mon[35048]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-12T03:18:13.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:13 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:13.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:13 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:13.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:13 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:15.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:15 smithi006 ceph-mon[30991]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:18:15.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:15 smithi161 ceph-mon[35485]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:18:15.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:15 smithi118 ceph-mon[35048]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:18:16.029 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 11 on host 'smithi161' 2024-09-12T03:18:16.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:16.027+0000 7f795cff9640 1 -- 172.21.15.161:0/3219627073 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7960062630 con 0x7f79480a2bb0 2024-09-12T03:18:16.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:16.029+0000 7f7966071640 1 -- 172.21.15.161:0/3219627073 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f79480a2bb0 msgr2=0x7f79480a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:16.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:16.029+0000 7f7966071640 1 --2- 172.21.15.161:0/3219627073 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f79480a2bb0 0x7f79480a5070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7954000c00 tx=0x7f7954031400 comp rx=0 tx=0).stop 2024-09-12T03:18:16.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:16.029+0000 7f7966071640 1 -- 172.21.15.161:0/3219627073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7960104fd0 msgr2=0x7f7960198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:16.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:16.029+0000 7f7966071640 1 --2- 172.21.15.161:0/3219627073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7960104fd0 0x7f7960198380 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f795000c970 tx=0x7f795000ce30 comp rx=0 tx=0).stop 2024-09-12T03:18:16.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:16.029+0000 7f7966071640 1 -- 172.21.15.161:0/3219627073 shutdown_connections 2024-09-12T03:18:16.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:16.029+0000 7f7966071640 1 --2- 172.21.15.161:0/3219627073 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f79480a2bb0 0x7f79480a5070 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:16.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:16.029+0000 7f7966071640 1 --2- 172.21.15.161:0/3219627073 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7960107900 0x7f796019f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:16.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:16.029+0000 7f7966071640 1 --2- 172.21.15.161:0/3219627073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7960104fd0 0x7f7960198380 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:16.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:16.029+0000 7f7966071640 1 --2- 172.21.15.161:0/3219627073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79601026a0 0x7f7960197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:16.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:16.029+0000 7f7966071640 1 -- 172.21.15.161:0/3219627073 >> 172.21.15.161:0/3219627073 conn(0x7f79600fc2a0 msgr2=0x7f7960102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:16.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:16.030+0000 7f7966071640 1 -- 172.21.15.161:0/3219627073 shutdown_connections 2024-09-12T03:18:16.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:16.030+0000 7f7966071640 1 -- 172.21.15.161:0/3219627073 wait complete. 2024-09-12T03:18:16.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:16 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:16.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:16 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:16.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:16 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:16.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:16 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:16.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:16 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:16.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:16 smithi161 ceph-mon[35485]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:18:16.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:16 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:16.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:16 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:16.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:16 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:16.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:16 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:16.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:16 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:16.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:16 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:16.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:16 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:16.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:16 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:16.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:16 smithi118 ceph-mon[35048]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:18:16.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:16 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:16.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:16 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:16.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:16 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:16.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:16 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:16.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:16 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:16.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:16 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:16.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:16 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:16.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:16 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:16.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:16 smithi006 ceph-mon[30991]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:18:16.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:16 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:16.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:16 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:16.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:16 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:16.779 DEBUG:teuthology.orchestra.run.smithi161:osd.11> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.11.service 2024-09-12T03:18:16.781 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-12T03:18:16.782 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T03:18:17.049 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:17.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.495+0000 7f1db1a68640 1 -- 172.21.15.6:0/3660694561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1dac076220 msgr2=0x7f1dac074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:17.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.495+0000 7f1db1a68640 1 --2- 172.21.15.6:0/3660694561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1dac076220 0x7f1dac074680 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009900 tx=0x7f1d9c02a410 comp rx=0 tx=0).stop 2024-09-12T03:18:17.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.495+0000 7f1db1a68640 1 -- 172.21.15.6:0/3660694561 shutdown_connections 2024-09-12T03:18:17.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.495+0000 7f1db1a68640 1 --2- 172.21.15.6:0/3660694561 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1dac109a80 0x7f1dac10bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.495+0000 7f1db1a68640 1 --2- 172.21.15.6:0/3660694561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dac074c50 0x7f1dac0750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.495+0000 7f1db1a68640 1 --2- 172.21.15.6:0/3660694561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1dac076220 0x7f1dac074680 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.495+0000 7f1db1a68640 1 -- 172.21.15.6:0/3660694561 >> 172.21.15.6:0/3660694561 conn(0x7f1dac0fd390 msgr2=0x7f1dac0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:17.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.495+0000 7f1db1a68640 1 -- 172.21.15.6:0/3660694561 shutdown_connections 2024-09-12T03:18:17.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.495+0000 7f1db1a68640 1 -- 172.21.15.6:0/3660694561 wait complete. 2024-09-12T03:18:17.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.495+0000 7f1db1a68640 1 Processor -- start 2024-09-12T03:18:17.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.496+0000 7f1db1a68640 1 -- start start 2024-09-12T03:18:17.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.496+0000 7f1db1a68640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1dac074c50 0x7f1dac19a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:17.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.496+0000 7f1db1a68640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1dac076220 0x7f1dac19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:17.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.496+0000 7f1db1a68640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dac109a80 0x7f1dac1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:17.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.496+0000 7f1db1a68640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac10fae0 con 0x7f1dac074c50 2024-09-12T03:18:17.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.496+0000 7f1db1a68640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac10f960 con 0x7f1dac076220 2024-09-12T03:18:17.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.496+0000 7f1db1a68640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac10fc60 con 0x7f1dac109a80 2024-09-12T03:18:17.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.496+0000 7f1dab7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1dac074c50 0x7f1dac19a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.496+0000 7f1dab7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1dac074c50 0x7f1dac19a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33228/0 (socket says 172.21.15.6:33228) 2024-09-12T03:18:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.496+0000 7f1dab7fe640 1 -- 172.21.15.6:0/4150599404 learned_addr learned my addr 172.21.15.6:0/4150599404 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:18:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.496+0000 7f1daaffd640 1 --2- 172.21.15.6:0/4150599404 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1dac076220 0x7f1dac19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.496+0000 7f1dabfff640 1 --2- 172.21.15.6:0/4150599404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dac109a80 0x7f1dac1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.496+0000 7f1dab7fe640 1 -- 172.21.15.6:0/4150599404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dac109a80 msgr2=0x7f1dac1a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.496+0000 7f1dab7fe640 1 --2- 172.21.15.6:0/4150599404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dac109a80 0x7f1dac1a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.496+0000 7f1dab7fe640 1 -- 172.21.15.6:0/4150599404 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1dac076220 msgr2=0x7f1dac19a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.496+0000 7f1dab7fe640 1 --2- 172.21.15.6:0/4150599404 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1dac076220 0x7f1dac19a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.496+0000 7f1dab7fe640 1 -- 172.21.15.6:0/4150599404 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dac1a1d70 con 0x7f1dac074c50 2024-09-12T03:18:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.497+0000 7f1dab7fe640 1 --2- 172.21.15.6:0/4150599404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1dac074c50 0x7f1dac19a140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c031610 tx=0x7f1d9c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.497+0000 7f1da8ff9640 1 -- 172.21.15.6:0/4150599404 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c03f030 con 0x7f1dac074c50 2024-09-12T03:18:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.497+0000 7f1da8ff9640 1 -- 172.21.15.6:0/4150599404 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d9c003ce0 con 0x7f1dac074c50 2024-09-12T03:18:17.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.497+0000 7f1db1a68640 1 -- 172.21.15.6:0/4150599404 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dac1a2000 con 0x7f1dac074c50 2024-09-12T03:18:17.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.497+0000 7f1da8ff9640 1 -- 172.21.15.6:0/4150599404 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c035070 con 0x7f1dac074c50 2024-09-12T03:18:17.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.497+0000 7f1db1a68640 1 -- 172.21.15.6:0/4150599404 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dac1a2560 con 0x7f1dac074c50 2024-09-12T03:18:17.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.498+0000 7f1db1a68640 1 -- 172.21.15.6:0/4150599404 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dac108800 con 0x7f1dac074c50 2024-09-12T03:18:17.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.499+0000 7f1da8ff9640 1 -- 172.21.15.6:0/4150599404 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f1d9c042030 con 0x7f1dac074c50 2024-09-12T03:18:17.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.500+0000 7f1da8ff9640 1 --2- 172.21.15.6:0/4150599404 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1d940a2bb0 0x7f1d940a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:17.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.500+0000 7f1daaffd640 1 --2- 172.21.15.6:0/4150599404 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1d940a2bb0 0x7f1d940a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:17.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.500+0000 7f1da8ff9640 1 -- 172.21.15.6:0/4150599404 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6809+0+0 (secure 0 0 0) 0x7f1d9c0eefb0 con 0x7f1dac074c50 2024-09-12T03:18:17.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.501+0000 7f1daaffd640 1 --2- 172.21.15.6:0/4150599404 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1d940a2bb0 0x7f1d940a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1dac19b660 tx=0x7f1da0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:17.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.504+0000 7f1da8ff9640 1 -- 172.21.15.6:0/4150599404 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d9c0bc450 con 0x7f1dac074c50 2024-09-12T03:18:17.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.678+0000 7f1db1a68640 1 -- 172.21.15.6:0/4150599404 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f1dac062df0 con 0x7f1dac074c50 2024-09-12T03:18:17.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.679+0000 7f1da8ff9640 1 -- 172.21.15.6:0/4150599404 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f1d9c0bbd50 con 0x7f1dac074c50 2024-09-12T03:18:17.679 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:18:17.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.681+0000 7f1db1a68640 1 -- 172.21.15.6:0/4150599404 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1d940a2bb0 msgr2=0x7f1d940a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:17.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.681+0000 7f1db1a68640 1 --2- 172.21.15.6:0/4150599404 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1d940a2bb0 0x7f1d940a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1dac19b660 tx=0x7f1da0009110 comp rx=0 tx=0).stop 2024-09-12T03:18:17.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.681+0000 7f1db1a68640 1 -- 172.21.15.6:0/4150599404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1dac074c50 msgr2=0x7f1dac19a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:17.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.681+0000 7f1db1a68640 1 --2- 172.21.15.6:0/4150599404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1dac074c50 0x7f1dac19a140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c031610 tx=0x7f1d9c0043b0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.681+0000 7f1db1a68640 1 -- 172.21.15.6:0/4150599404 shutdown_connections 2024-09-12T03:18:17.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.681+0000 7f1db1a68640 1 --2- 172.21.15.6:0/4150599404 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1d940a2bb0 0x7f1d940a5070 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.681+0000 7f1db1a68640 1 --2- 172.21.15.6:0/4150599404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dac109a80 0x7f1dac1a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.681+0000 7f1db1a68640 1 --2- 172.21.15.6:0/4150599404 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1dac076220 0x7f1dac19a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.681+0000 7f1db1a68640 1 --2- 172.21.15.6:0/4150599404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1dac074c50 0x7f1dac19a140 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.681+0000 7f1db1a68640 1 -- 172.21.15.6:0/4150599404 >> 172.21.15.6:0/4150599404 conn(0x7f1dac0fd390 msgr2=0x7f1dac1059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:17.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.681+0000 7f1db1a68640 1 -- 172.21.15.6:0/4150599404 shutdown_connections 2024-09-12T03:18:17.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:17.681+0000 7f1db1a68640 1 -- 172.21.15.6:0/4150599404 wait complete. 2024-09-12T03:18:18.379 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1726111077,"num_in_osds":12,"osd_in_since":1726111078,"num_remapped_pgs":0} 2024-09-12T03:18:18.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:18 smithi006 ceph-mon[30991]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:18:18.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:18 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/4150599404' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:18:18.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:18 smithi161 ceph-mon[35485]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:18:18.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:18 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/4150599404' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:18:18.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:18:18 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:18:18.396+0000 7f57e31d3640 -1 osd.11 0 log_to_monitors true 2024-09-12T03:18:18.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:18 smithi118 ceph-mon[35048]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:18:18.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:18 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/4150599404' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:18:19.381 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T03:18:19.548 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:19 smithi161 ceph-mon[35485]: from='osd.11 [v2:172.21.15.161:6824/2889795796,v1:172.21.15.161:6825/2889795796]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:18:19.549 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:19 smithi161 ceph-mon[35485]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:18:19.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:19 smithi006 ceph-mon[30991]: from='osd.11 [v2:172.21.15.161:6824/2889795796,v1:172.21.15.161:6825/2889795796]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:18:19.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:19 smithi006 ceph-mon[30991]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:18:19.653 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:19.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:19 smithi118 ceph-mon[35048]: from='osd.11 [v2:172.21.15.161:6824/2889795796,v1:172.21.15.161:6825/2889795796]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:18:19.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:19 smithi118 ceph-mon[35048]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:18:20.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.092+0000 7f25325b9640 1 -- 172.21.15.6:0/1539960485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f252c107900 msgr2=0x7f252c109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:20.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.092+0000 7f25325b9640 1 --2- 172.21.15.6:0/1539960485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f252c107900 0x7f252c109d50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2524009ca0 tx=0x7f252402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.092+0000 7f25325b9640 1 -- 172.21.15.6:0/1539960485 shutdown_connections 2024-09-12T03:18:20.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.092+0000 7f25325b9640 1 --2- 172.21.15.6:0/1539960485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f252c107900 0x7f252c109d50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.092+0000 7f25325b9640 1 --2- 172.21.15.6:0/1539960485 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f252c104fd0 0x7f252c1073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.092+0000 7f25325b9640 1 --2- 172.21.15.6:0/1539960485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f252c1026a0 0x7f252c104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.092+0000 7f25325b9640 1 -- 172.21.15.6:0/1539960485 >> 172.21.15.6:0/1539960485 conn(0x7f252c0fc2a0 msgr2=0x7f252c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:20.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.092+0000 7f25325b9640 1 -- 172.21.15.6:0/1539960485 shutdown_connections 2024-09-12T03:18:20.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.092+0000 7f25325b9640 1 -- 172.21.15.6:0/1539960485 wait complete. 2024-09-12T03:18:20.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.092+0000 7f25325b9640 1 Processor -- start 2024-09-12T03:18:20.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.093+0000 7f25325b9640 1 -- start start 2024-09-12T03:18:20.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.093+0000 7f25325b9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f252c1026a0 0x7f252c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:20.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.093+0000 7f25325b9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f252c104fd0 0x7f252c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:20.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.093+0000 7f25325b9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f252c107900 0x7f252c19f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:20.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.093+0000 7f25325b9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f252c10d8c0 con 0x7f252c1026a0 2024-09-12T03:18:20.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.093+0000 7f25325b9640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f252c10d740 con 0x7f252c104fd0 2024-09-12T03:18:20.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.093+0000 7f25325b9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f252c10da40 con 0x7f252c107900 2024-09-12T03:18:20.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.093+0000 7f252bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f252c1026a0 0x7f252c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:20.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.093+0000 7f252bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f252c1026a0 0x7f252c197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52940/0 (socket says 172.21.15.6:52940) 2024-09-12T03:18:20.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.093+0000 7f252bfff640 1 -- 172.21.15.6:0/1461232295 learned_addr learned my addr 172.21.15.6:0/1461232295 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:18:20.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.093+0000 7f2530b56640 1 --2- 172.21.15.6:0/1461232295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f252c107900 0x7f252c19f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:20.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.093+0000 7f252b7fe640 1 --2- 172.21.15.6:0/1461232295 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f252c104fd0 0x7f252c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:20.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.093+0000 7f252bfff640 1 -- 172.21.15.6:0/1461232295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f252c107900 msgr2=0x7f252c19f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:20.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.093+0000 7f252bfff640 1 --2- 172.21.15.6:0/1461232295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f252c107900 0x7f252c19f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.093+0000 7f252bfff640 1 -- 172.21.15.6:0/1461232295 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f252c104fd0 msgr2=0x7f252c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:20.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.093+0000 7f252bfff640 1 --2- 172.21.15.6:0/1461232295 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f252c104fd0 0x7f252c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.093+0000 7f252bfff640 1 -- 172.21.15.6:0/1461232295 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252c19fb20 con 0x7f252c1026a0 2024-09-12T03:18:20.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.094+0000 7f252bfff640 1 --2- 172.21.15.6:0/1461232295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f252c1026a0 0x7f252c197ef0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f251c00d770 tx=0x7f251c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:20.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.094+0000 7f25297fa640 1 -- 172.21.15.6:0/1461232295 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c017070 con 0x7f252c1026a0 2024-09-12T03:18:20.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.094+0000 7f25297fa640 1 -- 172.21.15.6:0/1461232295 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f251c005500 con 0x7f252c1026a0 2024-09-12T03:18:20.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.094+0000 7f25325b9640 1 -- 172.21.15.6:0/1461232295 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f252c19fe10 con 0x7f252c1026a0 2024-09-12T03:18:20.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.094+0000 7f25297fa640 1 -- 172.21.15.6:0/1461232295 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c004e00 con 0x7f252c1026a0 2024-09-12T03:18:20.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.094+0000 7f25325b9640 1 -- 172.21.15.6:0/1461232295 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f252c1a0260 con 0x7f252c1026a0 2024-09-12T03:18:20.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.095+0000 7f25325b9640 1 -- 172.21.15.6:0/1461232295 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f252c0fdd90 con 0x7f252c1026a0 2024-09-12T03:18:20.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.097+0000 7f25297fa640 1 -- 172.21.15.6:0/1461232295 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f251c004fa0 con 0x7f252c1026a0 2024-09-12T03:18:20.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.098+0000 7f25297fa640 1 --2- 172.21.15.6:0/1461232295 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f25140a2c30 0x7f25140a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:20.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.098+0000 7f252b7fe640 1 --2- 172.21.15.6:0/1461232295 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f25140a2c30 0x7f25140a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:20.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.098+0000 7f25297fa640 1 -- 172.21.15.6:0/1461232295 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6831+0+0 (secure 0 0 0) 0x7f251c0ce890 con 0x7f252c1026a0 2024-09-12T03:18:20.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.098+0000 7f252b7fe640 1 --2- 172.21.15.6:0/1461232295 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f25140a2c30 0x7f25140a50f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f252c199410 tx=0x7f252000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:20.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.101+0000 7f25297fa640 1 -- 172.21.15.6:0/1461232295 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f251c09be20 con 0x7f252c1026a0 2024-09-12T03:18:20.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.275+0000 7f25325b9640 1 -- 172.21.15.6:0/1461232295 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f252c02da10 con 0x7f252c1026a0 2024-09-12T03:18:20.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.276+0000 7f25297fa640 1 -- 172.21.15.6:0/1461232295 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f251c09b720 con 0x7f252c1026a0 2024-09-12T03:18:20.276 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:18:20.278 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.278+0000 7f25325b9640 1 -- 172.21.15.6:0/1461232295 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f25140a2c30 msgr2=0x7f25140a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:20.278 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.278+0000 7f25325b9640 1 --2- 172.21.15.6:0/1461232295 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f25140a2c30 0x7f25140a50f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f252c199410 tx=0x7f252000a4c0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.278 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.278+0000 7f25325b9640 1 -- 172.21.15.6:0/1461232295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f252c1026a0 msgr2=0x7f252c197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:20.278 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.278+0000 7f25325b9640 1 --2- 172.21.15.6:0/1461232295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f252c1026a0 0x7f252c197ef0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f251c00d770 tx=0x7f251c00dc30 comp rx=0 tx=0).stop 2024-09-12T03:18:20.278 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.278+0000 7f25325b9640 1 -- 172.21.15.6:0/1461232295 shutdown_connections 2024-09-12T03:18:20.278 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.278+0000 7f25325b9640 1 --2- 172.21.15.6:0/1461232295 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f25140a2c30 0x7f25140a50f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.278+0000 7f25325b9640 1 --2- 172.21.15.6:0/1461232295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f252c107900 0x7f252c19f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.278+0000 7f25325b9640 1 --2- 172.21.15.6:0/1461232295 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f252c104fd0 0x7f252c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.278+0000 7f25325b9640 1 --2- 172.21.15.6:0/1461232295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f252c1026a0 0x7f252c197ef0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.278+0000 7f25325b9640 1 -- 172.21.15.6:0/1461232295 >> 172.21.15.6:0/1461232295 conn(0x7f252c0fc2a0 msgr2=0x7f252c102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:20.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.278+0000 7f25325b9640 1 -- 172.21.15.6:0/1461232295 shutdown_connections 2024-09-12T03:18:20.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:20.278+0000 7f25325b9640 1 -- 172.21.15.6:0/1461232295 wait complete. 2024-09-12T03:18:20.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:20 smithi006 ceph-mon[30991]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:18:20.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:20 smithi006 ceph-mon[30991]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T03:18:20.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:20 smithi006 ceph-mon[30991]: from='osd.11 [v2:172.21.15.161:6824/2889795796,v1:172.21.15.161:6825/2889795796]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:18:20.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:20 smithi006 ceph-mon[30991]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T03:18:20.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:20.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:20 smithi006 ceph-mon[30991]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:18:20.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:20 smithi006 ceph-mon[30991]: Detected new or changed devices on smithi161 2024-09-12T03:18:20.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:20.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:20.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:18:20.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:18:20.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:18:20.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:18:20.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:20 smithi006 ceph-mon[30991]: Adjusting osd_memory_target on smithi161 to 4272M 2024-09-12T03:18:20.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:20.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:20.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:20.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:20.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:20 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1461232295' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:18:20.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:18:20 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:18:20.296+0000 7f57de933640 -1 osd.11 0 waiting for initial osdmap 2024-09-12T03:18:20.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:18:20 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:18:20.310+0000 7f57daf6d640 -1 osd.11 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:18:20.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:20 smithi161 ceph-mon[35485]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:18:20.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:20 smithi161 ceph-mon[35485]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T03:18:20.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:20 smithi161 ceph-mon[35485]: from='osd.11 [v2:172.21.15.161:6824/2889795796,v1:172.21.15.161:6825/2889795796]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:18:20.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:20 smithi161 ceph-mon[35485]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T03:18:20.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:20.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:20 smithi161 ceph-mon[35485]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:18:20.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:20 smithi161 ceph-mon[35485]: Detected new or changed devices on smithi161 2024-09-12T03:18:20.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:20.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:20.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:18:20.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:18:20.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:18:20.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:18:20.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:20 smithi161 ceph-mon[35485]: Adjusting osd_memory_target on smithi161 to 4272M 2024-09-12T03:18:20.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:20.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:20.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:20.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:20.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:20 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1461232295' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:18:20.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:20 smithi118 ceph-mon[35048]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:18:20.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:20 smithi118 ceph-mon[35048]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T03:18:20.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:20 smithi118 ceph-mon[35048]: from='osd.11 [v2:172.21.15.161:6824/2889795796,v1:172.21.15.161:6825/2889795796]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:18:20.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:20 smithi118 ceph-mon[35048]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T03:18:20.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:20.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:20 smithi118 ceph-mon[35048]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-12T03:18:20.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:20 smithi118 ceph-mon[35048]: Detected new or changed devices on smithi161 2024-09-12T03:18:20.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:20.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:20.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:18:20.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:18:20.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:18:20.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:18:20.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:20 smithi118 ceph-mon[35048]: Adjusting osd_memory_target on smithi161 to 4272M 2024-09-12T03:18:20.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:20.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:20.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:20.714 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:20.714 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:20 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1461232295' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:18:20.964 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1726111077,"num_in_osds":12,"osd_in_since":1726111078,"num_remapped_pgs":0} 2024-09-12T03:18:21.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:21 smithi006 ceph-mon[30991]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-12T03:18:21.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:21 smithi006 ceph-mon[30991]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T03:18:21.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:21 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:21.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:21 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:21.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:21 smithi161 ceph-mon[35485]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-12T03:18:21.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:21 smithi161 ceph-mon[35485]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T03:18:21.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:21 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:21.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:21 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:21.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:21 smithi118 ceph-mon[35048]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-12T03:18:21.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:21 smithi118 ceph-mon[35048]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T03:18:21.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:21 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:21.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:21 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:21.966 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T03:18:22.233 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:22.568 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:22 smithi006 ceph-mon[30991]: purged_snaps scrub starts 2024-09-12T03:18:22.568 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:22 smithi006 ceph-mon[30991]: purged_snaps scrub ok 2024-09-12T03:18:22.569 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:22 smithi006 ceph-mon[30991]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:18:22.569 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:22 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:22.569 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:22 smithi006 ceph-mon[30991]: osd.11 [v2:172.21.15.161:6824/2889795796,v1:172.21.15.161:6825/2889795796] boot 2024-09-12T03:18:22.569 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:22 smithi006 ceph-mon[30991]: osdmap e74: 12 total, 12 up, 12 in 2024-09-12T03:18:22.569 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:22 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:22.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:22 smithi161 ceph-mon[35485]: purged_snaps scrub starts 2024-09-12T03:18:22.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:22 smithi161 ceph-mon[35485]: purged_snaps scrub ok 2024-09-12T03:18:22.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:22 smithi161 ceph-mon[35485]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:18:22.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:22 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:22.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:22 smithi161 ceph-mon[35485]: osd.11 [v2:172.21.15.161:6824/2889795796,v1:172.21.15.161:6825/2889795796] boot 2024-09-12T03:18:22.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:22 smithi161 ceph-mon[35485]: osdmap e74: 12 total, 12 up, 12 in 2024-09-12T03:18:22.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:22 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:22.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.651+0000 7fa3c5abd640 1 -- 172.21.15.6:0/2301506671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3c0101c10 msgr2=0x7fa3c0102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:22.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.651+0000 7fa3c5abd640 1 --2- 172.21.15.6:0/2301506671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3c0101c10 0x7fa3c0102010 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009ca0 tx=0x7fa3b002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.651+0000 7fa3c5abd640 1 -- 172.21.15.6:0/2301506671 shutdown_connections 2024-09-12T03:18:22.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.651+0000 7fa3c5abd640 1 --2- 172.21.15.6:0/2301506671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3c01037d0 0x7fa3c010a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.651+0000 7fa3c5abd640 1 --2- 172.21.15.6:0/2301506671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3c0102e10 0x7fa3c0103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.651+0000 7fa3c5abd640 1 --2- 172.21.15.6:0/2301506671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3c0101c10 0x7fa3c0102010 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.651+0000 7fa3c5abd640 1 -- 172.21.15.6:0/2301506671 >> 172.21.15.6:0/2301506671 conn(0x7fa3c00fd3c0 msgr2=0x7fa3c00ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:22.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.651+0000 7fa3c5abd640 1 -- 172.21.15.6:0/2301506671 shutdown_connections 2024-09-12T03:18:22.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.651+0000 7fa3c5abd640 1 -- 172.21.15.6:0/2301506671 wait complete. 2024-09-12T03:18:22.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.651+0000 7fa3c5abd640 1 Processor -- start 2024-09-12T03:18:22.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.651+0000 7fa3c5abd640 1 -- start start 2024-09-12T03:18:22.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.652+0000 7fa3c5abd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3c0101c10 0x7fa3c0197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.652+0000 7fa3c5abd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3c0102e10 0x7fa3c0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.652+0000 7fa3c5abd640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3c01037d0 0x7fa3c019f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.652+0000 7fa3c5abd640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c010d870 con 0x7fa3c0101c10 2024-09-12T03:18:22.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.652+0000 7fa3c5abd640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c010d6f0 con 0x7fa3c01037d0 2024-09-12T03:18:22.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.652+0000 7fa3c5abd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c010d9f0 con 0x7fa3c0102e10 2024-09-12T03:18:22.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.652+0000 7fa3bf7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3c0101c10 0x7fa3c0197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.652+0000 7fa3bf7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3c0101c10 0x7fa3c0197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52964/0 (socket says 172.21.15.6:52964) 2024-09-12T03:18:22.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.652+0000 7fa3bf7fe640 1 -- 172.21.15.6:0/876511290 learned_addr learned my addr 172.21.15.6:0/876511290 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:18:22.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.652+0000 7fa3beffd640 1 --2- 172.21.15.6:0/876511290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3c0102e10 0x7fa3c0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.652+0000 7fa3bffff640 1 --2- 172.21.15.6:0/876511290 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3c01037d0 0x7fa3c019f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.652+0000 7fa3bf7fe640 1 -- 172.21.15.6:0/876511290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3c0102e10 msgr2=0x7fa3c0198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:22.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.652+0000 7fa3bf7fe640 1 --2- 172.21.15.6:0/876511290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3c0102e10 0x7fa3c0198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.652+0000 7fa3bf7fe640 1 -- 172.21.15.6:0/876511290 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3c01037d0 msgr2=0x7fa3c019f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:22.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.652+0000 7fa3bf7fe640 1 --2- 172.21.15.6:0/876511290 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3c01037d0 0x7fa3c019f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.652+0000 7fa3bf7fe640 1 -- 172.21.15.6:0/876511290 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c019fb30 con 0x7fa3c0101c10 2024-09-12T03:18:22.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.652+0000 7fa3bf7fe640 1 --2- 172.21.15.6:0/876511290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3c0101c10 0x7fa3c0197f00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0004af0 tx=0x7fa3b0031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:22.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.652+0000 7fa3bcff9640 1 -- 172.21.15.6:0/876511290 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b002fb60 con 0x7fa3c0101c10 2024-09-12T03:18:22.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.653+0000 7fa3c5abd640 1 -- 172.21.15.6:0/876511290 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c019fdc0 con 0x7fa3c0101c10 2024-09-12T03:18:22.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.653+0000 7fa3bcff9640 1 -- 172.21.15.6:0/876511290 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3b002fd00 con 0x7fa3c0101c10 2024-09-12T03:18:22.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.653+0000 7fa3bcff9640 1 -- 172.21.15.6:0/876511290 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0046920 con 0x7fa3c0101c10 2024-09-12T03:18:22.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.653+0000 7fa3c5abd640 1 -- 172.21.15.6:0/876511290 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c01a0220 con 0x7fa3c0101c10 2024-09-12T03:18:22.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.654+0000 7fa3c5abd640 1 -- 172.21.15.6:0/876511290 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c0109000 con 0x7fa3c0101c10 2024-09-12T03:18:22.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.655+0000 7fa3bcff9640 1 -- 172.21.15.6:0/876511290 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fa3b0030c30 con 0x7fa3c0101c10 2024-09-12T03:18:22.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.655+0000 7fa3bcff9640 1 --2- 172.21.15.6:0/876511290 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa3a80a2bb0 0x7fa3a80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.655+0000 7fa3beffd640 1 --2- 172.21.15.6:0/876511290 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa3a80a2bb0 0x7fa3a80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.656+0000 7fa3bcff9640 1 -- 172.21.15.6:0/876511290 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fa3b00effe0 con 0x7fa3c0101c10 2024-09-12T03:18:22.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.656+0000 7fa3beffd640 1 --2- 172.21.15.6:0/876511290 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa3a80a2bb0 0x7fa3a80a5070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0199420 tx=0x7fa3b400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:22.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.660+0000 7fa3bcff9640 1 -- 172.21.15.6:0/876511290 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa3b00bd3e0 con 0x7fa3c0101c10 2024-09-12T03:18:22.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:22 smithi118 ceph-mon[35048]: purged_snaps scrub starts 2024-09-12T03:18:22.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:22 smithi118 ceph-mon[35048]: purged_snaps scrub ok 2024-09-12T03:18:22.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:22 smithi118 ceph-mon[35048]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:18:22.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:22 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:22.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:22 smithi118 ceph-mon[35048]: osd.11 [v2:172.21.15.161:6824/2889795796,v1:172.21.15.161:6825/2889795796] boot 2024-09-12T03:18:22.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:22 smithi118 ceph-mon[35048]: osdmap e74: 12 total, 12 up, 12 in 2024-09-12T03:18:22.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:22 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:18:22.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.836+0000 7fa3c5abd640 1 -- 172.21.15.6:0/876511290 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fa3c002da10 con 0x7fa3c0101c10 2024-09-12T03:18:22.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.837+0000 7fa3bcff9640 1 -- 172.21.15.6:0/876511290 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7fa3b00bcce0 con 0x7fa3c0101c10 2024-09-12T03:18:22.837 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:18:22.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.840+0000 7fa3c5abd640 1 -- 172.21.15.6:0/876511290 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa3a80a2bb0 msgr2=0x7fa3a80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:22.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.840+0000 7fa3c5abd640 1 --2- 172.21.15.6:0/876511290 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa3a80a2bb0 0x7fa3a80a5070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0199420 tx=0x7fa3b400b390 comp rx=0 tx=0).stop 2024-09-12T03:18:22.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.840+0000 7fa3c5abd640 1 -- 172.21.15.6:0/876511290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3c0101c10 msgr2=0x7fa3c0197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:22.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.840+0000 7fa3c5abd640 1 --2- 172.21.15.6:0/876511290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3c0101c10 0x7fa3c0197f00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0004af0 tx=0x7fa3b0031ee0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.840+0000 7fa3c5abd640 1 -- 172.21.15.6:0/876511290 shutdown_connections 2024-09-12T03:18:22.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.840+0000 7fa3c5abd640 1 --2- 172.21.15.6:0/876511290 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa3a80a2bb0 0x7fa3a80a5070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.840+0000 7fa3c5abd640 1 --2- 172.21.15.6:0/876511290 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3c01037d0 0x7fa3c019f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.840+0000 7fa3c5abd640 1 --2- 172.21.15.6:0/876511290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3c0102e10 0x7fa3c0198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.840+0000 7fa3c5abd640 1 --2- 172.21.15.6:0/876511290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3c0101c10 0x7fa3c0197f00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.840+0000 7fa3c5abd640 1 -- 172.21.15.6:0/876511290 >> 172.21.15.6:0/876511290 conn(0x7fa3c00fd3c0 msgr2=0x7fa3c01061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:22.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.840+0000 7fa3c5abd640 1 -- 172.21.15.6:0/876511290 shutdown_connections 2024-09-12T03:18:22.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:22.840+0000 7fa3c5abd640 1 -- 172.21.15.6:0/876511290 wait complete. 2024-09-12T03:18:23.452 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1726111101,"num_in_osds":12,"osd_in_since":1726111078,"num_remapped_pgs":0} 2024-09-12T03:18:23.453 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:18:23.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:23 smithi006 ceph-mon[30991]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T03:18:23.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:23 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/876511290' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:18:23.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:23 smithi161 ceph-mon[35485]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T03:18:23.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:23 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/876511290' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:18:23.705 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:23.711 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:23 smithi118 ceph-mon[35048]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T03:18:23.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:23 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/876511290' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:18:24.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.148+0000 7fc3f5264640 1 -- 172.21.15.6:0/3994873272 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3f0106a80 msgr2=0x7fc3f0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:24.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.148+0000 7fc3f5264640 1 --2- 172.21.15.6:0/3994873272 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3f0106a80 0x7fc3f0106e60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009930 tx=0x7fc3e002a420 comp rx=0 tx=0).stop 2024-09-12T03:18:24.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.148+0000 7fc3f5264640 1 -- 172.21.15.6:0/3994873272 shutdown_connections 2024-09-12T03:18:24.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.148+0000 7fc3f5264640 1 --2- 172.21.15.6:0/3994873272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3f0101420 0x7fc3f010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:24.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.148+0000 7fc3f5264640 1 --2- 172.21.15.6:0/3994873272 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3f0100a80 0x7fc3f0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:24.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.148+0000 7fc3f5264640 1 --2- 172.21.15.6:0/3994873272 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3f0106a80 0x7fc3f0106e60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:24.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.148+0000 7fc3f5264640 1 -- 172.21.15.6:0/3994873272 >> 172.21.15.6:0/3994873272 conn(0x7fc3f00fc7a0 msgr2=0x7fc3f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:24.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.149+0000 7fc3f5264640 1 -- 172.21.15.6:0/3994873272 shutdown_connections 2024-09-12T03:18:24.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.149+0000 7fc3f5264640 1 -- 172.21.15.6:0/3994873272 wait complete. 2024-09-12T03:18:24.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.149+0000 7fc3f5264640 1 Processor -- start 2024-09-12T03:18:24.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.149+0000 7fc3f5264640 1 -- start start 2024-09-12T03:18:24.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.149+0000 7fc3f5264640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3f0100a80 0x7fc3f0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:24.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.149+0000 7fc3f5264640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3f0101420 0x7fc3f01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:24.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.149+0000 7fc3f5264640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3f0106a80 0x7fc3f019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:24.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.149+0000 7fc3f5264640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f0111f60 con 0x7fc3f0100a80 2024-09-12T03:18:24.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.149+0000 7fc3f5264640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f0111de0 con 0x7fc3f0106a80 2024-09-12T03:18:24.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.149+0000 7fc3f5264640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f01120e0 con 0x7fc3f0101420 2024-09-12T03:18:24.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.149+0000 7fc3eeffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3f0100a80 0x7fc3f0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:24.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.149+0000 7fc3eeffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3f0100a80 0x7fc3f0198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52984/0 (socket says 172.21.15.6:52984) 2024-09-12T03:18:24.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.149+0000 7fc3eeffd640 1 -- 172.21.15.6:0/2757915925 learned_addr learned my addr 172.21.15.6:0/2757915925 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:18:24.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.150+0000 7fc3e7fff640 1 --2- 172.21.15.6:0/2757915925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3f0101420 0x7fc3f01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:24.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.150+0000 7fc3ef7fe640 1 --2- 172.21.15.6:0/2757915925 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3f0106a80 0x7fc3f019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:24.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.150+0000 7fc3eeffd640 1 -- 172.21.15.6:0/2757915925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3f0101420 msgr2=0x7fc3f01989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:24.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.150+0000 7fc3eeffd640 1 --2- 172.21.15.6:0/2757915925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3f0101420 0x7fc3f01989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:24.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.150+0000 7fc3eeffd640 1 -- 172.21.15.6:0/2757915925 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3f0106a80 msgr2=0x7fc3f019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:24.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.150+0000 7fc3eeffd640 1 --2- 172.21.15.6:0/2757915925 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3f0106a80 0x7fc3f019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:24.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.150+0000 7fc3eeffd640 1 -- 172.21.15.6:0/2757915925 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f019d410 con 0x7fc3f0100a80 2024-09-12T03:18:24.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.150+0000 7fc3ef7fe640 1 --2- 172.21.15.6:0/2757915925 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3f0106a80 0x7fc3f019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:24.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.150+0000 7fc3eeffd640 1 --2- 172.21.15.6:0/2757915925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3f0100a80 0x7fc3f0198460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009230 tx=0x7fc3e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:24.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.150+0000 7fc3ecff9640 1 -- 172.21.15.6:0/2757915925 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e003f030 con 0x7fc3f0100a80 2024-09-12T03:18:24.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.150+0000 7fc3ecff9640 1 -- 172.21.15.6:0/2757915925 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3e0003ad0 con 0x7fc3f0100a80 2024-09-12T03:18:24.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.150+0000 7fc3f5264640 1 -- 172.21.15.6:0/2757915925 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f019d6a0 con 0x7fc3f0100a80 2024-09-12T03:18:24.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.150+0000 7fc3ecff9640 1 -- 172.21.15.6:0/2757915925 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0035070 con 0x7fc3f0100a80 2024-09-12T03:18:24.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.150+0000 7fc3f5264640 1 -- 172.21.15.6:0/2757915925 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f01a4fe0 con 0x7fc3f0100a80 2024-09-12T03:18:24.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.151+0000 7fc3f5264640 1 -- 172.21.15.6:0/2757915925 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f01021c0 con 0x7fc3f0100a80 2024-09-12T03:18:24.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.153+0000 7fc3ecff9640 1 -- 172.21.15.6:0/2757915925 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fc3e0042030 con 0x7fc3f0100a80 2024-09-12T03:18:24.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.154+0000 7fc3ecff9640 1 --2- 172.21.15.6:0/2757915925 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc3dc0a2b80 0x7fc3dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:24.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.154+0000 7fc3e7fff640 1 --2- 172.21.15.6:0/2757915925 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc3dc0a2b80 0x7fc3dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:24.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.154+0000 7fc3ecff9640 1 -- 172.21.15.6:0/2757915925 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fc3e00ef5a0 con 0x7fc3f0100a80 2024-09-12T03:18:24.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.154+0000 7fc3e7fff640 1 --2- 172.21.15.6:0/2757915925 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc3dc0a2b80 0x7fc3dc0a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8005fd0 tx=0x7fc3d8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:24.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.157+0000 7fc3ecff9640 1 -- 172.21.15.6:0/2757915925 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc3e00bc900 con 0x7fc3f0100a80 2024-09-12T03:18:24.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.332+0000 7fc3f5264640 1 -- 172.21.15.6:0/2757915925 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3f0108ff0 con 0x7fc3f0100a80 2024-09-12T03:18:24.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.333+0000 7fc3ecff9640 1 -- 172.21.15.6:0/2757915925 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+18409 (secure 0 0 0) 0x7fc3e00bc200 con 0x7fc3f0100a80 2024-09-12T03:18:24.333 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:18:24.334 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":75,"fsid":"89d99938-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:11:29.530545+0000","modified":"2024-09-12T03:18:22.302058+0000","last_up_change":"2024-09-12T03:18:21.292572+0000","last_in_change":"2024-09-12T03:17:58.013800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:14:37.662010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f55d68b-95a0-4ee2-a46f-bc422b38471e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6803","nonce":726729286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6805","nonce":726729286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6809","nonce":726729286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6807","nonce":726729286}]},"public_addr":"172.21.15.6:6803/726729286","cluster_addr":"172.21.15.6:6805/726729286","heartbeat_back_addr":"172.21.15.6:6809/726729286","heartbeat_front_addr":"172.21.15.6:6807/726729286","state":["exists","up"]},{"osd":1,"uuid":"83afdb01-c731-4de1-96c6-3f798549b1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6801","nonce":3377555415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6803","nonce":3377555415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6807","nonce":3377555415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6805","nonce":3377555415}]},"public_addr":"172.21.15.118:6801/3377555415","cluster_addr":"172.21.15.118:6803/3377555415","heartbeat_back_addr":"172.21.15.118:6807/3377555415","heartbeat_front_addr":"172.21.15.118:6805/3377555415","state":["exists","up"]},{"osd":2,"uuid":"aed718b4-b849-403a-b81c-3a140b5c380c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6801","nonce":392541009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6803","nonce":392541009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6807","nonce":392541009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6805","nonce":392541009}]},"public_addr":"172.21.15.161:6801/392541009","cluster_addr":"172.21.15.161:6803/392541009","heartbeat_back_addr":"172.21.15.161:6807/392541009","heartbeat_front_addr":"172.21.15.161:6805/392541009","state":["exists","up"]},{"osd":3,"uuid":"25ced4c1-bf31-402d-8397-d42fd57f615f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6811","nonce":1892039151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6813","nonce":1892039151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6817","nonce":1892039151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6815","nonce":1892039151}]},"public_addr":"172.21.15.6:6811/1892039151","cluster_addr":"172.21.15.6:6813/1892039151","heartbeat_back_addr":"172.21.15.6:6817/1892039151","heartbeat_front_addr":"172.21.15.6:6815/1892039151","state":["exists","up"]},{"osd":4,"uuid":"1d884604-920b-4c35-b41e-d629ad9f9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6809","nonce":150177024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6811","nonce":150177024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6815","nonce":150177024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6813","nonce":150177024}]},"public_addr":"172.21.15.118:6809/150177024","cluster_addr":"172.21.15.118:6811/150177024","heartbeat_back_addr":"172.21.15.118:6815/150177024","heartbeat_front_addr":"172.21.15.118:6813/150177024","state":["exists","up"]},{"osd":5,"uuid":"1d40c0ef-99f6-426f-90c7-27579edc66af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6809","nonce":3758167854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6811","nonce":3758167854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6815","nonce":3758167854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6813","nonce":3758167854}]},"public_addr":"172.21.15.161:6809/3758167854","cluster_addr":"172.21.15.161:6811/3758167854","heartbeat_back_addr":"172.21.15.161:6815/3758167854","heartbeat_front_addr":"172.21.15.161:6813/3758167854","state":["exists","up"]},{"osd":6,"uuid":"9d9f9510-b107-40c2-9a98-9012715944b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6819","nonce":2129224874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6821","nonce":2129224874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6825","nonce":2129224874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6823","nonce":2129224874}]},"public_addr":"172.21.15.6:6819/2129224874","cluster_addr":"172.21.15.6:6821/2129224874","heartbeat_back_addr":"172.21.15.6:6825/2129224874","heartbeat_front_addr":"172.21.15.6:6823/2129224874","state":["exists","up"]},{"osd":7,"uuid":"84635ae1-3240-4954-af14-76bb3aa162e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6817","nonce":269980353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6819","nonce":269980353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6823","nonce":269980353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6821","nonce":269980353}]},"public_addr":"172.21.15.118:6817/269980353","cluster_addr":"172.21.15.118:6819/269980353","heartbeat_back_addr":"172.21.15.118:6823/269980353","heartbeat_front_addr":"172.21.15.118:6821/269980353","state":["exists","up"]},{"osd":8,"uuid":"6fd58e55-1492-4595-8647-823ea3e3c5d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6817","nonce":1783416134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6819","nonce":1783416134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6823","nonce":1783416134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6821","nonce":1783416134}]},"public_addr":"172.21.15.161:6817/1783416134","cluster_addr":"172.21.15.161:6819/1783416134","heartbeat_back_addr":"172.21.15.161:6823/1783416134","heartbeat_front_addr":"172.21.15.161:6821/1783416134","state":["exists","up"]},{"osd":9,"uuid":"90508a6d-a49c-43c6-a600-5bbccd8702ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6827","nonce":2987332365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6829","nonce":2987332365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6833","nonce":2987332365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6831","nonce":2987332365}]},"public_addr":"172.21.15.6:6827/2987332365","cluster_addr":"172.21.15.6:6829/2987332365","heartbeat_back_addr":"172.21.15.6:6833/2987332365","heartbeat_front_addr":"172.21.15.6:6831/2987332365","state":["exists","up"]},{"osd":10,"uuid":"911e86d1-669a-4935-82c0-7da51d141df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6825","nonce":2167146341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6827","nonce":2167146341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6831","nonce":2167146341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6829","nonce":2167146341}]},"public_addr":"172.21.15.118:6825/2167146341","cluster_addr":"172.21.15.118:6827/2167146341","heartbeat_back_addr":"172.21.15.118:6831/2167146341","heartbeat_front_addr":"172.21.15.118:6829/2167146341","state":["exists","up"]},{"osd":11,"uuid":"5a15c6a7-ff43-473c-bb48-0c129c316df5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6825","nonce":2889795796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6827","nonce":2889795796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6831","nonce":2889795796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6829","nonce":2889795796}]},"public_addr":"172.21.15.161:6825/2889795796","cluster_addr":"172.21.15.161:6827/2889795796","heartbeat_back_addr":"172.21.15.161:6831/2889795796","heartbeat_front_addr":"172.21.15.161:6829/2889795796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:42.912205+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:08.410669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:34.299134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:59.093181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:24.116635+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:49.566433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:15.719320+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:40.541017+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:05.178178+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:31.163877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:55.405110+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:19.417156+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6800/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/2620491916":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/4037656392":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/1377116573":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/3062800125":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:6800/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/4145338968":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/825701128":"2024-09-13T03:11:55.244257+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:18:24.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.335+0000 7fc3f5264640 1 -- 172.21.15.6:0/2757915925 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc3dc0a2b80 msgr2=0x7fc3dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:24.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.335+0000 7fc3f5264640 1 --2- 172.21.15.6:0/2757915925 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc3dc0a2b80 0x7fc3dc0a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8005fd0 tx=0x7fc3d8005850 comp rx=0 tx=0).stop 2024-09-12T03:18:24.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.335+0000 7fc3f5264640 1 -- 172.21.15.6:0/2757915925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3f0100a80 msgr2=0x7fc3f0198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:24.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.335+0000 7fc3f5264640 1 --2- 172.21.15.6:0/2757915925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3f0100a80 0x7fc3f0198460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009230 tx=0x7fc3e0004270 comp rx=0 tx=0).stop 2024-09-12T03:18:24.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.336+0000 7fc3f5264640 1 -- 172.21.15.6:0/2757915925 shutdown_connections 2024-09-12T03:18:24.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.336+0000 7fc3f5264640 1 --2- 172.21.15.6:0/2757915925 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc3dc0a2b80 0x7fc3dc0a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:24.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.336+0000 7fc3f5264640 1 --2- 172.21.15.6:0/2757915925 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3f0106a80 0x7fc3f019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:24.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.336+0000 7fc3f5264640 1 --2- 172.21.15.6:0/2757915925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3f0101420 0x7fc3f01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:24.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.336+0000 7fc3f5264640 1 --2- 172.21.15.6:0/2757915925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3f0100a80 0x7fc3f0198460 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:24.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.336+0000 7fc3f5264640 1 -- 172.21.15.6:0/2757915925 >> 172.21.15.6:0/2757915925 conn(0x7fc3f00fc7a0 msgr2=0x7fc3f00fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:24.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.336+0000 7fc3f5264640 1 -- 172.21.15.6:0/2757915925 shutdown_connections 2024-09-12T03:18:24.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:24.336+0000 7fc3f5264640 1 -- 172.21.15.6:0/2757915925 wait complete. 2024-09-12T03:18:24.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:24 smithi006 ceph-mon[30991]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:24.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:24 smithi161 ceph-mon[35485]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:24.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:24 smithi118 ceph-mon[35048]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:24.961 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T03:14:37.662010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:18:24.961 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-12T03:18:25.216 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:25.582 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:25 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2757915925' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:18:25.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:25 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2757915925' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:18:25.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.667+0000 7ff3fa2a0640 1 -- 172.21.15.6:0/1507819692 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3f4106f10 msgr2=0x7ff3f410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:25.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.667+0000 7ff3fa2a0640 1 --2- 172.21.15.6:0/1507819692 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3f4106f10 0x7ff3f410ea90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009ca0 tx=0x7ff3ec02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.667+0000 7ff3fa2a0640 1 -- 172.21.15.6:0/1507819692 shutdown_connections 2024-09-12T03:18:25.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.667+0000 7ff3fa2a0640 1 --2- 172.21.15.6:0/1507819692 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3f4106f10 0x7ff3f410ea90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.667+0000 7ff3fa2a0640 1 --2- 172.21.15.6:0/1507819692 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3f4102790 0x7ff3f41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.667+0000 7ff3fa2a0640 1 --2- 172.21.15.6:0/1507819692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3f4101de0 0x7ff3f41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.667+0000 7ff3fa2a0640 1 -- 172.21.15.6:0/1507819692 >> 172.21.15.6:0/1507819692 conn(0x7ff3f40fd8c0 msgr2=0x7ff3f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:25.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.667+0000 7ff3fa2a0640 1 -- 172.21.15.6:0/1507819692 shutdown_connections 2024-09-12T03:18:25.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.667+0000 7ff3fa2a0640 1 -- 172.21.15.6:0/1507819692 wait complete. 2024-09-12T03:18:25.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.667+0000 7ff3fa2a0640 1 Processor -- start 2024-09-12T03:18:25.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.668+0000 7ff3fa2a0640 1 -- start start 2024-09-12T03:18:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.668+0000 7ff3fa2a0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3f4101de0 0x7ff3f41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.668+0000 7ff3fa2a0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3f4102790 0x7ff3f4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.668+0000 7ff3fa2a0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3f4106f10 0x7ff3f419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.668+0000 7ff3fa2a0640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f4111f70 con 0x7ff3f4106f10 2024-09-12T03:18:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.668+0000 7ff3fa2a0640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f4111df0 con 0x7ff3f4101de0 2024-09-12T03:18:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.668+0000 7ff3fa2a0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41120f0 con 0x7ff3f4102790 2024-09-12T03:18:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.668+0000 7ff3f883d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3f4106f10 0x7ff3f419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.668+0000 7ff3f883d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3f4106f10 0x7ff3f419ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53010/0 (socket says 172.21.15.6:53010) 2024-09-12T03:18:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.668+0000 7ff3f883d640 1 -- 172.21.15.6:0/1352104120 learned_addr learned my addr 172.21.15.6:0/1352104120 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:18:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.668+0000 7ff3ebfff640 1 --2- 172.21.15.6:0/1352104120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3f4102790 0x7ff3f4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.668+0000 7ff3f3fff640 1 --2- 172.21.15.6:0/1352104120 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3f4101de0 0x7ff3f41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.668+0000 7ff3f883d640 1 -- 172.21.15.6:0/1352104120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3f4102790 msgr2=0x7ff3f4198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.668+0000 7ff3f883d640 1 --2- 172.21.15.6:0/1352104120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3f4102790 0x7ff3f4198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.668+0000 7ff3f883d640 1 -- 172.21.15.6:0/1352104120 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3f4101de0 msgr2=0x7ff3f41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.668+0000 7ff3f883d640 1 --2- 172.21.15.6:0/1352104120 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3f4101de0 0x7ff3f41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.668+0000 7ff3f883d640 1 -- 172.21.15.6:0/1352104120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f419d390 con 0x7ff3f4106f10 2024-09-12T03:18:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.669+0000 7ff3f883d640 1 --2- 172.21.15.6:0/1352104120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3f4106f10 0x7ff3f419ccb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec031a20 tx=0x7ff3ec031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.669+0000 7ff3f1ffb640 1 -- 172.21.15.6:0/1352104120 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec003c30 con 0x7ff3f4106f10 2024-09-12T03:18:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.669+0000 7ff3f1ffb640 1 -- 172.21.15.6:0/1352104120 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3ec003dd0 con 0x7ff3f4106f10 2024-09-12T03:18:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.669+0000 7ff3fa2a0640 1 -- 172.21.15.6:0/1352104120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f419d620 con 0x7ff3f4106f10 2024-09-12T03:18:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.669+0000 7ff3f1ffb640 1 -- 172.21.15.6:0/1352104120 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec02fcf0 con 0x7ff3f4106f10 2024-09-12T03:18:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.669+0000 7ff3fa2a0640 1 -- 172.21.15.6:0/1352104120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f41a4f60 con 0x7ff3f4106f10 2024-09-12T03:18:25.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.670+0000 7ff3fa2a0640 1 -- 172.21.15.6:0/1352104120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f41021c0 con 0x7ff3f4106f10 2024-09-12T03:18:25.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.671+0000 7ff3f1ffb640 1 -- 172.21.15.6:0/1352104120 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ff3ec004050 con 0x7ff3f4106f10 2024-09-12T03:18:25.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.672+0000 7ff3f1ffb640 1 --2- 172.21.15.6:0/1352104120 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff3e00a2c80 0x7ff3e00a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.672+0000 7ff3f3fff640 1 --2- 172.21.15.6:0/1352104120 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff3e00a2c80 0x7ff3e00a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:25.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.672+0000 7ff3f1ffb640 1 -- 172.21.15.6:0/1352104120 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7ff3ec0f0070 con 0x7ff3f4106f10 2024-09-12T03:18:25.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.672+0000 7ff3f3fff640 1 --2- 172.21.15.6:0/1352104120 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff3e00a2c80 0x7ff3e00a5140 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4005fd0 tx=0x7ff3e400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:25.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.676+0000 7ff3f1ffb640 1 -- 172.21.15.6:0/1352104120 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff3ec0bd460 con 0x7ff3f4106f10 2024-09-12T03:18:25.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:25 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2757915925' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:18:25.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.852+0000 7ff3fa2a0640 1 -- 172.21.15.6:0/1352104120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff3f4109bc0 con 0x7ff3f4106f10 2024-09-12T03:18:25.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.853+0000 7ff3f1ffb640 1 -- 172.21.15.6:0/1352104120 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7ff3ec02fe90 con 0x7ff3f4106f10 2024-09-12T03:18:25.853 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-09-12T03:18:25.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.855+0000 7ff3fa2a0640 1 -- 172.21.15.6:0/1352104120 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff3e00a2c80 msgr2=0x7ff3e00a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:25.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.855+0000 7ff3fa2a0640 1 --2- 172.21.15.6:0/1352104120 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff3e00a2c80 0x7ff3e00a5140 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4005fd0 tx=0x7ff3e400b390 comp rx=0 tx=0).stop 2024-09-12T03:18:25.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.855+0000 7ff3fa2a0640 1 -- 172.21.15.6:0/1352104120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3f4106f10 msgr2=0x7ff3f419ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:25.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.855+0000 7ff3fa2a0640 1 --2- 172.21.15.6:0/1352104120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3f4106f10 0x7ff3f419ccb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec031a20 tx=0x7ff3ec031a50 comp rx=0 tx=0).stop 2024-09-12T03:18:25.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.855+0000 7ff3fa2a0640 1 -- 172.21.15.6:0/1352104120 shutdown_connections 2024-09-12T03:18:25.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.855+0000 7ff3fa2a0640 1 --2- 172.21.15.6:0/1352104120 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff3e00a2c80 0x7ff3e00a5140 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.855+0000 7ff3fa2a0640 1 --2- 172.21.15.6:0/1352104120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3f4106f10 0x7ff3f419ccb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.855+0000 7ff3fa2a0640 1 --2- 172.21.15.6:0/1352104120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3f4102790 0x7ff3f4198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.855+0000 7ff3fa2a0640 1 --2- 172.21.15.6:0/1352104120 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3f4101de0 0x7ff3f41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.855+0000 7ff3fa2a0640 1 -- 172.21.15.6:0/1352104120 >> 172.21.15.6:0/1352104120 conn(0x7ff3f40fd8c0 msgr2=0x7ff3f40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:25.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.855+0000 7ff3fa2a0640 1 -- 172.21.15.6:0/1352104120 shutdown_connections 2024-09-12T03:18:25.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:25.855+0000 7ff3fa2a0640 1 -- 172.21.15.6:0/1352104120 wait complete. 2024-09-12T03:18:26.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:26 smithi006 ceph-mon[30991]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:26.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:26 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1352104120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:18:26.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:26 smithi161 ceph-mon[35485]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:26.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:26 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1352104120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:18:26.692 INFO:tasks.cephadm:Adding mds.a on smithi006 2024-09-12T03:18:26.692 INFO:tasks.cephadm:Adding mds.d on smithi006 2024-09-12T03:18:26.692 INFO:tasks.cephadm:Adding mds.g on smithi006 2024-09-12T03:18:26.692 INFO:tasks.cephadm:Adding mds.j on smithi006 2024-09-12T03:18:26.692 INFO:tasks.cephadm:Adding mds.b on smithi118 2024-09-12T03:18:26.692 INFO:tasks.cephadm:Adding mds.e on smithi118 2024-09-12T03:18:26.692 INFO:tasks.cephadm:Adding mds.h on smithi118 2024-09-12T03:18:26.692 INFO:tasks.cephadm:Adding mds.k on smithi118 2024-09-12T03:18:26.692 INFO:tasks.cephadm:Adding mds.c on smithi161 2024-09-12T03:18:26.692 INFO:tasks.cephadm:Adding mds.f on smithi161 2024-09-12T03:18:26.692 INFO:tasks.cephadm:Adding mds.i on smithi161 2024-09-12T03:18:26.692 INFO:tasks.cephadm:Adding mds.l on smithi161 2024-09-12T03:18:26.693 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi118=b;smithi118=e;smithi118=h;smithi118=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l' 2024-09-12T03:18:26.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:26 smithi118 ceph-mon[35048]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:26.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:26 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1352104120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:18:26.953 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:18:27.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.667+0000 7f4b355ac640 1 -- 172.21.15.161:0/4105277843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b30102e00 msgr2=0x7f4b30103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:27.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.667+0000 7f4b355ac640 1 --2- 172.21.15.161:0/4105277843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b30102e00 0x7f4b30103280 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4b18009900 tx=0x7f4b1802a410 comp rx=0 tx=0).stop 2024-09-12T03:18:27.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.667+0000 7f4b355ac640 1 -- 172.21.15.161:0/4105277843 shutdown_connections 2024-09-12T03:18:27.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.667+0000 7f4b355ac640 1 --2- 172.21.15.161:0/4105277843 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4b301037c0 0x7f4b3010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.667+0000 7f4b355ac640 1 --2- 172.21.15.161:0/4105277843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b30102e00 0x7f4b30103280 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.667+0000 7f4b355ac640 1 --2- 172.21.15.161:0/4105277843 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b30101c00 0x7f4b30102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.667+0000 7f4b355ac640 1 -- 172.21.15.161:0/4105277843 >> 172.21.15.161:0/4105277843 conn(0x7f4b300fd390 msgr2=0x7f4b300ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:27.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.667+0000 7f4b355ac640 1 -- 172.21.15.161:0/4105277843 shutdown_connections 2024-09-12T03:18:27.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.667+0000 7f4b355ac640 1 -- 172.21.15.161:0/4105277843 wait complete. 2024-09-12T03:18:27.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.668+0000 7f4b355ac640 1 Processor -- start 2024-09-12T03:18:27.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.668+0000 7f4b355ac640 1 -- start start 2024-09-12T03:18:27.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.668+0000 7f4b355ac640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b30101c00 0x7f4b30070c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.668+0000 7f4b355ac640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b30102e00 0x7f4b30071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.668+0000 7f4b355ac640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4b301037c0 0x7f4b301a8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.668+0000 7f4b355ac640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3010d980 con 0x7f4b30101c00 2024-09-12T03:18:27.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.668+0000 7f4b355ac640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3010d800 con 0x7f4b301037c0 2024-09-12T03:18:27.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.668+0000 7f4b355ac640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3010db00 con 0x7f4b30102e00 2024-09-12T03:18:27.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.668+0000 7f4b27fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b30102e00 0x7f4b30071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:27.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.668+0000 7f4b27fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b30102e00 0x7f4b30071180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41444/0 (socket says 172.21.15.161:41444) 2024-09-12T03:18:27.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.668+0000 7f4b27fff640 1 -- 172.21.15.161:0/462360659 learned_addr learned my addr 172.21.15.161:0/462360659 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-12T03:18:27.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.669+0000 7f4b2effd640 1 --2- 172.21.15.161:0/462360659 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b30101c00 0x7f4b30070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:27.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.669+0000 7f4b2f7fe640 1 --2- 172.21.15.161:0/462360659 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4b301037c0 0x7f4b301a8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:27.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.669+0000 7f4b27fff640 1 -- 172.21.15.161:0/462360659 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4b301037c0 msgr2=0x7f4b301a8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:27.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.669+0000 7f4b27fff640 1 --2- 172.21.15.161:0/462360659 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4b301037c0 0x7f4b301a8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.669+0000 7f4b27fff640 1 -- 172.21.15.161:0/462360659 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b30101c00 msgr2=0x7f4b30070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:27.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.669+0000 7f4b27fff640 1 --2- 172.21.15.161:0/462360659 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b30101c00 0x7f4b30070c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.669+0000 7f4b27fff640 1 -- 172.21.15.161:0/462360659 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b301a85d0 con 0x7f4b30102e00 2024-09-12T03:18:27.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.669+0000 7f4b2effd640 1 --2- 172.21.15.161:0/462360659 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b30101c00 0x7f4b30070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:27.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.669+0000 7f4b27fff640 1 --2- 172.21.15.161:0/462360659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b30102e00 0x7f4b30071180 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4b18009230 tx=0x7f4b18004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:27.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.669+0000 7f4b2cff9640 1 -- 172.21.15.161:0/462360659 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1803f030 con 0x7f4b30102e00 2024-09-12T03:18:27.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.669+0000 7f4b2cff9640 1 -- 172.21.15.161:0/462360659 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b18003ce0 con 0x7f4b30102e00 2024-09-12T03:18:27.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.669+0000 7f4b2cff9640 1 -- 172.21.15.161:0/462360659 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18035070 con 0x7f4b30102e00 2024-09-12T03:18:27.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.669+0000 7f4b355ac640 1 -- 172.21.15.161:0/462360659 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b301a87a0 con 0x7f4b30102e00 2024-09-12T03:18:27.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.669+0000 7f4b355ac640 1 -- 172.21.15.161:0/462360659 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b301a8d00 con 0x7f4b30102e00 2024-09-12T03:18:27.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.670+0000 7f4b355ac640 1 -- 172.21.15.161:0/462360659 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b301090a0 con 0x7f4b30102e00 2024-09-12T03:18:27.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.671+0000 7f4b2cff9640 1 -- 172.21.15.161:0/462360659 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f4b18042030 con 0x7f4b30102e00 2024-09-12T03:18:27.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.672+0000 7f4b2cff9640 1 --2- 172.21.15.161:0/462360659 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4b1c0a2c00 0x7f4b1c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.672+0000 7f4b2cff9640 1 -- 172.21.15.161:0/462360659 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f4b180ef370 con 0x7f4b30102e00 2024-09-12T03:18:27.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.673+0000 7f4b2effd640 1 --2- 172.21.15.161:0/462360659 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4b1c0a2c00 0x7f4b1c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:27.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.673+0000 7f4b2effd640 1 --2- 172.21.15.161:0/462360659 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4b1c0a2c00 0x7f4b1c0a50c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4b30102c60 tx=0x7f4b20008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:27.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.677+0000 7f4b2cff9640 1 -- 172.21.15.161:0/462360659 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4b180bc770 con 0x7f4b30102e00 2024-09-12T03:18:27.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.852+0000 7f4b355ac640 1 -- 172.21.15.161:0/462360659 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi118=b;smithi118=e;smithi118=h;smithi118=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}) -- 0x7f4b30062730 con 0x7f4b1c0a2c00 2024-09-12T03:18:27.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.860+0000 7f4b2cff9640 1 -- 172.21.15.161:0/462360659 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f4b30062730 con 0x7f4b1c0a2c00 2024-09-12T03:18:27.860 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mds.all update... 2024-09-12T03:18:27.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.862+0000 7f4b355ac640 1 -- 172.21.15.161:0/462360659 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4b1c0a2c00 msgr2=0x7f4b1c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:27.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.862+0000 7f4b355ac640 1 --2- 172.21.15.161:0/462360659 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4b1c0a2c00 0x7f4b1c0a50c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4b30102c60 tx=0x7f4b20008040 comp rx=0 tx=0).stop 2024-09-12T03:18:27.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.862+0000 7f4b355ac640 1 -- 172.21.15.161:0/462360659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b30102e00 msgr2=0x7f4b30071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:27.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.862+0000 7f4b355ac640 1 --2- 172.21.15.161:0/462360659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b30102e00 0x7f4b30071180 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4b18009230 tx=0x7f4b18004430 comp rx=0 tx=0).stop 2024-09-12T03:18:27.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.862+0000 7f4b355ac640 1 -- 172.21.15.161:0/462360659 shutdown_connections 2024-09-12T03:18:27.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.862+0000 7f4b355ac640 1 --2- 172.21.15.161:0/462360659 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4b1c0a2c00 0x7f4b1c0a50c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.862+0000 7f4b355ac640 1 --2- 172.21.15.161:0/462360659 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4b301037c0 0x7f4b301a8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.862+0000 7f4b355ac640 1 --2- 172.21.15.161:0/462360659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b30102e00 0x7f4b30071180 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.862+0000 7f4b355ac640 1 --2- 172.21.15.161:0/462360659 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b30101c00 0x7f4b30070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.862+0000 7f4b355ac640 1 -- 172.21.15.161:0/462360659 >> 172.21.15.161:0/462360659 conn(0x7f4b300fd390 msgr2=0x7f4b30106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:27.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.862+0000 7f4b355ac640 1 -- 172.21.15.161:0/462360659 shutdown_connections 2024-09-12T03:18:27.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-12T03:18:27.862+0000 7f4b355ac640 1 -- 172.21.15.161:0/462360659 wait complete. 2024-09-12T03:18:28.485 DEBUG:teuthology.orchestra.run.smithi006:mds.a> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.a.service 2024-09-12T03:18:28.492 DEBUG:teuthology.orchestra.run.smithi006:mds.d> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.d.service 2024-09-12T03:18:28.535 DEBUG:teuthology.orchestra.run.smithi006:mds.g> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.g.service 2024-09-12T03:18:28.578 DEBUG:teuthology.orchestra.run.smithi006:mds.j> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.j.service 2024-09-12T03:18:28.622 DEBUG:teuthology.orchestra.run.smithi118:mds.b> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.b.service 2024-09-12T03:18:28.624 DEBUG:teuthology.orchestra.run.smithi118:mds.e> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.e.service 2024-09-12T03:18:28.670 DEBUG:teuthology.orchestra.run.smithi118:mds.h> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.h.service 2024-09-12T03:18:28.713 DEBUG:teuthology.orchestra.run.smithi118:mds.k> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.k.service 2024-09-12T03:18:28.756 DEBUG:teuthology.orchestra.run.smithi161:mds.c> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.c.service 2024-09-12T03:18:28.758 DEBUG:teuthology.orchestra.run.smithi161:mds.f> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.f.service 2024-09-12T03:18:28.800 DEBUG:teuthology.orchestra.run.smithi161:mds.i> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.i.service 2024-09-12T03:18:28.843 DEBUG:teuthology.orchestra.run.smithi161:mds.l> sudo journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.l.service 2024-09-12T03:18:28.887 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-12T03:18:28.887 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:18:28.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:28 smithi118 ceph-mon[35048]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:28.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:28 smithi118 ceph-mon[35048]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi118=b;smithi118=e;smithi118=h;smithi118=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:28.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:28 smithi118 ceph-mon[35048]: Saving service mds.all spec with placement smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi118=b;smithi118=e;smithi118=h;smithi118=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-09-12T03:18:28.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:28 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:28.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:28 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:28.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:28 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:28.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:28 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:28.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:28 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:28.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:28 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:28.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:28 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:28.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:28 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:29.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:28 smithi006 ceph-mon[30991]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:29.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:28 smithi006 ceph-mon[30991]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi118=b;smithi118=e;smithi118=h;smithi118=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:29.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:28 smithi006 ceph-mon[30991]: Saving service mds.all spec with placement smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi118=b;smithi118=e;smithi118=h;smithi118=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-09-12T03:18:29.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:28 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:29.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:28 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:29.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:28 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:29.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:28 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:29.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:28 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:29.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:28 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:29.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:28 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:29.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:28 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:29.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:28 smithi161 ceph-mon[35485]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:29.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:28 smithi161 ceph-mon[35485]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi118=b;smithi118=e;smithi118=h;smithi118=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:29.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:28 smithi161 ceph-mon[35485]: Saving service mds.all spec with placement smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi118=b;smithi118=e;smithi118=h;smithi118=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-09-12T03:18:29.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:28 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:29.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:28 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:29.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:28 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:29.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:28 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:29.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:28 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:29.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:28 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:29.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:28 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:29.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:28 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:29.145 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:29.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.606+0000 7f32b6cb3640 1 -- 172.21.15.6:0/1012549992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32b0102790 msgr2=0x7f32b01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:29.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.606+0000 7f32b6cb3640 1 --2- 172.21.15.6:0/1012549992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32b0102790 0x7f32b01067e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009900 tx=0x7f32a002a410 comp rx=0 tx=0).stop 2024-09-12T03:18:29.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.606+0000 7f32b6cb3640 1 -- 172.21.15.6:0/1012549992 shutdown_connections 2024-09-12T03:18:29.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.606+0000 7f32b6cb3640 1 --2- 172.21.15.6:0/1012549992 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32b0106f10 0x7f32b010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:29.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.606+0000 7f32b6cb3640 1 --2- 172.21.15.6:0/1012549992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32b0102790 0x7f32b01067e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:29.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.606+0000 7f32b6cb3640 1 --2- 172.21.15.6:0/1012549992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b0101de0 0x7f32b01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:29.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.606+0000 7f32b6cb3640 1 -- 172.21.15.6:0/1012549992 >> 172.21.15.6:0/1012549992 conn(0x7f32b00fd8c0 msgr2=0x7f32b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:29.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.606+0000 7f32b6cb3640 1 -- 172.21.15.6:0/1012549992 shutdown_connections 2024-09-12T03:18:29.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.606+0000 7f32b6cb3640 1 -- 172.21.15.6:0/1012549992 wait complete. 2024-09-12T03:18:29.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.607+0000 7f32b6cb3640 1 Processor -- start 2024-09-12T03:18:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.607+0000 7f32b6cb3640 1 -- start start 2024-09-12T03:18:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.607+0000 7f32b6cb3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32b0101de0 0x7f32b00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.607+0000 7f32b6cb3640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32b0102790 0x7f32b0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.607+0000 7f32b6cb3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b0106f10 0x7f32b0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.607+0000 7f32b6cb3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f32b0111f90 con 0x7f32b0101de0 2024-09-12T03:18:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.607+0000 7f32b6cb3640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f32b0111e10 con 0x7f32b0102790 2024-09-12T03:18:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.607+0000 7f32b6cb3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f32b0112110 con 0x7f32b0106f10 2024-09-12T03:18:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.607+0000 7f32b4a4f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32b0101de0 0x7f32b00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.607+0000 7f32b4a4f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32b0101de0 0x7f32b00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60736/0 (socket says 172.21.15.6:60736) 2024-09-12T03:18:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.607+0000 7f32b4a4f640 1 -- 172.21.15.6:0/784398052 learned_addr learned my addr 172.21.15.6:0/784398052 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:18:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.607+0000 7f32b5250640 1 --2- 172.21.15.6:0/784398052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b0106f10 0x7f32b0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.607+0000 7f32affff640 1 --2- 172.21.15.6:0/784398052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32b0102790 0x7f32b0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:29.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.607+0000 7f32b4a4f640 1 -- 172.21.15.6:0/784398052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b0106f10 msgr2=0x7f32b0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:29.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.607+0000 7f32b4a4f640 1 --2- 172.21.15.6:0/784398052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b0106f10 0x7f32b0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:29.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.608+0000 7f32b4a4f640 1 -- 172.21.15.6:0/784398052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32b0102790 msgr2=0x7f32b0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:29.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.608+0000 7f32b4a4f640 1 --2- 172.21.15.6:0/784398052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32b0102790 0x7f32b0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:29.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.608+0000 7f32b4a4f640 1 -- 172.21.15.6:0/784398052 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b01a68d0 con 0x7f32b0101de0 2024-09-12T03:18:29.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.608+0000 7f32affff640 1 --2- 172.21.15.6:0/784398052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32b0102790 0x7f32b0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:29.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.608+0000 7f32b5250640 1 --2- 172.21.15.6:0/784398052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b0106f10 0x7f32b0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:29.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.608+0000 7f32b4a4f640 1 --2- 172.21.15.6:0/784398052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32b0101de0 0x7f32b00744f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f329800b4a0 tx=0x7f329800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:29.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.608+0000 7f32adffb640 1 -- 172.21.15.6:0/784398052 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298014070 con 0x7f32b0101de0 2024-09-12T03:18:29.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.608+0000 7f32adffb640 1 -- 172.21.15.6:0/784398052 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3298010af0 con 0x7f32b0101de0 2024-09-12T03:18:29.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.608+0000 7f32b6cb3640 1 -- 172.21.15.6:0/784398052 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b01a6bc0 con 0x7f32b0101de0 2024-09-12T03:18:29.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.608+0000 7f32adffb640 1 -- 172.21.15.6:0/784398052 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298017d10 con 0x7f32b0101de0 2024-09-12T03:18:29.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.608+0000 7f32b6cb3640 1 -- 172.21.15.6:0/784398052 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b01a7150 con 0x7f32b0101de0 2024-09-12T03:18:29.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.609+0000 7f32b6cb3640 1 -- 172.21.15.6:0/784398052 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b01021c0 con 0x7f32b0101de0 2024-09-12T03:18:29.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.610+0000 7f32adffb640 1 -- 172.21.15.6:0/784398052 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f3298017070 con 0x7f32b0101de0 2024-09-12T03:18:29.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.611+0000 7f32adffb640 1 --2- 172.21.15.6:0/784398052 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f329c0a2c00 0x7f329c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:29.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.611+0000 7f32affff640 1 --2- 172.21.15.6:0/784398052 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f329c0a2c00 0x7f329c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:29.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.611+0000 7f32adffb640 1 -- 172.21.15.6:0/784398052 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f32980cdf40 con 0x7f32b0101de0 2024-09-12T03:18:29.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.612+0000 7f32affff640 1 --2- 172.21.15.6:0/784398052 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f329c0a2c00 0x7f329c0a50c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f32a0000c00 tx=0x7f32a0032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:29.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.616+0000 7f32adffb640 1 -- 172.21.15.6:0/784398052 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f329809b340 con 0x7f32b0101de0 2024-09-12T03:18:29.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.828+0000 7f32b6cb3640 1 -- 172.21.15.6:0/784398052 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32b010a0f0 con 0x7f32b0101de0 2024-09-12T03:18:29.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.828+0000 7f32adffb640 1 -- 172.21.15.6:0/784398052 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f329809ac40 con 0x7f32b0101de0 2024-09-12T03:18:29.829 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:18:29.829 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":1,"btime":"2024-09-12T03:11:29:530327+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-12T03:18:29.829 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 1 2024-09-12T03:18:29.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.831+0000 7f32b6cb3640 1 -- 172.21.15.6:0/784398052 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f329c0a2c00 msgr2=0x7f329c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:29.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.831+0000 7f32b6cb3640 1 --2- 172.21.15.6:0/784398052 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f329c0a2c00 0x7f329c0a50c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f32a0000c00 tx=0x7f32a0032d50 comp rx=0 tx=0).stop 2024-09-12T03:18:29.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.831+0000 7f32b6cb3640 1 -- 172.21.15.6:0/784398052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32b0101de0 msgr2=0x7f32b00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:29.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.831+0000 7f32b6cb3640 1 --2- 172.21.15.6:0/784398052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32b0101de0 0x7f32b00744f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f329800b4a0 tx=0x7f329800b960 comp rx=0 tx=0).stop 2024-09-12T03:18:29.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.831+0000 7f32b6cb3640 1 -- 172.21.15.6:0/784398052 shutdown_connections 2024-09-12T03:18:29.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.831+0000 7f32b6cb3640 1 --2- 172.21.15.6:0/784398052 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f329c0a2c00 0x7f329c0a50c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:29.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.831+0000 7f32b6cb3640 1 --2- 172.21.15.6:0/784398052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b0106f10 0x7f32b0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:29.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.831+0000 7f32b6cb3640 1 --2- 172.21.15.6:0/784398052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32b0102790 0x7f32b0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:29.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.831+0000 7f32b6cb3640 1 --2- 172.21.15.6:0/784398052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32b0101de0 0x7f32b00744f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:29.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.831+0000 7f32b6cb3640 1 -- 172.21.15.6:0/784398052 >> 172.21.15.6:0/784398052 conn(0x7f32b00fd8c0 msgr2=0x7f32b00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:29.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.831+0000 7f32b6cb3640 1 -- 172.21.15.6:0/784398052 shutdown_connections 2024-09-12T03:18:29.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:29.831+0000 7f32b6cb3640 1 -- 172.21.15.6:0/784398052 wait complete. 2024-09-12T03:18:29.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:29 smithi161 ceph-mon[35485]: Deploying daemon mds.l on smithi161 2024-09-12T03:18:29.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:29 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/784398052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:30.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:29 smithi006 ceph-mon[30991]: Deploying daemon mds.l on smithi161 2024-09-12T03:18:30.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:29 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/784398052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:30.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:29 smithi118 ceph-mon[35048]: Deploying daemon mds.l on smithi161 2024-09-12T03:18:30.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:29 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/784398052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:30.306 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 12 03:18:30 smithi161 systemd[1]: Starting Ceph mds.l for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:18:30.613 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 12 03:18:30 smithi161 podman[65446]: 2024-09-12 03:18:30.306580343 +0000 UTC m=+0.095437104 container create 456b8c4a58a3ac1ba9a389d56d24eeb6b977c18a53082232d712696b35a7edd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-l, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-12T03:18:30.613 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 12 03:18:30 smithi161 podman[65446]: 2024-09-12 03:18:30.227446217 +0000 UTC m=+0.016302978 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:18:30.613 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 12 03:18:30 smithi161 podman[65446]: 2024-09-12 03:18:30.525843122 +0000 UTC m=+0.314699882 container init 456b8c4a58a3ac1ba9a389d56d24eeb6b977c18a53082232d712696b35a7edd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-l, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-12T03:18:30.613 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 12 03:18:30 smithi161 podman[65446]: 2024-09-12 03:18:30.528571565 +0000 UTC m=+0.317428326 container start 456b8c4a58a3ac1ba9a389d56d24eeb6b977c18a53082232d712696b35a7edd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-l, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-12T03:18:30.614 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 12 03:18:30 smithi161 ceph-mds[65466]: -- 172.21.15.161:0/1475635848 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5566890ab860 con 0x556689088800 2024-09-12T03:18:30.614 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 12 03:18:30 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-l[65462]: starting mds.l at 2024-09-12T03:18:31.113 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 12 03:18:30 smithi161 bash[65446]: 456b8c4a58a3ac1ba9a389d56d24eeb6b977c18a53082232d712696b35a7edd0 2024-09-12T03:18:31.113 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 12 03:18:30 smithi161 systemd[1]: Started Ceph mds.l for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:18:31.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:30 smithi161 ceph-mon[35485]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:31.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:30 smithi161 ceph-mon[35485]: mds.? [v2:172.21.15.161:6832/1749185252,v1:172.21.15.161:6833/1749185252] up:boot 2024-09-12T03:18:31.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:30 smithi161 ceph-mon[35485]: fsmap 1 up:standby 2024-09-12T03:18:31.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:30 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:18:31.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:30 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:31.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:30 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:31.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:30 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:31.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:30 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:31.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:30 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:31.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:30 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:31.125 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:30 smithi118 ceph-mon[35048]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:31.126 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:30 smithi118 ceph-mon[35048]: mds.? [v2:172.21.15.161:6832/1749185252,v1:172.21.15.161:6833/1749185252] up:boot 2024-09-12T03:18:31.126 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:30 smithi118 ceph-mon[35048]: fsmap 1 up:standby 2024-09-12T03:18:31.126 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:30 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:18:31.126 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:30 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:31.126 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:30 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:31.126 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:30 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:31.126 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:30 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:31.126 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:30 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:31.126 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:30 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:31.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:30 smithi006 ceph-mon[30991]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:31.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:30 smithi006 ceph-mon[30991]: mds.? [v2:172.21.15.161:6832/1749185252,v1:172.21.15.161:6833/1749185252] up:boot 2024-09-12T03:18:31.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:30 smithi006 ceph-mon[30991]: fsmap 1 up:standby 2024-09-12T03:18:31.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:30 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:18:31.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:30 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:31.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:30 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:31.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:30 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:31.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:30 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:31.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:30 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:31.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:30 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:32.172 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:31 smithi118 ceph-mon[35048]: Deploying daemon mds.k on smithi118 2024-09-12T03:18:32.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:31 smithi006 ceph-mon[30991]: Deploying daemon mds.k on smithi118 2024-09-12T03:18:32.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:31 smithi161 ceph-mon[35485]: Deploying daemon mds.k on smithi118 2024-09-12T03:18:32.592 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:18:32.712 INFO:journalctl@ceph.mds.k.smithi118.stdout:Sep 12 03:18:32 smithi118 systemd[1]: Starting Ceph mds.k for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:18:32.850 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:32.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:32 smithi006 ceph-mon[30991]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:33.093 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:32 smithi118 ceph-mon[35048]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:33.094 INFO:journalctl@ceph.mds.k.smithi118.stdout:Sep 12 03:18:32 smithi118 podman[64570]: 2024-09-12 03:18:32.740631529 +0000 UTC m=+0.015953340 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:18:33.094 INFO:journalctl@ceph.mds.k.smithi118.stdout:Sep 12 03:18:32 smithi118 podman[64570]: 2024-09-12 03:18:32.867282382 +0000 UTC m=+0.142604222 container create 89deea317135b27996571d99c6023a62135d0ed66377239baf21718f956782bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-12T03:18:33.094 INFO:journalctl@ceph.mds.k.smithi118.stdout:Sep 12 03:18:33 smithi118 podman[64570]: 2024-09-12 03:18:33.094504914 +0000 UTC m=+0.369826709 container init 89deea317135b27996571d99c6023a62135d0ed66377239baf21718f956782bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-12T03:18:33.264 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:32 smithi161 ceph-mon[35485]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:33.364 INFO:journalctl@ceph.mds.k.smithi118.stdout:Sep 12 03:18:33 smithi118 podman[64570]: 2024-09-12 03:18:33.098106943 +0000 UTC m=+0.373428735 container start 89deea317135b27996571d99c6023a62135d0ed66377239baf21718f956782bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731) 2024-09-12T03:18:33.364 INFO:journalctl@ceph.mds.k.smithi118.stdout:Sep 12 03:18:33 smithi118 ceph-mds[64589]: -- 172.21.15.118:0/567354309 <== mon.2 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d39f5db860 con 0x55d39f5b9000 2024-09-12T03:18:33.364 INFO:journalctl@ceph.mds.k.smithi118.stdout:Sep 12 03:18:33 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-k[64585]: starting mds.k at 2024-09-12T03:18:33.365 INFO:journalctl@ceph.mds.k.smithi118.stdout:Sep 12 03:18:33 smithi118 bash[64570]: 89deea317135b27996571d99c6023a62135d0ed66377239baf21718f956782bf 2024-09-12T03:18:33.365 INFO:journalctl@ceph.mds.k.smithi118.stdout:Sep 12 03:18:33 smithi118 systemd[1]: Started Ceph mds.k for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:18:33.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.321+0000 7f6441121640 1 -- 172.21.15.6:0/482246419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f643c102ef0 msgr2=0x7f643c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:33.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.321+0000 7f6441121640 1 --2- 172.21.15.6:0/482246419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f643c102ef0 0x7f643c10da40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6434009ca0 tx=0x7f643402a950 comp rx=0 tx=0).stop 2024-09-12T03:18:33.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.322+0000 7f6441121640 1 -- 172.21.15.6:0/482246419 shutdown_connections 2024-09-12T03:18:33.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.322+0000 7f6441121640 1 --2- 172.21.15.6:0/482246419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f643c102ef0 0x7f643c10da40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:33.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.322+0000 7f6441121640 1 --2- 172.21.15.6:0/482246419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f643c102570 0x7f643c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:33.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.322+0000 7f6441121640 1 --2- 172.21.15.6:0/482246419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643c100e20 0x7f643c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:33.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.322+0000 7f6441121640 1 -- 172.21.15.6:0/482246419 >> 172.21.15.6:0/482246419 conn(0x7f643c0fc7d0 msgr2=0x7f643c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:33.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.322+0000 7f6441121640 1 -- 172.21.15.6:0/482246419 shutdown_connections 2024-09-12T03:18:33.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.322+0000 7f6441121640 1 -- 172.21.15.6:0/482246419 wait complete. 2024-09-12T03:18:33.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.323+0000 7f6441121640 1 Processor -- start 2024-09-12T03:18:33.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.323+0000 7f6441121640 1 -- start start 2024-09-12T03:18:33.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.323+0000 7f6441121640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643c100e20 0x7f643c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:33.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.323+0000 7f6441121640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f643c102570 0x7f643c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:33.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.323+0000 7f6441121640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f643c102ef0 0x7f643c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:33.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.323+0000 7f6441121640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f643c1120a0 con 0x7f643c102570 2024-09-12T03:18:33.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.323+0000 7f6441121640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f643c111f20 con 0x7f643c102ef0 2024-09-12T03:18:33.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.323+0000 7f6441121640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f643c112220 con 0x7f643c100e20 2024-09-12T03:18:33.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.323+0000 7f643a59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f643c102570 0x7f643c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:33.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.323+0000 7f643a59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f643c102570 0x7f643c1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60748/0 (socket says 172.21.15.6:60748) 2024-09-12T03:18:33.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.323+0000 7f643a59c640 1 -- 172.21.15.6:0/632831981 learned_addr learned my addr 172.21.15.6:0/632831981 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:18:33.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.323+0000 7f643b59e640 1 --2- 172.21.15.6:0/632831981 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f643c102ef0 0x7f643c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:33.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.323+0000 7f643ad9d640 1 --2- 172.21.15.6:0/632831981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643c100e20 0x7f643c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:33.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.324+0000 7f643b59e640 1 -- 172.21.15.6:0/632831981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643c100e20 msgr2=0x7f643c198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:33.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.324+0000 7f643b59e640 1 --2- 172.21.15.6:0/632831981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643c100e20 0x7f643c198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:33.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.324+0000 7f643b59e640 1 -- 172.21.15.6:0/632831981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f643c102570 msgr2=0x7f643c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:33.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.324+0000 7f643b59e640 1 --2- 172.21.15.6:0/632831981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f643c102570 0x7f643c1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:33.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.324+0000 7f643b59e640 1 -- 172.21.15.6:0/632831981 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c19d430 con 0x7f643c102ef0 2024-09-12T03:18:33.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.324+0000 7f643a59c640 1 --2- 172.21.15.6:0/632831981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f643c102570 0x7f643c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:18:33.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.324+0000 7f643b59e640 1 --2- 172.21.15.6:0/632831981 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f643c102ef0 0x7f643c19cd50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6434005cc0 tx=0x7f643402f3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:33.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.324+0000 7f6423fff640 1 -- 172.21.15.6:0/632831981 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434034040 con 0x7f643c102ef0 2024-09-12T03:18:33.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.324+0000 7f6423fff640 1 -- 172.21.15.6:0/632831981 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6434030920 con 0x7f643c102ef0 2024-09-12T03:18:33.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.324+0000 7f6423fff640 1 -- 172.21.15.6:0/632831981 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64340045b0 con 0x7f643c102ef0 2024-09-12T03:18:33.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.324+0000 7f6441121640 1 -- 172.21.15.6:0/632831981 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643c19d6c0 con 0x7f643c102ef0 2024-09-12T03:18:33.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.324+0000 7f6441121640 1 -- 172.21.15.6:0/632831981 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643c19db50 con 0x7f643c102ef0 2024-09-12T03:18:33.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.326+0000 7f6441121640 1 -- 172.21.15.6:0/632831981 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f643c101200 con 0x7f643c102ef0 2024-09-12T03:18:33.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.327+0000 7f6423fff640 1 -- 172.21.15.6:0/632831981 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f6434037020 con 0x7f643c102ef0 2024-09-12T03:18:33.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.328+0000 7f6423fff640 1 --2- 172.21.15.6:0/632831981 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f64240a2b80 0x7f64240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:33.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.328+0000 7f643ad9d640 1 --2- 172.21.15.6:0/632831981 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f64240a2b80 0x7f64240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:33.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.328+0000 7f6423fff640 1 -- 172.21.15.6:0/632831981 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f64340ef920 con 0x7f643c102ef0 2024-09-12T03:18:33.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.329+0000 7f643ad9d640 1 --2- 172.21.15.6:0/632831981 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f64240a2b80 0x7f64240a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f642c009bd0 tx=0x7f642c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:33.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.332+0000 7f6423fff640 1 -- 172.21.15.6:0/632831981 <== mon.2 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f64340bcd20 con 0x7f643c102ef0 2024-09-12T03:18:33.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.552+0000 7f6441121640 1 -- 172.21.15.6:0/632831981 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f643c1a5090 con 0x7f643c102ef0 2024-09-12T03:18:33.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.553+0000 7f6423fff640 1 -- 172.21.15.6:0/632831981 <== mon.2 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f64340098b0 con 0x7f643c102ef0 2024-09-12T03:18:33.554 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:18:33.554 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":2,"btime":"2024-09-12T03:18:30:557403+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T03:18:33.554 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 2 2024-09-12T03:18:33.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.556+0000 7f6441121640 1 -- 172.21.15.6:0/632831981 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f64240a2b80 msgr2=0x7f64240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:33.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.556+0000 7f6441121640 1 --2- 172.21.15.6:0/632831981 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f64240a2b80 0x7f64240a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f642c009bd0 tx=0x7f642c005f90 comp rx=0 tx=0).stop 2024-09-12T03:18:33.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.556+0000 7f6441121640 1 -- 172.21.15.6:0/632831981 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f643c102ef0 msgr2=0x7f643c19cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:33.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.556+0000 7f6441121640 1 --2- 172.21.15.6:0/632831981 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f643c102ef0 0x7f643c19cd50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6434005cc0 tx=0x7f643402f3a0 comp rx=0 tx=0).stop 2024-09-12T03:18:33.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.556+0000 7f6441121640 1 -- 172.21.15.6:0/632831981 shutdown_connections 2024-09-12T03:18:33.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.556+0000 7f6441121640 1 --2- 172.21.15.6:0/632831981 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f64240a2b80 0x7f64240a5040 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:33.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.556+0000 7f6441121640 1 --2- 172.21.15.6:0/632831981 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f643c102ef0 0x7f643c19cd50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:33.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.556+0000 7f6441121640 1 --2- 172.21.15.6:0/632831981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f643c102570 0x7f643c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:33.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.556+0000 7f6441121640 1 --2- 172.21.15.6:0/632831981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643c100e20 0x7f643c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:33.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.556+0000 7f6441121640 1 -- 172.21.15.6:0/632831981 >> 172.21.15.6:0/632831981 conn(0x7f643c0fc7d0 msgr2=0x7f643c106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:33.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.556+0000 7f6441121640 1 -- 172.21.15.6:0/632831981 shutdown_connections 2024-09-12T03:18:33.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:33.556+0000 7f6441121640 1 -- 172.21.15.6:0/632831981 wait complete. 2024-09-12T03:18:34.496 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:34 smithi006 ceph-mon[30991]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:34.496 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:34 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:34.496 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:34 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:34.496 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:34 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:34.496 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:34 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:34.497 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:34 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:34.497 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:34 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:34.497 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:34 smithi006 ceph-mon[30991]: Deploying daemon mds.j on smithi006 2024-09-12T03:18:34.497 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:34 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/632831981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:34.497 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:34 smithi006 ceph-mon[30991]: mds.? [v2:172.21.15.118:6832/3592007490,v1:172.21.15.118:6833/3592007490] up:boot 2024-09-12T03:18:34.497 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:34 smithi006 ceph-mon[30991]: fsmap 2 up:standby 2024-09-12T03:18:34.497 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:34 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:18:34.579 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:34 smithi118 ceph-mon[35048]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:34.579 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:34 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:34.579 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:34 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:34.579 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:34 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:34.579 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:34 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:34.579 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:34 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:34.579 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:34 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:34.579 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:34 smithi118 ceph-mon[35048]: Deploying daemon mds.j on smithi006 2024-09-12T03:18:34.580 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:34 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/632831981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:34.580 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:34 smithi118 ceph-mon[35048]: mds.? [v2:172.21.15.118:6832/3592007490,v1:172.21.15.118:6833/3592007490] up:boot 2024-09-12T03:18:34.580 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:34 smithi118 ceph-mon[35048]: fsmap 2 up:standby 2024-09-12T03:18:34.580 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:34 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:18:34.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:34 smithi161 ceph-mon[35485]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:34.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:34 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:34.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:34 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:34.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:34 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:34.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:34 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:34.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:34 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:34.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:34 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:34.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:34 smithi161 ceph-mon[35485]: Deploying daemon mds.j on smithi006 2024-09-12T03:18:34.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:34 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/632831981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:34.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:34 smithi161 ceph-mon[35485]: mds.? [v2:172.21.15.118:6832/3592007490,v1:172.21.15.118:6833/3592007490] up:boot 2024-09-12T03:18:34.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:34 smithi161 ceph-mon[35485]: fsmap 2 up:standby 2024-09-12T03:18:34.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:34 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:18:36.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:36 smithi006 ceph-mon[30991]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:36.362 INFO:journalctl@ceph.mds.j.smithi006.stdout:Sep 12 03:18:36 smithi006 systemd[1]: Started Ceph mds.j for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:18:36.528 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:18:36.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:36 smithi161 ceph-mon[35485]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:36.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:36 smithi118 ceph-mon[35048]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:37.405 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:37.529 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:37 smithi006 ceph-mon[30991]: mds.? [v2:172.21.15.6:6834/4139960526,v1:172.21.15.6:6835/4139960526] up:boot 2024-09-12T03:18:37.529 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:37 smithi006 ceph-mon[30991]: fsmap 3 up:standby 2024-09-12T03:18:37.529 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:37 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:18:37.529 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:37 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:37.529 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:37 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:37.530 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:37 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:37.530 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:37 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:37.530 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:37 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:37.530 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:37 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:37.530 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:37 smithi006 ceph-mon[30991]: Deploying daemon mds.d on smithi006 2024-09-12T03:18:37.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:37 smithi161 ceph-mon[35485]: mds.? [v2:172.21.15.6:6834/4139960526,v1:172.21.15.6:6835/4139960526] up:boot 2024-09-12T03:18:37.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:37 smithi161 ceph-mon[35485]: fsmap 3 up:standby 2024-09-12T03:18:37.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:37 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:18:37.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:37 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:37.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:37 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:37.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:37 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:37.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:37 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:37.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:37 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:37.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:37 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:37.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:37 smithi161 ceph-mon[35485]: Deploying daemon mds.d on smithi006 2024-09-12T03:18:37.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:37 smithi118 ceph-mon[35048]: mds.? [v2:172.21.15.6:6834/4139960526,v1:172.21.15.6:6835/4139960526] up:boot 2024-09-12T03:18:37.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:37 smithi118 ceph-mon[35048]: fsmap 3 up:standby 2024-09-12T03:18:37.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:37 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:18:37.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:37 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:37.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:37 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:37.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:37 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:37.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:37 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:37.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:37 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:37.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:37 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:37.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:37 smithi118 ceph-mon[35048]: Deploying daemon mds.d on smithi006 2024-09-12T03:18:38.508 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:38 smithi006 ceph-mon[30991]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:38.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.484+0000 7f5c0ffed640 1 -- 172.21.15.6:0/1338886841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c08100e20 msgr2=0x7f5c08101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:38.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.484+0000 7f5c0ffed640 1 --2- 172.21.15.6:0/1338886841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c08100e20 0x7f5c08101200 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009900 tx=0x7f5c0402a480 comp rx=0 tx=0).stop 2024-09-12T03:18:38.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.484+0000 7f5c0ffed640 1 -- 172.21.15.6:0/1338886841 shutdown_connections 2024-09-12T03:18:38.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.484+0000 7f5c0ffed640 1 --2- 172.21.15.6:0/1338886841 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c08102ef0 0x7f5c0810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.484+0000 7f5c0ffed640 1 --2- 172.21.15.6:0/1338886841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c08102570 0x7f5c081029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.484+0000 7f5c0ffed640 1 --2- 172.21.15.6:0/1338886841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c08100e20 0x7f5c08101200 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.484+0000 7f5c0ffed640 1 -- 172.21.15.6:0/1338886841 >> 172.21.15.6:0/1338886841 conn(0x7f5c080fc7d0 msgr2=0x7f5c080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:38.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.484+0000 7f5c0ffed640 1 -- 172.21.15.6:0/1338886841 shutdown_connections 2024-09-12T03:18:38.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.484+0000 7f5c0ffed640 1 -- 172.21.15.6:0/1338886841 wait complete. 2024-09-12T03:18:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.485+0000 7f5c0ffed640 1 Processor -- start 2024-09-12T03:18:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.485+0000 7f5c0ffed640 1 -- start start 2024-09-12T03:18:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.485+0000 7f5c0ffed640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c08100e20 0x7f5c081970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.485+0000 7f5c0ffed640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c08102570 0x7f5c08197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.485+0000 7f5c0ffed640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c08102ef0 0x7f5c08197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.485+0000 7f5c0ffed640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5c08110f80 con 0x7f5c08102ef0 2024-09-12T03:18:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.485+0000 7f5c0ffed640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f5c08110e00 con 0x7f5c08102570 2024-09-12T03:18:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.485+0000 7f5c0ffed640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5c08111100 con 0x7f5c08100e20 2024-09-12T03:18:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.485+0000 7f5c0e58a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c08102ef0 0x7f5c08197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.485+0000 7f5c0e58a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c08102ef0 0x7f5c08197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60794/0 (socket says 172.21.15.6:60794) 2024-09-12T03:18:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.485+0000 7f5c0e58a640 1 -- 172.21.15.6:0/1164828187 learned_addr learned my addr 172.21.15.6:0/1164828187 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:18:38.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.485+0000 7f5c0dd89640 1 --2- 172.21.15.6:0/1164828187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c08100e20 0x7f5c081970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:38.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.485+0000 7f5c0d588640 1 --2- 172.21.15.6:0/1164828187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c08102570 0x7f5c08197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:38.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.485+0000 7f5c0e58a640 1 -- 172.21.15.6:0/1164828187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c08100e20 msgr2=0x7f5c081970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:38.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.485+0000 7f5c0e58a640 1 --2- 172.21.15.6:0/1164828187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c08100e20 0x7f5c081970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.485+0000 7f5c0e58a640 1 -- 172.21.15.6:0/1164828187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c08102570 msgr2=0x7f5c08197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:38.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.485+0000 7f5c0e58a640 1 --2- 172.21.15.6:0/1164828187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c08102570 0x7f5c08197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.485+0000 7f5c0e58a640 1 -- 172.21.15.6:0/1164828187 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c081a6930 con 0x7f5c08102ef0 2024-09-12T03:18:38.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.486+0000 7f5c0e58a640 1 --2- 172.21.15.6:0/1164828187 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c08102ef0 0x7f5c08197b40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000cce0 tx=0x7f5c0000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:38.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.486+0000 7f5bfeffd640 1 -- 172.21.15.6:0/1164828187 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0000f450 con 0x7f5c08102ef0 2024-09-12T03:18:38.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.486+0000 7f5bfeffd640 1 -- 172.21.15.6:0/1164828187 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c0000fad0 con 0x7f5c08102ef0 2024-09-12T03:18:38.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.486+0000 7f5c0ffed640 1 -- 172.21.15.6:0/1164828187 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c081a6c20 con 0x7f5c08102ef0 2024-09-12T03:18:38.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.486+0000 7f5bfeffd640 1 -- 172.21.15.6:0/1164828187 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00018c10 con 0x7f5c08102ef0 2024-09-12T03:18:38.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.486+0000 7f5c0ffed640 1 -- 172.21.15.6:0/1164828187 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c081a7110 con 0x7f5c08102ef0 2024-09-12T03:18:38.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.487+0000 7f5c0ffed640 1 -- 172.21.15.6:0/1164828187 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c08101200 con 0x7f5c08102ef0 2024-09-12T03:18:38.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.489+0000 7f5bfeffd640 1 -- 172.21.15.6:0/1164828187 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f5c0000f5f0 con 0x7f5c08102ef0 2024-09-12T03:18:38.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.489+0000 7f5bfeffd640 1 --2- 172.21.15.6:0/1164828187 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5bf40a2b80 0x7f5bf40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.490+0000 7f5c0dd89640 1 --2- 172.21.15.6:0/1164828187 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5bf40a2b80 0x7f5bf40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:38.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.490+0000 7f5bfeffd640 1 -- 172.21.15.6:0/1164828187 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f5c000cf280 con 0x7f5c08102ef0 2024-09-12T03:18:38.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.490+0000 7f5c0dd89640 1 --2- 172.21.15.6:0/1164828187 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5bf40a2b80 0x7f5bf40a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009230 tx=0x7f5c040058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:38.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.494+0000 7f5bfeffd640 1 -- 172.21.15.6:0/1164828187 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c00014030 con 0x7f5c08102ef0 2024-09-12T03:18:38.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:38 smithi161 ceph-mon[35485]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:38.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:38 smithi118 ceph-mon[35048]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:38.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.718+0000 7f5c0ffed640 1 -- 172.21.15.6:0/1164828187 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c08108b00 con 0x7f5c08102ef0 2024-09-12T03:18:38.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.718+0000 7f5bfeffd640 1 -- 172.21.15.6:0/1164828187 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7f5c0009c5e0 con 0x7f5c08102ef0 2024-09-12T03:18:38.719 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:18:38.719 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":4,"btime":"2024-09-12T03:18:36:236561+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-12T03:18:38.719 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 4 2024-09-12T03:18:38.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.721+0000 7f5c0ffed640 1 -- 172.21.15.6:0/1164828187 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5bf40a2b80 msgr2=0x7f5bf40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:38.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.721+0000 7f5c0ffed640 1 --2- 172.21.15.6:0/1164828187 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5bf40a2b80 0x7f5bf40a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009230 tx=0x7f5c040058d0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.721+0000 7f5c0ffed640 1 -- 172.21.15.6:0/1164828187 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c08102ef0 msgr2=0x7f5c08197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:38.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.721+0000 7f5c0ffed640 1 --2- 172.21.15.6:0/1164828187 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c08102ef0 0x7f5c08197b40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000cce0 tx=0x7f5c0000a620 comp rx=0 tx=0).stop 2024-09-12T03:18:38.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.721+0000 7f5c0ffed640 1 -- 172.21.15.6:0/1164828187 shutdown_connections 2024-09-12T03:18:38.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.721+0000 7f5c0ffed640 1 --2- 172.21.15.6:0/1164828187 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5bf40a2b80 0x7f5bf40a5040 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.721+0000 7f5c0ffed640 1 --2- 172.21.15.6:0/1164828187 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c08102ef0 0x7f5c08197b40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.721+0000 7f5c0ffed640 1 --2- 172.21.15.6:0/1164828187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c08102570 0x7f5c08197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.721+0000 7f5c0ffed640 1 --2- 172.21.15.6:0/1164828187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c08100e20 0x7f5c081970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.721+0000 7f5c0ffed640 1 -- 172.21.15.6:0/1164828187 >> 172.21.15.6:0/1164828187 conn(0x7f5c080fc7d0 msgr2=0x7f5c080fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:38.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.721+0000 7f5c0ffed640 1 -- 172.21.15.6:0/1164828187 shutdown_connections 2024-09-12T03:18:38.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:38.721+0000 7f5c0ffed640 1 -- 172.21.15.6:0/1164828187 wait complete. 2024-09-12T03:18:39.507 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:39 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1164828187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:39.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:39 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1164828187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:39.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:39 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1164828187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:39.861 INFO:journalctl@ceph.mds.d.smithi006.stdout:Sep 12 03:18:39 smithi006 systemd[1]: Started Ceph mds.d for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:18:40.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:40 smithi118 ceph-mon[35048]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:40.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:40 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:40.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:40 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:40.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:40 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:40.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:40 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:40.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:40 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:40.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:40 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:40.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:40 smithi118 ceph-mon[35048]: Deploying daemon mds.e on smithi118 2024-09-12T03:18:40.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:40 smithi118 ceph-mon[35048]: mds.? [v2:172.21.15.6:6836/3369703756,v1:172.21.15.6:6837/3369703756] up:boot 2024-09-12T03:18:40.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:40 smithi118 ceph-mon[35048]: fsmap 4 up:standby 2024-09-12T03:18:40.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:40 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:18:41.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:40 smithi006 ceph-mon[30991]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:41.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:40 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:41.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:40 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:41.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:40 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:41.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:40 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:41.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:40 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:41.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:40 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:41.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:40 smithi006 ceph-mon[30991]: Deploying daemon mds.e on smithi118 2024-09-12T03:18:41.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:40 smithi006 ceph-mon[30991]: mds.? [v2:172.21.15.6:6836/3369703756,v1:172.21.15.6:6837/3369703756] up:boot 2024-09-12T03:18:41.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:40 smithi006 ceph-mon[30991]: fsmap 4 up:standby 2024-09-12T03:18:41.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:40 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:18:41.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:40 smithi161 ceph-mon[35485]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:41.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:40 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:41.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:40 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:41.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:40 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:41.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:40 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:41.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:40 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:41.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:40 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:41.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:40 smithi161 ceph-mon[35485]: Deploying daemon mds.e on smithi118 2024-09-12T03:18:41.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:40 smithi161 ceph-mon[35485]: mds.? [v2:172.21.15.6:6836/3369703756,v1:172.21.15.6:6837/3369703756] up:boot 2024-09-12T03:18:41.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:40 smithi161 ceph-mon[35485]: fsmap 4 up:standby 2024-09-12T03:18:41.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:40 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:18:41.539 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:18:41.811 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:41.962 INFO:journalctl@ceph.mds.e.smithi118.stdout:Sep 12 03:18:41 smithi118 systemd[1]: Starting Ceph mds.e for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:18:42.263 INFO:journalctl@ceph.mds.e.smithi118.stdout:Sep 12 03:18:42 smithi118 podman[64885]: 2024-09-12 03:18:41.909308058 +0000 UTC m=+0.016120080 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:18:42.263 INFO:journalctl@ceph.mds.e.smithi118.stdout:Sep 12 03:18:42 smithi118 podman[64885]: 2024-09-12 03:18:42.053383791 +0000 UTC m=+0.160195813 container create 3075a89eb6ea44a7e415849ca256c16068f0e9fb6387a5465c1a103169a43fc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:18:42.263 INFO:journalctl@ceph.mds.e.smithi118.stdout:Sep 12 03:18:42 smithi118 podman[64885]: 2024-09-12 03:18:42.263961287 +0000 UTC m=+0.370773309 container init 3075a89eb6ea44a7e415849ca256c16068f0e9fb6387a5465c1a103169a43fc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-e, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-12T03:18:42.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.243+0000 7f3894ee7640 1 -- 172.21.15.6:0/2014937280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38901012e0 msgr2=0x7f389010dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:42.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.243+0000 7f3894ee7640 1 --2- 172.21.15.6:0/2014937280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38901012e0 0x7f389010dfa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3884009900 tx=0x7f388402a410 comp rx=0 tx=0).stop 2024-09-12T03:18:42.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.243+0000 7f3894ee7640 1 -- 172.21.15.6:0/2014937280 shutdown_connections 2024-09-12T03:18:42.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.243+0000 7f3894ee7640 1 --2- 172.21.15.6:0/2014937280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f389010e4e0 0x7f38901108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:42.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.243+0000 7f3894ee7640 1 --2- 172.21.15.6:0/2014937280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38901012e0 0x7f389010dfa0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:42.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.243+0000 7f3894ee7640 1 --2- 172.21.15.6:0/2014937280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3890100930 0x7f3890100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:42.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.243+0000 7f3894ee7640 1 -- 172.21.15.6:0/2014937280 >> 172.21.15.6:0/2014937280 conn(0x7f38900fc7a0 msgr2=0x7f38900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:42.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.243+0000 7f3894ee7640 1 -- 172.21.15.6:0/2014937280 shutdown_connections 2024-09-12T03:18:42.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.244+0000 7f3894ee7640 1 -- 172.21.15.6:0/2014937280 wait complete. 2024-09-12T03:18:42.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.244+0000 7f3894ee7640 1 Processor -- start 2024-09-12T03:18:42.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.245+0000 7f3894ee7640 1 -- start start 2024-09-12T03:18:42.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.245+0000 7f3894ee7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3890100930 0x7f389019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:42.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.245+0000 7f3894ee7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38901012e0 0x7f389019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:42.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.245+0000 7f3894ee7640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f389010e4e0 0x7f389019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:42.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.245+0000 7f3894ee7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3890114160 con 0x7f38901012e0 2024-09-12T03:18:42.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.245+0000 7f3894ee7640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f3890113fe0 con 0x7f389010e4e0 2024-09-12T03:18:42.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.245+0000 7f3894ee7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f38901142e0 con 0x7f3890100930 2024-09-12T03:18:42.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.245+0000 7f388dd9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38901012e0 0x7f389019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:42.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.245+0000 7f388dd9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38901012e0 0x7f389019ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45856/0 (socket says 172.21.15.6:45856) 2024-09-12T03:18:42.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.245+0000 7f388dd9b640 1 -- 172.21.15.6:0/805813059 learned_addr learned my addr 172.21.15.6:0/805813059 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:18:42.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.245+0000 7f388e59c640 1 --2- 172.21.15.6:0/805813059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3890100930 0x7f389019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:42.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.245+0000 7f388ed9d640 1 --2- 172.21.15.6:0/805813059 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f389010e4e0 0x7f389019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:42.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.245+0000 7f388dd9b640 1 -- 172.21.15.6:0/805813059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3890100930 msgr2=0x7f389019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:42.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.245+0000 7f388dd9b640 1 --2- 172.21.15.6:0/805813059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3890100930 0x7f389019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:42.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.245+0000 7f388dd9b640 1 -- 172.21.15.6:0/805813059 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f389010e4e0 msgr2=0x7f389019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:42.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.245+0000 7f388dd9b640 1 --2- 172.21.15.6:0/805813059 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f389010e4e0 0x7f389019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:42.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.245+0000 7f388dd9b640 1 -- 172.21.15.6:0/805813059 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389019f670 con 0x7f38901012e0 2024-09-12T03:18:42.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.246+0000 7f388dd9b640 1 --2- 172.21.15.6:0/805813059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38901012e0 0x7f389019ac00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3884005ab0 tx=0x7f38840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:42.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.246+0000 7f387f7fe640 1 -- 172.21.15.6:0/805813059 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388403f030 con 0x7f38901012e0 2024-09-12T03:18:42.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.246+0000 7f3894ee7640 1 -- 172.21.15.6:0/805813059 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f389019f900 con 0x7f38901012e0 2024-09-12T03:18:42.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.246+0000 7f387f7fe640 1 -- 172.21.15.6:0/805813059 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3884003ce0 con 0x7f38901012e0 2024-09-12T03:18:42.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.246+0000 7f387f7fe640 1 -- 172.21.15.6:0/805813059 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884035070 con 0x7f38901012e0 2024-09-12T03:18:42.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.246+0000 7f3894ee7640 1 -- 172.21.15.6:0/805813059 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38901a7240 con 0x7f38901012e0 2024-09-12T03:18:42.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.247+0000 7f3894ee7640 1 -- 172.21.15.6:0/805813059 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f389006b010 con 0x7f38901012e0 2024-09-12T03:18:42.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.248+0000 7f387f7fe640 1 -- 172.21.15.6:0/805813059 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f3884042030 con 0x7f38901012e0 2024-09-12T03:18:42.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.249+0000 7f387f7fe640 1 --2- 172.21.15.6:0/805813059 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38780a2b80 0x7f38780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:42.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.249+0000 7f388e59c640 1 --2- 172.21.15.6:0/805813059 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38780a2b80 0x7f38780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:42.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.249+0000 7f387f7fe640 1 -- 172.21.15.6:0/805813059 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f38840ef740 con 0x7f38901012e0 2024-09-12T03:18:42.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.249+0000 7f388e59c640 1 --2- 172.21.15.6:0/805813059 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38780a2b80 0x7f38780a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f38800058c0 tx=0x7f388000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:42.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.252+0000 7f387f7fe640 1 -- 172.21.15.6:0/805813059 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f38840bcb40 con 0x7f38901012e0 2024-09-12T03:18:42.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.471+0000 7f3894ee7640 1 -- 172.21.15.6:0/805813059 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3890100d10 con 0x7f38901012e0 2024-09-12T03:18:42.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.472+0000 7f387f7fe640 1 -- 172.21.15.6:0/805813059 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3910 (secure 0 0 0) 0x7f3890100d10 con 0x7f38901012e0 2024-09-12T03:18:42.473 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:18:42.473 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":5,"btime":"2024-09-12T03:18:40:339273+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-12T03:18:42.474 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 5 2024-09-12T03:18:42.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.474+0000 7f3894ee7640 1 -- 172.21.15.6:0/805813059 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38780a2b80 msgr2=0x7f38780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:42.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.474+0000 7f3894ee7640 1 --2- 172.21.15.6:0/805813059 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38780a2b80 0x7f38780a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f38800058c0 tx=0x7f388000a2d0 comp rx=0 tx=0).stop 2024-09-12T03:18:42.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.474+0000 7f3894ee7640 1 -- 172.21.15.6:0/805813059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38901012e0 msgr2=0x7f389019ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:42.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.474+0000 7f3894ee7640 1 --2- 172.21.15.6:0/805813059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38901012e0 0x7f389019ac00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3884005ab0 tx=0x7f38840042f0 comp rx=0 tx=0).stop 2024-09-12T03:18:42.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.474+0000 7f3894ee7640 1 -- 172.21.15.6:0/805813059 shutdown_connections 2024-09-12T03:18:42.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.474+0000 7f3894ee7640 1 --2- 172.21.15.6:0/805813059 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38780a2b80 0x7f38780a5040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:42.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.474+0000 7f3894ee7640 1 --2- 172.21.15.6:0/805813059 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f389010e4e0 0x7f389019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:42.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.474+0000 7f3894ee7640 1 --2- 172.21.15.6:0/805813059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38901012e0 0x7f389019ac00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:42.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.474+0000 7f3894ee7640 1 --2- 172.21.15.6:0/805813059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3890100930 0x7f389019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:42.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.474+0000 7f3894ee7640 1 -- 172.21.15.6:0/805813059 >> 172.21.15.6:0/805813059 conn(0x7f38900fc7a0 msgr2=0x7f38900feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:42.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.474+0000 7f3894ee7640 1 -- 172.21.15.6:0/805813059 shutdown_connections 2024-09-12T03:18:42.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:42.474+0000 7f3894ee7640 1 -- 172.21.15.6:0/805813059 wait complete. 2024-09-12T03:18:42.712 INFO:journalctl@ceph.mds.e.smithi118.stdout:Sep 12 03:18:42 smithi118 podman[64885]: 2024-09-12 03:18:42.266858986 +0000 UTC m=+0.373671013 container start 3075a89eb6ea44a7e415849ca256c16068f0e9fb6387a5465c1a103169a43fc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-e, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:18:42.712 INFO:journalctl@ceph.mds.e.smithi118.stdout:Sep 12 03:18:42 smithi118 ceph-mds[64904]: -- 172.21.15.118:0/1634612798 <== mon.2 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cb69951860 con 0x55cb6992f400 2024-09-12T03:18:42.713 INFO:journalctl@ceph.mds.e.smithi118.stdout:Sep 12 03:18:42 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-e[64900]: starting mds.e at 2024-09-12T03:18:42.713 INFO:journalctl@ceph.mds.e.smithi118.stdout:Sep 12 03:18:42 smithi118 bash[64885]: 3075a89eb6ea44a7e415849ca256c16068f0e9fb6387a5465c1a103169a43fc0 2024-09-12T03:18:42.713 INFO:journalctl@ceph.mds.e.smithi118.stdout:Sep 12 03:18:42 smithi118 systemd[1]: Started Ceph mds.e for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:18:43.028 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:42 smithi161 ceph-mon[35485]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:43.028 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:43.028 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:43.028 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:43.028 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:43.028 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:43.028 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:42 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:43.029 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:42 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/805813059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:43.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:42 smithi006 ceph-mon[30991]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:43.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:43.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:43.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:43.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:43.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:43.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:42 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:43.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:42 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/805813059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:43.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:42 smithi118 ceph-mon[35048]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:43.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:42 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:43.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:42 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:43.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:42 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:43.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:42 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:43.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:42 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:43.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:42 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:43.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:42 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/805813059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:44.106 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:43 smithi161 ceph-mon[35485]: Deploying daemon mds.f on smithi161 2024-09-12T03:18:44.107 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:43 smithi161 ceph-mon[35485]: mds.? [v2:172.21.15.118:6834/1389654073,v1:172.21.15.118:6835/1389654073] up:boot 2024-09-12T03:18:44.107 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:43 smithi161 ceph-mon[35485]: fsmap 5 up:standby 2024-09-12T03:18:44.107 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:43 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:18:44.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:43 smithi006 ceph-mon[30991]: Deploying daemon mds.f on smithi161 2024-09-12T03:18:44.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:43 smithi006 ceph-mon[30991]: mds.? [v2:172.21.15.118:6834/1389654073,v1:172.21.15.118:6835/1389654073] up:boot 2024-09-12T03:18:44.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:43 smithi006 ceph-mon[30991]: fsmap 5 up:standby 2024-09-12T03:18:44.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:43 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:18:44.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:43 smithi118 ceph-mon[35048]: Deploying daemon mds.f on smithi161 2024-09-12T03:18:44.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:43 smithi118 ceph-mon[35048]: mds.? [v2:172.21.15.118:6834/1389654073,v1:172.21.15.118:6835/1389654073] up:boot 2024-09-12T03:18:44.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:43 smithi118 ceph-mon[35048]: fsmap 5 up:standby 2024-09-12T03:18:44.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:43 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:18:44.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:44 smithi118 ceph-mon[35048]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:45.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:44 smithi006 ceph-mon[30991]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:45.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:44 smithi161 ceph-mon[35485]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:45.113 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 12 03:18:44 smithi161 systemd[1]: Started Ceph mds.f for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:18:45.193 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:18:45.461 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:45.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:45 smithi006 ceph-mon[30991]: mds.? [v2:172.21.15.161:6834/955127607,v1:172.21.15.161:6835/955127607] up:boot 2024-09-12T03:18:45.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:45 smithi006 ceph-mon[30991]: fsmap 6 up:standby 2024-09-12T03:18:45.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:45 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:18:45.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:45 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:45.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:45 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:45.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:45 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:45.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:45 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:45.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:45 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:45.863 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:45 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:45.863 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:45 smithi006 ceph-mon[30991]: Deploying daemon mds.c on smithi161 2024-09-12T03:18:45.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.975+0000 7f0abebcc640 1 -- 172.21.15.6:0/1607414807 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab8106f10 msgr2=0x7f0ab810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:45.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.975+0000 7f0abebcc640 1 --2- 172.21.15.6:0/1607414807 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab8106f10 0x7f0ab810ea90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009ca0 tx=0x7f0ab002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:18:45.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.975+0000 7f0abebcc640 1 -- 172.21.15.6:0/1607414807 shutdown_connections 2024-09-12T03:18:45.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.975+0000 7f0abebcc640 1 --2- 172.21.15.6:0/1607414807 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab8106f10 0x7f0ab810ea90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:45.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.975+0000 7f0abebcc640 1 --2- 172.21.15.6:0/1607414807 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ab8102790 0x7f0ab81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:45.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.975+0000 7f0abebcc640 1 --2- 172.21.15.6:0/1607414807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ab8101de0 0x7f0ab81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:45.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.975+0000 7f0abebcc640 1 -- 172.21.15.6:0/1607414807 >> 172.21.15.6:0/1607414807 conn(0x7f0ab80fd8c0 msgr2=0x7f0ab80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:45.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.975+0000 7f0abebcc640 1 -- 172.21.15.6:0/1607414807 shutdown_connections 2024-09-12T03:18:45.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.975+0000 7f0abebcc640 1 -- 172.21.15.6:0/1607414807 wait complete. 2024-09-12T03:18:45.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.976+0000 7f0abebcc640 1 Processor -- start 2024-09-12T03:18:45.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.976+0000 7f0abebcc640 1 -- start start 2024-09-12T03:18:45.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.976+0000 7f0abebcc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ab8101de0 0x7f0ab80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:45.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.976+0000 7f0abebcc640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ab8102790 0x7f0ab8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:45.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.976+0000 7f0abebcc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab8106f10 0x7f0ab8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:45.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.976+0000 7f0abebcc640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab8111fb0 con 0x7f0ab8106f10 2024-09-12T03:18:45.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.976+0000 7f0abebcc640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab8111e30 con 0x7f0ab8102790 2024-09-12T03:18:45.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.976+0000 7f0abebcc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab8112130 con 0x7f0ab8101de0 2024-09-12T03:18:45.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.976+0000 7f0abd169640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab8106f10 0x7f0ab8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:45.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.977+0000 7f0abd169640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab8106f10 0x7f0ab8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45860/0 (socket says 172.21.15.6:45860) 2024-09-12T03:18:45.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.977+0000 7f0abd169640 1 -- 172.21.15.6:0/3458887968 learned_addr learned my addr 172.21.15.6:0/3458887968 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:18:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.977+0000 7f0abc968640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ab8101de0 0x7f0ab80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.977+0000 7f0ab7fff640 1 --2- 172.21.15.6:0/3458887968 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ab8102790 0x7f0ab8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.977+0000 7f0abd169640 1 -- 172.21.15.6:0/3458887968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ab8101de0 msgr2=0x7f0ab80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.977+0000 7f0abd169640 1 --2- 172.21.15.6:0/3458887968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ab8101de0 0x7f0ab80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.977+0000 7f0abd169640 1 -- 172.21.15.6:0/3458887968 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ab8102790 msgr2=0x7f0ab8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.977+0000 7f0abd169640 1 --2- 172.21.15.6:0/3458887968 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ab8102790 0x7f0ab8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.977+0000 7f0abd169640 1 -- 172.21.15.6:0/3458887968 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab81a6950 con 0x7f0ab8106f10 2024-09-12T03:18:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.977+0000 7f0abc968640 1 --2- 172.21.15.6:0/3458887968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ab8101de0 0x7f0ab80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:18:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.977+0000 7f0abd169640 1 --2- 172.21.15.6:0/3458887968 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab8106f10 0x7f0ab8074f70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0031a20 tx=0x7f0ab0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.977+0000 7f0ab5ffb640 1 -- 172.21.15.6:0/3458887968 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0003c30 con 0x7f0ab8106f10 2024-09-12T03:18:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.977+0000 7f0ab5ffb640 1 -- 172.21.15.6:0/3458887968 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ab0003dd0 con 0x7f0ab8106f10 2024-09-12T03:18:45.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.977+0000 7f0ab5ffb640 1 -- 172.21.15.6:0/3458887968 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab002fcf0 con 0x7f0ab8106f10 2024-09-12T03:18:45.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.977+0000 7f0abebcc640 1 -- 172.21.15.6:0/3458887968 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab81a6be0 con 0x7f0ab8106f10 2024-09-12T03:18:45.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.978+0000 7f0abebcc640 1 -- 172.21.15.6:0/3458887968 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab81a7170 con 0x7f0ab8106f10 2024-09-12T03:18:45.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.978+0000 7f0abebcc640 1 -- 172.21.15.6:0/3458887968 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ab81021c0 con 0x7f0ab8106f10 2024-09-12T03:18:45.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.980+0000 7f0ab5ffb640 1 -- 172.21.15.6:0/3458887968 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f0ab0004050 con 0x7f0ab8106f10 2024-09-12T03:18:45.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.980+0000 7f0ab5ffb640 1 --2- 172.21.15.6:0/3458887968 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0aa40a2c00 0x7f0aa40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:45.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.980+0000 7f0abc968640 1 --2- 172.21.15.6:0/3458887968 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0aa40a2c00 0x7f0aa40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:45.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.980+0000 7f0ab5ffb640 1 -- 172.21.15.6:0/3458887968 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f0ab00efeb0 con 0x7f0ab8106f10 2024-09-12T03:18:45.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.981+0000 7f0abc968640 1 --2- 172.21.15.6:0/3458887968 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0aa40a2c00 0x7f0aa40a50c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8005fd0 tx=0x7f0aa800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:45.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:45.985+0000 7f0ab5ffb640 1 -- 172.21.15.6:0/3458887968 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0ab00bd460 con 0x7f0ab8106f10 2024-09-12T03:18:46.000 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:45 smithi161 ceph-mon[35485]: mds.? [v2:172.21.15.161:6834/955127607,v1:172.21.15.161:6835/955127607] up:boot 2024-09-12T03:18:46.000 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:45 smithi161 ceph-mon[35485]: fsmap 6 up:standby 2024-09-12T03:18:46.001 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:45 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:18:46.001 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:45 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:46.001 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:45 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:46.001 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:45 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:46.001 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:45 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:46.001 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:45 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:46.001 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:45 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:46.001 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:45 smithi161 ceph-mon[35485]: Deploying daemon mds.c on smithi161 2024-09-12T03:18:46.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:46.195+0000 7f0abebcc640 1 -- 172.21.15.6:0/3458887968 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ab810a0f0 con 0x7f0ab8106f10 2024-09-12T03:18:46.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:46.196+0000 7f0ab5ffb640 1 -- 172.21.15.6:0/3458887968 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5563 (secure 0 0 0) 0x7f0ab002fe90 con 0x7f0ab8106f10 2024-09-12T03:18:46.196 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:18:46.196 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":7,"btime":"2024-09-12T03:18:44:782744+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-12T03:18:46.196 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 7 2024-09-12T03:18:46.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:46.198+0000 7f0abebcc640 1 -- 172.21.15.6:0/3458887968 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0aa40a2c00 msgr2=0x7f0aa40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:46.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:46.198+0000 7f0abebcc640 1 --2- 172.21.15.6:0/3458887968 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0aa40a2c00 0x7f0aa40a50c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8005fd0 tx=0x7f0aa800b390 comp rx=0 tx=0).stop 2024-09-12T03:18:46.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:46.198+0000 7f0abebcc640 1 -- 172.21.15.6:0/3458887968 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab8106f10 msgr2=0x7f0ab8074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:46.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:46.198+0000 7f0abebcc640 1 --2- 172.21.15.6:0/3458887968 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab8106f10 0x7f0ab8074f70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0031a20 tx=0x7f0ab0031a50 comp rx=0 tx=0).stop 2024-09-12T03:18:46.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:46.198+0000 7f0abebcc640 1 -- 172.21.15.6:0/3458887968 shutdown_connections 2024-09-12T03:18:46.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:46.198+0000 7f0abebcc640 1 --2- 172.21.15.6:0/3458887968 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0aa40a2c00 0x7f0aa40a50c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:46.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:46.198+0000 7f0abebcc640 1 --2- 172.21.15.6:0/3458887968 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab8106f10 0x7f0ab8074f70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:46.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:46.198+0000 7f0abebcc640 1 --2- 172.21.15.6:0/3458887968 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ab8102790 0x7f0ab8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:46.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:46.198+0000 7f0abebcc640 1 --2- 172.21.15.6:0/3458887968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ab8101de0 0x7f0ab80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:46.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:46.198+0000 7f0abebcc640 1 -- 172.21.15.6:0/3458887968 >> 172.21.15.6:0/3458887968 conn(0x7f0ab80fd8c0 msgr2=0x7f0ab80fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:46.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:46.198+0000 7f0abebcc640 1 -- 172.21.15.6:0/3458887968 shutdown_connections 2024-09-12T03:18:46.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:46.198+0000 7f0abebcc640 1 -- 172.21.15.6:0/3458887968 wait complete. 2024-09-12T03:18:46.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:45 smithi118 ceph-mon[35048]: mds.? [v2:172.21.15.161:6834/955127607,v1:172.21.15.161:6835/955127607] up:boot 2024-09-12T03:18:46.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:45 smithi118 ceph-mon[35048]: fsmap 6 up:standby 2024-09-12T03:18:46.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:45 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:18:46.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:45 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:46.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:45 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:46.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:45 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:46.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:45 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:46.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:45 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:46.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:45 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:46.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:45 smithi118 ceph-mon[35048]: Deploying daemon mds.c on smithi161 2024-09-12T03:18:47.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:46 smithi161 ceph-mon[35485]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:47.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:46 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3458887968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:47.116 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 12 03:18:46 smithi161 systemd[1]: Starting Ceph mds.c for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:18:47.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:46 smithi118 ceph-mon[35048]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:47.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:46 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3458887968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:47.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:46 smithi006 ceph-mon[30991]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:47.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:46 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3458887968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:47.440 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 12 03:18:47 smithi161 podman[66050]: 2024-09-12 03:18:47.080279365 +0000 UTC m=+0.015729780 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:18:47.440 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 12 03:18:47 smithi161 podman[66050]: 2024-09-12 03:18:47.17884168 +0000 UTC m=+0.114292082 container create 9abf9587f1ed710013ce0d4893f616920182d479a5655abe604af13a88120f57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-c, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-12T03:18:47.441 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 12 03:18:47 smithi161 podman[66050]: 2024-09-12 03:18:47.414792822 +0000 UTC m=+0.350243225 container init 9abf9587f1ed710013ce0d4893f616920182d479a5655abe604af13a88120f57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-c, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-12T03:18:47.441 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 12 03:18:47 smithi161 podman[66050]: 2024-09-12 03:18:47.417616685 +0000 UTC m=+0.353067088 container start 9abf9587f1ed710013ce0d4893f616920182d479a5655abe604af13a88120f57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2) 2024-09-12T03:18:47.441 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 12 03:18:47 smithi161 ceph-mds[66069]: -- 172.21.15.161:0/120954272 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5620eb52d860 con 0x5620eb50b000 2024-09-12T03:18:47.863 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 12 03:18:47 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-c[66065]: starting mds.c at 2024-09-12T03:18:47.863 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 12 03:18:47 smithi161 bash[66050]: 9abf9587f1ed710013ce0d4893f616920182d479a5655abe604af13a88120f57 2024-09-12T03:18:47.863 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 12 03:18:47 smithi161 systemd[1]: Started Ceph mds.c for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:18:48.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:48 smithi118 ceph-mon[35048]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:48.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:48 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:48.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:48 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:48.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:48 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:48.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:48 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:48.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:48 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:48.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:48 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:48.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:48 smithi118 ceph-mon[35048]: Deploying daemon mds.h on smithi118 2024-09-12T03:18:48.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:48 smithi118 ceph-mon[35048]: mds.? [v2:172.21.15.161:6836/283024944,v1:172.21.15.161:6837/283024944] up:boot 2024-09-12T03:18:48.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:48 smithi118 ceph-mon[35048]: fsmap 7 up:standby 2024-09-12T03:18:48.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:48 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:18:48.786 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:18:48.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:48 smithi006 ceph-mon[30991]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:48.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:48 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:48.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:48 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:48.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:48 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:48.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:48 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:48.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:48 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:48.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:48 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:48.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:48 smithi006 ceph-mon[30991]: Deploying daemon mds.h on smithi118 2024-09-12T03:18:48.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:48 smithi006 ceph-mon[30991]: mds.? [v2:172.21.15.161:6836/283024944,v1:172.21.15.161:6837/283024944] up:boot 2024-09-12T03:18:48.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:48 smithi006 ceph-mon[30991]: fsmap 7 up:standby 2024-09-12T03:18:48.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:48 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:18:48.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:48 smithi161 ceph-mon[35485]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:48.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:48 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:48.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:48 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:48.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:48 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:48.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:48 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:48.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:48 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:48.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:48 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:48.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:48 smithi161 ceph-mon[35485]: Deploying daemon mds.h on smithi118 2024-09-12T03:18:48.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:48 smithi161 ceph-mon[35485]: mds.? [v2:172.21.15.161:6836/283024944,v1:172.21.15.161:6837/283024944] up:boot 2024-09-12T03:18:48.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:48 smithi161 ceph-mon[35485]: fsmap 7 up:standby 2024-09-12T03:18:48.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:48 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:18:49.054 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:49.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.490+0000 7ff30470e640 1 -- 172.21.15.6:0/672827651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2fc103740 msgr2=0x7ff2fc107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:49.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.490+0000 7ff30470e640 1 --2- 172.21.15.6:0/672827651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2fc103740 0x7ff2fc107b80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009930 tx=0x7ff2ec02a420 comp rx=0 tx=0).stop 2024-09-12T03:18:49.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.490+0000 7ff30470e640 1 -- 172.21.15.6:0/672827651 shutdown_connections 2024-09-12T03:18:49.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.490+0000 7ff30470e640 1 --2- 172.21.15.6:0/672827651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2fc108290 0x7ff2fc10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:49.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.490+0000 7ff30470e640 1 --2- 172.21.15.6:0/672827651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2fc103740 0x7ff2fc107b80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:49.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.490+0000 7ff30470e640 1 --2- 172.21.15.6:0/672827651 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2fc102d90 0x7ff2fc103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:49.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.490+0000 7ff30470e640 1 -- 172.21.15.6:0/672827651 >> 172.21.15.6:0/672827651 conn(0x7ff2fc0fc7d0 msgr2=0x7ff2fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:49.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.490+0000 7ff30470e640 1 -- 172.21.15.6:0/672827651 shutdown_connections 2024-09-12T03:18:49.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.490+0000 7ff30470e640 1 -- 172.21.15.6:0/672827651 wait complete. 2024-09-12T03:18:49.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.491+0000 7ff30470e640 1 Processor -- start 2024-09-12T03:18:49.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.491+0000 7ff30470e640 1 -- start start 2024-09-12T03:18:49.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.491+0000 7ff30470e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2fc102d90 0x7ff2fc1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:49.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.491+0000 7ff30470e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2fc103740 0x7ff2fc100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:49.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.491+0000 7ff30470e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2fc108290 0x7ff2fc101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:49.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.491+0000 7ff30470e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc112090 con 0x7ff2fc103740 2024-09-12T03:18:49.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.491+0000 7ff30470e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc111f10 con 0x7ff2fc102d90 2024-09-12T03:18:49.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.491+0000 7ff30470e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc112210 con 0x7ff2fc108290 2024-09-12T03:18:49.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.491+0000 7ff301ca9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2fc103740 0x7ff2fc100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:49.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.492+0000 7ff301ca9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2fc103740 0x7ff2fc100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39246/0 (socket says 172.21.15.6:39246) 2024-09-12T03:18:49.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.492+0000 7ff3024aa640 1 --2- 172.21.15.6:0/1083130039 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2fc102d90 0x7ff2fc1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:49.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.492+0000 7ff301ca9640 1 -- 172.21.15.6:0/1083130039 learned_addr learned my addr 172.21.15.6:0/1083130039 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:18:49.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.492+0000 7ff302cab640 1 --2- 172.21.15.6:0/1083130039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2fc108290 0x7ff2fc101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:49.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.492+0000 7ff301ca9640 1 -- 172.21.15.6:0/1083130039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2fc108290 msgr2=0x7ff2fc101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:49.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.492+0000 7ff301ca9640 1 --2- 172.21.15.6:0/1083130039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2fc108290 0x7ff2fc101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:49.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.492+0000 7ff301ca9640 1 -- 172.21.15.6:0/1083130039 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2fc102d90 msgr2=0x7ff2fc1009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:49.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.492+0000 7ff301ca9640 1 --2- 172.21.15.6:0/1083130039 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2fc102d90 0x7ff2fc1009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:49.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.492+0000 7ff301ca9640 1 -- 172.21.15.6:0/1083130039 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2fc1a6a70 con 0x7ff2fc103740 2024-09-12T03:18:49.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.492+0000 7ff301ca9640 1 --2- 172.21.15.6:0/1083130039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2fc103740 0x7ff2fc100f30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec005940 tx=0x7ff2ec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:49.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.492+0000 7ff2f37fe640 1 -- 172.21.15.6:0/1083130039 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec03f030 con 0x7ff2fc103740 2024-09-12T03:18:49.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.492+0000 7ff30470e640 1 -- 172.21.15.6:0/1083130039 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2fc1a6d00 con 0x7ff2fc103740 2024-09-12T03:18:49.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.492+0000 7ff2f37fe640 1 -- 172.21.15.6:0/1083130039 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2ec003ce0 con 0x7ff2fc103740 2024-09-12T03:18:49.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.492+0000 7ff2f37fe640 1 -- 172.21.15.6:0/1083130039 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec035070 con 0x7ff2fc103740 2024-09-12T03:18:49.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.492+0000 7ff30470e640 1 -- 172.21.15.6:0/1083130039 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2fc1a7190 con 0x7ff2fc103740 2024-09-12T03:18:49.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.493+0000 7ff30470e640 1 -- 172.21.15.6:0/1083130039 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2fc103170 con 0x7ff2fc103740 2024-09-12T03:18:49.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.494+0000 7ff2f37fe640 1 -- 172.21.15.6:0/1083130039 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ff2ec042030 con 0x7ff2fc103740 2024-09-12T03:18:49.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.495+0000 7ff2f37fe640 1 --2- 172.21.15.6:0/1083130039 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff2e80a2b80 0x7ff2e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:49.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.495+0000 7ff3024aa640 1 --2- 172.21.15.6:0/1083130039 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff2e80a2b80 0x7ff2e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:49.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.496+0000 7ff2f37fe640 1 -- 172.21.15.6:0/1083130039 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7ff2ec0ef010 con 0x7ff2fc103740 2024-09-12T03:18:49.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.496+0000 7ff3024aa640 1 --2- 172.21.15.6:0/1083130039 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff2e80a2b80 0x7ff2e80a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800bbe0 tx=0x7ff2f800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:49.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.500+0000 7ff2f37fe640 1 -- 172.21.15.6:0/1083130039 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2ec0bbdc0 con 0x7ff2fc103740 2024-09-12T03:18:49.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.710+0000 7ff30470e640 1 -- 172.21.15.6:0/1083130039 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2fc10a2a0 con 0x7ff2fc103740 2024-09-12T03:18:49.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.711+0000 7ff2f37fe640 1 -- 172.21.15.6:0/1083130039 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6388 (secure 0 0 0) 0x7ff2ec0c00c0 con 0x7ff2fc103740 2024-09-12T03:18:49.711 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:18:49.711 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":8,"btime":"2024-09-12T03:18:47:913871+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-12T03:18:49.711 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 8 2024-09-12T03:18:49.712 INFO:journalctl@ceph.mds.h.smithi118.stdout:Sep 12 03:18:49 smithi118 systemd[1]: Starting Ceph mds.h for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:18:49.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.713+0000 7ff30470e640 1 -- 172.21.15.6:0/1083130039 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff2e80a2b80 msgr2=0x7ff2e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:49.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.713+0000 7ff30470e640 1 --2- 172.21.15.6:0/1083130039 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff2e80a2b80 0x7ff2e80a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800bbe0 tx=0x7ff2f800b390 comp rx=0 tx=0).stop 2024-09-12T03:18:49.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.713+0000 7ff30470e640 1 -- 172.21.15.6:0/1083130039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2fc103740 msgr2=0x7ff2fc100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:49.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.713+0000 7ff30470e640 1 --2- 172.21.15.6:0/1083130039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2fc103740 0x7ff2fc100f30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec005940 tx=0x7ff2ec0042f0 comp rx=0 tx=0).stop 2024-09-12T03:18:49.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.713+0000 7ff30470e640 1 -- 172.21.15.6:0/1083130039 shutdown_connections 2024-09-12T03:18:49.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.713+0000 7ff30470e640 1 --2- 172.21.15.6:0/1083130039 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff2e80a2b80 0x7ff2e80a5040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:49.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.713+0000 7ff30470e640 1 --2- 172.21.15.6:0/1083130039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2fc108290 0x7ff2fc101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:49.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.713+0000 7ff30470e640 1 --2- 172.21.15.6:0/1083130039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2fc103740 0x7ff2fc100f30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:49.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.713+0000 7ff30470e640 1 --2- 172.21.15.6:0/1083130039 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2fc102d90 0x7ff2fc1009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:49.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.713+0000 7ff30470e640 1 -- 172.21.15.6:0/1083130039 >> 172.21.15.6:0/1083130039 conn(0x7ff2fc0fc7d0 msgr2=0x7ff2fc106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:49.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.713+0000 7ff30470e640 1 -- 172.21.15.6:0/1083130039 shutdown_connections 2024-09-12T03:18:49.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:49.713+0000 7ff30470e640 1 -- 172.21.15.6:0/1083130039 wait complete. 2024-09-12T03:18:50.086 INFO:journalctl@ceph.mds.h.smithi118.stdout:Sep 12 03:18:49 smithi118 podman[65187]: 2024-09-12 03:18:49.720891006 +0000 UTC m=+0.016249359 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:18:50.086 INFO:journalctl@ceph.mds.h.smithi118.stdout:Sep 12 03:18:49 smithi118 podman[65187]: 2024-09-12 03:18:49.840727285 +0000 UTC m=+0.136085616 container create 8070a91c59c69526155be6cb0a1840fed9f6b0c27275e388ad1ca6fb9e195842 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-h, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-12T03:18:50.086 INFO:journalctl@ceph.mds.h.smithi118.stdout:Sep 12 03:18:50 smithi118 podman[65187]: 2024-09-12 03:18:50.05958739 +0000 UTC m=+0.354945721 container init 8070a91c59c69526155be6cb0a1840fed9f6b0c27275e388ad1ca6fb9e195842 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-h, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-12T03:18:50.087 INFO:journalctl@ceph.mds.h.smithi118.stdout:Sep 12 03:18:50 smithi118 podman[65187]: 2024-09-12 03:18:50.062438251 +0000 UTC m=+0.357796599 container start 8070a91c59c69526155be6cb0a1840fed9f6b0c27275e388ad1ca6fb9e195842 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD) 2024-09-12T03:18:50.087 INFO:journalctl@ceph.mds.h.smithi118.stdout:Sep 12 03:18:50 smithi118 ceph-mds[65207]: -- 172.21.15.118:0/1554896274 <== mon.2 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558e8848b860 con 0x558e88469400 2024-09-12T03:18:50.462 INFO:journalctl@ceph.mds.h.smithi118.stdout:Sep 12 03:18:50 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-h[65203]: starting mds.h at 2024-09-12T03:18:50.462 INFO:journalctl@ceph.mds.h.smithi118.stdout:Sep 12 03:18:50 smithi118 bash[65187]: 8070a91c59c69526155be6cb0a1840fed9f6b0c27275e388ad1ca6fb9e195842 2024-09-12T03:18:50.462 INFO:journalctl@ceph.mds.h.smithi118.stdout:Sep 12 03:18:50 smithi118 systemd[1]: Started Ceph mds.h for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:18:50.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:50 smithi006 ceph-mon[30991]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:50.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:50 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1083130039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:50.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:50 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:50.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:50 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:50.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:50 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:50.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:50 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:50.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:50 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:50.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:50 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:50.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:50 smithi161 ceph-mon[35485]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:50.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:50 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1083130039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:50.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:50 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:50.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:50 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:50.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:50 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:50.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:50 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:50.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:50 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:50.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:50 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:50.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:50 smithi118 ceph-mon[35048]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:50.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:50 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1083130039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:50.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:50 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:50.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:50 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:50.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:50 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:50.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:50 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:50.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:50 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:50.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:50 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:51.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:51 smithi006 ceph-mon[30991]: Deploying daemon mds.i on smithi161 2024-09-12T03:18:51.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:51 smithi006 ceph-mon[30991]: mds.? [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] up:boot 2024-09-12T03:18:51.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:51 smithi006 ceph-mon[30991]: fsmap 8 up:standby 2024-09-12T03:18:51.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:51 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:18:51.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:51 smithi161 ceph-mon[35485]: Deploying daemon mds.i on smithi161 2024-09-12T03:18:51.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:51 smithi161 ceph-mon[35485]: mds.? [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] up:boot 2024-09-12T03:18:51.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:51 smithi161 ceph-mon[35485]: fsmap 8 up:standby 2024-09-12T03:18:51.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:51 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:18:51.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:51 smithi118 ceph-mon[35048]: Deploying daemon mds.i on smithi161 2024-09-12T03:18:51.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:51 smithi118 ceph-mon[35048]: mds.? [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] up:boot 2024-09-12T03:18:51.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:51 smithi118 ceph-mon[35048]: fsmap 8 up:standby 2024-09-12T03:18:51.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:51 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:18:52.293 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:18:52.363 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 12 03:18:52 smithi161 systemd[1]: Starting Ceph mds.i for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:18:52.564 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:52.706 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:52 smithi161 ceph-mon[35485]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:52.706 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 12 03:18:52 smithi161 podman[66353]: 2024-09-12 03:18:52.331192766 +0000 UTC m=+0.016090421 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:18:52.706 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 12 03:18:52 smithi161 podman[66353]: 2024-09-12 03:18:52.440858222 +0000 UTC m=+0.125755912 container create 12bd8a72daec7123574e5cf07bbf51a9f89794f2d61fa69b0271e2c1918a6749 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-12T03:18:52.706 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 12 03:18:52 smithi161 podman[66353]: 2024-09-12 03:18:52.707067771 +0000 UTC m=+0.391965423 container init 12bd8a72daec7123574e5cf07bbf51a9f89794f2d61fa69b0271e2c1918a6749 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-i, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:18:52.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:52 smithi006 ceph-mon[30991]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:52.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:52 smithi118 ceph-mon[35048]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:53.114 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 12 03:18:52 smithi161 podman[66353]: 2024-09-12 03:18:52.710003887 +0000 UTC m=+0.394901536 container start 12bd8a72daec7123574e5cf07bbf51a9f89794f2d61fa69b0271e2c1918a6749 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-i, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-12T03:18:53.115 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 12 03:18:52 smithi161 ceph-mds[66372]: -- 172.21.15.161:0/16914679 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b7928dd860 con 0x55b7928bb000 2024-09-12T03:18:53.115 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 12 03:18:52 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-i[66368]: starting mds.i at 2024-09-12T03:18:53.115 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 12 03:18:52 smithi161 bash[66353]: 12bd8a72daec7123574e5cf07bbf51a9f89794f2d61fa69b0271e2c1918a6749 2024-09-12T03:18:53.115 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 12 03:18:52 smithi161 systemd[1]: Started Ceph mds.i for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:18:53.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.176+0000 7fd933a57640 1 -- 172.21.15.6:0/4099829819 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd92c102570 msgr2=0x7fd92c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:53.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.176+0000 7fd933a57640 1 --2- 172.21.15.6:0/4099829819 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd92c102570 0x7fd92c1029b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd920009900 tx=0x7fd92002a480 comp rx=0 tx=0).stop 2024-09-12T03:18:53.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.176+0000 7fd933a57640 1 -- 172.21.15.6:0/4099829819 shutdown_connections 2024-09-12T03:18:53.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.176+0000 7fd933a57640 1 --2- 172.21.15.6:0/4099829819 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd92c102ef0 0x7fd92c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:53.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.176+0000 7fd933a57640 1 --2- 172.21.15.6:0/4099829819 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd92c102570 0x7fd92c1029b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:53.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.176+0000 7fd933a57640 1 --2- 172.21.15.6:0/4099829819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd92c100e20 0x7fd92c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:53.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.176+0000 7fd933a57640 1 -- 172.21.15.6:0/4099829819 >> 172.21.15.6:0/4099829819 conn(0x7fd92c0fc7d0 msgr2=0x7fd92c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:53.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.176+0000 7fd933a57640 1 -- 172.21.15.6:0/4099829819 shutdown_connections 2024-09-12T03:18:53.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.176+0000 7fd933a57640 1 -- 172.21.15.6:0/4099829819 wait complete. 2024-09-12T03:18:53.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.177+0000 7fd933a57640 1 Processor -- start 2024-09-12T03:18:53.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.177+0000 7fd933a57640 1 -- start start 2024-09-12T03:18:53.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.177+0000 7fd933a57640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd92c100e20 0x7fd92c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:53.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.177+0000 7fd933a57640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd92c102570 0x7fd92c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:53.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.177+0000 7fd933a57640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd92c102ef0 0x7fd92c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:53.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.177+0000 7fd933a57640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c110f80 con 0x7fd92c102570 2024-09-12T03:18:53.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.177+0000 7fd933a57640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c110e00 con 0x7fd92c100e20 2024-09-12T03:18:53.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.177+0000 7fd933a57640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c111100 con 0x7fd92c102ef0 2024-09-12T03:18:53.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.177+0000 7fd930ff2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd92c102570 0x7fd92c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:53.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.177+0000 7fd930ff2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd92c102570 0x7fd92c0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39272/0 (socket says 172.21.15.6:39272) 2024-09-12T03:18:53.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.177+0000 7fd930ff2640 1 -- 172.21.15.6:0/2187515364 learned_addr learned my addr 172.21.15.6:0/2187515364 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:18:53.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.177+0000 7fd9317f3640 1 --2- 172.21.15.6:0/2187515364 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd92c100e20 0x7fd92c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:53.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.177+0000 7fd931ff4640 1 --2- 172.21.15.6:0/2187515364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd92c102ef0 0x7fd92c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:53.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.177+0000 7fd930ff2640 1 -- 172.21.15.6:0/2187515364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd92c102ef0 msgr2=0x7fd92c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:53.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.178+0000 7fd930ff2640 1 --2- 172.21.15.6:0/2187515364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd92c102ef0 0x7fd92c0fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:53.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.178+0000 7fd930ff2640 1 -- 172.21.15.6:0/2187515364 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd92c100e20 msgr2=0x7fd92c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:53.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.178+0000 7fd930ff2640 1 --2- 172.21.15.6:0/2187515364 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd92c100e20 0x7fd92c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:53.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.178+0000 7fd930ff2640 1 -- 172.21.15.6:0/2187515364 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd92c1a69b0 con 0x7fd92c102570 2024-09-12T03:18:53.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.178+0000 7fd930ff2640 1 --2- 172.21.15.6:0/2187515364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd92c102570 0x7fd92c0fd1c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd920006010 tx=0x7fd920003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:53.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.178+0000 7fd92a7fc640 1 -- 172.21.15.6:0/2187515364 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92003f070 con 0x7fd92c102570 2024-09-12T03:18:53.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.178+0000 7fd92a7fc640 1 -- 172.21.15.6:0/2187515364 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd920031c30 con 0x7fd92c102570 2024-09-12T03:18:53.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.178+0000 7fd933a57640 1 -- 172.21.15.6:0/2187515364 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd92c1a6c40 con 0x7fd92c102570 2024-09-12T03:18:53.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.178+0000 7fd92a7fc640 1 -- 172.21.15.6:0/2187515364 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd920035030 con 0x7fd92c102570 2024-09-12T03:18:53.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.178+0000 7fd933a57640 1 -- 172.21.15.6:0/2187515364 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd92c1a7170 con 0x7fd92c102570 2024-09-12T03:18:53.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.179+0000 7fd933a57640 1 -- 172.21.15.6:0/2187515364 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd92c101200 con 0x7fd92c102570 2024-09-12T03:18:53.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.180+0000 7fd92a7fc640 1 -- 172.21.15.6:0/2187515364 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd920042070 con 0x7fd92c102570 2024-09-12T03:18:53.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.181+0000 7fd92a7fc640 1 --2- 172.21.15.6:0/2187515364 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd9140a2c00 0x7fd9140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:53.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.181+0000 7fd9317f3640 1 --2- 172.21.15.6:0/2187515364 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd9140a2c00 0x7fd9140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:53.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.181+0000 7fd92a7fc640 1 -- 172.21.15.6:0/2187515364 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fd9200ef250 con 0x7fd92c102570 2024-09-12T03:18:53.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.182+0000 7fd9317f3640 1 --2- 172.21.15.6:0/2187515364 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd9140a2c00 0x7fd9140a50c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd91c005fd0 tx=0x7fd91c009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:53.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.186+0000 7fd92a7fc640 1 -- 172.21.15.6:0/2187515364 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd9200bc650 con 0x7fd92c102570 2024-09-12T03:18:53.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.408+0000 7fd933a57640 1 -- 172.21.15.6:0/2187515364 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd92c0fe1b0 con 0x7fd92c102570 2024-09-12T03:18:53.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.409+0000 7fd92a7fc640 1 -- 172.21.15.6:0/2187515364 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7216 (secure 0 0 0) 0x7fd92c0fe1b0 con 0x7fd92c102570 2024-09-12T03:18:53.410 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:18:53.410 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":9,"btime":"2024-09-12T03:18:50:574918+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-12T03:18:53.410 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 9 2024-09-12T03:18:53.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.411+0000 7fd933a57640 1 -- 172.21.15.6:0/2187515364 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd9140a2c00 msgr2=0x7fd9140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:53.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.411+0000 7fd933a57640 1 --2- 172.21.15.6:0/2187515364 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd9140a2c00 0x7fd9140a50c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd91c005fd0 tx=0x7fd91c009680 comp rx=0 tx=0).stop 2024-09-12T03:18:53.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.411+0000 7fd933a57640 1 -- 172.21.15.6:0/2187515364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd92c102570 msgr2=0x7fd92c0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:53.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.411+0000 7fd933a57640 1 --2- 172.21.15.6:0/2187515364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd92c102570 0x7fd92c0fd1c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd920006010 tx=0x7fd920003c10 comp rx=0 tx=0).stop 2024-09-12T03:18:53.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.411+0000 7fd933a57640 1 -- 172.21.15.6:0/2187515364 shutdown_connections 2024-09-12T03:18:53.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.411+0000 7fd933a57640 1 --2- 172.21.15.6:0/2187515364 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd9140a2c00 0x7fd9140a50c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:53.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.411+0000 7fd933a57640 1 --2- 172.21.15.6:0/2187515364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd92c102ef0 0x7fd92c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:53.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.411+0000 7fd933a57640 1 --2- 172.21.15.6:0/2187515364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd92c102570 0x7fd92c0fd1c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:53.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.411+0000 7fd933a57640 1 --2- 172.21.15.6:0/2187515364 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd92c100e20 0x7fd92c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:53.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.411+0000 7fd933a57640 1 -- 172.21.15.6:0/2187515364 >> 172.21.15.6:0/2187515364 conn(0x7fd92c0fc7d0 msgr2=0x7fd92c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:53.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.411+0000 7fd933a57640 1 -- 172.21.15.6:0/2187515364 shutdown_connections 2024-09-12T03:18:53.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:53.411+0000 7fd933a57640 1 -- 172.21.15.6:0/2187515364 wait complete. 2024-09-12T03:18:54.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:53 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:54.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:53 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:54.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:53 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:54.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:53 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:54.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:53 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:54.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:53 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:54.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:53 smithi118 ceph-mon[35048]: Deploying daemon mds.g on smithi006 2024-09-12T03:18:54.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:53 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2187515364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:54.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:53 smithi118 ceph-mon[35048]: mds.? [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] up:boot 2024-09-12T03:18:54.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:53 smithi118 ceph-mon[35048]: fsmap 9 up:standby 2024-09-12T03:18:54.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:53 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:18:54.280 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:53 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:54.280 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:53 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:54.281 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:53 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:54.281 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:53 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:54.281 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:53 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:54.281 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:53 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:54.281 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:53 smithi006 ceph-mon[30991]: Deploying daemon mds.g on smithi006 2024-09-12T03:18:54.281 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:53 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2187515364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:54.281 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:53 smithi006 ceph-mon[30991]: mds.? [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] up:boot 2024-09-12T03:18:54.281 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:53 smithi006 ceph-mon[30991]: fsmap 9 up:standby 2024-09-12T03:18:54.281 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:53 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:18:54.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:53 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:54.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:53 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:54.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:53 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:54.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:53 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:54.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:53 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:54.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:53 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:54.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:53 smithi161 ceph-mon[35485]: Deploying daemon mds.g on smithi006 2024-09-12T03:18:54.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:53 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2187515364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:54.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:53 smithi161 ceph-mon[35485]: mds.? [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] up:boot 2024-09-12T03:18:54.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:53 smithi161 ceph-mon[35485]: fsmap 9 up:standby 2024-09-12T03:18:54.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:53 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:18:55.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:54 smithi006 ceph-mon[30991]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:55.112 INFO:journalctl@ceph.mds.g.smithi006.stdout:Sep 12 03:18:55 smithi006 systemd[1]: Starting Ceph mds.g for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:18:55.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:54 smithi118 ceph-mon[35048]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:55.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:54 smithi161 ceph-mon[35485]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:55.554 INFO:journalctl@ceph.mds.g.smithi006.stdout:Sep 12 03:18:55 smithi006 podman[72049]: 2024-09-12 03:18:55.27766489 +0000 UTC m=+0.108015946 container create d6b829cebcbd88c7a9e1c83470276f567d7e4c75dd425c60f6d6e91346507042 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-12T03:18:55.554 INFO:journalctl@ceph.mds.g.smithi006.stdout:Sep 12 03:18:55 smithi006 podman[72049]: 2024-09-12 03:18:55.186499306 +0000 UTC m=+0.016850317 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:18:55.861 INFO:journalctl@ceph.mds.g.smithi006.stdout:Sep 12 03:18:55 smithi006 podman[72049]: 2024-09-12 03:18:55.554827397 +0000 UTC m=+0.385178408 container init d6b829cebcbd88c7a9e1c83470276f567d7e4c75dd425c60f6d6e91346507042 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-g, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T03:18:55.862 INFO:journalctl@ceph.mds.g.smithi006.stdout:Sep 12 03:18:55 smithi006 podman[72049]: 2024-09-12 03:18:55.557845897 +0000 UTC m=+0.388196908 container start d6b829cebcbd88c7a9e1c83470276f567d7e4c75dd425c60f6d6e91346507042 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-g, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T03:18:55.862 INFO:journalctl@ceph.mds.g.smithi006.stdout:Sep 12 03:18:55 smithi006 ceph-mds[72068]: -- 172.21.15.6:0/1366579092 <== mon.0 v2:172.21.15.6:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5638a1963c20 con 0x5638a1941000 2024-09-12T03:18:55.862 INFO:journalctl@ceph.mds.g.smithi006.stdout:Sep 12 03:18:55 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-g[72064]: starting mds.g at 2024-09-12T03:18:55.862 INFO:journalctl@ceph.mds.g.smithi006.stdout:Sep 12 03:18:55 smithi006 bash[72049]: d6b829cebcbd88c7a9e1c83470276f567d7e4c75dd425c60f6d6e91346507042 2024-09-12T03:18:55.862 INFO:journalctl@ceph.mds.g.smithi006.stdout:Sep 12 03:18:55 smithi006 systemd[1]: Started Ceph mds.g for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:18:56.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:56 smithi006 ceph-mon[30991]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:56.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:56 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:56.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:56 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:56.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:56 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:56.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:56 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:56.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:56 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:56.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:56 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:56.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:56 smithi006 ceph-mon[30991]: Deploying daemon mds.a on smithi006 2024-09-12T03:18:56.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:56 smithi006 ceph-mon[30991]: mds.? [v2:172.21.15.6:6838/3301072029,v1:172.21.15.6:6839/3301072029] up:boot 2024-09-12T03:18:56.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:56 smithi006 ceph-mon[30991]: fsmap 10 up:standby 2024-09-12T03:18:56.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:56 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:18:57.012 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:18:57.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:56 smithi161 ceph-mon[35485]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:57.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:56 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:57.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:56 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:57.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:56 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:57.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:56 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:57.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:56 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:57.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:56 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:57.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:56 smithi161 ceph-mon[35485]: Deploying daemon mds.a on smithi006 2024-09-12T03:18:57.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:56 smithi161 ceph-mon[35485]: mds.? [v2:172.21.15.6:6838/3301072029,v1:172.21.15.6:6839/3301072029] up:boot 2024-09-12T03:18:57.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:56 smithi161 ceph-mon[35485]: fsmap 10 up:standby 2024-09-12T03:18:57.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:56 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:18:57.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:56 smithi118 ceph-mon[35048]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:57.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:56 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:57.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:56 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:57.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:56 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:57.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:56 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:57.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:56 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:57.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:56 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:57.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:56 smithi118 ceph-mon[35048]: Deploying daemon mds.a on smithi006 2024-09-12T03:18:57.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:56 smithi118 ceph-mon[35048]: mds.? [v2:172.21.15.6:6838/3301072029,v1:172.21.15.6:6839/3301072029] up:boot 2024-09-12T03:18:57.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:56 smithi118 ceph-mon[35048]: fsmap 10 up:standby 2024-09-12T03:18:57.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:56 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:18:57.277 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:57.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.891+0000 7fd7ad674640 1 -- 172.21.15.6:0/4157065359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7a8102790 msgr2=0x7fd7a81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:57.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.891+0000 7fd7ad674640 1 --2- 172.21.15.6:0/4157065359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7a8102790 0x7fd7a81067e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009900 tx=0x7fd79c02a3f0 comp rx=0 tx=0).stop 2024-09-12T03:18:57.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.891+0000 7fd7ad674640 1 -- 172.21.15.6:0/4157065359 shutdown_connections 2024-09-12T03:18:57.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.891+0000 7fd7ad674640 1 --2- 172.21.15.6:0/4157065359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a8106f10 0x7fd7a810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:57.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.891+0000 7fd7ad674640 1 --2- 172.21.15.6:0/4157065359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7a8102790 0x7fd7a81067e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:57.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.891+0000 7fd7ad674640 1 --2- 172.21.15.6:0/4157065359 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7a8101de0 0x7fd7a81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:57.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.891+0000 7fd7ad674640 1 -- 172.21.15.6:0/4157065359 >> 172.21.15.6:0/4157065359 conn(0x7fd7a80fd8c0 msgr2=0x7fd7a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:57.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.891+0000 7fd7ad674640 1 -- 172.21.15.6:0/4157065359 shutdown_connections 2024-09-12T03:18:57.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.891+0000 7fd7ad674640 1 -- 172.21.15.6:0/4157065359 wait complete. 2024-09-12T03:18:57.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.892+0000 7fd7ad674640 1 Processor -- start 2024-09-12T03:18:57.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.892+0000 7fd7ad674640 1 -- start start 2024-09-12T03:18:57.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.892+0000 7fd7ad674640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7a8101de0 0x7fd7a8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:57.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.892+0000 7fd7ad674640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7a8102790 0x7fd7a8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:57.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.892+0000 7fd7ad674640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a8106f10 0x7fd7a819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:57.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.892+0000 7fd7ad674640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a8111f70 con 0x7fd7a8101de0 2024-09-12T03:18:57.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.892+0000 7fd7ad674640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a8111df0 con 0x7fd7a8102790 2024-09-12T03:18:57.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.892+0000 7fd7ad674640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a81120f0 con 0x7fd7a8106f10 2024-09-12T03:18:57.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.892+0000 7fd7a6ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7a8101de0 0x7fd7a8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:57.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.892+0000 7fd7a6ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7a8101de0 0x7fd7a8198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39302/0 (socket says 172.21.15.6:39302) 2024-09-12T03:18:57.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.892+0000 7fd7a6ffd640 1 -- 172.21.15.6:0/1516532565 learned_addr learned my addr 172.21.15.6:0/1516532565 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:18:57.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.892+0000 7fd7a67fc640 1 --2- 172.21.15.6:0/1516532565 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7a8102790 0x7fd7a8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:57.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.892+0000 7fd7a77fe640 1 --2- 172.21.15.6:0/1516532565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a8106f10 0x7fd7a819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:57.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.892+0000 7fd7a6ffd640 1 -- 172.21.15.6:0/1516532565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a8106f10 msgr2=0x7fd7a819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:57.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.892+0000 7fd7a6ffd640 1 --2- 172.21.15.6:0/1516532565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a8106f10 0x7fd7a819cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:57.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.892+0000 7fd7a6ffd640 1 -- 172.21.15.6:0/1516532565 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7a8102790 msgr2=0x7fd7a8198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:57.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.892+0000 7fd7a6ffd640 1 --2- 172.21.15.6:0/1516532565 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7a8102790 0x7fd7a8198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:57.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.892+0000 7fd7a6ffd640 1 -- 172.21.15.6:0/1516532565 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a819d400 con 0x7fd7a8101de0 2024-09-12T03:18:57.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.893+0000 7fd7a6ffd640 1 --2- 172.21.15.6:0/1516532565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7a8101de0 0x7fd7a8198450 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd79800c920 tx=0x7fd79800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:57.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.893+0000 7fd78ffff640 1 -- 172.21.15.6:0/1516532565 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798007c70 con 0x7fd7a8101de0 2024-09-12T03:18:57.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.893+0000 7fd7ad674640 1 -- 172.21.15.6:0/1516532565 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a819d6f0 con 0x7fd7a8101de0 2024-09-12T03:18:57.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.893+0000 7fd78ffff640 1 -- 172.21.15.6:0/1516532565 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd798007e10 con 0x7fd7a8101de0 2024-09-12T03:18:57.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.893+0000 7fd78ffff640 1 -- 172.21.15.6:0/1516532565 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798018890 con 0x7fd7a8101de0 2024-09-12T03:18:57.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.893+0000 7fd7ad674640 1 -- 172.21.15.6:0/1516532565 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a81a4fd0 con 0x7fd7a8101de0 2024-09-12T03:18:57.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.894+0000 7fd7ad674640 1 -- 172.21.15.6:0/1516532565 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7a81021c0 con 0x7fd7a8101de0 2024-09-12T03:18:57.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.896+0000 7fd78ffff640 1 -- 172.21.15.6:0/1516532565 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd798018a50 con 0x7fd7a8101de0 2024-09-12T03:18:57.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.896+0000 7fd78ffff640 1 --2- 172.21.15.6:0/1516532565 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd7900a2b80 0x7fd7900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:57.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.897+0000 7fd7a67fc640 1 --2- 172.21.15.6:0/1516532565 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd7900a2b80 0x7fd7900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:57.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.897+0000 7fd78ffff640 1 -- 172.21.15.6:0/1516532565 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fd7980cf070 con 0x7fd7a8101de0 2024-09-12T03:18:57.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.897+0000 7fd7a67fc640 1 --2- 172.21.15.6:0/1516532565 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd7900a2b80 0x7fd7900a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd7a81087d0 tx=0x7fd79c031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:57.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:57.900+0000 7fd78ffff640 1 -- 172.21.15.6:0/1516532565 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd798014030 con 0x7fd7a8101de0 2024-09-12T03:18:58.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:58.119+0000 7fd7ad674640 1 -- 172.21.15.6:0/1516532565 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7a81997b0 con 0x7fd7a8101de0 2024-09-12T03:18:58.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:58.120+0000 7fd78ffff640 1 -- 172.21.15.6:0/1516532565 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8869 (secure 0 0 0) 0x7fd7a81997b0 con 0x7fd7a8101de0 2024-09-12T03:18:58.120 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:18:58.120 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":11,"btime":"2024-09-12T03:18:55:891062+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-12T03:18:58.120 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 11 2024-09-12T03:18:58.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:58.122+0000 7fd7ad674640 1 -- 172.21.15.6:0/1516532565 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd7900a2b80 msgr2=0x7fd7900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:58.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:58.122+0000 7fd7ad674640 1 --2- 172.21.15.6:0/1516532565 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd7900a2b80 0x7fd7900a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd7a81087d0 tx=0x7fd79c031490 comp rx=0 tx=0).stop 2024-09-12T03:18:58.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:58.122+0000 7fd7ad674640 1 -- 172.21.15.6:0/1516532565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7a8101de0 msgr2=0x7fd7a8198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:58.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:58.122+0000 7fd7ad674640 1 --2- 172.21.15.6:0/1516532565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7a8101de0 0x7fd7a8198450 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd79800c920 tx=0x7fd79800cde0 comp rx=0 tx=0).stop 2024-09-12T03:18:58.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:58.122+0000 7fd7ad674640 1 -- 172.21.15.6:0/1516532565 shutdown_connections 2024-09-12T03:18:58.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:58.122+0000 7fd7ad674640 1 --2- 172.21.15.6:0/1516532565 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd7900a2b80 0x7fd7900a5040 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:58.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:58.122+0000 7fd7ad674640 1 --2- 172.21.15.6:0/1516532565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a8106f10 0x7fd7a819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:58.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:58.122+0000 7fd7ad674640 1 --2- 172.21.15.6:0/1516532565 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7a8102790 0x7fd7a8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:58.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:58.122+0000 7fd7ad674640 1 --2- 172.21.15.6:0/1516532565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7a8101de0 0x7fd7a8198450 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:58.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:58.122+0000 7fd7ad674640 1 -- 172.21.15.6:0/1516532565 >> 172.21.15.6:0/1516532565 conn(0x7fd7a80fd8c0 msgr2=0x7fd7a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:58.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:58.122+0000 7fd7ad674640 1 -- 172.21.15.6:0/1516532565 shutdown_connections 2024-09-12T03:18:58.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:18:58.122+0000 7fd7ad674640 1 -- 172.21.15.6:0/1516532565 wait complete. 2024-09-12T03:18:58.789 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:58 smithi006 ceph-mon[30991]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:58.789 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:58 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1516532565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:58.789 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:58 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:58.789 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:58 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:58.790 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:58 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:58.790 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:58 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:58.790 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:58 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:58.790 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:58 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:59.079 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:58 smithi118 ceph-mon[35048]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:59.079 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:58 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1516532565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:59.079 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:58 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:59.080 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:58 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:59.080 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:58 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:59.080 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:58 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:59.080 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:58 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:59.080 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:58 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:59.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:58 smithi161 ceph-mon[35485]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:59.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:58 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1516532565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:59.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:58 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:59.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:58 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:59.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:58 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:18:59.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:58 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:18:59.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:58 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:18:59.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:58 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:00.107 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:59 smithi118 ceph-mon[35048]: Deploying daemon mds.b on smithi118 2024-09-12T03:19:00.108 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:59 smithi118 ceph-mon[35048]: mds.? [v2:172.21.15.6:6840/2670411625,v1:172.21.15.6:6841/2670411625] up:boot 2024-09-12T03:19:00.108 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:59 smithi118 ceph-mon[35048]: fsmap 11 up:standby 2024-09-12T03:19:00.108 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:18:59 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:19:00.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:59 smithi006 ceph-mon[30991]: Deploying daemon mds.b on smithi118 2024-09-12T03:19:00.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:59 smithi006 ceph-mon[30991]: mds.? [v2:172.21.15.6:6840/2670411625,v1:172.21.15.6:6841/2670411625] up:boot 2024-09-12T03:19:00.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:59 smithi006 ceph-mon[30991]: fsmap 11 up:standby 2024-09-12T03:19:00.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:18:59 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:19:00.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:59 smithi161 ceph-mon[35485]: Deploying daemon mds.b on smithi118 2024-09-12T03:19:00.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:59 smithi161 ceph-mon[35485]: mds.? [v2:172.21.15.6:6840/2670411625,v1:172.21.15.6:6841/2670411625] up:boot 2024-09-12T03:19:00.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:59 smithi161 ceph-mon[35485]: fsmap 11 up:standby 2024-09-12T03:19:00.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:18:59 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:19:00.712 INFO:journalctl@ceph.mds.b.smithi118.stdout:Sep 12 03:19:00 smithi118 systemd[1]: Starting Ceph mds.b for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:19:00.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:00 smithi006 ceph-mon[30991]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:19:01.028 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:00 smithi118 ceph-mon[35048]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:19:01.028 INFO:journalctl@ceph.mds.b.smithi118.stdout:Sep 12 03:19:00 smithi118 podman[65490]: 2024-09-12 03:19:00.70189456 +0000 UTC m=+0.015704503 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:19:01.029 INFO:journalctl@ceph.mds.b.smithi118.stdout:Sep 12 03:19:00 smithi118 podman[65490]: 2024-09-12 03:19:00.810474048 +0000 UTC m=+0.124283983 container create e937849e5ddb6f857a968dc3070fe4c29cbdc6c40ada7db7c1d62e89c823f53b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-12T03:19:01.029 INFO:journalctl@ceph.mds.b.smithi118.stdout:Sep 12 03:19:01 smithi118 podman[65490]: 2024-09-12 03:19:01.029146519 +0000 UTC m=+0.342956465 container init e937849e5ddb6f857a968dc3070fe4c29cbdc6c40ada7db7c1d62e89c823f53b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-b, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-12T03:19:01.101 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:19:01.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:00 smithi161 ceph-mon[35485]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:19:01.372 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:01.462 INFO:journalctl@ceph.mds.b.smithi118.stdout:Sep 12 03:19:01 smithi118 podman[65490]: 2024-09-12 03:19:01.031960245 +0000 UTC m=+0.345770186 container start e937849e5ddb6f857a968dc3070fe4c29cbdc6c40ada7db7c1d62e89c823f53b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:19:01.462 INFO:journalctl@ceph.mds.b.smithi118.stdout:Sep 12 03:19:01 smithi118 ceph-mds[65509]: -- 172.21.15.118:0/2931806748 <== mon.2 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a7a0ac9860 con 0x55a7a0aa7000 2024-09-12T03:19:01.462 INFO:journalctl@ceph.mds.b.smithi118.stdout:Sep 12 03:19:01 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-b[65505]: starting mds.b at 2024-09-12T03:19:01.462 INFO:journalctl@ceph.mds.b.smithi118.stdout:Sep 12 03:19:01 smithi118 bash[65490]: e937849e5ddb6f857a968dc3070fe4c29cbdc6c40ada7db7c1d62e89c823f53b 2024-09-12T03:19:01.462 INFO:journalctl@ceph.mds.b.smithi118.stdout:Sep 12 03:19:01 smithi118 systemd[1]: Started Ceph mds.b for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:19:01.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.837+0000 7fb1983a8640 1 -- 172.21.15.6:0/1405494060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1901036e0 msgr2=0x7fb190107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:01.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.837+0000 7fb1983a8640 1 --2- 172.21.15.6:0/1405494060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1901036e0 0x7fb190107b90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb180009930 tx=0x7fb18002a420 comp rx=0 tx=0).stop 2024-09-12T03:19:01.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.837+0000 7fb1983a8640 1 -- 172.21.15.6:0/1405494060 shutdown_connections 2024-09-12T03:19:01.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.837+0000 7fb1983a8640 1 --2- 172.21.15.6:0/1405494060 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1901082a0 0x7fb19010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:01.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.837+0000 7fb1983a8640 1 --2- 172.21.15.6:0/1405494060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1901036e0 0x7fb190107b90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:01.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.837+0000 7fb1983a8640 1 --2- 172.21.15.6:0/1405494060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb190102d30 0x7fb190103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:01.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.837+0000 7fb1983a8640 1 -- 172.21.15.6:0/1405494060 >> 172.21.15.6:0/1405494060 conn(0x7fb1900fc7d0 msgr2=0x7fb1900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:01.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.837+0000 7fb1983a8640 1 -- 172.21.15.6:0/1405494060 shutdown_connections 2024-09-12T03:19:01.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.837+0000 7fb1983a8640 1 -- 172.21.15.6:0/1405494060 wait complete. 2024-09-12T03:19:01.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.838+0000 7fb1983a8640 1 Processor -- start 2024-09-12T03:19:01.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.838+0000 7fb1983a8640 1 -- start start 2024-09-12T03:19:01.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.838+0000 7fb1983a8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb190102d30 0x7fb190100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:01.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.838+0000 7fb1983a8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1901036e0 0x7fb190100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:01.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.838+0000 7fb1983a8640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1901082a0 0x7fb190101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:01.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.838+0000 7fb1983a8640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901120a0 con 0x7fb190102d30 2024-09-12T03:19:01.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.838+0000 7fb1983a8640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fb190111f20 con 0x7fb1901082a0 2024-09-12T03:19:01.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.838+0000 7fb1983a8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb190112220 con 0x7fb1901036e0 2024-09-12T03:19:01.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.838+0000 7fb196144640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb190102d30 0x7fb190100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:01.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.838+0000 7fb196144640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb190102d30 0x7fb190100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36722/0 (socket says 172.21.15.6:36722) 2024-09-12T03:19:01.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.838+0000 7fb196144640 1 -- 172.21.15.6:0/1536364632 learned_addr learned my addr 172.21.15.6:0/1536364632 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:01.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.838+0000 7fb196945640 1 --2- 172.21.15.6:0/1536364632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1901082a0 0x7fb190101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:01.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.838+0000 7fb195943640 1 --2- 172.21.15.6:0/1536364632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1901036e0 0x7fb190100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:01.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.839+0000 7fb196144640 1 -- 172.21.15.6:0/1536364632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1901036e0 msgr2=0x7fb190100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:01.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.839+0000 7fb196144640 1 --2- 172.21.15.6:0/1536364632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1901036e0 0x7fb190100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:01.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.839+0000 7fb196144640 1 -- 172.21.15.6:0/1536364632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1901082a0 msgr2=0x7fb190101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:01.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.839+0000 7fb196144640 1 --2- 172.21.15.6:0/1536364632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1901082a0 0x7fb190101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:01.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.839+0000 7fb196144640 1 -- 172.21.15.6:0/1536364632 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1901a6a40 con 0x7fb190102d30 2024-09-12T03:19:01.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.839+0000 7fb196144640 1 --2- 172.21.15.6:0/1536364632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb190102d30 0x7fb190100990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00e9f0 tx=0x7fb18c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:01.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.839+0000 7fb1877fe640 1 -- 172.21.15.6:0/1536364632 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c00cd80 con 0x7fb190102d30 2024-09-12T03:19:01.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.839+0000 7fb1877fe640 1 -- 172.21.15.6:0/1536364632 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb18c010910 con 0x7fb190102d30 2024-09-12T03:19:01.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.839+0000 7fb1983a8640 1 -- 172.21.15.6:0/1536364632 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1901a6d30 con 0x7fb190102d30 2024-09-12T03:19:01.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.839+0000 7fb1877fe640 1 -- 172.21.15.6:0/1536364632 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c0188f0 con 0x7fb190102d30 2024-09-12T03:19:01.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.839+0000 7fb1983a8640 1 -- 172.21.15.6:0/1536364632 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1901a71e0 con 0x7fb190102d30 2024-09-12T03:19:01.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.840+0000 7fb1983a8640 1 -- 172.21.15.6:0/1536364632 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb190103110 con 0x7fb190102d30 2024-09-12T03:19:01.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.842+0000 7fb1877fe640 1 -- 172.21.15.6:0/1536364632 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb18c010ac0 con 0x7fb190102d30 2024-09-12T03:19:01.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.842+0000 7fb1877fe640 1 --2- 172.21.15.6:0/1536364632 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb17c0a2b80 0x7fb17c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:01.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.843+0000 7fb195943640 1 --2- 172.21.15.6:0/1536364632 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb17c0a2b80 0x7fb17c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:01.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.843+0000 7fb1877fe640 1 -- 172.21.15.6:0/1536364632 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fb18c0cf400 con 0x7fb190102d30 2024-09-12T03:19:01.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.843+0000 7fb195943640 1 --2- 172.21.15.6:0/1536364632 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb17c0a2b80 0x7fb17c0a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb180009900 tx=0x7fb180005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:01.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:01.849+0000 7fb1877fe640 1 -- 172.21.15.6:0/1536364632 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb18c09c760 con 0x7fb190102d30 2024-09-12T03:19:02.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:02.081+0000 7fb1983a8640 1 -- 172.21.15.6:0/1536364632 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb190101ec0 con 0x7fb190102d30 2024-09-12T03:19:02.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:02.082+0000 7fb1877fe640 1 -- 172.21.15.6:0/1536364632 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10521 (secure 0 0 0) 0x7fb18c09c060 con 0x7fb190102d30 2024-09-12T03:19:02.083 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:19:02.083 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":13,"btime":"2024-09-12T03:19:01:742206+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-12T03:19:02.083 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 13 2024-09-12T03:19:02.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:02.084+0000 7fb1983a8640 1 -- 172.21.15.6:0/1536364632 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb17c0a2b80 msgr2=0x7fb17c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:02.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:02.084+0000 7fb1983a8640 1 --2- 172.21.15.6:0/1536364632 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb17c0a2b80 0x7fb17c0a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb180009900 tx=0x7fb180005e50 comp rx=0 tx=0).stop 2024-09-12T03:19:02.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:02.084+0000 7fb1983a8640 1 -- 172.21.15.6:0/1536364632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb190102d30 msgr2=0x7fb190100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:02.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:02.084+0000 7fb1983a8640 1 --2- 172.21.15.6:0/1536364632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb190102d30 0x7fb190100990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00e9f0 tx=0x7fb18c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:02.084+0000 7fb1983a8640 1 -- 172.21.15.6:0/1536364632 shutdown_connections 2024-09-12T03:19:02.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:02.084+0000 7fb1983a8640 1 --2- 172.21.15.6:0/1536364632 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb17c0a2b80 0x7fb17c0a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:02.084+0000 7fb1983a8640 1 --2- 172.21.15.6:0/1536364632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1901082a0 0x7fb190101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:02.084+0000 7fb1983a8640 1 --2- 172.21.15.6:0/1536364632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1901036e0 0x7fb190100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:02.084+0000 7fb1983a8640 1 --2- 172.21.15.6:0/1536364632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb190102d30 0x7fb190100990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:02.084+0000 7fb1983a8640 1 -- 172.21.15.6:0/1536364632 >> 172.21.15.6:0/1536364632 conn(0x7fb1900fc7d0 msgr2=0x7fb190105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:02.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:02.084+0000 7fb1983a8640 1 -- 172.21.15.6:0/1536364632 shutdown_connections 2024-09-12T03:19:02.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:02.084+0000 7fb1983a8640 1 -- 172.21.15.6:0/1536364632 wait complete. 2024-09-12T03:19:02.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:02 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:02.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:02 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:02.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:02 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:02.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:02 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:02.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:02 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:02.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:02 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:02.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:02 smithi118 ceph-mon[35048]: mds.? [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] up:boot 2024-09-12T03:19:02.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:02 smithi118 ceph-mon[35048]: fsmap 12 up:standby 2024-09-12T03:19:02.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:02 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:19:02.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:02 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1536364632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:02.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:02 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:02.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:02 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:02.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:02 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:02.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:02 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:02.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:02 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:02.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:02 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:02.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:02 smithi006 ceph-mon[30991]: mds.? [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] up:boot 2024-09-12T03:19:02.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:02 smithi006 ceph-mon[30991]: fsmap 12 up:standby 2024-09-12T03:19:02.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:02 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:19:02.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:02 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1536364632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:02.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:02 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:02.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:02 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:02.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:02 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:02.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:02 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:02.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:02 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:02.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:02 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:02.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:02 smithi161 ceph-mon[35485]: mds.? [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] up:boot 2024-09-12T03:19:02.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:02 smithi161 ceph-mon[35485]: fsmap 12 up:standby 2024-09-12T03:19:02.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:02 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:19:02.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:02 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1536364632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:03.083 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-12T03:19:03.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:03 smithi006 ceph-mon[30991]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:19:03.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:03 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:03.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:03 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:03.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:03 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:03.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:03 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:03.425 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:03.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:03 smithi118 ceph-mon[35048]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:19:03.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:03 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:03.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:03 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:03.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:03 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:03.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:03 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:03.515 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:03 smithi161 ceph-mon[35485]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:19:03.515 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:03 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:03.515 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:03 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:03.515 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:03 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:03.516 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:03 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:03.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.935+0000 7fd12d119640 1 -- 172.21.15.6:0/2849603438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd128106a80 msgr2=0x7fd128106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:03.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.935+0000 7fd12d119640 1 --2- 172.21.15.6:0/2849603438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd128106a80 0x7fd128106e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd118009930 tx=0x7fd11802a420 comp rx=0 tx=0).stop 2024-09-12T03:19:03.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.935+0000 7fd12d119640 1 -- 172.21.15.6:0/2849603438 shutdown_connections 2024-09-12T03:19:03.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.935+0000 7fd12d119640 1 --2- 172.21.15.6:0/2849603438 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd128101420 0x7fd12810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.935+0000 7fd12d119640 1 --2- 172.21.15.6:0/2849603438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd128100a80 0x7fd128100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.935+0000 7fd12d119640 1 --2- 172.21.15.6:0/2849603438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd128106a80 0x7fd128106e60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.935+0000 7fd12d119640 1 -- 172.21.15.6:0/2849603438 >> 172.21.15.6:0/2849603438 conn(0x7fd1280fc7a0 msgr2=0x7fd1280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:03.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.935+0000 7fd12d119640 1 -- 172.21.15.6:0/2849603438 shutdown_connections 2024-09-12T03:19:03.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.935+0000 7fd12d119640 1 -- 172.21.15.6:0/2849603438 wait complete. 2024-09-12T03:19:03.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.936+0000 7fd12d119640 1 Processor -- start 2024-09-12T03:19:03.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.937+0000 7fd12d119640 1 -- start start 2024-09-12T03:19:03.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.937+0000 7fd12d119640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd128100a80 0x7fd128197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.937+0000 7fd12d119640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd128101420 0x7fd128197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.937+0000 7fd12d119640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd128106a80 0x7fd12819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.937+0000 7fd12d119640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd128110e80 con 0x7fd128101420 2024-09-12T03:19:03.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.937+0000 7fd12d119640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fd128110d00 con 0x7fd128100a80 2024-09-12T03:19:03.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.937+0000 7fd12d119640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd128111000 con 0x7fd128106a80 2024-09-12T03:19:03.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.937+0000 7fd12659c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd128101420 0x7fd128197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:03.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.937+0000 7fd12659c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd128101420 0x7fd128197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36744/0 (socket says 172.21.15.6:36744) 2024-09-12T03:19:03.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.937+0000 7fd12659c640 1 -- 172.21.15.6:0/782612758 learned_addr learned my addr 172.21.15.6:0/782612758 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:03.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.937+0000 7fd12759e640 1 --2- 172.21.15.6:0/782612758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd128106a80 0x7fd12819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:03.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.937+0000 7fd126d9d640 1 --2- 172.21.15.6:0/782612758 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd128100a80 0x7fd128197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:03.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.937+0000 7fd12659c640 1 -- 172.21.15.6:0/782612758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd128106a80 msgr2=0x7fd12819bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:03.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.937+0000 7fd12659c640 1 --2- 172.21.15.6:0/782612758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd128106a80 0x7fd12819bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.937+0000 7fd12659c640 1 -- 172.21.15.6:0/782612758 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd128100a80 msgr2=0x7fd128197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:03.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.937+0000 7fd12659c640 1 --2- 172.21.15.6:0/782612758 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd128100a80 0x7fd128197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.937+0000 7fd12659c640 1 -- 172.21.15.6:0/782612758 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12819c2b0 con 0x7fd128101420 2024-09-12T03:19:03.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.938+0000 7fd12659c640 1 --2- 172.21.15.6:0/782612758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd128101420 0x7fd128197840 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00d770 tx=0x7fd11c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:03.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.938+0000 7fd10ffff640 1 -- 172.21.15.6:0/782612758 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c017070 con 0x7fd128101420 2024-09-12T03:19:03.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.938+0000 7fd12d119640 1 -- 172.21.15.6:0/782612758 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd12819c5a0 con 0x7fd128101420 2024-09-12T03:19:03.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.938+0000 7fd10ffff640 1 -- 172.21.15.6:0/782612758 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd11c005500 con 0x7fd128101420 2024-09-12T03:19:03.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.938+0000 7fd10ffff640 1 -- 172.21.15.6:0/782612758 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c004e00 con 0x7fd128101420 2024-09-12T03:19:03.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.938+0000 7fd12d119640 1 -- 172.21.15.6:0/782612758 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1281a3e80 con 0x7fd128101420 2024-09-12T03:19:03.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.939+0000 7fd12d119640 1 -- 172.21.15.6:0/782612758 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd128100ee0 con 0x7fd128101420 2024-09-12T03:19:03.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.940+0000 7fd10ffff640 1 -- 172.21.15.6:0/782612758 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd11c004fa0 con 0x7fd128101420 2024-09-12T03:19:03.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.941+0000 7fd10ffff640 1 --2- 172.21.15.6:0/782612758 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd1100a2b80 0x7fd1100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.941+0000 7fd126d9d640 1 --2- 172.21.15.6:0/782612758 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd1100a2b80 0x7fd1100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:03.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.941+0000 7fd10ffff640 1 -- 172.21.15.6:0/782612758 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fd11c0ce950 con 0x7fd128101420 2024-09-12T03:19:03.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.941+0000 7fd126d9d640 1 --2- 172.21.15.6:0/782612758 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd1100a2b80 0x7fd1100a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd118009900 tx=0x7fd1180058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:03.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:03.944+0000 7fd10ffff640 1 -- 172.21.15.6:0/782612758 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd11c09b700 con 0x7fd128101420 2024-09-12T03:19:04.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:04.155+0000 7fd12d119640 1 -- 172.21.15.6:0/782612758 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd128109020 con 0x7fd128101420 2024-09-12T03:19:04.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:04.156+0000 7fd10ffff640 1 -- 172.21.15.6:0/782612758 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) ==== 84+0+65 (secure 0 0 0) 0x7fd11c09fa00 con 0x7fd128101420 2024-09-12T03:19:04.156 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:19:04.156 INFO:teuthology.orchestra.run.smithi006.stdout:[ 2024-09-12T03:19:04.157 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:19:04.157 INFO:teuthology.orchestra.run.smithi006.stdout: "poolnum": 1, 2024-09-12T03:19:04.157 INFO:teuthology.orchestra.run.smithi006.stdout: "poolname": ".mgr" 2024-09-12T03:19:04.157 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:19:04.157 INFO:teuthology.orchestra.run.smithi006.stdout:] 2024-09-12T03:19:04.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:04.158+0000 7fd12d119640 1 -- 172.21.15.6:0/782612758 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd1100a2b80 msgr2=0x7fd1100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:04.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:04.158+0000 7fd12d119640 1 --2- 172.21.15.6:0/782612758 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd1100a2b80 0x7fd1100a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd118009900 tx=0x7fd1180058d0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:04.158+0000 7fd12d119640 1 -- 172.21.15.6:0/782612758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd128101420 msgr2=0x7fd128197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:04.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:04.158+0000 7fd12d119640 1 --2- 172.21.15.6:0/782612758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd128101420 0x7fd128197840 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00d770 tx=0x7fd11c00dc30 comp rx=0 tx=0).stop 2024-09-12T03:19:04.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:04.158+0000 7fd12d119640 1 -- 172.21.15.6:0/782612758 shutdown_connections 2024-09-12T03:19:04.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:04.158+0000 7fd12d119640 1 --2- 172.21.15.6:0/782612758 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd1100a2b80 0x7fd1100a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:04.158+0000 7fd12d119640 1 --2- 172.21.15.6:0/782612758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd128106a80 0x7fd12819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:04.158+0000 7fd12d119640 1 --2- 172.21.15.6:0/782612758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd128101420 0x7fd128197840 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:04.158+0000 7fd12d119640 1 --2- 172.21.15.6:0/782612758 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd128100a80 0x7fd128197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:04.158+0000 7fd12d119640 1 -- 172.21.15.6:0/782612758 >> 172.21.15.6:0/782612758 conn(0x7fd1280fc7a0 msgr2=0x7fd1280fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:04.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:04.158+0000 7fd12d119640 1 -- 172.21.15.6:0/782612758 shutdown_connections 2024-09-12T03:19:04.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:04.158+0000 7fd12d119640 1 -- 172.21.15.6:0/782612758 wait complete. 2024-09-12T03:19:04.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:04 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/782612758' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T03:19:04.579 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:04 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/782612758' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T03:19:04.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:04 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/782612758' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T03:19:04.797 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T03:19:04.797 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T03:19:05.060 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:05.419 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:05 smithi006 ceph-mon[30991]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:19:05.419 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:05 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:05.419 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:05 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:05.419 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:05 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:05.419 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:05 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:05.419 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:05 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:05.419 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:05 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:05.419 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:05 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:05.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:05 smithi118 ceph-mon[35048]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:19:05.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:05 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:05.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:05 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:05.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:05 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:05.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:05 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:05.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:05 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:05.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:05 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:05.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:05 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:05.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.504+0000 7f38b22a4640 1 -- 172.21.15.6:0/3508416849 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38ac106f10 msgr2=0x7f38ac10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:05.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.504+0000 7f38b22a4640 1 --2- 172.21.15.6:0/3508416849 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38ac106f10 0x7f38ac10ea90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009ca0 tx=0x7f38a402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.504+0000 7f38b22a4640 1 -- 172.21.15.6:0/3508416849 shutdown_connections 2024-09-12T03:19:05.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.504+0000 7f38b22a4640 1 --2- 172.21.15.6:0/3508416849 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38ac106f10 0x7f38ac10ea90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.504+0000 7f38b22a4640 1 --2- 172.21.15.6:0/3508416849 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38ac102790 0x7f38ac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.504+0000 7f38b22a4640 1 --2- 172.21.15.6:0/3508416849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38ac101de0 0x7f38ac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.504+0000 7f38b22a4640 1 -- 172.21.15.6:0/3508416849 >> 172.21.15.6:0/3508416849 conn(0x7f38ac0fd8e0 msgr2=0x7f38ac0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:05.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.504+0000 7f38b22a4640 1 -- 172.21.15.6:0/3508416849 shutdown_connections 2024-09-12T03:19:05.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.504+0000 7f38b22a4640 1 -- 172.21.15.6:0/3508416849 wait complete. 2024-09-12T03:19:05.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.505+0000 7f38b22a4640 1 Processor -- start 2024-09-12T03:19:05.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.505+0000 7f38b22a4640 1 -- start start 2024-09-12T03:19:05.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.505+0000 7f38b22a4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38ac101de0 0x7f38ac1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.506+0000 7f38b22a4640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38ac102790 0x7f38ac1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.506+0000 7f38b22a4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38ac106f10 0x7f38ac19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.506+0000 7f38b22a4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac111f70 con 0x7f38ac106f10 2024-09-12T03:19:05.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.506+0000 7f38b22a4640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac111df0 con 0x7f38ac102790 2024-09-12T03:19:05.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.506+0000 7f38b22a4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac1120f0 con 0x7f38ac101de0 2024-09-12T03:19:05.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.506+0000 7f38b0841640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38ac106f10 0x7f38ac19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.506+0000 7f38b0841640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38ac106f10 0x7f38ac19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36766/0 (socket says 172.21.15.6:36766) 2024-09-12T03:19:05.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.506+0000 7f38b0841640 1 -- 172.21.15.6:0/2826759080 learned_addr learned my addr 172.21.15.6:0/2826759080 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:05.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.506+0000 7f38ab7fe640 1 --2- 172.21.15.6:0/2826759080 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38ac102790 0x7f38ac1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.506+0000 7f38abfff640 1 --2- 172.21.15.6:0/2826759080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38ac101de0 0x7f38ac1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.506+0000 7f38b0841640 1 -- 172.21.15.6:0/2826759080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38ac101de0 msgr2=0x7f38ac1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:05.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.506+0000 7f38b0841640 1 --2- 172.21.15.6:0/2826759080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38ac101de0 0x7f38ac1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.506+0000 7f38b0841640 1 -- 172.21.15.6:0/2826759080 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38ac102790 msgr2=0x7f38ac1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:05.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.506+0000 7f38b0841640 1 --2- 172.21.15.6:0/2826759080 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38ac102790 0x7f38ac1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.506+0000 7f38b0841640 1 -- 172.21.15.6:0/2826759080 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ac19d360 con 0x7f38ac106f10 2024-09-12T03:19:05.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.506+0000 7f38ab7fe640 1 --2- 172.21.15.6:0/2826759080 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38ac102790 0x7f38ac1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:05.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.506+0000 7f38b0841640 1 --2- 172.21.15.6:0/2826759080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38ac106f10 0x7f38ac19cc80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f38a4031a20 tx=0x7f38a4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:05.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.507+0000 7f38a97fa640 1 -- 172.21.15.6:0/2826759080 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4003c30 con 0x7f38ac106f10 2024-09-12T03:19:05.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.507+0000 7f38b22a4640 1 -- 172.21.15.6:0/2826759080 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ac19d5f0 con 0x7f38ac106f10 2024-09-12T03:19:05.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.507+0000 7f38a97fa640 1 -- 172.21.15.6:0/2826759080 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38a4003dd0 con 0x7f38ac106f10 2024-09-12T03:19:05.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.507+0000 7f38a97fa640 1 -- 172.21.15.6:0/2826759080 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a402fcf0 con 0x7f38ac106f10 2024-09-12T03:19:05.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.507+0000 7f38b22a4640 1 -- 172.21.15.6:0/2826759080 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ac1a4f30 con 0x7f38ac106f10 2024-09-12T03:19:05.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.508+0000 7f38b22a4640 1 -- 172.21.15.6:0/2826759080 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38ac1021c0 con 0x7f38ac106f10 2024-09-12T03:19:05.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.510+0000 7f38a97fa640 1 -- 172.21.15.6:0/2826759080 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f38a4004050 con 0x7f38ac106f10 2024-09-12T03:19:05.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.510+0000 7f38a97fa640 1 --2- 172.21.15.6:0/2826759080 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38940a2b80 0x7f38940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.511+0000 7f38abfff640 1 --2- 172.21.15.6:0/2826759080 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38940a2b80 0x7f38940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.511+0000 7f38a97fa640 1 -- 172.21.15.6:0/2826759080 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f38a40f00a0 con 0x7f38ac106f10 2024-09-12T03:19:05.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.511+0000 7f38abfff640 1 --2- 172.21.15.6:0/2826759080 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38940a2b80 0x7f38940a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f389c0058c0 tx=0x7f389c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:05.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.513+0000 7f38a97fa640 1 -- 172.21.15.6:0/2826759080 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f38a40bd530 con 0x7f38ac106f10 2024-09-12T03:19:05.575 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:05 smithi161 ceph-mon[35485]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:19:05.575 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:05 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:05.575 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:05 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:05.575 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:05 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:05.576 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:05 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:05.576 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:05 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:05.576 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:05 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:05.576 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:05 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:05.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:05.710+0000 7f38b22a4640 1 -- 172.21.15.6:0/2826759080 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f38ac10a150 con 0x7f38ac106f10 2024-09-12T03:19:06.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:06.193+0000 7f38a97fa640 1 -- 172.21.15.6:0/2826759080 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) ==== 138+0+0 (secure 0 0 0) 0x7f38a402fe90 con 0x7f38ac106f10 2024-09-12T03:19:06.193 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_metadata' created 2024-09-12T03:19:06.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:06.195+0000 7f38b22a4640 1 -- 172.21.15.6:0/2826759080 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38940a2b80 msgr2=0x7f38940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:06.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:06.195+0000 7f38b22a4640 1 --2- 172.21.15.6:0/2826759080 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38940a2b80 0x7f38940a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f389c0058c0 tx=0x7f389c00a2d0 comp rx=0 tx=0).stop 2024-09-12T03:19:06.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:06.195+0000 7f38b22a4640 1 -- 172.21.15.6:0/2826759080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38ac106f10 msgr2=0x7f38ac19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:06.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:06.195+0000 7f38b22a4640 1 --2- 172.21.15.6:0/2826759080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38ac106f10 0x7f38ac19cc80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f38a4031a20 tx=0x7f38a4031a50 comp rx=0 tx=0).stop 2024-09-12T03:19:06.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:06.195+0000 7f38b22a4640 1 -- 172.21.15.6:0/2826759080 shutdown_connections 2024-09-12T03:19:06.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:06.195+0000 7f38b22a4640 1 --2- 172.21.15.6:0/2826759080 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38940a2b80 0x7f38940a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:06.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:06.195+0000 7f38b22a4640 1 --2- 172.21.15.6:0/2826759080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38ac106f10 0x7f38ac19cc80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:06.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:06.195+0000 7f38b22a4640 1 --2- 172.21.15.6:0/2826759080 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38ac102790 0x7f38ac1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:06.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:06.195+0000 7f38b22a4640 1 --2- 172.21.15.6:0/2826759080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38ac101de0 0x7f38ac1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:06.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:06.195+0000 7f38b22a4640 1 -- 172.21.15.6:0/2826759080 >> 172.21.15.6:0/2826759080 conn(0x7f38ac0fd8e0 msgr2=0x7f38ac0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:06.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:06.195+0000 7f38b22a4640 1 -- 172.21.15.6:0/2826759080 shutdown_connections 2024-09-12T03:19:06.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:06.195+0000 7f38b22a4640 1 -- 172.21.15.6:0/2826759080 wait complete. 2024-09-12T03:19:06.287 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:06 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2826759080' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T03:19:06.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:06 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2826759080' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T03:19:06.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:06 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2826759080' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T03:19:06.921 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T03:19:07.213 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:07.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:07 smithi006 ceph-mon[30991]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:19:07.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:07 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2826759080' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T03:19:07.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:07 smithi006 ceph-mon[30991]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T03:19:07.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:07 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:07.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:07 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:07.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:07 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:07.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:07 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:07 smithi118 ceph-mon[35048]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:19:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:07 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2826759080' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T03:19:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:07 smithi118 ceph-mon[35048]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T03:19:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:07 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:07 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:07 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:07 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:07.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:07 smithi161 ceph-mon[35485]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:19:07.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:07 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2826759080' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T03:19:07.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:07 smithi161 ceph-mon[35485]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T03:19:07.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:07 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:07.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:07 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:07.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:07 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:07.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:07 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:07.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.660+0000 7fe7cde2d640 1 -- 172.21.15.6:0/2469321968 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7c8101de0 msgr2=0x7fe7c81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.660+0000 7fe7cde2d640 1 --2- 172.21.15.6:0/2469321968 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7c8101de0 0x7fe7c81021c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009900 tx=0x7fe7b802a410 comp rx=0 tx=0).stop 2024-09-12T03:19:07.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.660+0000 7fe7cde2d640 1 -- 172.21.15.6:0/2469321968 shutdown_connections 2024-09-12T03:19:07.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.660+0000 7fe7cde2d640 1 --2- 172.21.15.6:0/2469321968 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7c8106f10 0x7fe7c810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.660+0000 7fe7cde2d640 1 --2- 172.21.15.6:0/2469321968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7c8102790 0x7fe7c81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.660+0000 7fe7cde2d640 1 --2- 172.21.15.6:0/2469321968 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7c8101de0 0x7fe7c81021c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.660+0000 7fe7cde2d640 1 -- 172.21.15.6:0/2469321968 >> 172.21.15.6:0/2469321968 conn(0x7fe7c80fd8c0 msgr2=0x7fe7c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:07.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.660+0000 7fe7cde2d640 1 -- 172.21.15.6:0/2469321968 shutdown_connections 2024-09-12T03:19:07.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.660+0000 7fe7cde2d640 1 -- 172.21.15.6:0/2469321968 wait complete. 2024-09-12T03:19:07.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.661+0000 7fe7cde2d640 1 Processor -- start 2024-09-12T03:19:07.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.661+0000 7fe7cde2d640 1 -- start start 2024-09-12T03:19:07.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.661+0000 7fe7cde2d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7c8101de0 0x7fe7c8198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.661+0000 7fe7cde2d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7c8102790 0x7fe7c8198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.661+0000 7fe7cde2d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7c8106f10 0x7fe7c819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.661+0000 7fe7cde2d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c8112010 con 0x7fe7c8106f10 2024-09-12T03:19:07.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.661+0000 7fe7cde2d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c8111e90 con 0x7fe7c8101de0 2024-09-12T03:19:07.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.661+0000 7fe7cde2d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c8112190 con 0x7fe7c8102790 2024-09-12T03:19:07.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.661+0000 7fe7c7fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7c8106f10 0x7fe7c819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.661+0000 7fe7c7fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7c8106f10 0x7fe7c819ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36794/0 (socket says 172.21.15.6:36794) 2024-09-12T03:19:07.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.661+0000 7fe7c7fff640 1 -- 172.21.15.6:0/1551992638 learned_addr learned my addr 172.21.15.6:0/1551992638 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:07.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.661+0000 7fe7c6ffd640 1 --2- 172.21.15.6:0/1551992638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7c8102790 0x7fe7c8198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.661+0000 7fe7c77fe640 1 --2- 172.21.15.6:0/1551992638 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7c8101de0 0x7fe7c8198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.662+0000 7fe7c7fff640 1 -- 172.21.15.6:0/1551992638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7c8102790 msgr2=0x7fe7c8198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.662+0000 7fe7c7fff640 1 --2- 172.21.15.6:0/1551992638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7c8102790 0x7fe7c8198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.662+0000 7fe7c7fff640 1 -- 172.21.15.6:0/1551992638 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7c8101de0 msgr2=0x7fe7c8198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.662+0000 7fe7c7fff640 1 --2- 172.21.15.6:0/1551992638 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7c8101de0 0x7fe7c8198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.662+0000 7fe7c7fff640 1 -- 172.21.15.6:0/1551992638 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c819d540 con 0x7fe7c8106f10 2024-09-12T03:19:07.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.662+0000 7fe7c77fe640 1 --2- 172.21.15.6:0/1551992638 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7c8101de0 0x7fe7c8198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:07.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.662+0000 7fe7c7fff640 1 --2- 172.21.15.6:0/1551992638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7c8106f10 0x7fe7c819ce60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000b970 tx=0x7fe7c000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:07.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.662+0000 7fe7c4ff9640 1 -- 172.21.15.6:0/1551992638 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0019070 con 0x7fe7c8106f10 2024-09-12T03:19:07.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.662+0000 7fe7cde2d640 1 -- 172.21.15.6:0/1551992638 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c819d830 con 0x7fe7c8106f10 2024-09-12T03:19:07.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.662+0000 7fe7c4ff9640 1 -- 172.21.15.6:0/1551992638 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7c000cb40 con 0x7fe7c8106f10 2024-09-12T03:19:07.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.662+0000 7fe7c4ff9640 1 -- 172.21.15.6:0/1551992638 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0007650 con 0x7fe7c8106f10 2024-09-12T03:19:07.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.662+0000 7fe7cde2d640 1 -- 172.21.15.6:0/1551992638 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c81a5110 con 0x7fe7c8106f10 2024-09-12T03:19:07.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.663+0000 7fe7cde2d640 1 -- 172.21.15.6:0/1551992638 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7c8102240 con 0x7fe7c8106f10 2024-09-12T03:19:07.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.664+0000 7fe7c4ff9640 1 -- 172.21.15.6:0/1551992638 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe7c001c070 con 0x7fe7c8106f10 2024-09-12T03:19:07.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.665+0000 7fe7c4ff9640 1 --2- 172.21.15.6:0/1551992638 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe7b40a2b80 0x7fe7b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.665+0000 7fe7c77fe640 1 --2- 172.21.15.6:0/1551992638 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe7b40a2b80 0x7fe7b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.665+0000 7fe7c4ff9640 1 -- 172.21.15.6:0/1551992638 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7497+0+0 (secure 0 0 0) 0x7fe7c00ce3b0 con 0x7fe7c8106f10 2024-09-12T03:19:07.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.666+0000 7fe7c77fe640 1 --2- 172.21.15.6:0/1551992638 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe7b40a2b80 0x7fe7b40a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8000c00 tx=0x7fe7b8032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:07.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.669+0000 7fe7c4ff9640 1 -- 172.21.15.6:0/1551992638 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe7c009b640 con 0x7fe7c8106f10 2024-09-12T03:19:07.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:07.872+0000 7fe7cde2d640 1 -- 172.21.15.6:0/1551992638 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe7c8109b60 con 0x7fe7c8106f10 2024-09-12T03:19:08.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:08.311+0000 7fe7c4ff9640 1 -- 172.21.15.6:0/1551992638 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) ==== 170+0+0 (secure 0 0 0) 0x7fe7c009af40 con 0x7fe7c8106f10 2024-09-12T03:19:08.311 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data' created 2024-09-12T03:19:08.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:08.313+0000 7fe7cde2d640 1 -- 172.21.15.6:0/1551992638 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe7b40a2b80 msgr2=0x7fe7b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:08.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:08.313+0000 7fe7cde2d640 1 --2- 172.21.15.6:0/1551992638 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe7b40a2b80 0x7fe7b40a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8000c00 tx=0x7fe7b8032d50 comp rx=0 tx=0).stop 2024-09-12T03:19:08.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:08.313+0000 7fe7cde2d640 1 -- 172.21.15.6:0/1551992638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7c8106f10 msgr2=0x7fe7c819ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:08.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:08.313+0000 7fe7cde2d640 1 --2- 172.21.15.6:0/1551992638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7c8106f10 0x7fe7c819ce60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000b970 tx=0x7fe7c000be30 comp rx=0 tx=0).stop 2024-09-12T03:19:08.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:08.313+0000 7fe7cde2d640 1 -- 172.21.15.6:0/1551992638 shutdown_connections 2024-09-12T03:19:08.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:08.313+0000 7fe7cde2d640 1 --2- 172.21.15.6:0/1551992638 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe7b40a2b80 0x7fe7b40a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:08.313+0000 7fe7cde2d640 1 --2- 172.21.15.6:0/1551992638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7c8106f10 0x7fe7c819ce60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:08.313+0000 7fe7cde2d640 1 --2- 172.21.15.6:0/1551992638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7c8102790 0x7fe7c8198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:08.313+0000 7fe7cde2d640 1 --2- 172.21.15.6:0/1551992638 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7c8101de0 0x7fe7c8198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:08.313+0000 7fe7cde2d640 1 -- 172.21.15.6:0/1551992638 >> 172.21.15.6:0/1551992638 conn(0x7fe7c80fd8c0 msgr2=0x7fe7c80fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:08.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:08.314+0000 7fe7cde2d640 1 -- 172.21.15.6:0/1551992638 shutdown_connections 2024-09-12T03:19:08.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:08.314+0000 7fe7cde2d640 1 -- 172.21.15.6:0/1551992638 wait complete. 2024-09-12T03:19:08.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:08 smithi006 ceph-mon[30991]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T03:19:08.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:08 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:08.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:08 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:08.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:08 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:08.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:08 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:08.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:08 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:08.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:08 smithi161 ceph-mon[35485]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T03:19:08.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:08 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:08.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:08 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:08.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:08 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:08.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:08 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:08.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:08 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:08.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:08 smithi118 ceph-mon[35048]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T03:19:08.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:08 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:08.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:08 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:08.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:08 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:08.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:08 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:08.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:08 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:19:08.896 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T03:19:09.154 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:09.545 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:09 smithi006 ceph-mon[30991]: pgmap v271: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:19:09.545 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:09 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1551992638' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T03:19:09.545 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:09 smithi006 ceph-mon[30991]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:19:09.546 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:09 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1551992638' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T03:19:09.546 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:09 smithi006 ceph-mon[30991]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T03:19:09.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:09 smithi161 ceph-mon[35485]: pgmap v271: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:19:09.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:09 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1551992638' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T03:19:09.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:09 smithi161 ceph-mon[35485]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:19:09.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:09 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1551992638' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T03:19:09.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:09 smithi161 ceph-mon[35485]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T03:19:09.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.629+0000 7f6857fed640 1 -- 172.21.15.6:0/1211690946 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6850106a80 msgr2=0x7f6850106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:09.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.629+0000 7f6857fed640 1 --2- 172.21.15.6:0/1211690946 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6850106a80 0x7f6850106e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f684c009930 tx=0x7f684c02a420 comp rx=0 tx=0).stop 2024-09-12T03:19:09.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.629+0000 7f6857fed640 1 -- 172.21.15.6:0/1211690946 shutdown_connections 2024-09-12T03:19:09.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.629+0000 7f6857fed640 1 --2- 172.21.15.6:0/1211690946 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6850101420 0x7f685010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.629+0000 7f6857fed640 1 --2- 172.21.15.6:0/1211690946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6850100a80 0x7f6850100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.629+0000 7f6857fed640 1 --2- 172.21.15.6:0/1211690946 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6850106a80 0x7f6850106e60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.629+0000 7f6857fed640 1 -- 172.21.15.6:0/1211690946 >> 172.21.15.6:0/1211690946 conn(0x7f68500fc7a0 msgr2=0x7f68500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:09.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.629+0000 7f6857fed640 1 -- 172.21.15.6:0/1211690946 shutdown_connections 2024-09-12T03:19:09.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.629+0000 7f6857fed640 1 -- 172.21.15.6:0/1211690946 wait complete. 2024-09-12T03:19:09.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.630+0000 7f6857fed640 1 Processor -- start 2024-09-12T03:19:09.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.630+0000 7f6857fed640 1 -- start start 2024-09-12T03:19:09.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.630+0000 7f6857fed640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6850100a80 0x7f68501992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.630+0000 7f6857fed640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6850101420 0x7f68501997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.630+0000 7f6857fed640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6850106a80 0x7f68501a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.630+0000 7f6857fed640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6850110e40 con 0x7f6850101420 2024-09-12T03:19:09.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.630+0000 7f6857fed640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f6850110cc0 con 0x7f6850106a80 2024-09-12T03:19:09.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.630+0000 7f6857fed640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6850110fc0 con 0x7f6850100a80 2024-09-12T03:19:09.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.630+0000 7f6855588640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6850101420 0x7f68501997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:09.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.630+0000 7f6855588640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6850101420 0x7f68501997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40466/0 (socket says 172.21.15.6:40466) 2024-09-12T03:19:09.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.630+0000 7f6855588640 1 -- 172.21.15.6:0/2307814995 learned_addr learned my addr 172.21.15.6:0/2307814995 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:09.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.630+0000 7f685658a640 1 --2- 172.21.15.6:0/2307814995 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6850106a80 0x7f68501a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:09.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.630+0000 7f6855d89640 1 --2- 172.21.15.6:0/2307814995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6850100a80 0x7f68501992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:09.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.630+0000 7f6855588640 1 -- 172.21.15.6:0/2307814995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6850100a80 msgr2=0x7f68501992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:09.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.630+0000 7f6855588640 1 --2- 172.21.15.6:0/2307814995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6850100a80 0x7f68501992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.630+0000 7f6855588640 1 -- 172.21.15.6:0/2307814995 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6850106a80 msgr2=0x7f68501a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:09.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.630+0000 7f6855588640 1 --2- 172.21.15.6:0/2307814995 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6850106a80 0x7f68501a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.631+0000 7f6855588640 1 -- 172.21.15.6:0/2307814995 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68501a4860 con 0x7f6850101420 2024-09-12T03:19:09.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.631+0000 7f6855588640 1 --2- 172.21.15.6:0/2307814995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6850101420 0x7f68501997e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f684000c920 tx=0x7f684000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:09.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.631+0000 7f6846ffd640 1 -- 172.21.15.6:0/2307814995 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840007c70 con 0x7f6850101420 2024-09-12T03:19:09.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.631+0000 7f6846ffd640 1 -- 172.21.15.6:0/2307814995 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6840007e10 con 0x7f6850101420 2024-09-12T03:19:09.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.631+0000 7f6846ffd640 1 -- 172.21.15.6:0/2307814995 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840018890 con 0x7f6850101420 2024-09-12T03:19:09.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.631+0000 7f6857fed640 1 -- 172.21.15.6:0/2307814995 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68501a4a30 con 0x7f6850101420 2024-09-12T03:19:09.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.631+0000 7f6857fed640 1 -- 172.21.15.6:0/2307814995 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68501a4ef0 con 0x7f6850101420 2024-09-12T03:19:09.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.632+0000 7f6857fed640 1 -- 172.21.15.6:0/2307814995 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6850100ee0 con 0x7f6850101420 2024-09-12T03:19:09.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.633+0000 7f6846ffd640 1 -- 172.21.15.6:0/2307814995 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f6840018a50 con 0x7f6850101420 2024-09-12T03:19:09.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.634+0000 7f6846ffd640 1 --2- 172.21.15.6:0/2307814995 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f683c0a2c00 0x7f683c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.634+0000 7f6846ffd640 1 -- 172.21.15.6:0/2307814995 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7f68400cf280 con 0x7f6850101420 2024-09-12T03:19:09.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.634+0000 7f6855d89640 1 --2- 172.21.15.6:0/2307814995 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f683c0a2c00 0x7f683c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:09.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.635+0000 7f6855d89640 1 --2- 172.21.15.6:0/2307814995 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f683c0a2c00 0x7f683c0a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f684c000c00 tx=0x7f684c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:09.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.638+0000 7f6846ffd640 1 -- 172.21.15.6:0/2307814995 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6840014030 con 0x7f6850101420 2024-09-12T03:19:09.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:09 smithi118 ceph-mon[35048]: pgmap v271: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:19:09.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:09 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1551992638' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T03:19:09.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:09 smithi118 ceph-mon[35048]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:19:09.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:09 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1551992638' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T03:19:09.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:09 smithi118 ceph-mon[35048]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T03:19:09.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:09.848+0000 7f6857fed640 1 -- 172.21.15.6:0/2307814995 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f685006ad80 con 0x7f6850101420 2024-09-12T03:19:10.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:10.320+0000 7f6846ffd640 1 -- 172.21.15.6:0/2307814995 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f685006ad80 con 0x7f6850101420 2024-09-12T03:19:10.321 INFO:teuthology.orchestra.run.smithi006.stderr:new fs with metadata pool 2 and data pool 3 2024-09-12T03:19:10.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:10.322+0000 7f6857fed640 1 -- 172.21.15.6:0/2307814995 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f683c0a2c00 msgr2=0x7f683c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:10.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:10.322+0000 7f6857fed640 1 --2- 172.21.15.6:0/2307814995 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f683c0a2c00 0x7f683c0a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f684c000c00 tx=0x7f684c005a20 comp rx=0 tx=0).stop 2024-09-12T03:19:10.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:10.322+0000 7f6857fed640 1 -- 172.21.15.6:0/2307814995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6850101420 msgr2=0x7f68501997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:10.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:10.322+0000 7f6857fed640 1 --2- 172.21.15.6:0/2307814995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6850101420 0x7f68501997e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f684000c920 tx=0x7f684000cde0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:10.322+0000 7f6857fed640 1 -- 172.21.15.6:0/2307814995 shutdown_connections 2024-09-12T03:19:10.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:10.322+0000 7f6857fed640 1 --2- 172.21.15.6:0/2307814995 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f683c0a2c00 0x7f683c0a50c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:10.322+0000 7f6857fed640 1 --2- 172.21.15.6:0/2307814995 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6850106a80 0x7f68501a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:10.322+0000 7f6857fed640 1 --2- 172.21.15.6:0/2307814995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6850101420 0x7f68501997e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:10.322+0000 7f6857fed640 1 --2- 172.21.15.6:0/2307814995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6850100a80 0x7f68501992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:10.322+0000 7f6857fed640 1 -- 172.21.15.6:0/2307814995 >> 172.21.15.6:0/2307814995 conn(0x7f68500fc7a0 msgr2=0x7f68500fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:10.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:10.322+0000 7f6857fed640 1 -- 172.21.15.6:0/2307814995 shutdown_connections 2024-09-12T03:19:10.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:10.322+0000 7f6857fed640 1 -- 172.21.15.6:0/2307814995 wait complete. 2024-09-12T03:19:10.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:10 smithi006 ceph-mon[30991]: pgmap v273: 97 pgs: 19 active+clean, 8 creating+peering, 70 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:19:10.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:10 smithi006 ceph-mon[30991]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T03:19:10.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:10 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2307814995' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T03:19:10.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:10 smithi006 ceph-mon[30991]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T03:19:10.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:10 smithi006 ceph-mon[30991]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T03:19:10.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:10 smithi161 ceph-mon[35485]: pgmap v273: 97 pgs: 19 active+clean, 8 creating+peering, 70 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:19:10.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:10 smithi161 ceph-mon[35485]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T03:19:10.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:10 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2307814995' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T03:19:10.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:10 smithi161 ceph-mon[35485]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T03:19:10.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:10 smithi161 ceph-mon[35485]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T03:19:10.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:10 smithi118 ceph-mon[35048]: pgmap v273: 97 pgs: 19 active+clean, 8 creating+peering, 70 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:19:10.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:10 smithi118 ceph-mon[35048]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T03:19:10.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:10 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2307814995' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T03:19:10.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:10 smithi118 ceph-mon[35048]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T03:19:10.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:10 smithi118 ceph-mon[35048]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T03:19:10.960 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:19:11.223 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:11.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:11 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2307814995' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T03:19:11.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:11 smithi006 ceph-mon[30991]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T03:19:11.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:11 smithi006 ceph-mon[30991]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T03:19:11.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:11 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2307814995' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T03:19:11.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:11 smithi161 ceph-mon[35485]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T03:19:11.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:11 smithi161 ceph-mon[35485]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T03:19:11.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.632+0000 7f5b4f5b6640 1 -- 172.21.15.6:0/2658177415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b48102db0 msgr2=0x7f5b48103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:11.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.632+0000 7f5b4f5b6640 1 --2- 172.21.15.6:0/2658177415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b48102db0 0x7f5b48103190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009930 tx=0x7f5b4002a420 comp rx=0 tx=0).stop 2024-09-12T03:19:11.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.633+0000 7f5b4f5b6640 1 -- 172.21.15.6:0/2658177415 shutdown_connections 2024-09-12T03:19:11.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.633+0000 7f5b4f5b6640 1 --2- 172.21.15.6:0/2658177415 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b481082e0 0x7f5b4810ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:11.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.633+0000 7f5b4f5b6640 1 --2- 172.21.15.6:0/2658177415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b48103760 0x7f5b48107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:11.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.633+0000 7f5b4f5b6640 1 --2- 172.21.15.6:0/2658177415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b48102db0 0x7f5b48103190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:11.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.633+0000 7f5b4f5b6640 1 -- 172.21.15.6:0/2658177415 >> 172.21.15.6:0/2658177415 conn(0x7f5b480fc7d0 msgr2=0x7f5b480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:11.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.633+0000 7f5b4f5b6640 1 -- 172.21.15.6:0/2658177415 shutdown_connections 2024-09-12T03:19:11.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.633+0000 7f5b4f5b6640 1 -- 172.21.15.6:0/2658177415 wait complete. 2024-09-12T03:19:11.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.633+0000 7f5b4f5b6640 1 Processor -- start 2024-09-12T03:19:11.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.633+0000 7f5b4f5b6640 1 -- start start 2024-09-12T03:19:11.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b4f5b6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b48102db0 0x7f5b481985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:11.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b4f5b6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b48103760 0x7f5b48198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:11.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b4f5b6640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b481082e0 0x7f5b4819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:11.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b4f5b6640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5b481120b0 con 0x7f5b48102db0 2024-09-12T03:19:11.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b4f5b6640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f5b48111f30 con 0x7f5b481082e0 2024-09-12T03:19:11.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b4f5b6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5b48112230 con 0x7f5b48103760 2024-09-12T03:19:11.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b4d352640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b48102db0 0x7f5b481985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:11.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b4d352640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b48102db0 0x7f5b481985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40492/0 (socket says 172.21.15.6:40492) 2024-09-12T03:19:11.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b4d352640 1 -- 172.21.15.6:0/3819419332 learned_addr learned my addr 172.21.15.6:0/3819419332 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:11.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b4db53640 1 --2- 172.21.15.6:0/3819419332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b481082e0 0x7f5b4819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:11.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b4cb51640 1 --2- 172.21.15.6:0/3819419332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b48103760 0x7f5b48198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:11.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b4d352640 1 -- 172.21.15.6:0/3819419332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b48103760 msgr2=0x7f5b48198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:11.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b4d352640 1 --2- 172.21.15.6:0/3819419332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b48103760 0x7f5b48198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:11.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b4d352640 1 -- 172.21.15.6:0/3819419332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b481082e0 msgr2=0x7f5b4819cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:11.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b4d352640 1 --2- 172.21.15.6:0/3819419332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b481082e0 0x7f5b4819cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:11.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b4d352640 1 -- 172.21.15.6:0/3819419332 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4819d580 con 0x7f5b48102db0 2024-09-12T03:19:11.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b4db53640 1 --2- 172.21.15.6:0/3819419332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b481082e0 0x7f5b4819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:11.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b4d352640 1 --2- 172.21.15.6:0/3819419332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b48102db0 0x7f5b481985d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5b40004770 tx=0x7f5b40003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:11.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b467fc640 1 -- 172.21.15.6:0/3819419332 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4003f070 con 0x7f5b48102db0 2024-09-12T03:19:11.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b467fc640 1 -- 172.21.15.6:0/3819419332 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b40031c20 con 0x7f5b48102db0 2024-09-12T03:19:11.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b4f5b6640 1 -- 172.21.15.6:0/3819419332 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4819d810 con 0x7f5b48102db0 2024-09-12T03:19:11.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b467fc640 1 -- 172.21.15.6:0/3819419332 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40035030 con 0x7f5b48102db0 2024-09-12T03:19:11.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.634+0000 7f5b4f5b6640 1 -- 172.21.15.6:0/3819419332 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b481a5150 con 0x7f5b48102db0 2024-09-12T03:19:11.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.635+0000 7f5b4f5b6640 1 -- 172.21.15.6:0/3819419332 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b48103190 con 0x7f5b48102db0 2024-09-12T03:19:11.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.637+0000 7f5b467fc640 1 -- 172.21.15.6:0/3819419332 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f5b40042070 con 0x7f5b48102db0 2024-09-12T03:19:11.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.637+0000 7f5b467fc640 1 --2- 172.21.15.6:0/3819419332 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5b300a2b80 0x7f5b300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:11.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.637+0000 7f5b4cb51640 1 --2- 172.21.15.6:0/3819419332 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5b300a2b80 0x7f5b300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:11.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.638+0000 7f5b467fc640 1 -- 172.21.15.6:0/3819419332 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f5b400f0080 con 0x7f5b48102db0 2024-09-12T03:19:11.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.638+0000 7f5b4cb51640 1 --2- 172.21.15.6:0/3819419332 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5b300a2b80 0x7f5b300a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c005fd0 tx=0x7f5b3c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:11.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.641+0000 7f5b467fc640 1 -- 172.21.15.6:0/3819419332 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5b400bd130 con 0x7f5b48102db0 2024-09-12T03:19:11.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:11 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2307814995' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T03:19:11.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:11 smithi118 ceph-mon[35048]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T03:19:11.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:11 smithi118 ceph-mon[35048]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T03:19:11.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.819+0000 7f5b4f5b6640 1 -- 172.21.15.6:0/3819419332 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b4810a2c0 con 0x7f5b48102db0 2024-09-12T03:19:11.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.820+0000 7f5b467fc640 1 -- 172.21.15.6:0/3819419332 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22220 (secure 0 0 0) 0x7f5b400bca30 con 0x7f5b48102db0 2024-09-12T03:19:11.822 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:19:11.822 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":80,"fsid":"89d99938-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:11:29.530545+0000","modified":"2024-09-12T03:19:09.851106+0000","last_up_change":"2024-09-12T03:18:21.292572+0000","last_in_change":"2024-09-12T03:17:58.013800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:14:37.662010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:06.143398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:08.305279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f55d68b-95a0-4ee2-a46f-bc422b38471e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6803","nonce":726729286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6805","nonce":726729286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6809","nonce":726729286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6807","nonce":726729286}]},"public_addr":"172.21.15.6:6803/726729286","cluster_addr":"172.21.15.6:6805/726729286","heartbeat_back_addr":"172.21.15.6:6809/726729286","heartbeat_front_addr":"172.21.15.6:6807/726729286","state":["exists","up"]},{"osd":1,"uuid":"83afdb01-c731-4de1-96c6-3f798549b1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6801","nonce":3377555415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6803","nonce":3377555415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6807","nonce":3377555415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6805","nonce":3377555415}]},"public_addr":"172.21.15.118:6801/3377555415","cluster_addr":"172.21.15.118:6803/3377555415","heartbeat_back_addr":"172.21.15.118:6807/3377555415","heartbeat_front_addr":"172.21.15.118:6805/3377555415","state":["exists","up"]},{"osd":2,"uuid":"aed718b4-b849-403a-b81c-3a140b5c380c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6801","nonce":392541009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6803","nonce":392541009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6807","nonce":392541009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6805","nonce":392541009}]},"public_addr":"172.21.15.161:6801/392541009","cluster_addr":"172.21.15.161:6803/392541009","heartbeat_back_addr":"172.21.15.161:6807/392541009","heartbeat_front_addr":"172.21.15.161:6805/392541009","state":["exists","up"]},{"osd":3,"uuid":"25ced4c1-bf31-402d-8397-d42fd57f615f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6811","nonce":1892039151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6813","nonce":1892039151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6817","nonce":1892039151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6815","nonce":1892039151}]},"public_addr":"172.21.15.6:6811/1892039151","cluster_addr":"172.21.15.6:6813/1892039151","heartbeat_back_addr":"172.21.15.6:6817/1892039151","heartbeat_front_addr":"172.21.15.6:6815/1892039151","state":["exists","up"]},{"osd":4,"uuid":"1d884604-920b-4c35-b41e-d629ad9f9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6809","nonce":150177024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6811","nonce":150177024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6815","nonce":150177024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6813","nonce":150177024}]},"public_addr":"172.21.15.118:6809/150177024","cluster_addr":"172.21.15.118:6811/150177024","heartbeat_back_addr":"172.21.15.118:6815/150177024","heartbeat_front_addr":"172.21.15.118:6813/150177024","state":["exists","up"]},{"osd":5,"uuid":"1d40c0ef-99f6-426f-90c7-27579edc66af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6809","nonce":3758167854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6811","nonce":3758167854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6815","nonce":3758167854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6813","nonce":3758167854}]},"public_addr":"172.21.15.161:6809/3758167854","cluster_addr":"172.21.15.161:6811/3758167854","heartbeat_back_addr":"172.21.15.161:6815/3758167854","heartbeat_front_addr":"172.21.15.161:6813/3758167854","state":["exists","up"]},{"osd":6,"uuid":"9d9f9510-b107-40c2-9a98-9012715944b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6819","nonce":2129224874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6821","nonce":2129224874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6825","nonce":2129224874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6823","nonce":2129224874}]},"public_addr":"172.21.15.6:6819/2129224874","cluster_addr":"172.21.15.6:6821/2129224874","heartbeat_back_addr":"172.21.15.6:6825/2129224874","heartbeat_front_addr":"172.21.15.6:6823/2129224874","state":["exists","up"]},{"osd":7,"uuid":"84635ae1-3240-4954-af14-76bb3aa162e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6817","nonce":269980353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6819","nonce":269980353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6823","nonce":269980353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6821","nonce":269980353}]},"public_addr":"172.21.15.118:6817/269980353","cluster_addr":"172.21.15.118:6819/269980353","heartbeat_back_addr":"172.21.15.118:6823/269980353","heartbeat_front_addr":"172.21.15.118:6821/269980353","state":["exists","up"]},{"osd":8,"uuid":"6fd58e55-1492-4595-8647-823ea3e3c5d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6817","nonce":1783416134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6819","nonce":1783416134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6823","nonce":1783416134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6821","nonce":1783416134}]},"public_addr":"172.21.15.161:6817/1783416134","cluster_addr":"172.21.15.161:6819/1783416134","heartbeat_back_addr":"172.21.15.161:6823/1783416134","heartbeat_front_addr":"172.21.15.161:6821/1783416134","state":["exists","up"]},{"osd":9,"uuid":"90508a6d-a49c-43c6-a600-5bbccd8702ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6827","nonce":2987332365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6829","nonce":2987332365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6833","nonce":2987332365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6831","nonce":2987332365}]},"public_addr":"172.21.15.6:6827/2987332365","cluster_addr":"172.21.15.6:6829/2987332365","heartbeat_back_addr":"172.21.15.6:6833/2987332365","heartbeat_front_addr":"172.21.15.6:6831/2987332365","state":["exists","up"]},{"osd":10,"uuid":"911e86d1-669a-4935-82c0-7da51d141df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6825","nonce":2167146341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6827","nonce":2167146341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6831","nonce":2167146341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6829","nonce":2167146341}]},"public_addr":"172.21.15.118:6825/2167146341","cluster_addr":"172.21.15.118:6827/2167146341","heartbeat_back_addr":"172.21.15.118:6831/2167146341","heartbeat_front_addr":"172.21.15.118:6829/2167146341","state":["exists","up"]},{"osd":11,"uuid":"5a15c6a7-ff43-473c-bb48-0c129c316df5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6825","nonce":2889795796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6827","nonce":2889795796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6831","nonce":2889795796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6829","nonce":2889795796}]},"public_addr":"172.21.15.161:6825/2889795796","cluster_addr":"172.21.15.161:6827/2889795796","heartbeat_back_addr":"172.21.15.161:6831/2889795796","heartbeat_front_addr":"172.21.15.161:6829/2889795796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:42.912205+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:08.410669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:34.299134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:59.093181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:24.116635+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:49.566433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:15.719320+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:40.541017+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:05.178178+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:31.163877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:55.405110+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:19.417156+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6800/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/2620491916":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/4037656392":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/1377116573":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/3062800125":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:6800/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/4145338968":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/825701128":"2024-09-13T03:11:55.244257+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:11.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.823+0000 7f5b4f5b6640 1 -- 172.21.15.6:0/3819419332 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5b300a2b80 msgr2=0x7f5b300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:11.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.823+0000 7f5b4f5b6640 1 --2- 172.21.15.6:0/3819419332 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5b300a2b80 0x7f5b300a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c005fd0 tx=0x7f5b3c005850 comp rx=0 tx=0).stop 2024-09-12T03:19:11.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.823+0000 7f5b4f5b6640 1 -- 172.21.15.6:0/3819419332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b48102db0 msgr2=0x7f5b481985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:11.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.823+0000 7f5b4f5b6640 1 --2- 172.21.15.6:0/3819419332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b48102db0 0x7f5b481985d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5b40004770 tx=0x7f5b40003b90 comp rx=0 tx=0).stop 2024-09-12T03:19:11.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.823+0000 7f5b4f5b6640 1 -- 172.21.15.6:0/3819419332 shutdown_connections 2024-09-12T03:19:11.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.823+0000 7f5b4f5b6640 1 --2- 172.21.15.6:0/3819419332 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5b300a2b80 0x7f5b300a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:11.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.823+0000 7f5b4f5b6640 1 --2- 172.21.15.6:0/3819419332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b481082e0 0x7f5b4819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:11.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.823+0000 7f5b4f5b6640 1 --2- 172.21.15.6:0/3819419332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b48103760 0x7f5b48198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:11.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.823+0000 7f5b4f5b6640 1 --2- 172.21.15.6:0/3819419332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b48102db0 0x7f5b481985d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:11.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.823+0000 7f5b4f5b6640 1 -- 172.21.15.6:0/3819419332 >> 172.21.15.6:0/3819419332 conn(0x7f5b480fc7d0 msgr2=0x7f5b48105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:11.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.823+0000 7f5b4f5b6640 1 -- 172.21.15.6:0/3819419332 shutdown_connections 2024-09-12T03:19:11.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:11.823+0000 7f5b4f5b6640 1 -- 172.21.15.6:0/3819419332 wait complete. 2024-09-12T03:19:12.212 INFO:journalctl@ceph.mds.b.smithi118.stdout:Sep 12 03:19:11 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-b[65505]: 2024-09-12T03:19:11.748+0000 7f8b3c921640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:19:12.480 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:12 smithi006 ceph-mon[30991]: pgmap v276: 97 pgs: 54 active+clean, 8 creating+peering, 35 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 506 B/s wr, 1 op/s 2024-09-12T03:19:12.481 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:12 smithi006 ceph-mon[30991]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T03:19:12.481 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:12 smithi006 ceph-mon[30991]: Cluster is now healthy 2024-09-12T03:19:12.481 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:12 smithi006 ceph-mon[30991]: mds.? [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] up:active 2024-09-12T03:19:12.481 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:12 smithi006 ceph-mon[30991]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:19:12.481 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:12 smithi006 ceph-mon[30991]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:19:12.481 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:12 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3819419332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:12.508 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:19:12.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:12 smithi161 ceph-mon[35485]: pgmap v276: 97 pgs: 54 active+clean, 8 creating+peering, 35 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 506 B/s wr, 1 op/s 2024-09-12T03:19:12.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:12 smithi161 ceph-mon[35485]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T03:19:12.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:12 smithi161 ceph-mon[35485]: Cluster is now healthy 2024-09-12T03:19:12.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:12 smithi161 ceph-mon[35485]: mds.? [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] up:active 2024-09-12T03:19:12.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:12 smithi161 ceph-mon[35485]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:19:12.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:12 smithi161 ceph-mon[35485]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:19:12.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:12 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3819419332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:12.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:12 smithi118 ceph-mon[35048]: pgmap v276: 97 pgs: 54 active+clean, 8 creating+peering, 35 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 506 B/s wr, 1 op/s 2024-09-12T03:19:12.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:12 smithi118 ceph-mon[35048]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T03:19:12.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:12 smithi118 ceph-mon[35048]: Cluster is now healthy 2024-09-12T03:19:12.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:12 smithi118 ceph-mon[35048]: mds.? [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] up:active 2024-09-12T03:19:12.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:12 smithi118 ceph-mon[35048]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:19:12.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:12 smithi118 ceph-mon[35048]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:19:12.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:12 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3819419332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:12.763 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:13.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.190+0000 7f9508dda640 1 -- 172.21.15.6:0/801785418 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9504101de0 msgr2=0x7f95041021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.190+0000 7f9508dda640 1 --2- 172.21.15.6:0/801785418 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9504101de0 0x7f95041021c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009ca0 tx=0x7f94fc02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.191+0000 7f9508dda640 1 -- 172.21.15.6:0/801785418 shutdown_connections 2024-09-12T03:19:13.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.191+0000 7f9508dda640 1 --2- 172.21.15.6:0/801785418 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9504106f10 0x7f950410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.191+0000 7f9508dda640 1 --2- 172.21.15.6:0/801785418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9504102790 0x7f95041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.191+0000 7f9508dda640 1 --2- 172.21.15.6:0/801785418 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9504101de0 0x7f95041021c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.191+0000 7f9508dda640 1 -- 172.21.15.6:0/801785418 >> 172.21.15.6:0/801785418 conn(0x7f95040fd8c0 msgr2=0x7f95040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:13.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.191+0000 7f9508dda640 1 -- 172.21.15.6:0/801785418 shutdown_connections 2024-09-12T03:19:13.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.191+0000 7f9508dda640 1 -- 172.21.15.6:0/801785418 wait complete. 2024-09-12T03:19:13.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.191+0000 7f9508dda640 1 Processor -- start 2024-09-12T03:19:13.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.192+0000 7f9508dda640 1 -- start start 2024-09-12T03:19:13.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.192+0000 7f9508dda640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9504101de0 0x7f95040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.192+0000 7f9508dda640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9504102790 0x7f9504074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.192+0000 7f9508dda640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9504106f10 0x7f9504074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.192+0000 7f9508dda640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9504111f30 con 0x7f9504101de0 2024-09-12T03:19:13.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.192+0000 7f9508dda640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f9504111db0 con 0x7f9504102790 2024-09-12T03:19:13.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.192+0000 7f9508dda640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f95041120b0 con 0x7f9504106f10 2024-09-12T03:19:13.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.192+0000 7f950259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9504101de0 0x7f95040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.192+0000 7f950259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9504101de0 0x7f95040744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40518/0 (socket says 172.21.15.6:40518) 2024-09-12T03:19:13.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.192+0000 7f950259c640 1 -- 172.21.15.6:0/2788983185 learned_addr learned my addr 172.21.15.6:0/2788983185 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:13.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.192+0000 7f9502d9d640 1 --2- 172.21.15.6:0/2788983185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9504106f10 0x7f9504074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.192+0000 7f9501d9b640 1 --2- 172.21.15.6:0/2788983185 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9504102790 0x7f9504074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.192+0000 7f950259c640 1 -- 172.21.15.6:0/2788983185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9504106f10 msgr2=0x7f9504074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.192+0000 7f950259c640 1 --2- 172.21.15.6:0/2788983185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9504106f10 0x7f9504074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.192+0000 7f950259c640 1 -- 172.21.15.6:0/2788983185 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9504102790 msgr2=0x7f9504074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.192+0000 7f950259c640 1 --2- 172.21.15.6:0/2788983185 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9504102790 0x7f9504074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.192+0000 7f950259c640 1 -- 172.21.15.6:0/2788983185 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95041a6980 con 0x7f9504101de0 2024-09-12T03:19:13.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.192+0000 7f9502d9d640 1 --2- 172.21.15.6:0/2788983185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9504106f10 0x7f9504074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:13.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.192+0000 7f950259c640 1 --2- 172.21.15.6:0/2788983185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9504101de0 0x7f95040744f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f94fc02ade0 tx=0x7f94fc0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:13.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.193+0000 7f94fb7fe640 1 -- 172.21.15.6:0/2788983185 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc02fa20 con 0x7f9504101de0 2024-09-12T03:19:13.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.193+0000 7f9508dda640 1 -- 172.21.15.6:0/2788983185 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95041a6c10 con 0x7f9504101de0 2024-09-12T03:19:13.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.193+0000 7f94fb7fe640 1 -- 172.21.15.6:0/2788983185 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94fc02fbc0 con 0x7f9504101de0 2024-09-12T03:19:13.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.193+0000 7f94fb7fe640 1 -- 172.21.15.6:0/2788983185 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc0467b0 con 0x7f9504101de0 2024-09-12T03:19:13.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.193+0000 7f9508dda640 1 -- 172.21.15.6:0/2788983185 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95041a71a0 con 0x7f9504101de0 2024-09-12T03:19:13.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.194+0000 7f9508dda640 1 -- 172.21.15.6:0/2788983185 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95041021c0 con 0x7f9504101de0 2024-09-12T03:19:13.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.195+0000 7f94fb7fe640 1 -- 172.21.15.6:0/2788983185 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f94fc02fd70 con 0x7f9504101de0 2024-09-12T03:19:13.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.196+0000 7f94fb7fe640 1 --2- 172.21.15.6:0/2788983185 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f94f00a2b80 0x7f94f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.196+0000 7f94fb7fe640 1 -- 172.21.15.6:0/2788983185 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f94fc0f0710 con 0x7f9504101de0 2024-09-12T03:19:13.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.196+0000 7f9501d9b640 1 --2- 172.21.15.6:0/2788983185 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f94f00a2b80 0x7f94f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.196+0000 7f9501d9b640 1 --2- 172.21.15.6:0/2788983185 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f94f00a2b80 0x7f94f00a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f95041087b0 tx=0x7f94ec005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:13.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.199+0000 7f94fb7fe640 1 -- 172.21.15.6:0/2788983185 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f94fc0453e0 con 0x7f9504101de0 2024-09-12T03:19:13.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.376+0000 7f9508dda640 1 -- 172.21.15.6:0/2788983185 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f950410a0f0 con 0x7f9504101de0 2024-09-12T03:19:13.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.378+0000 7f94fb7fe640 1 -- 172.21.15.6:0/2788983185 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22220 (secure 0 0 0) 0x7f94fc0bd850 con 0x7f9504101de0 2024-09-12T03:19:13.379 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:19:13.379 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":80,"fsid":"89d99938-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:11:29.530545+0000","modified":"2024-09-12T03:19:09.851106+0000","last_up_change":"2024-09-12T03:18:21.292572+0000","last_in_change":"2024-09-12T03:17:58.013800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:14:37.662010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:06.143398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:08.305279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f55d68b-95a0-4ee2-a46f-bc422b38471e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6803","nonce":726729286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6805","nonce":726729286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6809","nonce":726729286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6807","nonce":726729286}]},"public_addr":"172.21.15.6:6803/726729286","cluster_addr":"172.21.15.6:6805/726729286","heartbeat_back_addr":"172.21.15.6:6809/726729286","heartbeat_front_addr":"172.21.15.6:6807/726729286","state":["exists","up"]},{"osd":1,"uuid":"83afdb01-c731-4de1-96c6-3f798549b1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6801","nonce":3377555415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6803","nonce":3377555415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6807","nonce":3377555415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6805","nonce":3377555415}]},"public_addr":"172.21.15.118:6801/3377555415","cluster_addr":"172.21.15.118:6803/3377555415","heartbeat_back_addr":"172.21.15.118:6807/3377555415","heartbeat_front_addr":"172.21.15.118:6805/3377555415","state":["exists","up"]},{"osd":2,"uuid":"aed718b4-b849-403a-b81c-3a140b5c380c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6801","nonce":392541009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6803","nonce":392541009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6807","nonce":392541009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6805","nonce":392541009}]},"public_addr":"172.21.15.161:6801/392541009","cluster_addr":"172.21.15.161:6803/392541009","heartbeat_back_addr":"172.21.15.161:6807/392541009","heartbeat_front_addr":"172.21.15.161:6805/392541009","state":["exists","up"]},{"osd":3,"uuid":"25ced4c1-bf31-402d-8397-d42fd57f615f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6811","nonce":1892039151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6813","nonce":1892039151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6817","nonce":1892039151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6815","nonce":1892039151}]},"public_addr":"172.21.15.6:6811/1892039151","cluster_addr":"172.21.15.6:6813/1892039151","heartbeat_back_addr":"172.21.15.6:6817/1892039151","heartbeat_front_addr":"172.21.15.6:6815/1892039151","state":["exists","up"]},{"osd":4,"uuid":"1d884604-920b-4c35-b41e-d629ad9f9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6809","nonce":150177024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6811","nonce":150177024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6815","nonce":150177024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6813","nonce":150177024}]},"public_addr":"172.21.15.118:6809/150177024","cluster_addr":"172.21.15.118:6811/150177024","heartbeat_back_addr":"172.21.15.118:6815/150177024","heartbeat_front_addr":"172.21.15.118:6813/150177024","state":["exists","up"]},{"osd":5,"uuid":"1d40c0ef-99f6-426f-90c7-27579edc66af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6809","nonce":3758167854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6811","nonce":3758167854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6815","nonce":3758167854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6813","nonce":3758167854}]},"public_addr":"172.21.15.161:6809/3758167854","cluster_addr":"172.21.15.161:6811/3758167854","heartbeat_back_addr":"172.21.15.161:6815/3758167854","heartbeat_front_addr":"172.21.15.161:6813/3758167854","state":["exists","up"]},{"osd":6,"uuid":"9d9f9510-b107-40c2-9a98-9012715944b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6819","nonce":2129224874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6821","nonce":2129224874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6825","nonce":2129224874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6823","nonce":2129224874}]},"public_addr":"172.21.15.6:6819/2129224874","cluster_addr":"172.21.15.6:6821/2129224874","heartbeat_back_addr":"172.21.15.6:6825/2129224874","heartbeat_front_addr":"172.21.15.6:6823/2129224874","state":["exists","up"]},{"osd":7,"uuid":"84635ae1-3240-4954-af14-76bb3aa162e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6817","nonce":269980353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6819","nonce":269980353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6823","nonce":269980353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6821","nonce":269980353}]},"public_addr":"172.21.15.118:6817/269980353","cluster_addr":"172.21.15.118:6819/269980353","heartbeat_back_addr":"172.21.15.118:6823/269980353","heartbeat_front_addr":"172.21.15.118:6821/269980353","state":["exists","up"]},{"osd":8,"uuid":"6fd58e55-1492-4595-8647-823ea3e3c5d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6817","nonce":1783416134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6819","nonce":1783416134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6823","nonce":1783416134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6821","nonce":1783416134}]},"public_addr":"172.21.15.161:6817/1783416134","cluster_addr":"172.21.15.161:6819/1783416134","heartbeat_back_addr":"172.21.15.161:6823/1783416134","heartbeat_front_addr":"172.21.15.161:6821/1783416134","state":["exists","up"]},{"osd":9,"uuid":"90508a6d-a49c-43c6-a600-5bbccd8702ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6827","nonce":2987332365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6829","nonce":2987332365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6833","nonce":2987332365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6831","nonce":2987332365}]},"public_addr":"172.21.15.6:6827/2987332365","cluster_addr":"172.21.15.6:6829/2987332365","heartbeat_back_addr":"172.21.15.6:6833/2987332365","heartbeat_front_addr":"172.21.15.6:6831/2987332365","state":["exists","up"]},{"osd":10,"uuid":"911e86d1-669a-4935-82c0-7da51d141df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6825","nonce":2167146341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6827","nonce":2167146341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6831","nonce":2167146341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6829","nonce":2167146341}]},"public_addr":"172.21.15.118:6825/2167146341","cluster_addr":"172.21.15.118:6827/2167146341","heartbeat_back_addr":"172.21.15.118:6831/2167146341","heartbeat_front_addr":"172.21.15.118:6829/2167146341","state":["exists","up"]},{"osd":11,"uuid":"5a15c6a7-ff43-473c-bb48-0c129c316df5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6825","nonce":2889795796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6827","nonce":2889795796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6831","nonce":2889795796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6829","nonce":2889795796}]},"public_addr":"172.21.15.161:6825/2889795796","cluster_addr":"172.21.15.161:6827/2889795796","heartbeat_back_addr":"172.21.15.161:6831/2889795796","heartbeat_front_addr":"172.21.15.161:6829/2889795796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:42.912205+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:08.410669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:34.299134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:59.093181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:24.116635+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:49.566433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:15.719320+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:40.541017+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:05.178178+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:31.163877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:55.405110+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:19.417156+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6800/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/2620491916":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/4037656392":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/1377116573":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/3062800125":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:6800/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/4145338968":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/825701128":"2024-09-13T03:11:55.244257+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:13.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.380+0000 7f9508dda640 1 -- 172.21.15.6:0/2788983185 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f94f00a2b80 msgr2=0x7f94f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.380+0000 7f9508dda640 1 --2- 172.21.15.6:0/2788983185 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f94f00a2b80 0x7f94f00a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f95041087b0 tx=0x7f94ec005c60 comp rx=0 tx=0).stop 2024-09-12T03:19:13.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.380+0000 7f9508dda640 1 -- 172.21.15.6:0/2788983185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9504101de0 msgr2=0x7f95040744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.380+0000 7f9508dda640 1 --2- 172.21.15.6:0/2788983185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9504101de0 0x7f95040744f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f94fc02ade0 tx=0x7f94fc0319f0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.380+0000 7f9508dda640 1 -- 172.21.15.6:0/2788983185 shutdown_connections 2024-09-12T03:19:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.380+0000 7f9508dda640 1 --2- 172.21.15.6:0/2788983185 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f94f00a2b80 0x7f94f00a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.380+0000 7f9508dda640 1 --2- 172.21.15.6:0/2788983185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9504106f10 0x7f9504074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.380+0000 7f9508dda640 1 --2- 172.21.15.6:0/2788983185 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9504102790 0x7f9504074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.380+0000 7f9508dda640 1 --2- 172.21.15.6:0/2788983185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9504101de0 0x7f95040744f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.380+0000 7f9508dda640 1 -- 172.21.15.6:0/2788983185 >> 172.21.15.6:0/2788983185 conn(0x7f95040fd8c0 msgr2=0x7f95040fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.380+0000 7f9508dda640 1 -- 172.21.15.6:0/2788983185 shutdown_connections 2024-09-12T03:19:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:13.380+0000 7f9508dda640 1 -- 172.21.15.6:0/2788983185 wait complete. 2024-09-12T03:19:13.999 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-12T03:19:14.253 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:14.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:14 smithi006 ceph-mon[30991]: pgmap v277: 97 pgs: 95 active+clean, 2 creating+peering; 578 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-12T03:19:14.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:14 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2788983185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:14.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:14 smithi161 ceph-mon[35485]: pgmap v277: 97 pgs: 95 active+clean, 2 creating+peering; 578 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-12T03:19:14.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:14 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2788983185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:14.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:14 smithi118 ceph-mon[35048]: pgmap v277: 97 pgs: 95 active+clean, 2 creating+peering; 578 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-12T03:19:14.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:14 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2788983185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.711+0000 7f99cb69f640 1 -- 172.21.15.6:0/1346077762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99c410e480 msgr2=0x7f99c4110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.711+0000 7f99cb69f640 1 --2- 172.21.15.6:0/1346077762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99c410e480 0x7f99c4110870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009ca0 tx=0x7f99bc02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.711+0000 7f99cb69f640 1 -- 172.21.15.6:0/1346077762 shutdown_connections 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.711+0000 7f99cb69f640 1 --2- 172.21.15.6:0/1346077762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99c410e480 0x7f99c4110870 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.711+0000 7f99cb69f640 1 --2- 172.21.15.6:0/1346077762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99c4101280 0x7f99c410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.711+0000 7f99cb69f640 1 --2- 172.21.15.6:0/1346077762 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99c41008d0 0x7f99c4100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.711+0000 7f99cb69f640 1 -- 172.21.15.6:0/1346077762 >> 172.21.15.6:0/1346077762 conn(0x7f99c40fc7a0 msgr2=0x7f99c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.711+0000 7f99cb69f640 1 -- 172.21.15.6:0/1346077762 shutdown_connections 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.711+0000 7f99cb69f640 1 -- 172.21.15.6:0/1346077762 wait complete. 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.711+0000 7f99cb69f640 1 Processor -- start 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.711+0000 7f99cb69f640 1 -- start start 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.712+0000 7f99cb69f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99c41008d0 0x7f99c419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.712+0000 7f99cb69f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99c4101280 0x7f99c419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.712+0000 7f99cb69f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99c410e480 0x7f99c419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.712+0000 7f99cb69f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f99c4114160 con 0x7f99c4101280 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.712+0000 7f99cb69f640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f99c4113fe0 con 0x7f99c41008d0 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.712+0000 7f99cb69f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f99c41142e0 con 0x7f99c410e480 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.712+0000 7f99c8c3a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99c4101280 0x7f99c419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.712+0000 7f99c8c3a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99c4101280 0x7f99c419ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40540/0 (socket says 172.21.15.6:40540) 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.712+0000 7f99c8c3a640 1 -- 172.21.15.6:0/3220207634 learned_addr learned my addr 172.21.15.6:0/3220207634 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.712+0000 7f99c943b640 1 --2- 172.21.15.6:0/3220207634 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99c41008d0 0x7f99c419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.712+0000 7f99c9c3c640 1 --2- 172.21.15.6:0/3220207634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99c410e480 0x7f99c419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.712+0000 7f99c8c3a640 1 -- 172.21.15.6:0/3220207634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99c410e480 msgr2=0x7f99c419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.712+0000 7f99c8c3a640 1 --2- 172.21.15.6:0/3220207634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99c410e480 0x7f99c419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.712+0000 7f99c8c3a640 1 -- 172.21.15.6:0/3220207634 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99c41008d0 msgr2=0x7f99c419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.712+0000 7f99c8c3a640 1 --2- 172.21.15.6:0/3220207634 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99c41008d0 0x7f99c419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.712+0000 7f99c8c3a640 1 -- 172.21.15.6:0/3220207634 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c419f670 con 0x7f99c4101280 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.712+0000 7f99c943b640 1 --2- 172.21.15.6:0/3220207634 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99c41008d0 0x7f99c419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.712+0000 7f99c8c3a640 1 --2- 172.21.15.6:0/3220207634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99c4101280 0x7f99c419ac00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f99b400c920 tx=0x7f99b400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.713+0000 7f99ba7fc640 1 -- 172.21.15.6:0/3220207634 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b4007c70 con 0x7f99c4101280 2024-09-12T03:19:14.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.713+0000 7f99ba7fc640 1 -- 172.21.15.6:0/3220207634 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99b4007e10 con 0x7f99c4101280 2024-09-12T03:19:14.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.713+0000 7f99cb69f640 1 -- 172.21.15.6:0/3220207634 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c419f960 con 0x7f99c4101280 2024-09-12T03:19:14.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.713+0000 7f99ba7fc640 1 -- 172.21.15.6:0/3220207634 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b4018890 con 0x7f99c4101280 2024-09-12T03:19:14.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.713+0000 7f99cb69f640 1 -- 172.21.15.6:0/3220207634 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c41a7240 con 0x7f99c4101280 2024-09-12T03:19:14.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.714+0000 7f99cb69f640 1 -- 172.21.15.6:0/3220207634 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c4100cb0 con 0x7f99c4101280 2024-09-12T03:19:14.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.715+0000 7f99ba7fc640 1 -- 172.21.15.6:0/3220207634 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f99b4018a70 con 0x7f99c4101280 2024-09-12T03:19:14.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.715+0000 7f99ba7fc640 1 --2- 172.21.15.6:0/3220207634 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f99b00a2c00 0x7f99b00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.716+0000 7f99c943b640 1 --2- 172.21.15.6:0/3220207634 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f99b00a2c00 0x7f99b00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.716+0000 7f99ba7fc640 1 -- 172.21.15.6:0/3220207634 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f99b40ceea0 con 0x7f99c4101280 2024-09-12T03:19:14.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.716+0000 7f99c943b640 1 --2- 172.21.15.6:0/3220207634 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f99b00a2c00 0x7f99b00a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f99c00058c0 tx=0x7f99c000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:14.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.719+0000 7f99ba7fc640 1 -- 172.21.15.6:0/3220207634 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99b4014030 con 0x7f99c4101280 2024-09-12T03:19:14.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:14.935+0000 7f99cb69f640 1 -- 172.21.15.6:0/3220207634 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f99c4109080 con 0x7f99c4101280 2024-09-12T03:19:15.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:15.337+0000 7f99ba7fc640 1 -- 172.21.15.6:0/3220207634 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f99b409bf50 con 0x7f99c4101280 2024-09-12T03:19:15.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:15.339+0000 7f99cb69f640 1 -- 172.21.15.6:0/3220207634 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f99b00a2c00 msgr2=0x7f99b00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:15.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:15.339+0000 7f99cb69f640 1 --2- 172.21.15.6:0/3220207634 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f99b00a2c00 0x7f99b00a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f99c00058c0 tx=0x7f99c000a2d0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:15.339+0000 7f99cb69f640 1 -- 172.21.15.6:0/3220207634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99c4101280 msgr2=0x7f99c419ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:15.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:15.339+0000 7f99cb69f640 1 --2- 172.21.15.6:0/3220207634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99c4101280 0x7f99c419ac00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f99b400c920 tx=0x7f99b400cde0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:15.340+0000 7f99cb69f640 1 -- 172.21.15.6:0/3220207634 shutdown_connections 2024-09-12T03:19:15.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:15.340+0000 7f99cb69f640 1 --2- 172.21.15.6:0/3220207634 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f99b00a2c00 0x7f99b00a50c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:15.340+0000 7f99cb69f640 1 --2- 172.21.15.6:0/3220207634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99c410e480 0x7f99c419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:15.340+0000 7f99cb69f640 1 --2- 172.21.15.6:0/3220207634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99c4101280 0x7f99c419ac00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:15.340+0000 7f99cb69f640 1 --2- 172.21.15.6:0/3220207634 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99c41008d0 0x7f99c419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:15.340+0000 7f99cb69f640 1 -- 172.21.15.6:0/3220207634 >> 172.21.15.6:0/3220207634 conn(0x7f99c40fc7a0 msgr2=0x7f99c40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:15.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:15.340+0000 7f99cb69f640 1 -- 172.21.15.6:0/3220207634 shutdown_connections 2024-09-12T03:19:15.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:15.340+0000 7f99cb69f640 1 -- 172.21.15.6:0/3220207634 wait complete. 2024-09-12T03:19:15.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:15 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3220207634' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T03:19:15.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:15 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3220207634' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T03:19:15.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:15 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3220207634' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T03:19:15.984 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-12T03:19:15.985 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-12T03:19:16.248 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:16.597 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:16 smithi006 ceph-mon[30991]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-12T03:19:16.597 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:16 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3220207634' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T03:19:16.597 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:16 smithi006 ceph-mon[30991]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:19:16.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:16 smithi161 ceph-mon[35485]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-12T03:19:16.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:16 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3220207634' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T03:19:16.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:16 smithi161 ceph-mon[35485]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:19:16.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.680+0000 7f2e1709c640 1 -- 172.21.15.6:0/208125428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e10100a80 msgr2=0x7f2e10100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.680+0000 7f2e1709c640 1 --2- 172.21.15.6:0/208125428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e10100a80 0x7f2e10100ee0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009cd0 tx=0x7f2e0002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.680+0000 7f2e1709c640 1 -- 172.21.15.6:0/208125428 shutdown_connections 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.680+0000 7f2e1709c640 1 --2- 172.21.15.6:0/208125428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e10101420 0x7f2e1010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.680+0000 7f2e1709c640 1 --2- 172.21.15.6:0/208125428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e10100a80 0x7f2e10100ee0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.680+0000 7f2e1709c640 1 --2- 172.21.15.6:0/208125428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e10106a80 0x7f2e10106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.680+0000 7f2e1709c640 1 -- 172.21.15.6:0/208125428 >> 172.21.15.6:0/208125428 conn(0x7f2e100fc7a0 msgr2=0x7f2e100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.680+0000 7f2e1709c640 1 -- 172.21.15.6:0/208125428 shutdown_connections 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.680+0000 7f2e1709c640 1 -- 172.21.15.6:0/208125428 wait complete. 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.681+0000 7f2e1709c640 1 Processor -- start 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.681+0000 7f2e1709c640 1 -- start start 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.681+0000 7f2e1709c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e10100a80 0x7f2e10197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.681+0000 7f2e1709c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e10101420 0x7f2e101979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:16.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.681+0000 7f2e1709c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e10106a80 0x7f2e1019bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:16.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.681+0000 7f2e1709c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2e10110f70 con 0x7f2e10106a80 2024-09-12T03:19:16.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.681+0000 7f2e1709c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f2e10110df0 con 0x7f2e10101420 2024-09-12T03:19:16.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.681+0000 7f2e1709c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101110f0 con 0x7f2e10100a80 2024-09-12T03:19:16.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.681+0000 7f2e15639640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e10106a80 0x7f2e1019bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:16.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.681+0000 7f2e15639640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e10106a80 0x7f2e1019bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40554/0 (socket says 172.21.15.6:40554) 2024-09-12T03:19:16.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.681+0000 7f2e15639640 1 -- 172.21.15.6:0/1078991734 learned_addr learned my addr 172.21.15.6:0/1078991734 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:16.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.681+0000 7f2e14e38640 1 --2- 172.21.15.6:0/1078991734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e10100a80 0x7f2e10197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:16.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.681+0000 7f2e0ffff640 1 --2- 172.21.15.6:0/1078991734 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e10101420 0x7f2e101979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:16.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.681+0000 7f2e15639640 1 -- 172.21.15.6:0/1078991734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e10100a80 msgr2=0x7f2e10197490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:16.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.681+0000 7f2e15639640 1 --2- 172.21.15.6:0/1078991734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e10100a80 0x7f2e10197490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.681+0000 7f2e15639640 1 -- 172.21.15.6:0/1078991734 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e10101420 msgr2=0x7f2e101979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:16.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.681+0000 7f2e15639640 1 --2- 172.21.15.6:0/1078991734 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e10101420 0x7f2e101979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.681+0000 7f2e15639640 1 -- 172.21.15.6:0/1078991734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1019c440 con 0x7f2e10106a80 2024-09-12T03:19:16.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.682+0000 7f2e14e38640 1 --2- 172.21.15.6:0/1078991734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e10100a80 0x7f2e10197490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:16.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.682+0000 7f2e0ffff640 1 --2- 172.21.15.6:0/1078991734 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e10101420 0x7f2e101979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:16.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.682+0000 7f2e15639640 1 --2- 172.21.15.6:0/1078991734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e10106a80 0x7f2e1019bd60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009fd0 tx=0x7f2df800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:16.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.682+0000 7f2e0dffb640 1 -- 172.21.15.6:0/1078991734 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8011830 con 0x7f2e10106a80 2024-09-12T03:19:16.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.682+0000 7f2e0dffb640 1 -- 172.21.15.6:0/1078991734 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2df8016070 con 0x7f2e10106a80 2024-09-12T03:19:16.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.682+0000 7f2e0dffb640 1 -- 172.21.15.6:0/1078991734 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df800f2e0 con 0x7f2e10106a80 2024-09-12T03:19:16.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.682+0000 7f2e1709c640 1 -- 172.21.15.6:0/1078991734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e1019c730 con 0x7f2e10106a80 2024-09-12T03:19:16.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.682+0000 7f2e1709c640 1 -- 172.21.15.6:0/1078991734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e101a4010 con 0x7f2e10106a80 2024-09-12T03:19:16.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.683+0000 7f2e1709c640 1 -- 172.21.15.6:0/1078991734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e10100f60 con 0x7f2e10106a80 2024-09-12T03:19:16.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.684+0000 7f2e0dffb640 1 -- 172.21.15.6:0/1078991734 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f2df80119d0 con 0x7f2e10106a80 2024-09-12T03:19:16.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.685+0000 7f2e0dffb640 1 --2- 172.21.15.6:0/1078991734 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2dfc0a2c00 0x7f2dfc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:16.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.685+0000 7f2e14e38640 1 --2- 172.21.15.6:0/1078991734 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2dfc0a2c00 0x7f2dfc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:16.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.685+0000 7f2e0dffb640 1 -- 172.21.15.6:0/1078991734 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f2df80ce5d0 con 0x7f2e10106a80 2024-09-12T03:19:16.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.686+0000 7f2e14e38640 1 --2- 172.21.15.6:0/1078991734 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2dfc0a2c00 0x7f2dfc0a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2e08006fd0 tx=0x7f2e08008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:16.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.689+0000 7f2e0dffb640 1 -- 172.21.15.6:0/1078991734 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2df809b680 con 0x7f2e10106a80 2024-09-12T03:19:16.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:16 smithi118 ceph-mon[35048]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-12T03:19:16.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:16 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3220207634' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T03:19:16.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:16 smithi118 ceph-mon[35048]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:19:16.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:16.902+0000 7f2e1709c640 1 -- 172.21.15.6:0/1078991734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f2e1002d7c0 con 0x7f2e10106a80 2024-09-12T03:19:17.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:17.348+0000 7f2e0dffb640 1 -- 172.21.15.6:0/1078991734 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f2df809af80 con 0x7f2e10106a80 2024-09-12T03:19:17.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:17.351+0000 7f2e1709c640 1 -- 172.21.15.6:0/1078991734 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2dfc0a2c00 msgr2=0x7f2dfc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:17.351+0000 7f2e1709c640 1 --2- 172.21.15.6:0/1078991734 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2dfc0a2c00 0x7f2dfc0a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2e08006fd0 tx=0x7f2e08008040 comp rx=0 tx=0).stop 2024-09-12T03:19:17.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:17.351+0000 7f2e1709c640 1 -- 172.21.15.6:0/1078991734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e10106a80 msgr2=0x7f2e1019bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:17.351+0000 7f2e1709c640 1 --2- 172.21.15.6:0/1078991734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e10106a80 0x7f2e1019bd60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009fd0 tx=0x7f2df800b8e0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:17.351+0000 7f2e1709c640 1 -- 172.21.15.6:0/1078991734 shutdown_connections 2024-09-12T03:19:17.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:17.351+0000 7f2e1709c640 1 --2- 172.21.15.6:0/1078991734 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2dfc0a2c00 0x7f2dfc0a50c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:17.351+0000 7f2e1709c640 1 --2- 172.21.15.6:0/1078991734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e10106a80 0x7f2e1019bd60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:17.351+0000 7f2e1709c640 1 --2- 172.21.15.6:0/1078991734 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e10101420 0x7f2e101979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:17.351+0000 7f2e1709c640 1 --2- 172.21.15.6:0/1078991734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e10100a80 0x7f2e10197490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:17.351+0000 7f2e1709c640 1 -- 172.21.15.6:0/1078991734 >> 172.21.15.6:0/1078991734 conn(0x7f2e100fc7a0 msgr2=0x7f2e100fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:17.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:17.351+0000 7f2e1709c640 1 -- 172.21.15.6:0/1078991734 shutdown_connections 2024-09-12T03:19:17.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:17.351+0000 7f2e1709c640 1 -- 172.21.15.6:0/1078991734 wait complete. 2024-09-12T03:19:17.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:17 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1078991734' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T03:19:17.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:17 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1078991734' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T03:19:17.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:17 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1078991734' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T03:19:17.994 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-12T03:19:18.258 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:18.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:18 smithi006 ceph-mon[30991]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-12T03:19:18.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:18 smithi006 ceph-mon[30991]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T03:19:18.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:18 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1078991734' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T03:19:18.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:18 smithi006 ceph-mon[30991]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T03:19:18.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:18 smithi006 ceph-mon[30991]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:19:18.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:18 smithi006 ceph-mon[30991]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-12T03:19:18.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:18 smithi006 ceph-mon[30991]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-12T03:19:18.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:18 smithi161 ceph-mon[35485]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-12T03:19:18.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:18 smithi161 ceph-mon[35485]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T03:19:18.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:18 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1078991734' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T03:19:18.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:18 smithi161 ceph-mon[35485]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T03:19:18.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:18 smithi161 ceph-mon[35485]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:19:18.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:18 smithi161 ceph-mon[35485]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-12T03:19:18.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:18 smithi161 ceph-mon[35485]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-12T03:19:18.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:18 smithi118 ceph-mon[35048]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-12T03:19:18.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:18 smithi118 ceph-mon[35048]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T03:19:18.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:18 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1078991734' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T03:19:18.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:18 smithi118 ceph-mon[35048]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T03:19:18.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:18 smithi118 ceph-mon[35048]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:19:18.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:18 smithi118 ceph-mon[35048]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-12T03:19:18.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:18 smithi118 ceph-mon[35048]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-12T03:19:18.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.739+0000 7fa2f1d32640 1 -- 172.21.15.6:0/1826400589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2ec102570 msgr2=0x7fa2ec1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.739+0000 7fa2f1d32640 1 --2- 172.21.15.6:0/1826400589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2ec102570 0x7fa2ec1029b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009900 tx=0x7fa2e002a480 comp rx=0 tx=0).stop 2024-09-12T03:19:18.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.739+0000 7fa2f1d32640 1 -- 172.21.15.6:0/1826400589 shutdown_connections 2024-09-12T03:19:18.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.739+0000 7fa2f1d32640 1 --2- 172.21.15.6:0/1826400589 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2ec102ef0 0x7fa2ec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.739+0000 7fa2f1d32640 1 --2- 172.21.15.6:0/1826400589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2ec102570 0x7fa2ec1029b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.739+0000 7fa2f1d32640 1 --2- 172.21.15.6:0/1826400589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2ec100e20 0x7fa2ec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.739+0000 7fa2f1d32640 1 -- 172.21.15.6:0/1826400589 >> 172.21.15.6:0/1826400589 conn(0x7fa2ec0fc7d0 msgr2=0x7fa2ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:18.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.739+0000 7fa2f1d32640 1 -- 172.21.15.6:0/1826400589 shutdown_connections 2024-09-12T03:19:18.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.739+0000 7fa2f1d32640 1 -- 172.21.15.6:0/1826400589 wait complete. 2024-09-12T03:19:18.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.739+0000 7fa2f1d32640 1 Processor -- start 2024-09-12T03:19:18.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.740+0000 7fa2f1d32640 1 -- start start 2024-09-12T03:19:18.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.740+0000 7fa2f1d32640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2ec100e20 0x7fa2ec19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:18.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.740+0000 7fa2f1d32640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2ec102570 0x7fa2ec19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:18.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.740+0000 7fa2f1d32640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2ec102ef0 0x7fa2ec19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:18.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.740+0000 7fa2f1d32640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec110f50 con 0x7fa2ec102ef0 2024-09-12T03:19:18.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.740+0000 7fa2f1d32640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec110dd0 con 0x7fa2ec100e20 2024-09-12T03:19:18.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.740+0000 7fa2f1d32640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec1110d0 con 0x7fa2ec102570 2024-09-12T03:19:18.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.740+0000 7fa2ebfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2ec102ef0 0x7fa2ec19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:18.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.740+0000 7fa2ebfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2ec102ef0 0x7fa2ec19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40576/0 (socket says 172.21.15.6:40576) 2024-09-12T03:19:18.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.740+0000 7fa2ebfff640 1 -- 172.21.15.6:0/422807144 learned_addr learned my addr 172.21.15.6:0/422807144 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:18.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.740+0000 7fa2eaffd640 1 --2- 172.21.15.6:0/422807144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2ec102570 0x7fa2ec19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:18.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.740+0000 7fa2eb7fe640 1 --2- 172.21.15.6:0/422807144 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2ec100e20 0x7fa2ec19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:18.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.740+0000 7fa2ebfff640 1 -- 172.21.15.6:0/422807144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2ec102570 msgr2=0x7fa2ec19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.740+0000 7fa2ebfff640 1 --2- 172.21.15.6:0/422807144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2ec102570 0x7fa2ec19abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.740+0000 7fa2ebfff640 1 -- 172.21.15.6:0/422807144 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2ec100e20 msgr2=0x7fa2ec19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.740+0000 7fa2ebfff640 1 --2- 172.21.15.6:0/422807144 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2ec100e20 0x7fa2ec19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.740+0000 7fa2ebfff640 1 -- 172.21.15.6:0/422807144 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2ec19f650 con 0x7fa2ec102ef0 2024-09-12T03:19:18.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.740+0000 7fa2eb7fe640 1 --2- 172.21.15.6:0/422807144 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2ec100e20 0x7fa2ec19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:18.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.740+0000 7fa2ebfff640 1 --2- 172.21.15.6:0/422807144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2ec102ef0 0x7fa2ec19ef70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400eaa0 tx=0x7fa2e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:18.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.741+0000 7fa2e8ff9640 1 -- 172.21.15.6:0/422807144 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e400cdb0 con 0x7fa2ec102ef0 2024-09-12T03:19:18.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.741+0000 7fa2f1d32640 1 -- 172.21.15.6:0/422807144 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2ec19f940 con 0x7fa2ec102ef0 2024-09-12T03:19:18.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.741+0000 7fa2e8ff9640 1 -- 172.21.15.6:0/422807144 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2e4005020 con 0x7fa2ec102ef0 2024-09-12T03:19:18.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.741+0000 7fa2e8ff9640 1 -- 172.21.15.6:0/422807144 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e40116b0 con 0x7fa2ec102ef0 2024-09-12T03:19:18.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.741+0000 7fa2f1d32640 1 -- 172.21.15.6:0/422807144 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2ec1a7220 con 0x7fa2ec102ef0 2024-09-12T03:19:18.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.742+0000 7fa2f1d32640 1 -- 172.21.15.6:0/422807144 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2ec101200 con 0x7fa2ec102ef0 2024-09-12T03:19:18.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.743+0000 7fa2e8ff9640 1 -- 172.21.15.6:0/422807144 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fa2e4005560 con 0x7fa2ec102ef0 2024-09-12T03:19:18.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.745+0000 7fa2e8ff9640 1 --2- 172.21.15.6:0/422807144 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa2d40a2b80 0x7fa2d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:18.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.745+0000 7fa2eb7fe640 1 --2- 172.21.15.6:0/422807144 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa2d40a2b80 0x7fa2d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:18.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.745+0000 7fa2e8ff9640 1 -- 172.21.15.6:0/422807144 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fa2e40cf220 con 0x7fa2ec102ef0 2024-09-12T03:19:18.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.745+0000 7fa2eb7fe640 1 --2- 172.21.15.6:0/422807144 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa2d40a2b80 0x7fa2d40a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00baa0 tx=0x7fa2dc005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:18.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.748+0000 7fa2e8ff9640 1 -- 172.21.15.6:0/422807144 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa2e4016030 con 0x7fa2ec102ef0 2024-09-12T03:19:18.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:18.958+0000 7fa2f1d32640 1 -- 172.21.15.6:0/422807144 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fa2ec1a73f0 con 0x7fa2ec102ef0 2024-09-12T03:19:19.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:19.364+0000 7fa2e8ff9640 1 -- 172.21.15.6:0/422807144 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7fa2e409c2d0 con 0x7fa2ec102ef0 2024-09-12T03:19:19.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:19.367+0000 7fa2f1d32640 1 -- 172.21.15.6:0/422807144 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa2d40a2b80 msgr2=0x7fa2d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:19.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:19.367+0000 7fa2f1d32640 1 --2- 172.21.15.6:0/422807144 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa2d40a2b80 0x7fa2d40a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00baa0 tx=0x7fa2dc005d90 comp rx=0 tx=0).stop 2024-09-12T03:19:19.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:19.367+0000 7fa2f1d32640 1 -- 172.21.15.6:0/422807144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2ec102ef0 msgr2=0x7fa2ec19ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:19.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:19.367+0000 7fa2f1d32640 1 --2- 172.21.15.6:0/422807144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2ec102ef0 0x7fa2ec19ef70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400eaa0 tx=0x7fa2e400ef60 comp rx=0 tx=0).stop 2024-09-12T03:19:19.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:19.367+0000 7fa2f1d32640 1 -- 172.21.15.6:0/422807144 shutdown_connections 2024-09-12T03:19:19.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:19.367+0000 7fa2f1d32640 1 --2- 172.21.15.6:0/422807144 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa2d40a2b80 0x7fa2d40a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:19.367+0000 7fa2f1d32640 1 --2- 172.21.15.6:0/422807144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2ec102ef0 0x7fa2ec19ef70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:19.367+0000 7fa2f1d32640 1 --2- 172.21.15.6:0/422807144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2ec102570 0x7fa2ec19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:19.367+0000 7fa2f1d32640 1 --2- 172.21.15.6:0/422807144 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2ec100e20 0x7fa2ec19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:19.367+0000 7fa2f1d32640 1 -- 172.21.15.6:0/422807144 >> 172.21.15.6:0/422807144 conn(0x7fa2ec0fc7d0 msgr2=0x7fa2ec069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:19.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:19.368+0000 7fa2f1d32640 1 -- 172.21.15.6:0/422807144 shutdown_connections 2024-09-12T03:19:19.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:19.368+0000 7fa2f1d32640 1 -- 172.21.15.6:0/422807144 wait complete. 2024-09-12T03:19:19.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:19 smithi006 ceph-mon[30991]: mds.? [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] up:active 2024-09-12T03:19:19.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:19 smithi006 ceph-mon[30991]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T03:19:19.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:19 smithi006 ceph-mon[30991]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T03:19:19.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:19 smithi006 ceph-mon[30991]: Cluster is now healthy 2024-09-12T03:19:19.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:19 smithi006 ceph-mon[30991]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-12T03:19:19.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:19 smithi006 ceph-mon[30991]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-12T03:19:19.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:19 smithi006 ceph-mon[30991]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-12T03:19:19.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:19 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/422807144' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T03:19:19.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-12T03:19:19.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-12T03:19:19.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-09-12T03:19:19.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-12T03:19:19.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-09-12T03:19:19.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-09-12T03:19:19.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-09-12T03:19:19.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:19 smithi161 ceph-mon[35485]: mds.? [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] up:active 2024-09-12T03:19:19.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:19 smithi161 ceph-mon[35485]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T03:19:19.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:19 smithi161 ceph-mon[35485]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T03:19:19.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:19 smithi161 ceph-mon[35485]: Cluster is now healthy 2024-09-12T03:19:19.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:19 smithi161 ceph-mon[35485]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-12T03:19:19.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:19 smithi161 ceph-mon[35485]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-12T03:19:19.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:19 smithi161 ceph-mon[35485]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-12T03:19:19.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:19 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/422807144' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T03:19:19.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-12T03:19:19.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-12T03:19:19.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-09-12T03:19:19.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-12T03:19:19.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-09-12T03:19:19.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-09-12T03:19:19.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-09-12T03:19:19.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:19 smithi118 ceph-mon[35048]: mds.? [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] up:active 2024-09-12T03:19:19.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:19 smithi118 ceph-mon[35048]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T03:19:19.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:19 smithi118 ceph-mon[35048]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T03:19:19.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:19 smithi118 ceph-mon[35048]: Cluster is now healthy 2024-09-12T03:19:19.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:19 smithi118 ceph-mon[35048]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-12T03:19:19.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:19 smithi118 ceph-mon[35048]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-12T03:19:19.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:19 smithi118 ceph-mon[35048]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-12T03:19:19.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:19 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/422807144' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T03:19:19.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-12T03:19:19.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-12T03:19:19.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-09-12T03:19:19.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-12T03:19:19.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-09-12T03:19:19.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-09-12T03:19:19.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-09-12T03:19:20.010 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-12T03:19:20.275 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:20.693 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:20 smithi006 ceph-mon[30991]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 7 op/s 2024-09-12T03:19:20.694 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:20 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/422807144' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T03:19:20.694 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-12T03:19:20.694 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-12T03:19:20.694 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-12T03:19:20.694 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-12T03:19:20.694 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-09-12T03:19:20.694 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-09-12T03:19:20.694 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:20 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-12T03:19:20.694 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:20 smithi006 ceph-mon[30991]: mds.? [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] up:active 2024-09-12T03:19:20.694 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:20 smithi006 ceph-mon[30991]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-12T03:19:20.694 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:20 smithi006 ceph-mon[30991]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T03:19:20.694 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:20 smithi006 ceph-mon[30991]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:19:20.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.696+0000 7fbbe6d8c640 1 -- 172.21.15.6:0/3381257135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbe0102ef0 msgr2=0x7fbbe010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:20.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.696+0000 7fbbe6d8c640 1 --2- 172.21.15.6:0/3381257135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbe0102ef0 0x7fbbe010da40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009ca0 tx=0x7fbbd802a950 comp rx=0 tx=0).stop 2024-09-12T03:19:20.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.696+0000 7fbbe6d8c640 1 -- 172.21.15.6:0/3381257135 shutdown_connections 2024-09-12T03:19:20.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.696+0000 7fbbe6d8c640 1 --2- 172.21.15.6:0/3381257135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbe0102ef0 0x7fbbe010da40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.696+0000 7fbbe6d8c640 1 --2- 172.21.15.6:0/3381257135 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbe0102570 0x7fbbe01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.696+0000 7fbbe6d8c640 1 --2- 172.21.15.6:0/3381257135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe0100e20 0x7fbbe0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.696+0000 7fbbe6d8c640 1 -- 172.21.15.6:0/3381257135 >> 172.21.15.6:0/3381257135 conn(0x7fbbe00fc7d0 msgr2=0x7fbbe00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:20.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.696+0000 7fbbe6d8c640 1 -- 172.21.15.6:0/3381257135 shutdown_connections 2024-09-12T03:19:20.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.696+0000 7fbbe6d8c640 1 -- 172.21.15.6:0/3381257135 wait complete. 2024-09-12T03:19:20.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.697+0000 7fbbe6d8c640 1 Processor -- start 2024-09-12T03:19:20.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.697+0000 7fbbe6d8c640 1 -- start start 2024-09-12T03:19:20.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.697+0000 7fbbe6d8c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbe0100e20 0x7fbbe01993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:20.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.697+0000 7fbbe6d8c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbe0102570 0x7fbbe01998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:20.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.697+0000 7fbbe6d8c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe0102ef0 0x7fbbe01a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:20.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.697+0000 7fbbe6d8c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe0110fa0 con 0x7fbbe0100e20 2024-09-12T03:19:20.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.697+0000 7fbbe6d8c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe0110e20 con 0x7fbbe0102570 2024-09-12T03:19:20.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.697+0000 7fbbe6d8c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe0111120 con 0x7fbbe0102ef0 2024-09-12T03:19:20.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.697+0000 7fbbe4b28640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbe0100e20 0x7fbbe01993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:20.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.697+0000 7fbbe4b28640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbe0100e20 0x7fbbe01993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51488/0 (socket says 172.21.15.6:51488) 2024-09-12T03:19:20.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.698+0000 7fbbe4b28640 1 -- 172.21.15.6:0/820978774 learned_addr learned my addr 172.21.15.6:0/820978774 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.698+0000 7fbbdffff640 1 --2- 172.21.15.6:0/820978774 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbe0102570 0x7fbbe01998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.698+0000 7fbbe5329640 1 --2- 172.21.15.6:0/820978774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe0102ef0 0x7fbbe01a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.698+0000 7fbbe4b28640 1 -- 172.21.15.6:0/820978774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe0102ef0 msgr2=0x7fbbe01a4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.698+0000 7fbbe4b28640 1 --2- 172.21.15.6:0/820978774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe0102ef0 0x7fbbe01a4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.698+0000 7fbbe4b28640 1 -- 172.21.15.6:0/820978774 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbe0102570 msgr2=0x7fbbe01998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.698+0000 7fbbe4b28640 1 --2- 172.21.15.6:0/820978774 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbe0102570 0x7fbbe01998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.698+0000 7fbbe4b28640 1 -- 172.21.15.6:0/820978774 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe01a4970 con 0x7fbbe0100e20 2024-09-12T03:19:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.698+0000 7fbbdffff640 1 --2- 172.21.15.6:0/820978774 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbe0102570 0x7fbbe01998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.698+0000 7fbbe4b28640 1 --2- 172.21.15.6:0/820978774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbe0100e20 0x7fbbe01993b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400b4a0 tx=0x7fbbd400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.698+0000 7fbbddffb640 1 -- 172.21.15.6:0/820978774 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4014070 con 0x7fbbe0100e20 2024-09-12T03:19:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.698+0000 7fbbe6d8c640 1 -- 172.21.15.6:0/820978774 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe01a4b40 con 0x7fbbe0100e20 2024-09-12T03:19:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.698+0000 7fbbddffb640 1 -- 172.21.15.6:0/820978774 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd4010af0 con 0x7fbbe0100e20 2024-09-12T03:19:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.698+0000 7fbbddffb640 1 -- 172.21.15.6:0/820978774 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4017d10 con 0x7fbbe0100e20 2024-09-12T03:19:20.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.698+0000 7fbbe6d8c640 1 -- 172.21.15.6:0/820978774 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe01a5080 con 0x7fbbe0100e20 2024-09-12T03:19:20.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.699+0000 7fbbe6d8c640 1 -- 172.21.15.6:0/820978774 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe0101200 con 0x7fbbe0100e20 2024-09-12T03:19:20.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.700+0000 7fbbddffb640 1 -- 172.21.15.6:0/820978774 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fbbd4017070 con 0x7fbbe0100e20 2024-09-12T03:19:20.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.701+0000 7fbbddffb640 1 --2- 172.21.15.6:0/820978774 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fbbd00a2b80 0x7fbbd00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:20.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.701+0000 7fbbdffff640 1 --2- 172.21.15.6:0/820978774 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fbbd00a2b80 0x7fbbd00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:20.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.701+0000 7fbbddffb640 1 -- 172.21.15.6:0/820978774 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fbbd40ce430 con 0x7fbbe0100e20 2024-09-12T03:19:20.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.702+0000 7fbbdffff640 1 --2- 172.21.15.6:0/820978774 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fbbd00a2b80 0x7fbbd00a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbbe006aac0 tx=0x7fbbcc005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:20.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.705+0000 7fbbddffb640 1 -- 172.21.15.6:0/820978774 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbbd409b3c0 con 0x7fbbe0100e20 2024-09-12T03:19:20.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:20 smithi118 ceph-mon[35048]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 7 op/s 2024-09-12T03:19:20.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:20 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/422807144' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T03:19:20.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-12T03:19:20.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-12T03:19:20.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-12T03:19:20.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-12T03:19:20.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-09-12T03:19:20.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-09-12T03:19:20.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:20 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-12T03:19:20.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:20 smithi118 ceph-mon[35048]: mds.? [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] up:active 2024-09-12T03:19:20.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:20 smithi118 ceph-mon[35048]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-12T03:19:20.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:20 smithi118 ceph-mon[35048]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T03:19:20.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:20 smithi118 ceph-mon[35048]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:19:20.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:20 smithi161 ceph-mon[35485]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 7 op/s 2024-09-12T03:19:20.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:20 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/422807144' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T03:19:20.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-12T03:19:20.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-12T03:19:20.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-12T03:19:20.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-12T03:19:20.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-09-12T03:19:20.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-09-12T03:19:20.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:20 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-12T03:19:20.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:20 smithi161 ceph-mon[35485]: mds.? [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] up:active 2024-09-12T03:19:20.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:20 smithi161 ceph-mon[35485]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-12T03:19:20.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:20 smithi161 ceph-mon[35485]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T03:19:20.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:20 smithi161 ceph-mon[35485]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:19:20.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:20.917+0000 7fbbe6d8c640 1 -- 172.21.15.6:0/820978774 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fbbe01a52b0 con 0x7fbbe0100e20 2024-09-12T03:19:21.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:21.379+0000 7fbbddffb640 1 -- 172.21.15.6:0/820978774 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7fbbd409acc0 con 0x7fbbe0100e20 2024-09-12T03:19:21.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:21.382+0000 7fbbe6d8c640 1 -- 172.21.15.6:0/820978774 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fbbd00a2b80 msgr2=0x7fbbd00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:21.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:21.382+0000 7fbbe6d8c640 1 --2- 172.21.15.6:0/820978774 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fbbd00a2b80 0x7fbbd00a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbbe006aac0 tx=0x7fbbcc005ec0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:21.382+0000 7fbbe6d8c640 1 -- 172.21.15.6:0/820978774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbe0100e20 msgr2=0x7fbbe01993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:21.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:21.382+0000 7fbbe6d8c640 1 --2- 172.21.15.6:0/820978774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbe0100e20 0x7fbbe01993b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400b4a0 tx=0x7fbbd400b960 comp rx=0 tx=0).stop 2024-09-12T03:19:21.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:21.382+0000 7fbbe6d8c640 1 -- 172.21.15.6:0/820978774 shutdown_connections 2024-09-12T03:19:21.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:21.382+0000 7fbbe6d8c640 1 --2- 172.21.15.6:0/820978774 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fbbd00a2b80 0x7fbbd00a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:21.382+0000 7fbbe6d8c640 1 --2- 172.21.15.6:0/820978774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe0102ef0 0x7fbbe01a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:21.382+0000 7fbbe6d8c640 1 --2- 172.21.15.6:0/820978774 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbe0102570 0x7fbbe01998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:21.382+0000 7fbbe6d8c640 1 --2- 172.21.15.6:0/820978774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbe0100e20 0x7fbbe01993b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:21.382+0000 7fbbe6d8c640 1 -- 172.21.15.6:0/820978774 >> 172.21.15.6:0/820978774 conn(0x7fbbe00fc7d0 msgr2=0x7fbbe0106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:21.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:21.383+0000 7fbbe6d8c640 1 -- 172.21.15.6:0/820978774 shutdown_connections 2024-09-12T03:19:21.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:21.383+0000 7fbbe6d8c640 1 -- 172.21.15.6:0/820978774 wait complete. 2024-09-12T03:19:21.680 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:21 smithi006 ceph-mon[30991]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T03:19:21.680 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:21 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/820978774' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T03:19:21.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:21 smithi118 ceph-mon[35048]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T03:19:21.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:21 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/820978774' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T03:19:21.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:21 smithi161 ceph-mon[35485]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T03:19:21.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:21 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/820978774' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T03:19:22.027 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-12T03:19:22.027 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-12T03:19:22.305 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:22.668 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:22 smithi006 ceph-mon[30991]: pgmap v283: 97 pgs: 97 active+clean; 580 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 895 B/s wr, 6 op/s 2024-09-12T03:19:22.668 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:22 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/820978774' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T03:19:22.668 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:22 smithi006 ceph-mon[30991]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:19:22.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:22 smithi118 ceph-mon[35048]: pgmap v283: 97 pgs: 97 active+clean; 580 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 895 B/s wr, 6 op/s 2024-09-12T03:19:22.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:22 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/820978774' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T03:19:22.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:22 smithi118 ceph-mon[35048]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:19:22.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.755+0000 7f6a1cc2f640 1 -- 172.21.15.6:0/4291426788 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a18100a80 msgr2=0x7f6a18100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.755+0000 7f6a1cc2f640 1 --2- 172.21.15.6:0/4291426788 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a18100a80 0x7f6a18100ee0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009930 tx=0x7f6a0c02a420 comp rx=0 tx=0).stop 2024-09-12T03:19:22.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.755+0000 7f6a1cc2f640 1 -- 172.21.15.6:0/4291426788 shutdown_connections 2024-09-12T03:19:22.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.755+0000 7f6a1cc2f640 1 --2- 172.21.15.6:0/4291426788 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a18101420 0x7f6a1810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.755+0000 7f6a1cc2f640 1 --2- 172.21.15.6:0/4291426788 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a18100a80 0x7f6a18100ee0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.755+0000 7f6a1cc2f640 1 --2- 172.21.15.6:0/4291426788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a18106a80 0x7f6a18106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.755+0000 7f6a1cc2f640 1 -- 172.21.15.6:0/4291426788 >> 172.21.15.6:0/4291426788 conn(0x7f6a180fc7a0 msgr2=0x7f6a180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:22.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.755+0000 7f6a1cc2f640 1 -- 172.21.15.6:0/4291426788 shutdown_connections 2024-09-12T03:19:22.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.755+0000 7f6a1cc2f640 1 -- 172.21.15.6:0/4291426788 wait complete. 2024-09-12T03:19:22.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.756+0000 7f6a1cc2f640 1 Processor -- start 2024-09-12T03:19:22.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.756+0000 7f6a1cc2f640 1 -- start start 2024-09-12T03:19:22.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.756+0000 7f6a1cc2f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a18100a80 0x7f6a1819d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.756+0000 7f6a1cc2f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a18101420 0x7f6a1819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.756+0000 7f6a1cc2f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a18106a80 0x7f6a18198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.756+0000 7f6a1cc2f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6a18110ea0 con 0x7f6a18101420 2024-09-12T03:19:22.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.756+0000 7f6a1cc2f640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f6a18110d20 con 0x7f6a18106a80 2024-09-12T03:19:22.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.756+0000 7f6a1cc2f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6a18111020 con 0x7f6a18100a80 2024-09-12T03:19:22.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.756+0000 7f6a15d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a18101420 0x7f6a1819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.757+0000 7f6a15d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a18101420 0x7f6a1819d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51506/0 (socket says 172.21.15.6:51506) 2024-09-12T03:19:22.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.757+0000 7f6a15d9b640 1 -- 172.21.15.6:0/1575074893 learned_addr learned my addr 172.21.15.6:0/1575074893 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:22.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.757+0000 7f6a16d9d640 1 --2- 172.21.15.6:0/1575074893 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a18106a80 0x7f6a18198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.757+0000 7f6a15d9b640 1 -- 172.21.15.6:0/1575074893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a18100a80 msgr2=0x7f6a1819d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.757+0000 7f6a15d9b640 1 --2- 172.21.15.6:0/1575074893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a18100a80 0x7f6a1819d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.757+0000 7f6a15d9b640 1 -- 172.21.15.6:0/1575074893 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a18106a80 msgr2=0x7f6a18198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.757+0000 7f6a15d9b640 1 --2- 172.21.15.6:0/1575074893 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a18106a80 0x7f6a18198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.757+0000 7f6a15d9b640 1 -- 172.21.15.6:0/1575074893 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a18198d00 con 0x7f6a18101420 2024-09-12T03:19:22.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.757+0000 7f6a15d9b640 1 --2- 172.21.15.6:0/1575074893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a18101420 0x7f6a1819d5a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009a50 tx=0x7f6a0c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:22.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.757+0000 7f6a077fe640 1 -- 172.21.15.6:0/1575074893 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c03f030 con 0x7f6a18101420 2024-09-12T03:19:22.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.757+0000 7f6a077fe640 1 -- 172.21.15.6:0/1575074893 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a0c003ce0 con 0x7f6a18101420 2024-09-12T03:19:22.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.757+0000 7f6a1cc2f640 1 -- 172.21.15.6:0/1575074893 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a18198f90 con 0x7f6a18101420 2024-09-12T03:19:22.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.757+0000 7f6a077fe640 1 -- 172.21.15.6:0/1575074893 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c035070 con 0x7f6a18101420 2024-09-12T03:19:22.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.757+0000 7f6a1cc2f640 1 -- 172.21.15.6:0/1575074893 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a181a4f90 con 0x7f6a18101420 2024-09-12T03:19:22.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.758+0000 7f6a1cc2f640 1 -- 172.21.15.6:0/1575074893 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a181021c0 con 0x7f6a18101420 2024-09-12T03:19:22.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.759+0000 7f6a077fe640 1 -- 172.21.15.6:0/1575074893 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f6a0c042030 con 0x7f6a18101420 2024-09-12T03:19:22.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.760+0000 7f6a077fe640 1 --2- 172.21.15.6:0/1575074893 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f6a000a2b80 0x7f6a000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.760+0000 7f6a1659c640 1 --2- 172.21.15.6:0/1575074893 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f6a000a2b80 0x7f6a000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.760+0000 7f6a077fe640 1 -- 172.21.15.6:0/1575074893 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f6a0c0efa70 con 0x7f6a18101420 2024-09-12T03:19:22.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.760+0000 7f6a1659c640 1 --2- 172.21.15.6:0/1575074893 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f6a000a2b80 0x7f6a000a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6a1806b850 tx=0x7f6a08006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:22.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.763+0000 7f6a077fe640 1 -- 172.21.15.6:0/1575074893 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6a0c0bca00 con 0x7f6a18101420 2024-09-12T03:19:22.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:22 smithi161 ceph-mon[35485]: pgmap v283: 97 pgs: 97 active+clean; 580 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 895 B/s wr, 6 op/s 2024-09-12T03:19:22.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:22 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/820978774' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T03:19:22.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:22 smithi161 ceph-mon[35485]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:19:22.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.963+0000 7f6a1cc2f640 1 -- 172.21.15.6:0/1575074893 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6a18104ca0 con 0x7f6a000a2b80 2024-09-12T03:19:22.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.987+0000 7f6a077fe640 1 -- 172.21.15.6:0/1575074893 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6a18104ca0 con 0x7f6a000a2b80 2024-09-12T03:19:22.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.989+0000 7f6a1cc2f640 1 -- 172.21.15.6:0/1575074893 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f6a000a2b80 msgr2=0x7f6a000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.989+0000 7f6a1cc2f640 1 --2- 172.21.15.6:0/1575074893 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f6a000a2b80 0x7f6a000a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6a1806b850 tx=0x7f6a08006c90 comp rx=0 tx=0).stop 2024-09-12T03:19:22.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.989+0000 7f6a1cc2f640 1 -- 172.21.15.6:0/1575074893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a18101420 msgr2=0x7f6a1819d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.989+0000 7f6a1cc2f640 1 --2- 172.21.15.6:0/1575074893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a18101420 0x7f6a1819d5a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009a50 tx=0x7f6a0c004270 comp rx=0 tx=0).stop 2024-09-12T03:19:22.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.989+0000 7f6a1cc2f640 1 -- 172.21.15.6:0/1575074893 shutdown_connections 2024-09-12T03:19:22.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.989+0000 7f6a1cc2f640 1 --2- 172.21.15.6:0/1575074893 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f6a000a2b80 0x7f6a000a5040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.989+0000 7f6a1cc2f640 1 --2- 172.21.15.6:0/1575074893 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a18106a80 0x7f6a18198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.989+0000 7f6a1cc2f640 1 --2- 172.21.15.6:0/1575074893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a18101420 0x7f6a1819d5a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.989+0000 7f6a1cc2f640 1 --2- 172.21.15.6:0/1575074893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a18100a80 0x7f6a1819d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.989+0000 7f6a1cc2f640 1 -- 172.21.15.6:0/1575074893 >> 172.21.15.6:0/1575074893 conn(0x7f6a180fc7a0 msgr2=0x7f6a18104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:22.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.989+0000 7f6a1cc2f640 1 -- 172.21.15.6:0/1575074893 shutdown_connections 2024-09-12T03:19:22.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:22.990+0000 7f6a1cc2f640 1 -- 172.21.15.6:0/1575074893 wait complete. 2024-09-12T03:19:23.295 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:19:22 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:19:22.968+0000 7f8aa508c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:19:23.295 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:19:22 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:19:22.968+0000 7f8aa508c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:19:23.295 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:19:22 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:19:22.968+0000 7f8aa508c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:19:23.295 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:19:22 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:19:22.968+0000 7f8aa508c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:19:23.295 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:19:22 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:19:22.968+0000 7f8aa508c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:19:23.579 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:23 smithi006 ceph-mon[30991]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:23.607 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-12T03:19:23.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:23 smithi118 ceph-mon[35048]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:23.862 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:23.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:23 smithi161 ceph-mon[35485]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:24.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.327+0000 7efd2d717640 1 -- 172.21.15.6:0/890990176 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd281082a0 msgr2=0x7efd2810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:24.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.327+0000 7efd2d717640 1 --2- 172.21.15.6:0/890990176 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd281082a0 0x7efd2810ec40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efd18009930 tx=0x7efd1802a420 comp rx=0 tx=0).stop 2024-09-12T03:19:24.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.327+0000 7efd2d717640 1 -- 172.21.15.6:0/890990176 shutdown_connections 2024-09-12T03:19:24.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.327+0000 7efd2d717640 1 --2- 172.21.15.6:0/890990176 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd281082a0 0x7efd2810ec40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.327+0000 7efd2d717640 1 --2- 172.21.15.6:0/890990176 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efd281036e0 0x7efd28107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.327+0000 7efd2d717640 1 --2- 172.21.15.6:0/890990176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd28102d30 0x7efd28103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.327+0000 7efd2d717640 1 -- 172.21.15.6:0/890990176 >> 172.21.15.6:0/890990176 conn(0x7efd280fc7d0 msgr2=0x7efd280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:24.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.327+0000 7efd2d717640 1 -- 172.21.15.6:0/890990176 shutdown_connections 2024-09-12T03:19:24.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.327+0000 7efd2d717640 1 -- 172.21.15.6:0/890990176 wait complete. 2024-09-12T03:19:24.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.327+0000 7efd2d717640 1 Processor -- start 2024-09-12T03:19:24.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.328+0000 7efd2d717640 1 -- start start 2024-09-12T03:19:24.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.328+0000 7efd2d717640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd28102d30 0x7efd281a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.328+0000 7efd2d717640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efd281036e0 0x7efd281a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.328+0000 7efd2d717640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd281082a0 0x7efd2819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.328+0000 7efd2d717640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7efd281120a0 con 0x7efd281082a0 2024-09-12T03:19:24.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.328+0000 7efd2d717640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7efd28111f20 con 0x7efd281036e0 2024-09-12T03:19:24.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.328+0000 7efd2d717640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efd28112220 con 0x7efd28102d30 2024-09-12T03:19:24.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.328+0000 7efd277fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd281082a0 0x7efd2819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.328+0000 7efd277fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd281082a0 0x7efd2819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51530/0 (socket says 172.21.15.6:51530) 2024-09-12T03:19:24.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.328+0000 7efd277fe640 1 -- 172.21.15.6:0/972887714 learned_addr learned my addr 172.21.15.6:0/972887714 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:24.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.328+0000 7efd26ffd640 1 --2- 172.21.15.6:0/972887714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd28102d30 0x7efd281a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.328+0000 7efd267fc640 1 --2- 172.21.15.6:0/972887714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efd281036e0 0x7efd281a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.328+0000 7efd277fe640 1 -- 172.21.15.6:0/972887714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd28102d30 msgr2=0x7efd281a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:24.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.328+0000 7efd277fe640 1 --2- 172.21.15.6:0/972887714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd28102d30 0x7efd281a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.328+0000 7efd277fe640 1 -- 172.21.15.6:0/972887714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efd281036e0 msgr2=0x7efd281a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:24.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.328+0000 7efd277fe640 1 --2- 172.21.15.6:0/972887714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efd281036e0 0x7efd281a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.328+0000 7efd277fe640 1 -- 172.21.15.6:0/972887714 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2819acc0 con 0x7efd281082a0 2024-09-12T03:19:24.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.328+0000 7efd277fe640 1 --2- 172.21.15.6:0/972887714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd281082a0 0x7efd2819a590 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efd18005ab0 tx=0x7efd180040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:24.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.329+0000 7efd1ffff640 1 -- 172.21.15.6:0/972887714 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1803f030 con 0x7efd281082a0 2024-09-12T03:19:24.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.329+0000 7efd1ffff640 1 -- 172.21.15.6:0/972887714 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd180044c0 con 0x7efd281082a0 2024-09-12T03:19:24.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.329+0000 7efd2d717640 1 -- 172.21.15.6:0/972887714 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd2819af50 con 0x7efd281082a0 2024-09-12T03:19:24.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.329+0000 7efd1ffff640 1 -- 172.21.15.6:0/972887714 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18035070 con 0x7efd281082a0 2024-09-12T03:19:24.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.329+0000 7efd2d717640 1 -- 172.21.15.6:0/972887714 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd281a72a0 con 0x7efd281082a0 2024-09-12T03:19:24.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.330+0000 7efd2d717640 1 -- 172.21.15.6:0/972887714 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd28103110 con 0x7efd281082a0 2024-09-12T03:19:24.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.331+0000 7efd1ffff640 1 -- 172.21.15.6:0/972887714 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7efd18042030 con 0x7efd281082a0 2024-09-12T03:19:24.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.332+0000 7efd1ffff640 1 --2- 172.21.15.6:0/972887714 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efd140a2c10 0x7efd140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.332+0000 7efd26ffd640 1 --2- 172.21.15.6:0/972887714 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efd140a2c10 0x7efd140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.332+0000 7efd1ffff640 1 -- 172.21.15.6:0/972887714 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7efd180ef9f0 con 0x7efd281082a0 2024-09-12T03:19:24.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.332+0000 7efd26ffd640 1 --2- 172.21.15.6:0/972887714 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efd140a2c10 0x7efd140a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efd20005fd0 tx=0x7efd20009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:24.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.335+0000 7efd1ffff640 1 -- 172.21.15.6:0/972887714 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efd180bc820 con 0x7efd281082a0 2024-09-12T03:19:24.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.535+0000 7efd2d717640 1 -- 172.21.15.6:0/972887714 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7efd28062630 con 0x7efd140a2c10 2024-09-12T03:19:24.536 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:24 smithi006 ceph-mon[30991]: pgmap v284: 97 pgs: 97 active+clean; 588 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 4.0 KiB/s wr, 14 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-09-12T03:19:24.536 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:24 smithi006 ceph-mon[30991]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T03:19:24.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.562+0000 7efd1ffff640 1 -- 172.21.15.6:0/972887714 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efd28062630 con 0x7efd140a2c10 2024-09-12T03:19:24.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.564+0000 7efd2d717640 1 -- 172.21.15.6:0/972887714 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efd140a2c10 msgr2=0x7efd140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:24.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.564+0000 7efd2d717640 1 --2- 172.21.15.6:0/972887714 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efd140a2c10 0x7efd140a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efd20005fd0 tx=0x7efd20009500 comp rx=0 tx=0).stop 2024-09-12T03:19:24.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.564+0000 7efd2d717640 1 -- 172.21.15.6:0/972887714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd281082a0 msgr2=0x7efd2819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:24.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.564+0000 7efd2d717640 1 --2- 172.21.15.6:0/972887714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd281082a0 0x7efd2819a590 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efd18005ab0 tx=0x7efd180040c0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.564+0000 7efd2d717640 1 -- 172.21.15.6:0/972887714 shutdown_connections 2024-09-12T03:19:24.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.564+0000 7efd2d717640 1 --2- 172.21.15.6:0/972887714 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efd140a2c10 0x7efd140a50d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.564+0000 7efd2d717640 1 --2- 172.21.15.6:0/972887714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd281082a0 0x7efd2819a590 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.564+0000 7efd2d717640 1 --2- 172.21.15.6:0/972887714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efd281036e0 0x7efd281a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.564+0000 7efd2d717640 1 --2- 172.21.15.6:0/972887714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd28102d30 0x7efd281a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.564+0000 7efd2d717640 1 -- 172.21.15.6:0/972887714 >> 172.21.15.6:0/972887714 conn(0x7efd280fc7d0 msgr2=0x7efd280fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:24.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.564+0000 7efd2d717640 1 -- 172.21.15.6:0/972887714 shutdown_connections 2024-09-12T03:19:24.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:24.564+0000 7efd2d717640 1 -- 172.21.15.6:0/972887714 wait complete. 2024-09-12T03:19:24.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:24 smithi118 ceph-mon[35048]: pgmap v284: 97 pgs: 97 active+clean; 588 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 4.0 KiB/s wr, 14 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-09-12T03:19:24.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:24 smithi118 ceph-mon[35048]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T03:19:24.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:24 smithi161 ceph-mon[35485]: pgmap v284: 97 pgs: 97 active+clean; 588 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 4.0 KiB/s wr, 14 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-09-12T03:19:24.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:24 smithi161 ceph-mon[35485]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T03:19:25.191 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-12T03:19:25.449 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:25.569 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:25 smithi006 ceph-mon[30991]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:25.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:25 smithi118 ceph-mon[35048]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:25.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:25 smithi161 ceph-mon[35485]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:25.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.900+0000 7fdc645a3640 1 -- 172.21.15.6:0/3258554420 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc5c106f30 msgr2=0x7fdc5c10eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:25.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.900+0000 7fdc645a3640 1 --2- 172.21.15.6:0/3258554420 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc5c106f30 0x7fdc5c10eab0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009ca0 tx=0x7fdc5402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:19:25.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.900+0000 7fdc645a3640 1 -- 172.21.15.6:0/3258554420 shutdown_connections 2024-09-12T03:19:25.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.900+0000 7fdc645a3640 1 --2- 172.21.15.6:0/3258554420 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc5c106f30 0x7fdc5c10eab0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:25.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.900+0000 7fdc645a3640 1 --2- 172.21.15.6:0/3258554420 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdc5c100cc0 0x7fdc5c106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:25.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.900+0000 7fdc645a3640 1 --2- 172.21.15.6:0/3258554420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc5c100310 0x7fdc5c1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:25.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.900+0000 7fdc645a3640 1 -- 172.21.15.6:0/3258554420 >> 172.21.15.6:0/3258554420 conn(0x7fdc5c077210 msgr2=0x7fdc5c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:25.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.900+0000 7fdc645a3640 1 -- 172.21.15.6:0/3258554420 shutdown_connections 2024-09-12T03:19:25.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.900+0000 7fdc645a3640 1 -- 172.21.15.6:0/3258554420 wait complete. 2024-09-12T03:19:25.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.901+0000 7fdc645a3640 1 Processor -- start 2024-09-12T03:19:25.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.901+0000 7fdc645a3640 1 -- start start 2024-09-12T03:19:25.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.901+0000 7fdc645a3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc5c100310 0x7fdc5c101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:25.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.901+0000 7fdc645a3640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdc5c100cc0 0x7fdc5c1017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:25.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.901+0000 7fdc645a3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc5c106f30 0x7fdc5c101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:25.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.901+0000 7fdc645a3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c111fb0 con 0x7fdc5c106f30 2024-09-12T03:19:25.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.901+0000 7fdc645a3640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c111e30 con 0x7fdc5c100cc0 2024-09-12T03:19:25.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.901+0000 7fdc645a3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c112130 con 0x7fdc5c100310 2024-09-12T03:19:25.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.901+0000 7fdc62b40640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc5c106f30 0x7fdc5c101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:25.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.901+0000 7fdc62b40640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc5c106f30 0x7fdc5c101d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51552/0 (socket says 172.21.15.6:51552) 2024-09-12T03:19:25.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.901+0000 7fdc62b40640 1 -- 172.21.15.6:0/3814655914 learned_addr learned my addr 172.21.15.6:0/3814655914 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:25.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.901+0000 7fdc61b3e640 1 --2- 172.21.15.6:0/3814655914 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdc5c100cc0 0x7fdc5c1017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:25.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.901+0000 7fdc6233f640 1 --2- 172.21.15.6:0/3814655914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc5c100310 0x7fdc5c101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:25.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.902+0000 7fdc62b40640 1 -- 172.21.15.6:0/3814655914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc5c100310 msgr2=0x7fdc5c101290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:25.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.902+0000 7fdc62b40640 1 --2- 172.21.15.6:0/3814655914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc5c100310 0x7fdc5c101290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:25.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.902+0000 7fdc62b40640 1 -- 172.21.15.6:0/3814655914 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdc5c100cc0 msgr2=0x7fdc5c1017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:25.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.902+0000 7fdc62b40640 1 --2- 172.21.15.6:0/3814655914 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdc5c100cc0 0x7fdc5c1017d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:25.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.902+0000 7fdc62b40640 1 -- 172.21.15.6:0/3814655914 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5c1a68a0 con 0x7fdc5c106f30 2024-09-12T03:19:25.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.902+0000 7fdc6233f640 1 --2- 172.21.15.6:0/3814655914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc5c100310 0x7fdc5c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:25.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.902+0000 7fdc62b40640 1 --2- 172.21.15.6:0/3814655914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc5c106f30 0x7fdc5c101d10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdc54031a60 tx=0x7fdc54031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:25.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.902+0000 7fdc537fe640 1 -- 172.21.15.6:0/3814655914 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54003b70 con 0x7fdc5c106f30 2024-09-12T03:19:25.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.902+0000 7fdc537fe640 1 -- 172.21.15.6:0/3814655914 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc54003d10 con 0x7fdc5c106f30 2024-09-12T03:19:25.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.902+0000 7fdc645a3640 1 -- 172.21.15.6:0/3814655914 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc5c1a6b30 con 0x7fdc5c106f30 2024-09-12T03:19:25.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.902+0000 7fdc537fe640 1 -- 172.21.15.6:0/3814655914 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54004770 con 0x7fdc5c106f30 2024-09-12T03:19:25.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.902+0000 7fdc645a3640 1 -- 172.21.15.6:0/3814655914 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc5c1a70c0 con 0x7fdc5c106f30 2024-09-12T03:19:25.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.903+0000 7fdc645a3640 1 -- 172.21.15.6:0/3814655914 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc5c1006f0 con 0x7fdc5c106f30 2024-09-12T03:19:25.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.904+0000 7fdc537fe640 1 -- 172.21.15.6:0/3814655914 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fdc5402f790 con 0x7fdc5c106f30 2024-09-12T03:19:25.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.905+0000 7fdc537fe640 1 --2- 172.21.15.6:0/3814655914 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fdc480a2c90 0x7fdc480a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:25.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.905+0000 7fdc6233f640 1 --2- 172.21.15.6:0/3814655914 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fdc480a2c90 0x7fdc480a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:25.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.905+0000 7fdc537fe640 1 -- 172.21.15.6:0/3814655914 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fdc540f0340 con 0x7fdc5c106f30 2024-09-12T03:19:25.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.906+0000 7fdc6233f640 1 --2- 172.21.15.6:0/3814655914 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fdc480a2c90 0x7fdc480a5150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009930 tx=0x7fdc58008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:25.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:25.909+0000 7fdc537fe640 1 -- 172.21.15.6:0/3814655914 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdc540bd280 con 0x7fdc5c106f30 2024-09-12T03:19:26.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:26.112+0000 7fdc645a3640 1 -- 172.21.15.6:0/3814655914 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fdc5c062630 con 0x7fdc480a2c90 2024-09-12T03:19:26.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:26.117+0000 7fdc537fe640 1 -- 172.21.15.6:0/3814655914 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fdc5c062630 con 0x7fdc480a2c90 2024-09-12T03:19:26.117 INFO:teuthology.orchestra.run.smithi006.stdout:/volumes/qa/sv_0/23311ad9-6ec5-4a54-8110-c4c54b674ebb 2024-09-12T03:19:26.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:26.120+0000 7fdc645a3640 1 -- 172.21.15.6:0/3814655914 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fdc480a2c90 msgr2=0x7fdc480a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:26.120+0000 7fdc645a3640 1 --2- 172.21.15.6:0/3814655914 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fdc480a2c90 0x7fdc480a5150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009930 tx=0x7fdc58008040 comp rx=0 tx=0).stop 2024-09-12T03:19:26.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:26.120+0000 7fdc645a3640 1 -- 172.21.15.6:0/3814655914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc5c106f30 msgr2=0x7fdc5c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:26.120+0000 7fdc645a3640 1 --2- 172.21.15.6:0/3814655914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc5c106f30 0x7fdc5c101d10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdc54031a60 tx=0x7fdc54031a90 comp rx=0 tx=0).stop 2024-09-12T03:19:26.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:26.120+0000 7fdc645a3640 1 -- 172.21.15.6:0/3814655914 shutdown_connections 2024-09-12T03:19:26.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:26.120+0000 7fdc645a3640 1 --2- 172.21.15.6:0/3814655914 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fdc480a2c90 0x7fdc480a5150 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:26.120+0000 7fdc645a3640 1 --2- 172.21.15.6:0/3814655914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc5c106f30 0x7fdc5c101d10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:26.120+0000 7fdc645a3640 1 --2- 172.21.15.6:0/3814655914 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdc5c100cc0 0x7fdc5c1017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:26.120+0000 7fdc645a3640 1 --2- 172.21.15.6:0/3814655914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc5c100310 0x7fdc5c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:26.120+0000 7fdc645a3640 1 -- 172.21.15.6:0/3814655914 >> 172.21.15.6:0/3814655914 conn(0x7fdc5c077210 msgr2=0x7fdc5c1056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:26.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:26.120+0000 7fdc645a3640 1 -- 172.21.15.6:0/3814655914 shutdown_connections 2024-09-12T03:19:26.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:26.120+0000 7fdc645a3640 1 -- 172.21.15.6:0/3814655914 wait complete. 2024-09-12T03:19:26.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:26 smithi118 ceph-mon[35048]: pgmap v285: 97 pgs: 97 active+clean; 588 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.1 KiB/s wr, 15 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-09-12T03:19:26.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:26 smithi118 ceph-mon[35048]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:26.741 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:26 smithi006 ceph-mon[30991]: pgmap v285: 97 pgs: 97 active+clean; 588 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.1 KiB/s wr, 15 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-09-12T03:19:26.741 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:26 smithi006 ceph-mon[30991]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:26.770 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-12T03:19:26.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:26 smithi161 ceph-mon[35485]: pgmap v285: 97 pgs: 97 active+clean; 588 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.1 KiB/s wr, 15 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-09-12T03:19:26.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:26 smithi161 ceph-mon[35485]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:27.024 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:27.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.457+0000 7f6268929640 1 -- 172.21.15.6:0/2069023670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6264101de0 msgr2=0x7f62641021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:27.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.457+0000 7f6268929640 1 --2- 172.21.15.6:0/2069023670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6264101de0 0x7f62641021c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6254009900 tx=0x7f625402a410 comp rx=0 tx=0).stop 2024-09-12T03:19:27.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.457+0000 7f6268929640 1 -- 172.21.15.6:0/2069023670 shutdown_connections 2024-09-12T03:19:27.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.457+0000 7f6268929640 1 --2- 172.21.15.6:0/2069023670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6264106f10 0x7f626410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.457+0000 7f6268929640 1 --2- 172.21.15.6:0/2069023670 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6264102790 0x7f62641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.457+0000 7f6268929640 1 --2- 172.21.15.6:0/2069023670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6264101de0 0x7f62641021c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.457+0000 7f6268929640 1 -- 172.21.15.6:0/2069023670 >> 172.21.15.6:0/2069023670 conn(0x7f62640fd8c0 msgr2=0x7f62640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:27.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.457+0000 7f6268929640 1 -- 172.21.15.6:0/2069023670 shutdown_connections 2024-09-12T03:19:27.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.457+0000 7f6268929640 1 -- 172.21.15.6:0/2069023670 wait complete. 2024-09-12T03:19:27.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.458+0000 7f6268929640 1 Processor -- start 2024-09-12T03:19:27.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.458+0000 7f6268929640 1 -- start start 2024-09-12T03:19:27.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.458+0000 7f6268929640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6264101de0 0x7f62641983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:27.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.458+0000 7f6268929640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6264102790 0x7f6264198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:27.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.458+0000 7f6268929640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6264106f10 0x7f626419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:27.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.458+0000 7f6268929640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6264111fe0 con 0x7f6264102790 2024-09-12T03:19:27.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.458+0000 7f6268929640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f6264111e60 con 0x7f6264106f10 2024-09-12T03:19:27.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.458+0000 7f6268929640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6264112160 con 0x7f6264101de0 2024-09-12T03:19:27.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.458+0000 7f6259d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6264102790 0x7f6264198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:27.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.458+0000 7f6259d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6264102790 0x7f6264198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51574/0 (socket says 172.21.15.6:51574) 2024-09-12T03:19:27.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.458+0000 7f6259d9b640 1 -- 172.21.15.6:0/2718431210 learned_addr learned my addr 172.21.15.6:0/2718431210 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:27.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.458+0000 7f626259c640 1 --2- 172.21.15.6:0/2718431210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6264101de0 0x7f62641983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:27.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.458+0000 7f6259d9b640 1 -- 172.21.15.6:0/2718431210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6264101de0 msgr2=0x7f62641983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:27.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.458+0000 7f6259d9b640 1 --2- 172.21.15.6:0/2718431210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6264101de0 0x7f62641983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.458+0000 7f6259d9b640 1 -- 172.21.15.6:0/2718431210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6264106f10 msgr2=0x7f626419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:27.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.458+0000 7f6262d9d640 1 --2- 172.21.15.6:0/2718431210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6264106f10 0x7f626419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:27.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.458+0000 7f6259d9b640 1 --2- 172.21.15.6:0/2718431210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6264106f10 0x7f626419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.458+0000 7f6259d9b640 1 -- 172.21.15.6:0/2718431210 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626419d380 con 0x7f6264102790 2024-09-12T03:19:27.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.459+0000 7f6259d9b640 1 --2- 172.21.15.6:0/2718431210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6264102790 0x7f6264198910 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f624c00c920 tx=0x7f624c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:27.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.459+0000 7f625bfff640 1 -- 172.21.15.6:0/2718431210 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c007c70 con 0x7f6264102790 2024-09-12T03:19:27.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.459+0000 7f6268929640 1 -- 172.21.15.6:0/2718431210 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626419d670 con 0x7f6264102790 2024-09-12T03:19:27.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.459+0000 7f625bfff640 1 -- 172.21.15.6:0/2718431210 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f624c007e10 con 0x7f6264102790 2024-09-12T03:19:27.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.459+0000 7f625bfff640 1 -- 172.21.15.6:0/2718431210 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c018890 con 0x7f6264102790 2024-09-12T03:19:27.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.459+0000 7f6268929640 1 -- 172.21.15.6:0/2718431210 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62641a4f50 con 0x7f6264102790 2024-09-12T03:19:27.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.460+0000 7f6268929640 1 -- 172.21.15.6:0/2718431210 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62641021c0 con 0x7f6264102790 2024-09-12T03:19:27.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.461+0000 7f625bfff640 1 -- 172.21.15.6:0/2718431210 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f624c018a70 con 0x7f6264102790 2024-09-12T03:19:27.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.462+0000 7f625bfff640 1 --2- 172.21.15.6:0/2718431210 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f62500a2c10 0x7f62500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:27.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.462+0000 7f626259c640 1 --2- 172.21.15.6:0/2718431210 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f62500a2c10 0x7f62500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:27.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.462+0000 7f625bfff640 1 -- 172.21.15.6:0/2718431210 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f624c0cf3a0 con 0x7f6264102790 2024-09-12T03:19:27.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.463+0000 7f626259c640 1 --2- 172.21.15.6:0/2718431210 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f62500a2c10 0x7f62500a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f625402a930 tx=0x7f6254005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:27.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.466+0000 7f625bfff640 1 -- 172.21.15.6:0/2718431210 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f624c014030 con 0x7f6264102790 2024-09-12T03:19:27.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.668+0000 7f6268929640 1 -- 172.21.15.6:0/2718431210 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6264062630 con 0x7f62500a2c10 2024-09-12T03:19:27.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.692+0000 7f625bfff640 1 -- 172.21.15.6:0/2718431210 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6264062630 con 0x7f62500a2c10 2024-09-12T03:19:27.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.695+0000 7f6268929640 1 -- 172.21.15.6:0/2718431210 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f62500a2c10 msgr2=0x7f62500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:27.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.695+0000 7f6268929640 1 --2- 172.21.15.6:0/2718431210 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f62500a2c10 0x7f62500a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f625402a930 tx=0x7f6254005fd0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.695+0000 7f6268929640 1 -- 172.21.15.6:0/2718431210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6264102790 msgr2=0x7f6264198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:27.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.695+0000 7f6268929640 1 --2- 172.21.15.6:0/2718431210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6264102790 0x7f6264198910 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f624c00c920 tx=0x7f624c00cde0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.695+0000 7f6268929640 1 -- 172.21.15.6:0/2718431210 shutdown_connections 2024-09-12T03:19:27.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.695+0000 7f6268929640 1 --2- 172.21.15.6:0/2718431210 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f62500a2c10 0x7f62500a50d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.695+0000 7f6268929640 1 --2- 172.21.15.6:0/2718431210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6264106f10 0x7f626419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.695+0000 7f6268929640 1 --2- 172.21.15.6:0/2718431210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6264102790 0x7f6264198910 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.695+0000 7f6268929640 1 --2- 172.21.15.6:0/2718431210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6264101de0 0x7f62641983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.695+0000 7f6268929640 1 -- 172.21.15.6:0/2718431210 >> 172.21.15.6:0/2718431210 conn(0x7f62640fd8c0 msgr2=0x7f62640fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:27.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.695+0000 7f6268929640 1 -- 172.21.15.6:0/2718431210 shutdown_connections 2024-09-12T03:19:27.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:27.695+0000 7f6268929640 1 -- 172.21.15.6:0/2718431210 wait complete. 2024-09-12T03:19:28.389 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-12T03:19:28.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:28 smithi006 ceph-mon[30991]: pgmap v286: 97 pgs: 97 active+clean; 588 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.0 KiB/s wr, 13 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-09-12T03:19:28.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:28 smithi006 ceph-mon[30991]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:28.648 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:28.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:28 smithi118 ceph-mon[35048]: pgmap v286: 97 pgs: 97 active+clean; 588 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.0 KiB/s wr, 13 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-09-12T03:19:28.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:28 smithi118 ceph-mon[35048]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:28.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:28 smithi161 ceph-mon[35485]: pgmap v286: 97 pgs: 97 active+clean; 588 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.0 KiB/s wr, 13 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-09-12T03:19:28.866 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:28 smithi161 ceph-mon[35485]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:29.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.128+0000 7fa9663e3640 1 -- 172.21.15.6:0/3419896988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9601082a0 msgr2=0x7fa96010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.128+0000 7fa9663e3640 1 --2- 172.21.15.6:0/3419896988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9601082a0 0x7fa96010ec40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa958009cd0 tx=0x7fa95802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.128+0000 7fa9663e3640 1 -- 172.21.15.6:0/3419896988 shutdown_connections 2024-09-12T03:19:29.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.128+0000 7fa9663e3640 1 --2- 172.21.15.6:0/3419896988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9601082a0 0x7fa96010ec40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.128+0000 7fa9663e3640 1 --2- 172.21.15.6:0/3419896988 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9601036e0 0x7fa960107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.128+0000 7fa9663e3640 1 --2- 172.21.15.6:0/3419896988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa960102d30 0x7fa960103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.128+0000 7fa9663e3640 1 -- 172.21.15.6:0/3419896988 >> 172.21.15.6:0/3419896988 conn(0x7fa9600fc7d0 msgr2=0x7fa9600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:29.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.128+0000 7fa9663e3640 1 -- 172.21.15.6:0/3419896988 shutdown_connections 2024-09-12T03:19:29.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.128+0000 7fa9663e3640 1 -- 172.21.15.6:0/3419896988 wait complete. 2024-09-12T03:19:29.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.128+0000 7fa9663e3640 1 Processor -- start 2024-09-12T03:19:29.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.129+0000 7fa9663e3640 1 -- start start 2024-09-12T03:19:29.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.129+0000 7fa9663e3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa960102d30 0x7fa960198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.129+0000 7fa9663e3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9601036e0 0x7fa960198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.129+0000 7fa9663e3640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9601082a0 0x7fa96019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.129+0000 7fa9663e3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa960112100 con 0x7fa9601036e0 2024-09-12T03:19:29.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.129+0000 7fa9663e3640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa960111f80 con 0x7fa9601082a0 2024-09-12T03:19:29.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.129+0000 7fa9663e3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa960112280 con 0x7fa960102d30 2024-09-12T03:19:29.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.129+0000 7fa957fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9601036e0 0x7fa960198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.129+0000 7fa957fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9601036e0 0x7fa960198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59236/0 (socket says 172.21.15.6:59236) 2024-09-12T03:19:29.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.129+0000 7fa957fff640 1 -- 172.21.15.6:0/4108327726 learned_addr learned my addr 172.21.15.6:0/4108327726 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:29.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.129+0000 7fa95ffff640 1 --2- 172.21.15.6:0/4108327726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa960102d30 0x7fa960198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.129+0000 7fa964980640 1 --2- 172.21.15.6:0/4108327726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9601082a0 0x7fa96019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.129+0000 7fa957fff640 1 -- 172.21.15.6:0/4108327726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa960102d30 msgr2=0x7fa960198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.129+0000 7fa957fff640 1 --2- 172.21.15.6:0/4108327726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa960102d30 0x7fa960198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.129+0000 7fa957fff640 1 -- 172.21.15.6:0/4108327726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9601082a0 msgr2=0x7fa96019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.129+0000 7fa957fff640 1 --2- 172.21.15.6:0/4108327726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9601082a0 0x7fa96019ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.129+0000 7fa957fff640 1 -- 172.21.15.6:0/4108327726 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96019d500 con 0x7fa9601036e0 2024-09-12T03:19:29.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.129+0000 7fa95ffff640 1 --2- 172.21.15.6:0/4108327726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa960102d30 0x7fa960198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:29.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.129+0000 7fa957fff640 1 --2- 172.21.15.6:0/4108327726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9601036e0 0x7fa960198a90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa94800c8c0 tx=0x7fa94800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:29.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.129+0000 7fa95dffb640 1 -- 172.21.15.6:0/4108327726 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948007bb0 con 0x7fa9601036e0 2024-09-12T03:19:29.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.130+0000 7fa95dffb640 1 -- 172.21.15.6:0/4108327726 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa948007d50 con 0x7fa9601036e0 2024-09-12T03:19:29.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.130+0000 7fa9663e3640 1 -- 172.21.15.6:0/4108327726 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96019d7f0 con 0x7fa9601036e0 2024-09-12T03:19:29.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.130+0000 7fa95dffb640 1 -- 172.21.15.6:0/4108327726 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948018830 con 0x7fa9601036e0 2024-09-12T03:19:29.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.130+0000 7fa9663e3640 1 -- 172.21.15.6:0/4108327726 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9601a50d0 con 0x7fa9601036e0 2024-09-12T03:19:29.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.131+0000 7fa9663e3640 1 -- 172.21.15.6:0/4108327726 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa960103110 con 0x7fa9601036e0 2024-09-12T03:19:29.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.132+0000 7fa95dffb640 1 -- 172.21.15.6:0/4108327726 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa9480189f0 con 0x7fa9601036e0 2024-09-12T03:19:29.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.132+0000 7fa95dffb640 1 --2- 172.21.15.6:0/4108327726 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa94c0a2c90 0x7fa94c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.132+0000 7fa95ffff640 1 --2- 172.21.15.6:0/4108327726 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa94c0a2c90 0x7fa94c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.133+0000 7fa95dffb640 1 -- 172.21.15.6:0/4108327726 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa9480cf2e0 con 0x7fa9601036e0 2024-09-12T03:19:29.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.133+0000 7fa95ffff640 1 --2- 172.21.15.6:0/4108327726 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa94c0a2c90 0x7fa94c0a5150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa950009930 tx=0x7fa950008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:29.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.136+0000 7fa95dffb640 1 -- 172.21.15.6:0/4108327726 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa948014030 con 0x7fa9601036e0 2024-09-12T03:19:29.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.341+0000 7fa9663e3640 1 -- 172.21.15.6:0/4108327726 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fa96010a2e0 con 0x7fa94c0a2c90 2024-09-12T03:19:29.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.346+0000 7fa95dffb640 1 -- 172.21.15.6:0/4108327726 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fa96010a2e0 con 0x7fa94c0a2c90 2024-09-12T03:19:29.347 INFO:teuthology.orchestra.run.smithi006.stdout:/volumes/qa/sv_1/43932019-504c-4abf-b8ca-9aefdc54d27f 2024-09-12T03:19:29.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.349+0000 7fa9663e3640 1 -- 172.21.15.6:0/4108327726 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa94c0a2c90 msgr2=0x7fa94c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.349+0000 7fa9663e3640 1 --2- 172.21.15.6:0/4108327726 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa94c0a2c90 0x7fa94c0a5150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa950009930 tx=0x7fa950008040 comp rx=0 tx=0).stop 2024-09-12T03:19:29.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.349+0000 7fa9663e3640 1 -- 172.21.15.6:0/4108327726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9601036e0 msgr2=0x7fa960198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.349+0000 7fa9663e3640 1 --2- 172.21.15.6:0/4108327726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9601036e0 0x7fa960198a90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa94800c8c0 tx=0x7fa94800cd80 comp rx=0 tx=0).stop 2024-09-12T03:19:29.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.349+0000 7fa9663e3640 1 -- 172.21.15.6:0/4108327726 shutdown_connections 2024-09-12T03:19:29.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.349+0000 7fa9663e3640 1 --2- 172.21.15.6:0/4108327726 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa94c0a2c90 0x7fa94c0a5150 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.349+0000 7fa9663e3640 1 --2- 172.21.15.6:0/4108327726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9601082a0 0x7fa96019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.349+0000 7fa9663e3640 1 --2- 172.21.15.6:0/4108327726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9601036e0 0x7fa960198a90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.349+0000 7fa9663e3640 1 --2- 172.21.15.6:0/4108327726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa960102d30 0x7fa960198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.349+0000 7fa9663e3640 1 -- 172.21.15.6:0/4108327726 >> 172.21.15.6:0/4108327726 conn(0x7fa9600fc7d0 msgr2=0x7fa960106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:29.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.349+0000 7fa9663e3640 1 -- 172.21.15.6:0/4108327726 shutdown_connections 2024-09-12T03:19:29.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:29.349+0000 7fa9663e3640 1 -- 172.21.15.6:0/4108327726 wait complete. 2024-09-12T03:19:30.029 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:19:30.293 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:30.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.756+0000 7fccf7c65640 1 -- 172.21.15.6:0/1834985807 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf0102790 msgr2=0x7fccf01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:30.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.756+0000 7fccf7c65640 1 --2- 172.21.15.6:0/1834985807 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf0102790 0x7fccf01067e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009900 tx=0x7fcce402a410 comp rx=0 tx=0).stop 2024-09-12T03:19:30.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.756+0000 7fccf7c65640 1 -- 172.21.15.6:0/1834985807 shutdown_connections 2024-09-12T03:19:30.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.756+0000 7fccf7c65640 1 --2- 172.21.15.6:0/1834985807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccf0106f10 0x7fccf010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.756+0000 7fccf7c65640 1 --2- 172.21.15.6:0/1834985807 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf0102790 0x7fccf01067e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.756+0000 7fccf7c65640 1 --2- 172.21.15.6:0/1834985807 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fccf0101de0 0x7fccf01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.756+0000 7fccf7c65640 1 -- 172.21.15.6:0/1834985807 >> 172.21.15.6:0/1834985807 conn(0x7fccf00fd8c0 msgr2=0x7fccf00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:30.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.756+0000 7fccf7c65640 1 -- 172.21.15.6:0/1834985807 shutdown_connections 2024-09-12T03:19:30.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.756+0000 7fccf7c65640 1 -- 172.21.15.6:0/1834985807 wait complete. 2024-09-12T03:19:30.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.757+0000 7fccf7c65640 1 Processor -- start 2024-09-12T03:19:30.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.757+0000 7fccf7c65640 1 -- start start 2024-09-12T03:19:30.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.757+0000 7fccf7c65640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf0101de0 0x7fccf01984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:30.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.757+0000 7fccf7c65640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fccf0102790 0x7fccf0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:30.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.757+0000 7fccf7c65640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccf0106f10 0x7fccf019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:30.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.757+0000 7fccf7c65640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fccf0111f70 con 0x7fccf0101de0 2024-09-12T03:19:30.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.757+0000 7fccf7c65640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fccf0111df0 con 0x7fccf0102790 2024-09-12T03:19:30.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.757+0000 7fccf7c65640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fccf01120f0 con 0x7fccf0106f10 2024-09-12T03:19:30.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.757+0000 7fccf5a01640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf0101de0 0x7fccf01984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:30.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.757+0000 7fccf5a01640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf0101de0 0x7fccf01984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59254/0 (socket says 172.21.15.6:59254) 2024-09-12T03:19:30.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.757+0000 7fccf5a01640 1 -- 172.21.15.6:0/736744845 learned_addr learned my addr 172.21.15.6:0/736744845 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:30.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.757+0000 7fccf6202640 1 --2- 172.21.15.6:0/736744845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccf0106f10 0x7fccf019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:30.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.757+0000 7fccf5200640 1 --2- 172.21.15.6:0/736744845 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fccf0102790 0x7fccf0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:30.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.757+0000 7fccf5a01640 1 -- 172.21.15.6:0/736744845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccf0106f10 msgr2=0x7fccf019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:30.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.757+0000 7fccf5a01640 1 --2- 172.21.15.6:0/736744845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccf0106f10 0x7fccf019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.757+0000 7fccf5a01640 1 -- 172.21.15.6:0/736744845 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fccf0102790 msgr2=0x7fccf0198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:30.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.757+0000 7fccf5a01640 1 --2- 172.21.15.6:0/736744845 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fccf0102790 0x7fccf0198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.757+0000 7fccf5a01640 1 -- 172.21.15.6:0/736744845 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf019d480 con 0x7fccf0101de0 2024-09-12T03:19:30.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.757+0000 7fccf5200640 1 --2- 172.21.15.6:0/736744845 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fccf0102790 0x7fccf0198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:30.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.757+0000 7fccf5a01640 1 --2- 172.21.15.6:0/736744845 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf0101de0 0x7fccf01984d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcce000cce0 tx=0x7fcce000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:30.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.758+0000 7fcceeffd640 1 -- 172.21.15.6:0/736744845 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0004b20 con 0x7fccf0101de0 2024-09-12T03:19:30.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.758+0000 7fcceeffd640 1 -- 172.21.15.6:0/736744845 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce0004cc0 con 0x7fccf0101de0 2024-09-12T03:19:30.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.758+0000 7fcceeffd640 1 -- 172.21.15.6:0/736744845 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce000f7d0 con 0x7fccf0101de0 2024-09-12T03:19:30.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.758+0000 7fccf7c65640 1 -- 172.21.15.6:0/736744845 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf019d770 con 0x7fccf0101de0 2024-09-12T03:19:30.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.758+0000 7fccf7c65640 1 -- 172.21.15.6:0/736744845 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf01a5050 con 0x7fccf0101de0 2024-09-12T03:19:30.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.759+0000 7fccf7c65640 1 -- 172.21.15.6:0/736744845 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf01021c0 con 0x7fccf0101de0 2024-09-12T03:19:30.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.760+0000 7fcceeffd640 1 -- 172.21.15.6:0/736744845 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fcce000f970 con 0x7fccf0101de0 2024-09-12T03:19:30.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.760+0000 7fcceeffd640 1 --2- 172.21.15.6:0/736744845 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fccd80a2c90 0x7fccd80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:30.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.761+0000 7fccf5200640 1 --2- 172.21.15.6:0/736744845 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fccd80a2c90 0x7fccd80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:30.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.761+0000 7fcceeffd640 1 -- 172.21.15.6:0/736744845 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fcce00cf2f0 con 0x7fccf0101de0 2024-09-12T03:19:30.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.761+0000 7fccf5200640 1 --2- 172.21.15.6:0/736744845 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fccd80a2c90 0x7fccd80a5150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcce4000c00 tx=0x7fcce4032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:30.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.765+0000 7fcceeffd640 1 -- 172.21.15.6:0/736744845 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcce0014030 con 0x7fccf0101de0 2024-09-12T03:19:30.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.975+0000 7fccf7c65640 1 -- 172.21.15.6:0/736744845 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccf010a150 con 0x7fccf0101de0 2024-09-12T03:19:30.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.976+0000 7fcceeffd640 1 -- 172.21.15.6:0/736744845 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11908 (secure 0 0 0) 0x7fcce009c0c0 con 0x7fccf0101de0 2024-09-12T03:19:30.976 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:19:30.976 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":24,"btime":"2024-09-12T03:19:21:375379+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:19:20.919190+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24463,24464,24494]},"id":1}]} 2024-09-12T03:19:30.976 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 24 2024-09-12T03:19:30.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.978+0000 7fccf7c65640 1 -- 172.21.15.6:0/736744845 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fccd80a2c90 msgr2=0x7fccd80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:30.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.978+0000 7fccf7c65640 1 --2- 172.21.15.6:0/736744845 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fccd80a2c90 0x7fccd80a5150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcce4000c00 tx=0x7fcce4032d50 comp rx=0 tx=0).stop 2024-09-12T03:19:30.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.978+0000 7fccf7c65640 1 -- 172.21.15.6:0/736744845 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf0101de0 msgr2=0x7fccf01984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:30.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.978+0000 7fccf7c65640 1 --2- 172.21.15.6:0/736744845 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf0101de0 0x7fccf01984d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcce000cce0 tx=0x7fcce000a620 comp rx=0 tx=0).stop 2024-09-12T03:19:30.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.978+0000 7fccf7c65640 1 -- 172.21.15.6:0/736744845 shutdown_connections 2024-09-12T03:19:30.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.978+0000 7fccf7c65640 1 --2- 172.21.15.6:0/736744845 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fccd80a2c90 0x7fccd80a5150 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.978+0000 7fccf7c65640 1 --2- 172.21.15.6:0/736744845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccf0106f10 0x7fccf019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.978+0000 7fccf7c65640 1 --2- 172.21.15.6:0/736744845 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fccf0102790 0x7fccf0198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.978+0000 7fccf7c65640 1 --2- 172.21.15.6:0/736744845 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf0101de0 0x7fccf01984d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.978+0000 7fccf7c65640 1 -- 172.21.15.6:0/736744845 >> 172.21.15.6:0/736744845 conn(0x7fccf00fd8c0 msgr2=0x7fccf00fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:30.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.978+0000 7fccf7c65640 1 -- 172.21.15.6:0/736744845 shutdown_connections 2024-09-12T03:19:30.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:30.978+0000 7fccf7c65640 1 -- 172.21.15.6:0/736744845 wait complete. 2024-09-12T03:19:31.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:31 smithi006 ceph-mon[30991]: pgmap v287: 97 pgs: 97 active+clean; 621 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.2 KiB/s wr, 15 op/s; 3 B/s, 2 keys/s, 0 objects/s recovering 2024-09-12T03:19:31.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:31 smithi006 ceph-mon[30991]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:31.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:31 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/736744845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:31.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:31 smithi118 ceph-mon[35048]: pgmap v287: 97 pgs: 97 active+clean; 621 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.2 KiB/s wr, 15 op/s; 3 B/s, 2 keys/s, 0 objects/s recovering 2024-09-12T03:19:31.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:31 smithi118 ceph-mon[35048]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:31.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:31 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/736744845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:31.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:31 smithi161 ceph-mon[35485]: pgmap v287: 97 pgs: 97 active+clean; 621 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.2 KiB/s wr, 15 op/s; 3 B/s, 2 keys/s, 0 objects/s recovering 2024-09-12T03:19:31.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:31 smithi161 ceph-mon[35485]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:31.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:31 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/736744845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:31.700 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:19:31.955 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:32.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:32 smithi006 ceph-mon[30991]: pgmap v288: 97 pgs: 97 active+clean; 621 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.5 KiB/s wr, 14 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T03:19:32.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.381+0000 7fa059cc5640 1 -- 172.21.15.6:0/1929549372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa054102d30 msgr2=0x7fa054103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:32.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.381+0000 7fa059cc5640 1 --2- 172.21.15.6:0/1929549372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa054102d30 0x7fa054103110 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa044009930 tx=0x7fa04402a420 comp rx=0 tx=0).stop 2024-09-12T03:19:32.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.381+0000 7fa059cc5640 1 -- 172.21.15.6:0/1929549372 shutdown_connections 2024-09-12T03:19:32.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.381+0000 7fa059cc5640 1 --2- 172.21.15.6:0/1929549372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0541082a0 0x7fa05410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:32.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.381+0000 7fa059cc5640 1 --2- 172.21.15.6:0/1929549372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0541036e0 0x7fa054107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:32.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.381+0000 7fa059cc5640 1 --2- 172.21.15.6:0/1929549372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa054102d30 0x7fa054103110 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:32.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.381+0000 7fa059cc5640 1 -- 172.21.15.6:0/1929549372 >> 172.21.15.6:0/1929549372 conn(0x7fa0540fc7d0 msgr2=0x7fa0540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:32.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.381+0000 7fa059cc5640 1 -- 172.21.15.6:0/1929549372 shutdown_connections 2024-09-12T03:19:32.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.381+0000 7fa059cc5640 1 -- 172.21.15.6:0/1929549372 wait complete. 2024-09-12T03:19:32.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.381+0000 7fa059cc5640 1 Processor -- start 2024-09-12T03:19:32.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.382+0000 7fa059cc5640 1 -- start start 2024-09-12T03:19:32.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.382+0000 7fa059cc5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa054102d30 0x7fa054100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:32.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.382+0000 7fa059cc5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0541036e0 0x7fa054100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:32.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.382+0000 7fa059cc5640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0541082a0 0x7fa054101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:32.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.382+0000 7fa059cc5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa054112100 con 0x7fa054102d30 2024-09-12T03:19:32.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.382+0000 7fa059cc5640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa054111f80 con 0x7fa0541082a0 2024-09-12T03:19:32.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.382+0000 7fa059cc5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa054112280 con 0x7fa0541036e0 2024-09-12T03:19:32.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.382+0000 7fa0537fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa054102d30 0x7fa054100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:32.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.382+0000 7fa0537fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa054102d30 0x7fa054100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59272/0 (socket says 172.21.15.6:59272) 2024-09-12T03:19:32.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.382+0000 7fa0537fe640 1 -- 172.21.15.6:0/1377514913 learned_addr learned my addr 172.21.15.6:0/1377514913 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:32.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.382+0000 7fa052ffd640 1 --2- 172.21.15.6:0/1377514913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0541036e0 0x7fa054100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:32.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.382+0000 7fa053fff640 1 --2- 172.21.15.6:0/1377514913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0541082a0 0x7fa054101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:32.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.382+0000 7fa0537fe640 1 -- 172.21.15.6:0/1377514913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0541036e0 msgr2=0x7fa054100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:32.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.382+0000 7fa0537fe640 1 --2- 172.21.15.6:0/1377514913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0541036e0 0x7fa054100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:32.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.382+0000 7fa0537fe640 1 -- 172.21.15.6:0/1377514913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0541082a0 msgr2=0x7fa054101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:32.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.382+0000 7fa0537fe640 1 --2- 172.21.15.6:0/1377514913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0541082a0 0x7fa054101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:32.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.382+0000 7fa0537fe640 1 -- 172.21.15.6:0/1377514913 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0541a6ac0 con 0x7fa054102d30 2024-09-12T03:19:32.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.382+0000 7fa052ffd640 1 --2- 172.21.15.6:0/1377514913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0541036e0 0x7fa054100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:32.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.382+0000 7fa0537fe640 1 --2- 172.21.15.6:0/1377514913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa054102d30 0x7fa054100990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa044009230 tx=0x7fa044004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:32.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.383+0000 7fa050ff9640 1 -- 172.21.15.6:0/1377514913 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04403f030 con 0x7fa054102d30 2024-09-12T03:19:32.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.383+0000 7fa050ff9640 1 -- 172.21.15.6:0/1377514913 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa044003ad0 con 0x7fa054102d30 2024-09-12T03:19:32.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.383+0000 7fa050ff9640 1 -- 172.21.15.6:0/1377514913 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa044035070 con 0x7fa054102d30 2024-09-12T03:19:32.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.383+0000 7fa059cc5640 1 -- 172.21.15.6:0/1377514913 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0541a6d50 con 0x7fa054102d30 2024-09-12T03:19:32.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.383+0000 7fa059cc5640 1 -- 172.21.15.6:0/1377514913 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0541a71d0 con 0x7fa054102d30 2024-09-12T03:19:32.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.384+0000 7fa059cc5640 1 -- 172.21.15.6:0/1377514913 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa054103110 con 0x7fa054102d30 2024-09-12T03:19:32.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.385+0000 7fa050ff9640 1 -- 172.21.15.6:0/1377514913 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa044042030 con 0x7fa054102d30 2024-09-12T03:19:32.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.385+0000 7fa050ff9640 1 --2- 172.21.15.6:0/1377514913 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa03c0a2c10 0x7fa03c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:32.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.386+0000 7fa052ffd640 1 --2- 172.21.15.6:0/1377514913 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa03c0a2c10 0x7fa03c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:32.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.386+0000 7fa050ff9640 1 -- 172.21.15.6:0/1377514913 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa0440ef700 con 0x7fa054102d30 2024-09-12T03:19:32.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.386+0000 7fa052ffd640 1 --2- 172.21.15.6:0/1377514913 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa03c0a2c10 0x7fa03c0a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa054102420 tx=0x7fa04800b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:32.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.390+0000 7fa050ff9640 1 -- 172.21.15.6:0/1377514913 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa0440bc4d0 con 0x7fa054102d30 2024-09-12T03:19:32.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:32 smithi118 ceph-mon[35048]: pgmap v288: 97 pgs: 97 active+clean; 621 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.5 KiB/s wr, 14 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T03:19:32.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.564+0000 7fa059cc5640 1 -- 172.21.15.6:0/1377514913 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa05410a2b0 con 0x7fa054102d30 2024-09-12T03:19:32.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.566+0000 7fa050ff9640 1 -- 172.21.15.6:0/1377514913 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22607 (secure 0 0 0) 0x7fa0440bbdd0 con 0x7fa054102d30 2024-09-12T03:19:32.566 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:19:32.566 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":82,"fsid":"89d99938-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:11:29.530545+0000","modified":"2024-09-12T03:19:20.366710+0000","last_up_change":"2024-09-12T03:18:21.292572+0000","last_in_change":"2024-09-12T03:17:58.013800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:14:37.662010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:06.143398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:08.305279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f55d68b-95a0-4ee2-a46f-bc422b38471e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6803","nonce":726729286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6805","nonce":726729286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6809","nonce":726729286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6807","nonce":726729286}]},"public_addr":"172.21.15.6:6803/726729286","cluster_addr":"172.21.15.6:6805/726729286","heartbeat_back_addr":"172.21.15.6:6809/726729286","heartbeat_front_addr":"172.21.15.6:6807/726729286","state":["exists","up"]},{"osd":1,"uuid":"83afdb01-c731-4de1-96c6-3f798549b1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6801","nonce":3377555415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6803","nonce":3377555415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6807","nonce":3377555415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6805","nonce":3377555415}]},"public_addr":"172.21.15.118:6801/3377555415","cluster_addr":"172.21.15.118:6803/3377555415","heartbeat_back_addr":"172.21.15.118:6807/3377555415","heartbeat_front_addr":"172.21.15.118:6805/3377555415","state":["exists","up"]},{"osd":2,"uuid":"aed718b4-b849-403a-b81c-3a140b5c380c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6801","nonce":392541009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6803","nonce":392541009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6807","nonce":392541009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6805","nonce":392541009}]},"public_addr":"172.21.15.161:6801/392541009","cluster_addr":"172.21.15.161:6803/392541009","heartbeat_back_addr":"172.21.15.161:6807/392541009","heartbeat_front_addr":"172.21.15.161:6805/392541009","state":["exists","up"]},{"osd":3,"uuid":"25ced4c1-bf31-402d-8397-d42fd57f615f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6811","nonce":1892039151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6813","nonce":1892039151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6817","nonce":1892039151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6815","nonce":1892039151}]},"public_addr":"172.21.15.6:6811/1892039151","cluster_addr":"172.21.15.6:6813/1892039151","heartbeat_back_addr":"172.21.15.6:6817/1892039151","heartbeat_front_addr":"172.21.15.6:6815/1892039151","state":["exists","up"]},{"osd":4,"uuid":"1d884604-920b-4c35-b41e-d629ad9f9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6809","nonce":150177024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6811","nonce":150177024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6815","nonce":150177024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6813","nonce":150177024}]},"public_addr":"172.21.15.118:6809/150177024","cluster_addr":"172.21.15.118:6811/150177024","heartbeat_back_addr":"172.21.15.118:6815/150177024","heartbeat_front_addr":"172.21.15.118:6813/150177024","state":["exists","up"]},{"osd":5,"uuid":"1d40c0ef-99f6-426f-90c7-27579edc66af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6809","nonce":3758167854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6811","nonce":3758167854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6815","nonce":3758167854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6813","nonce":3758167854}]},"public_addr":"172.21.15.161:6809/3758167854","cluster_addr":"172.21.15.161:6811/3758167854","heartbeat_back_addr":"172.21.15.161:6815/3758167854","heartbeat_front_addr":"172.21.15.161:6813/3758167854","state":["exists","up"]},{"osd":6,"uuid":"9d9f9510-b107-40c2-9a98-9012715944b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6819","nonce":2129224874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6821","nonce":2129224874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6825","nonce":2129224874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6823","nonce":2129224874}]},"public_addr":"172.21.15.6:6819/2129224874","cluster_addr":"172.21.15.6:6821/2129224874","heartbeat_back_addr":"172.21.15.6:6825/2129224874","heartbeat_front_addr":"172.21.15.6:6823/2129224874","state":["exists","up"]},{"osd":7,"uuid":"84635ae1-3240-4954-af14-76bb3aa162e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6817","nonce":269980353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6819","nonce":269980353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6823","nonce":269980353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6821","nonce":269980353}]},"public_addr":"172.21.15.118:6817/269980353","cluster_addr":"172.21.15.118:6819/269980353","heartbeat_back_addr":"172.21.15.118:6823/269980353","heartbeat_front_addr":"172.21.15.118:6821/269980353","state":["exists","up"]},{"osd":8,"uuid":"6fd58e55-1492-4595-8647-823ea3e3c5d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6817","nonce":1783416134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6819","nonce":1783416134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6823","nonce":1783416134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6821","nonce":1783416134}]},"public_addr":"172.21.15.161:6817/1783416134","cluster_addr":"172.21.15.161:6819/1783416134","heartbeat_back_addr":"172.21.15.161:6823/1783416134","heartbeat_front_addr":"172.21.15.161:6821/1783416134","state":["exists","up"]},{"osd":9,"uuid":"90508a6d-a49c-43c6-a600-5bbccd8702ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6827","nonce":2987332365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6829","nonce":2987332365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6833","nonce":2987332365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6831","nonce":2987332365}]},"public_addr":"172.21.15.6:6827/2987332365","cluster_addr":"172.21.15.6:6829/2987332365","heartbeat_back_addr":"172.21.15.6:6833/2987332365","heartbeat_front_addr":"172.21.15.6:6831/2987332365","state":["exists","up"]},{"osd":10,"uuid":"911e86d1-669a-4935-82c0-7da51d141df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6825","nonce":2167146341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6827","nonce":2167146341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6831","nonce":2167146341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6829","nonce":2167146341}]},"public_addr":"172.21.15.118:6825/2167146341","cluster_addr":"172.21.15.118:6827/2167146341","heartbeat_back_addr":"172.21.15.118:6831/2167146341","heartbeat_front_addr":"172.21.15.118:6829/2167146341","state":["exists","up"]},{"osd":11,"uuid":"5a15c6a7-ff43-473c-bb48-0c129c316df5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6825","nonce":2889795796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6827","nonce":2889795796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6831","nonce":2889795796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6829","nonce":2889795796}]},"public_addr":"172.21.15.161:6825/2889795796","cluster_addr":"172.21.15.161:6827/2889795796","heartbeat_back_addr":"172.21.15.161:6831/2889795796","heartbeat_front_addr":"172.21.15.161:6829/2889795796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:42.912205+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:08.410669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:34.299134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:59.093181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:24.116635+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:49.566433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:15.719320+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:40.541017+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:05.178178+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:31.163877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:55.405110+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:19.417156+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6800/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/2620491916":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/4037656392":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/1377116573":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/3062800125":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:6800/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/4145338968":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/825701128":"2024-09-13T03:11:55.244257+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:32.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.568+0000 7fa059cc5640 1 -- 172.21.15.6:0/1377514913 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa03c0a2c10 msgr2=0x7fa03c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:32.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.568+0000 7fa059cc5640 1 --2- 172.21.15.6:0/1377514913 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa03c0a2c10 0x7fa03c0a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa054102420 tx=0x7fa04800b5f0 comp rx=0 tx=0).stop 2024-09-12T03:19:32.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.568+0000 7fa059cc5640 1 -- 172.21.15.6:0/1377514913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa054102d30 msgr2=0x7fa054100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:32.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.568+0000 7fa059cc5640 1 --2- 172.21.15.6:0/1377514913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa054102d30 0x7fa054100990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa044009230 tx=0x7fa044004270 comp rx=0 tx=0).stop 2024-09-12T03:19:32.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.568+0000 7fa059cc5640 1 -- 172.21.15.6:0/1377514913 shutdown_connections 2024-09-12T03:19:32.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.568+0000 7fa059cc5640 1 --2- 172.21.15.6:0/1377514913 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa03c0a2c10 0x7fa03c0a50d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:32.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.568+0000 7fa059cc5640 1 --2- 172.21.15.6:0/1377514913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0541082a0 0x7fa054101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:32.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.568+0000 7fa059cc5640 1 --2- 172.21.15.6:0/1377514913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0541036e0 0x7fa054100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:32.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.568+0000 7fa059cc5640 1 --2- 172.21.15.6:0/1377514913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa054102d30 0x7fa054100990 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:32.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.568+0000 7fa059cc5640 1 -- 172.21.15.6:0/1377514913 >> 172.21.15.6:0/1377514913 conn(0x7fa0540fc7d0 msgr2=0x7fa054105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:32.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.568+0000 7fa059cc5640 1 -- 172.21.15.6:0/1377514913 shutdown_connections 2024-09-12T03:19:32.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:32.568+0000 7fa059cc5640 1 -- 172.21.15.6:0/1377514913 wait complete. 2024-09-12T03:19:32.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:32 smithi161 ceph-mon[35485]: pgmap v288: 97 pgs: 97 active+clean; 621 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.5 KiB/s wr, 14 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T03:19:33.333 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T03:19:33.333 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:19:33.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:33 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1377514913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:33.514 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:33 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1377514913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:33.598 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:33.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:33 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1377514913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:34.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.055+0000 7fe444a39640 1 -- 172.21.15.6:0/4172521581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4401082a0 msgr2=0x7fe44010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:34.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.055+0000 7fe444a39640 1 --2- 172.21.15.6:0/4172521581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4401082a0 0x7fe44010ec40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe438009cd0 tx=0x7fe43802a910 comp rx=0 tx=0).stop 2024-09-12T03:19:34.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.055+0000 7fe444a39640 1 -- 172.21.15.6:0/4172521581 shutdown_connections 2024-09-12T03:19:34.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.055+0000 7fe444a39640 1 --2- 172.21.15.6:0/4172521581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4401082a0 0x7fe44010ec40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.055+0000 7fe444a39640 1 --2- 172.21.15.6:0/4172521581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4401036e0 0x7fe440107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.055+0000 7fe444a39640 1 --2- 172.21.15.6:0/4172521581 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe440102d30 0x7fe440103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.055+0000 7fe444a39640 1 -- 172.21.15.6:0/4172521581 >> 172.21.15.6:0/4172521581 conn(0x7fe4400fc7d0 msgr2=0x7fe4400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:34.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.056+0000 7fe444a39640 1 -- 172.21.15.6:0/4172521581 shutdown_connections 2024-09-12T03:19:34.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.056+0000 7fe444a39640 1 -- 172.21.15.6:0/4172521581 wait complete. 2024-09-12T03:19:34.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.056+0000 7fe444a39640 1 Processor -- start 2024-09-12T03:19:34.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.056+0000 7fe444a39640 1 -- start start 2024-09-12T03:19:34.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.056+0000 7fe444a39640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe440102d30 0x7fe440100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.056+0000 7fe444a39640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4401036e0 0x7fe440100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.056+0000 7fe444a39640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4401082a0 0x7fe440101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.057+0000 7fe444a39640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe440112160 con 0x7fe4401036e0 2024-09-12T03:19:34.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.057+0000 7fe444a39640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe440111fe0 con 0x7fe440102d30 2024-09-12T03:19:34.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.057+0000 7fe444a39640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401122e0 con 0x7fe4401082a0 2024-09-12T03:19:34.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.057+0000 7fe437fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4401036e0 0x7fe440100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.057+0000 7fe437fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4401036e0 0x7fe440100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59292/0 (socket says 172.21.15.6:59292) 2024-09-12T03:19:34.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.057+0000 7fe437fff640 1 -- 172.21.15.6:0/2765952887 learned_addr learned my addr 172.21.15.6:0/2765952887 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:34.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.057+0000 7fe43e59c640 1 --2- 172.21.15.6:0/2765952887 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe440102d30 0x7fe440100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.057+0000 7fe43ed9d640 1 --2- 172.21.15.6:0/2765952887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4401082a0 0x7fe440101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.057+0000 7fe437fff640 1 -- 172.21.15.6:0/2765952887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4401082a0 msgr2=0x7fe440101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:34.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.057+0000 7fe437fff640 1 --2- 172.21.15.6:0/2765952887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4401082a0 0x7fe440101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.057+0000 7fe437fff640 1 -- 172.21.15.6:0/2765952887 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe440102d30 msgr2=0x7fe440100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:34.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.057+0000 7fe437fff640 1 --2- 172.21.15.6:0/2765952887 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe440102d30 0x7fe440100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.057+0000 7fe437fff640 1 -- 172.21.15.6:0/2765952887 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4401a6c00 con 0x7fe4401036e0 2024-09-12T03:19:34.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.057+0000 7fe437fff640 1 --2- 172.21.15.6:0/2765952887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4401036e0 0x7fe440100ed0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe428006fd0 tx=0x7fe42800b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:34.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.057+0000 7fe4377fe640 1 -- 172.21.15.6:0/2765952887 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe428015020 con 0x7fe4401036e0 2024-09-12T03:19:34.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.057+0000 7fe4377fe640 1 -- 172.21.15.6:0/2765952887 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe428011d10 con 0x7fe4401036e0 2024-09-12T03:19:34.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.057+0000 7fe444a39640 1 -- 172.21.15.6:0/2765952887 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4401a6ef0 con 0x7fe4401036e0 2024-09-12T03:19:34.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.057+0000 7fe4377fe640 1 -- 172.21.15.6:0/2765952887 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42800f2e0 con 0x7fe4401036e0 2024-09-12T03:19:34.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.057+0000 7fe444a39640 1 -- 172.21.15.6:0/2765952887 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4401a7350 con 0x7fe4401036e0 2024-09-12T03:19:34.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.058+0000 7fe444a39640 1 -- 172.21.15.6:0/2765952887 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe440103190 con 0x7fe4401036e0 2024-09-12T03:19:34.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.060+0000 7fe4377fe640 1 -- 172.21.15.6:0/2765952887 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe428018020 con 0x7fe4401036e0 2024-09-12T03:19:34.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.060+0000 7fe4377fe640 1 --2- 172.21.15.6:0/2765952887 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe42c0a2c10 0x7fe42c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.061+0000 7fe43e59c640 1 --2- 172.21.15.6:0/2765952887 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe42c0a2c10 0x7fe42c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.061+0000 7fe4377fe640 1 -- 172.21.15.6:0/2765952887 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe4280ceed0 con 0x7fe4401036e0 2024-09-12T03:19:34.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.061+0000 7fe43e59c640 1 --2- 172.21.15.6:0/2765952887 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe42c0a2c10 0x7fe42c0a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe430009990 tx=0x7fe430009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:34.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.067+0000 7fe4377fe640 1 -- 172.21.15.6:0/2765952887 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe42801f020 con 0x7fe4401036e0 2024-09-12T03:19:34.244 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:34 smithi006 ceph-mon[30991]: pgmap v289: 97 pgs: 97 active+clean; 630 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.5 KiB/s wr, 13 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T03:19:34.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.244+0000 7fe444a39640 1 -- 172.21.15.6:0/2765952887 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe440062630 con 0x7fe42c0a2c10 2024-09-12T03:19:34.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.251+0000 7fe4377fe640 1 -- 172.21.15.6:0/2765952887 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258722 (secure 0 0 0) 0x7fe440062630 con 0x7fe42c0a2c10 2024-09-12T03:19:34.254 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:19:34.255 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-09-12T03:19:34.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.254+0000 7fe444a39640 1 -- 172.21.15.6:0/2765952887 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe42c0a2c10 msgr2=0x7fe42c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.254+0000 7fe444a39640 1 --2- 172.21.15.6:0/2765952887 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe42c0a2c10 0x7fe42c0a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe430009990 tx=0x7fe430009110 comp rx=0 tx=0).stop 2024-09-12T03:19:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.254+0000 7fe444a39640 1 -- 172.21.15.6:0/2765952887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4401036e0 msgr2=0x7fe440100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.254+0000 7fe444a39640 1 --2- 172.21.15.6:0/2765952887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4401036e0 0x7fe440100ed0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe428006fd0 tx=0x7fe42800b820 comp rx=0 tx=0).stop 2024-09-12T03:19:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.254+0000 7fe444a39640 1 -- 172.21.15.6:0/2765952887 shutdown_connections 2024-09-12T03:19:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.254+0000 7fe444a39640 1 --2- 172.21.15.6:0/2765952887 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe42c0a2c10 0x7fe42c0a50d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.254+0000 7fe444a39640 1 --2- 172.21.15.6:0/2765952887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4401082a0 0x7fe440101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.254+0000 7fe444a39640 1 --2- 172.21.15.6:0/2765952887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4401036e0 0x7fe440100ed0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.254+0000 7fe444a39640 1 --2- 172.21.15.6:0/2765952887 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe440102d30 0x7fe440100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.254+0000 7fe444a39640 1 -- 172.21.15.6:0/2765952887 >> 172.21.15.6:0/2765952887 conn(0x7fe4400fc7d0 msgr2=0x7fe440106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.254+0000 7fe444a39640 1 -- 172.21.15.6:0/2765952887 shutdown_connections 2024-09-12T03:19:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:34.254+0000 7fe444a39640 1 -- 172.21.15.6:0/2765952887 wait complete. 2024-09-12T03:19:34.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:34 smithi118 ceph-mon[35048]: pgmap v289: 97 pgs: 97 active+clean; 630 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.5 KiB/s wr, 13 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T03:19:34.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:34 smithi161 ceph-mon[35485]: pgmap v289: 97 pgs: 97 active+clean; 630 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.5 KiB/s wr, 13 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T03:19:34.903 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-12T03:19:33.241786+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644717,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":229,"num_read_kb":219,"num_write":334,"num_write_kb":3036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270,"ondisk_log_size":270,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382560,"kb_used_data":12492,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559264,"statfs":{"total":1151940427776,"available":1151548686336,"internally_reserved":0,"allocated":12791808,"data_stored":7109309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":50533,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":97,"num_read_kb":106,"num_write":69,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002903"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.371249+0000","last_change":"2024-09-12T03:19:09.343829+0000","last_active":"2024-09-12T03:19:20.371249+0000","last_peered":"2024-09-12T03:19:20.371249+0000","last_clean":"2024-09-12T03:19:20.371249+0000","last_became_active":"2024-09-12T03:19:09.343228+0000","last_became_peered":"2024-09-12T03:19:09.343228+0000","last_unstale":"2024-09-12T03:19:20.371249+0000","last_undegraded":"2024-09-12T03:19:20.371249+0000","last_fullsized":"2024-09-12T03:19:20.371249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:08.720934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408528+0000","last_change":"2024-09-12T03:19:09.331829+0000","last_active":"2024-09-12T03:19:20.408528+0000","last_peered":"2024-09-12T03:19:20.408528+0000","last_clean":"2024-09-12T03:19:20.408528+0000","last_became_active":"2024-09-12T03:19:09.331276+0000","last_became_peered":"2024-09-12T03:19:09.331276+0000","last_unstale":"2024-09-12T03:19:20.408528+0000","last_undegraded":"2024-09-12T03:19:20.408528+0000","last_fullsized":"2024-09-12T03:19:20.408528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:48.672085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408509+0000","last_change":"2024-09-12T03:19:09.338120+0000","last_active":"2024-09-12T03:19:20.408509+0000","last_peered":"2024-09-12T03:19:20.408509+0000","last_clean":"2024-09-12T03:19:20.408509+0000","last_became_active":"2024-09-12T03:19:09.337773+0000","last_became_peered":"2024-09-12T03:19:09.337773+0000","last_unstale":"2024-09-12T03:19:20.408509+0000","last_undegraded":"2024-09-12T03:19:20.408509+0000","last_fullsized":"2024-09-12T03:19:20.408509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:13.716047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382087+0000","last_change":"2024-09-12T03:19:09.339404+0000","last_active":"2024-09-12T03:19:20.382087+0000","last_peered":"2024-09-12T03:19:20.382087+0000","last_clean":"2024-09-12T03:19:20.382087+0000","last_became_active":"2024-09-12T03:19:09.339105+0000","last_became_peered":"2024-09-12T03:19:09.339105+0000","last_unstale":"2024-09-12T03:19:20.382087+0000","last_undegraded":"2024-09-12T03:19:20.382087+0000","last_fullsized":"2024-09-12T03:19:20.382087+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:37.930963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373401+0000","last_change":"2024-09-12T03:19:09.326285+0000","last_active":"2024-09-12T03:19:20.373401+0000","last_peered":"2024-09-12T03:19:20.373401+0000","last_clean":"2024-09-12T03:19:20.373401+0000","last_became_active":"2024-09-12T03:19:09.325852+0000","last_became_peered":"2024-09-12T03:19:09.325852+0000","last_unstale":"2024-09-12T03:19:20.373401+0000","last_undegraded":"2024-09-12T03:19:20.373401+0000","last_fullsized":"2024-09-12T03:19:20.373401+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:44:25.387014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.376197+0000","last_change":"2024-09-12T03:19:20.376197+0000","last_active":"2024-09-12T03:19:20.376197+0000","last_peered":"2024-09-12T03:19:20.376197+0000","last_clean":"2024-09-12T03:19:20.376197+0000","last_became_active":"2024-09-12T03:19:20.375744+0000","last_became_peered":"2024-09-12T03:19:20.375744+0000","last_unstale":"2024-09-12T03:19:20.376197+0000","last_undegraded":"2024-09-12T03:19:20.376197+0000","last_fullsized":"2024-09-12T03:19:20.376197+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:08.663121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372152+0000","last_change":"2024-09-12T03:19:09.335284+0000","last_active":"2024-09-12T03:19:20.372152+0000","last_peered":"2024-09-12T03:19:20.372152+0000","last_clean":"2024-09-12T03:19:20.372152+0000","last_became_active":"2024-09-12T03:19:09.334948+0000","last_became_peered":"2024-09-12T03:19:09.334948+0000","last_unstale":"2024-09-12T03:19:20.372152+0000","last_undegraded":"2024-09-12T03:19:20.372152+0000","last_fullsized":"2024-09-12T03:19:20.372152+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:32.778433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372122+0000","last_change":"2024-09-12T03:19:09.338398+0000","last_active":"2024-09-12T03:19:20.372122+0000","last_peered":"2024-09-12T03:19:20.372122+0000","last_clean":"2024-09-12T03:19:20.372122+0000","last_became_active":"2024-09-12T03:19:09.337750+0000","last_became_peered":"2024-09-12T03:19:09.337750+0000","last_unstale":"2024-09-12T03:19:20.372122+0000","last_undegraded":"2024-09-12T03:19:20.372122+0000","last_fullsized":"2024-09-12T03:19:20.372122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:07.542322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370699+0000","last_change":"2024-09-12T03:19:09.340703+0000","last_active":"2024-09-12T03:19:20.370699+0000","last_peered":"2024-09-12T03:19:20.370699+0000","last_clean":"2024-09-12T03:19:20.370699+0000","last_became_active":"2024-09-12T03:19:09.340386+0000","last_became_peered":"2024-09-12T03:19:09.340386+0000","last_unstale":"2024-09-12T03:19:20.370699+0000","last_undegraded":"2024-09-12T03:19:20.370699+0000","last_fullsized":"2024-09-12T03:19:20.370699+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:19.990894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374331+0000","last_change":"2024-09-12T03:19:09.341489+0000","last_active":"2024-09-12T03:19:20.374331+0000","last_peered":"2024-09-12T03:19:20.374331+0000","last_clean":"2024-09-12T03:19:20.374331+0000","last_became_active":"2024-09-12T03:19:09.341183+0000","last_became_peered":"2024-09-12T03:19:09.341183+0000","last_unstale":"2024-09-12T03:19:20.374331+0000","last_undegraded":"2024-09-12T03:19:20.374331+0000","last_fullsized":"2024-09-12T03:19:20.374331+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:05.688402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372589+0000","last_change":"2024-09-12T03:19:09.339289+0000","last_active":"2024-09-12T03:19:20.372589+0000","last_peered":"2024-09-12T03:19:20.372589+0000","last_clean":"2024-09-12T03:19:20.372589+0000","last_became_active":"2024-09-12T03:19:09.338466+0000","last_became_peered":"2024-09-12T03:19:09.338466+0000","last_unstale":"2024-09-12T03:19:20.372589+0000","last_undegraded":"2024-09-12T03:19:20.372589+0000","last_fullsized":"2024-09-12T03:19:20.372589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:47:13.070057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382092+0000","last_change":"2024-09-12T03:19:09.337754+0000","last_active":"2024-09-12T03:19:20.382092+0000","last_peered":"2024-09-12T03:19:20.382092+0000","last_clean":"2024-09-12T03:19:20.382092+0000","last_became_active":"2024-09-12T03:19:09.332789+0000","last_became_peered":"2024-09-12T03:19:09.332789+0000","last_unstale":"2024-09-12T03:19:20.382092+0000","last_undegraded":"2024-09-12T03:19:20.382092+0000","last_fullsized":"2024-09-12T03:19:20.382092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:34.622136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375297+0000","last_change":"2024-09-12T03:19:09.333577+0000","last_active":"2024-09-12T03:19:20.375297+0000","last_peered":"2024-09-12T03:19:20.375297+0000","last_clean":"2024-09-12T03:19:20.375297+0000","last_became_active":"2024-09-12T03:19:09.333234+0000","last_became_peered":"2024-09-12T03:19:09.333234+0000","last_unstale":"2024-09-12T03:19:20.375297+0000","last_undegraded":"2024-09-12T03:19:20.375297+0000","last_fullsized":"2024-09-12T03:19:20.375297+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:22.604640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408267+0000","last_change":"2024-09-12T03:19:09.331607+0000","last_active":"2024-09-12T03:19:20.408267+0000","last_peered":"2024-09-12T03:19:20.408267+0000","last_clean":"2024-09-12T03:19:20.408267+0000","last_became_active":"2024-09-12T03:19:09.331253+0000","last_became_peered":"2024-09-12T03:19:09.331253+0000","last_unstale":"2024-09-12T03:19:20.408267+0000","last_undegraded":"2024-09-12T03:19:20.408267+0000","last_fullsized":"2024-09-12T03:19:20.408267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:11.969265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408245+0000","last_change":"2024-09-12T03:19:09.340768+0000","last_active":"2024-09-12T03:19:20.408245+0000","last_peered":"2024-09-12T03:19:20.408245+0000","last_clean":"2024-09-12T03:19:20.408245+0000","last_became_active":"2024-09-12T03:19:09.340437+0000","last_became_peered":"2024-09-12T03:19:09.340437+0000","last_unstale":"2024-09-12T03:19:20.408245+0000","last_undegraded":"2024-09-12T03:19:20.408245+0000","last_fullsized":"2024-09-12T03:19:20.408245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:32.645691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370081+0000","last_change":"2024-09-12T03:19:09.333829+0000","last_active":"2024-09-12T03:19:20.370081+0000","last_peered":"2024-09-12T03:19:20.370081+0000","last_clean":"2024-09-12T03:19:20.370081+0000","last_became_active":"2024-09-12T03:19:09.333462+0000","last_became_peered":"2024-09-12T03:19:09.333462+0000","last_unstale":"2024-09-12T03:19:20.370081+0000","last_undegraded":"2024-09-12T03:19:20.370081+0000","last_fullsized":"2024-09-12T03:19:20.370081+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:18:57.586836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.371273+0000","last_change":"2024-09-12T03:19:09.344068+0000","last_active":"2024-09-12T03:19:20.371273+0000","last_peered":"2024-09-12T03:19:20.371273+0000","last_clean":"2024-09-12T03:19:20.371273+0000","last_became_active":"2024-09-12T03:19:09.343751+0000","last_became_peered":"2024-09-12T03:19:09.343751+0000","last_unstale":"2024-09-12T03:19:20.371273+0000","last_undegraded":"2024-09-12T03:19:20.371273+0000","last_fullsized":"2024-09-12T03:19:20.371273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:52.425547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372644+0000","last_change":"2024-09-12T03:19:09.330828+0000","last_active":"2024-09-12T03:19:20.372644+0000","last_peered":"2024-09-12T03:19:20.372644+0000","last_clean":"2024-09-12T03:19:20.372644+0000","last_became_active":"2024-09-12T03:19:09.330497+0000","last_became_peered":"2024-09-12T03:19:09.330497+0000","last_unstale":"2024-09-12T03:19:20.372644+0000","last_undegraded":"2024-09-12T03:19:20.372644+0000","last_fullsized":"2024-09-12T03:19:20.372644+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:21:20.064177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375155+0000","last_change":"2024-09-12T03:19:09.344403+0000","last_active":"2024-09-12T03:19:20.375155+0000","last_peered":"2024-09-12T03:19:20.375155+0000","last_clean":"2024-09-12T03:19:20.375155+0000","last_became_active":"2024-09-12T03:19:09.343946+0000","last_became_peered":"2024-09-12T03:19:09.343946+0000","last_unstale":"2024-09-12T03:19:20.375155+0000","last_undegraded":"2024-09-12T03:19:20.375155+0000","last_fullsized":"2024-09-12T03:19:20.375155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:48.386117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370884+0000","last_change":"2024-09-12T03:19:09.341481+0000","last_active":"2024-09-12T03:19:20.370884+0000","last_peered":"2024-09-12T03:19:20.370884+0000","last_clean":"2024-09-12T03:19:20.370884+0000","last_became_active":"2024-09-12T03:19:09.341142+0000","last_became_peered":"2024-09-12T03:19:09.341142+0000","last_unstale":"2024-09-12T03:19:20.370884+0000","last_undegraded":"2024-09-12T03:19:20.370884+0000","last_fullsized":"2024-09-12T03:19:20.370884+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:42.371221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375324+0000","last_change":"2024-09-12T03:19:09.330621+0000","last_active":"2024-09-12T03:19:20.375324+0000","last_peered":"2024-09-12T03:19:20.375324+0000","last_clean":"2024-09-12T03:19:20.375324+0000","last_became_active":"2024-09-12T03:19:09.330169+0000","last_became_peered":"2024-09-12T03:19:09.330169+0000","last_unstale":"2024-09-12T03:19:20.375324+0000","last_undegraded":"2024-09-12T03:19:20.375324+0000","last_fullsized":"2024-09-12T03:19:20.375324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:16.634141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370190+0000","last_change":"2024-09-12T03:19:09.332817+0000","last_active":"2024-09-12T03:19:20.370190+0000","last_peered":"2024-09-12T03:19:20.370190+0000","last_clean":"2024-09-12T03:19:20.370190+0000","last_became_active":"2024-09-12T03:19:09.332236+0000","last_became_peered":"2024-09-12T03:19:09.332236+0000","last_unstale":"2024-09-12T03:19:20.370190+0000","last_undegraded":"2024-09-12T03:19:20.370190+0000","last_fullsized":"2024-09-12T03:19:20.370190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:28.024294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372100+0000","last_change":"2024-09-12T03:19:09.337910+0000","last_active":"2024-09-12T03:19:20.372100+0000","last_peered":"2024-09-12T03:19:20.372100+0000","last_clean":"2024-09-12T03:19:20.372100+0000","last_became_active":"2024-09-12T03:19:09.337176+0000","last_became_peered":"2024-09-12T03:19:09.337176+0000","last_unstale":"2024-09-12T03:19:20.372100+0000","last_undegraded":"2024-09-12T03:19:20.372100+0000","last_fullsized":"2024-09-12T03:19:20.372100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:53.795898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.385889+0000","last_change":"2024-09-12T03:19:09.337902+0000","last_active":"2024-09-12T03:19:20.385889+0000","last_peered":"2024-09-12T03:19:20.385889+0000","last_clean":"2024-09-12T03:19:20.385889+0000","last_became_active":"2024-09-12T03:19:09.337567+0000","last_became_peered":"2024-09-12T03:19:09.337567+0000","last_unstale":"2024-09-12T03:19:20.385889+0000","last_undegraded":"2024-09-12T03:19:20.385889+0000","last_fullsized":"2024-09-12T03:19:20.385889+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:08.471313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372957+0000","last_change":"2024-09-12T03:19:09.339850+0000","last_active":"2024-09-12T03:19:20.372957+0000","last_peered":"2024-09-12T03:19:20.372957+0000","last_clean":"2024-09-12T03:19:20.372957+0000","last_became_active":"2024-09-12T03:19:09.339563+0000","last_became_peered":"2024-09-12T03:19:09.339563+0000","last_unstale":"2024-09-12T03:19:20.372957+0000","last_undegraded":"2024-09-12T03:19:20.372957+0000","last_fullsized":"2024-09-12T03:19:20.372957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:27.473351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373965+0000","last_change":"2024-09-12T03:19:09.330555+0000","last_active":"2024-09-12T03:19:20.373965+0000","last_peered":"2024-09-12T03:19:20.373965+0000","last_clean":"2024-09-12T03:19:20.373965+0000","last_became_active":"2024-09-12T03:19:09.330183+0000","last_became_peered":"2024-09-12T03:19:09.330183+0000","last_unstale":"2024-09-12T03:19:20.373965+0000","last_undegraded":"2024-09-12T03:19:20.373965+0000","last_fullsized":"2024-09-12T03:19:20.373965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:39.334116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.371187+0000","last_change":"2024-09-12T03:19:09.338601+0000","last_active":"2024-09-12T03:19:20.371187+0000","last_peered":"2024-09-12T03:19:20.371187+0000","last_clean":"2024-09-12T03:19:20.371187+0000","last_became_active":"2024-09-12T03:19:09.337903+0000","last_became_peered":"2024-09-12T03:19:09.337903+0000","last_unstale":"2024-09-12T03:19:20.371187+0000","last_undegraded":"2024-09-12T03:19:20.371187+0000","last_fullsized":"2024-09-12T03:19:20.371187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:17.810139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375192+0000","last_change":"2024-09-12T03:19:09.326629+0000","last_active":"2024-09-12T03:19:20.375192+0000","last_peered":"2024-09-12T03:19:20.375192+0000","last_clean":"2024-09-12T03:19:20.375192+0000","last_became_active":"2024-09-12T03:19:09.326333+0000","last_became_peered":"2024-09-12T03:19:09.326333+0000","last_unstale":"2024-09-12T03:19:20.375192+0000","last_undegraded":"2024-09-12T03:19:20.375192+0000","last_fullsized":"2024-09-12T03:19:20.375192+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:20.807593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408435+0000","last_change":"2024-09-12T03:19:09.339919+0000","last_active":"2024-09-12T03:19:20.408435+0000","last_peered":"2024-09-12T03:19:20.408435+0000","last_clean":"2024-09-12T03:19:20.408435+0000","last_became_active":"2024-09-12T03:19:09.339618+0000","last_became_peered":"2024-09-12T03:19:09.339618+0000","last_unstale":"2024-09-12T03:19:20.408435+0000","last_undegraded":"2024-09-12T03:19:20.408435+0000","last_fullsized":"2024-09-12T03:19:20.408435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:11:03.740079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374805+0000","last_change":"2024-09-12T03:19:09.336148+0000","last_active":"2024-09-12T03:19:20.374805+0000","last_peered":"2024-09-12T03:19:20.374805+0000","last_clean":"2024-09-12T03:19:20.374805+0000","last_became_active":"2024-09-12T03:19:09.335779+0000","last_became_peered":"2024-09-12T03:19:09.335779+0000","last_unstale":"2024-09-12T03:19:20.374805+0000","last_undegraded":"2024-09-12T03:19:20.374805+0000","last_fullsized":"2024-09-12T03:19:20.374805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:18.234320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382536+0000","last_change":"2024-09-12T03:19:20.382536+0000","last_active":"2024-09-12T03:19:20.382536+0000","last_peered":"2024-09-12T03:19:20.382536+0000","last_clean":"2024-09-12T03:19:20.382536+0000","last_became_active":"2024-09-12T03:19:20.382118+0000","last_became_peered":"2024-09-12T03:19:20.382118+0000","last_unstale":"2024-09-12T03:19:20.382536+0000","last_undegraded":"2024-09-12T03:19:20.382536+0000","last_fullsized":"2024-09-12T03:19:20.382536+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:55.836646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370826+0000","last_change":"2024-09-12T03:19:09.345064+0000","last_active":"2024-09-12T03:19:20.370826+0000","last_peered":"2024-09-12T03:19:20.370826+0000","last_clean":"2024-09-12T03:19:20.370826+0000","last_became_active":"2024-09-12T03:19:09.344628+0000","last_became_peered":"2024-09-12T03:19:09.344628+0000","last_unstale":"2024-09-12T03:19:20.370826+0000","last_undegraded":"2024-09-12T03:19:20.370826+0000","last_fullsized":"2024-09-12T03:19:20.370826+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:50.011751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373663+0000","last_change":"2024-09-12T03:19:09.338052+0000","last_active":"2024-09-12T03:19:20.373663+0000","last_peered":"2024-09-12T03:19:20.373663+0000","last_clean":"2024-09-12T03:19:20.373663+0000","last_became_active":"2024-09-12T03:19:09.337710+0000","last_became_peered":"2024-09-12T03:19:09.337710+0000","last_unstale":"2024-09-12T03:19:20.373663+0000","last_undegraded":"2024-09-12T03:19:20.373663+0000","last_fullsized":"2024-09-12T03:19:20.373663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:36.521269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:29.397897+0000","last_change":"2024-09-12T03:19:07.208341+0000","last_active":"2024-09-12T03:19:29.397897+0000","last_peered":"2024-09-12T03:19:29.397897+0000","last_clean":"2024-09-12T03:19:29.397897+0000","last_became_active":"2024-09-12T03:19:07.207385+0000","last_became_peered":"2024-09-12T03:19:07.207385+0000","last_unstale":"2024-09-12T03:19:29.397897+0000","last_undegraded":"2024-09-12T03:19:29.397897+0000","last_fullsized":"2024-09-12T03:19:29.397897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:18.476795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386347+0000","last_change":"2024-09-12T03:19:09.339997+0000","last_active":"2024-09-12T03:19:20.386347+0000","last_peered":"2024-09-12T03:19:20.386347+0000","last_clean":"2024-09-12T03:19:20.386347+0000","last_became_active":"2024-09-12T03:19:09.339261+0000","last_became_peered":"2024-09-12T03:19:09.339261+0000","last_unstale":"2024-09-12T03:19:20.386347+0000","last_undegraded":"2024-09-12T03:19:20.386347+0000","last_fullsized":"2024-09-12T03:19:20.386347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:07.670345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374030+0000","last_change":"2024-09-12T03:19:07.200222+0000","last_active":"2024-09-12T03:19:20.374030+0000","last_peered":"2024-09-12T03:19:20.374030+0000","last_clean":"2024-09-12T03:19:20.374030+0000","last_became_active":"2024-09-12T03:19:07.199819+0000","last_became_peered":"2024-09-12T03:19:07.199819+0000","last_unstale":"2024-09-12T03:19:20.374030+0000","last_undegraded":"2024-09-12T03:19:20.374030+0000","last_fullsized":"2024-09-12T03:19:20.374030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:39:28.933609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375686+0000","last_change":"2024-09-12T03:19:07.207005+0000","last_active":"2024-09-12T03:19:20.375686+0000","last_peered":"2024-09-12T03:19:20.375686+0000","last_clean":"2024-09-12T03:19:20.375686+0000","last_became_active":"2024-09-12T03:19:07.206685+0000","last_became_peered":"2024-09-12T03:19:07.206685+0000","last_unstale":"2024-09-12T03:19:20.375686+0000","last_undegraded":"2024-09-12T03:19:20.375686+0000","last_fullsized":"2024-09-12T03:19:20.375686+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:31.111706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372028+0000","last_change":"2024-09-12T03:19:09.335871+0000","last_active":"2024-09-12T03:19:20.372028+0000","last_peered":"2024-09-12T03:19:20.372028+0000","last_clean":"2024-09-12T03:19:20.372028+0000","last_became_active":"2024-09-12T03:19:09.335568+0000","last_became_peered":"2024-09-12T03:19:09.335568+0000","last_unstale":"2024-09-12T03:19:20.372028+0000","last_undegraded":"2024-09-12T03:19:20.372028+0000","last_fullsized":"2024-09-12T03:19:20.372028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:52:34.471512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386049+0000","last_change":"2024-09-12T03:19:07.208320+0000","last_active":"2024-09-12T03:19:20.386049+0000","last_peered":"2024-09-12T03:19:20.386049+0000","last_clean":"2024-09-12T03:19:20.386049+0000","last_became_active":"2024-09-12T03:19:07.207873+0000","last_became_peered":"2024-09-12T03:19:07.207873+0000","last_unstale":"2024-09-12T03:19:20.386049+0000","last_undegraded":"2024-09-12T03:19:20.386049+0000","last_fullsized":"2024-09-12T03:19:20.386049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:00.128595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372467+0000","last_change":"2024-09-12T03:19:09.331930+0000","last_active":"2024-09-12T03:19:20.372467+0000","last_peered":"2024-09-12T03:19:20.372467+0000","last_clean":"2024-09-12T03:19:20.372467+0000","last_became_active":"2024-09-12T03:19:09.331558+0000","last_became_peered":"2024-09-12T03:19:09.331558+0000","last_unstale":"2024-09-12T03:19:20.372467+0000","last_undegraded":"2024-09-12T03:19:20.372467+0000","last_fullsized":"2024-09-12T03:19:20.372467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:00:27.935717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:31.399745+0000","last_change":"2024-09-12T03:19:07.205473+0000","last_active":"2024-09-12T03:19:31.399745+0000","last_peered":"2024-09-12T03:19:31.399745+0000","last_clean":"2024-09-12T03:19:31.399745+0000","last_became_active":"2024-09-12T03:19:07.205134+0000","last_became_peered":"2024-09-12T03:19:07.205134+0000","last_unstale":"2024-09-12T03:19:31.399745+0000","last_undegraded":"2024-09-12T03:19:31.399745+0000","last_fullsized":"2024-09-12T03:19:31.399745+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:23.443957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370511+0000","last_change":"2024-09-12T03:19:09.333868+0000","last_active":"2024-09-12T03:19:20.370511+0000","last_peered":"2024-09-12T03:19:20.370511+0000","last_clean":"2024-09-12T03:19:20.370511+0000","last_became_active":"2024-09-12T03:19:09.333484+0000","last_became_peered":"2024-09-12T03:19:09.333484+0000","last_unstale":"2024-09-12T03:19:20.370511+0000","last_undegraded":"2024-09-12T03:19:20.370511+0000","last_fullsized":"2024-09-12T03:19:20.370511+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:21:10.226821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386303+0000","last_change":"2024-09-12T03:19:20.386303+0000","last_active":"2024-09-12T03:19:20.386303+0000","last_peered":"2024-09-12T03:19:20.386303+0000","last_clean":"2024-09-12T03:19:20.386303+0000","last_became_active":"2024-09-12T03:19:20.383198+0000","last_became_peered":"2024-09-12T03:19:20.383198+0000","last_unstale":"2024-09-12T03:19:20.386303+0000","last_undegraded":"2024-09-12T03:19:20.386303+0000","last_fullsized":"2024-09-12T03:19:20.386303+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:42.224849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408655+0000","last_change":"2024-09-12T03:19:09.340849+0000","last_active":"2024-09-12T03:19:20.408655+0000","last_peered":"2024-09-12T03:19:20.408655+0000","last_clean":"2024-09-12T03:19:20.408655+0000","last_became_active":"2024-09-12T03:19:09.340517+0000","last_became_peered":"2024-09-12T03:19:09.340517+0000","last_unstale":"2024-09-12T03:19:20.408655+0000","last_undegraded":"2024-09-12T03:19:20.408655+0000","last_fullsized":"2024-09-12T03:19:20.408655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:42.018269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386240+0000","last_change":"2024-09-12T03:19:07.389147+0000","last_active":"2024-09-12T03:19:20.386240+0000","last_peered":"2024-09-12T03:19:20.386240+0000","last_clean":"2024-09-12T03:19:20.386240+0000","last_became_active":"2024-09-12T03:19:07.388744+0000","last_became_peered":"2024-09-12T03:19:07.388744+0000","last_unstale":"2024-09-12T03:19:20.386240+0000","last_undegraded":"2024-09-12T03:19:20.386240+0000","last_fullsized":"2024-09-12T03:19:20.386240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:16.700644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370280+0000","last_change":"2024-09-12T03:19:09.329243+0000","last_active":"2024-09-12T03:19:20.370280+0000","last_peered":"2024-09-12T03:19:20.370280+0000","last_clean":"2024-09-12T03:19:20.370280+0000","last_became_active":"2024-09-12T03:19:09.328610+0000","last_became_peered":"2024-09-12T03:19:09.328610+0000","last_unstale":"2024-09-12T03:19:20.370280+0000","last_undegraded":"2024-09-12T03:19:20.370280+0000","last_fullsized":"2024-09-12T03:19:20.370280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:49.452942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'25","reported_seq":80,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:32.401687+0000","last_change":"2024-09-12T03:19:07.207804+0000","last_active":"2024-09-12T03:19:32.401687+0000","last_peered":"2024-09-12T03:19:32.401687+0000","last_clean":"2024-09-12T03:19:32.401687+0000","last_became_active":"2024-09-12T03:19:07.207409+0000","last_became_peered":"2024-09-12T03:19:07.207409+0000","last_unstale":"2024-09-12T03:19:32.401687+0000","last_undegraded":"2024-09-12T03:19:32.401687+0000","last_fullsized":"2024-09-12T03:19:32.401687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:10.732423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50112,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":64,"num_write":26,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373666+0000","last_change":"2024-09-12T03:19:09.333102+0000","last_active":"2024-09-12T03:19:20.373666+0000","last_peered":"2024-09-12T03:19:20.373666+0000","last_clean":"2024-09-12T03:19:20.373666+0000","last_became_active":"2024-09-12T03:19:09.332712+0000","last_became_peered":"2024-09-12T03:19:09.332712+0000","last_unstale":"2024-09-12T03:19:20.373666+0000","last_undegraded":"2024-09-12T03:19:20.373666+0000","last_fullsized":"2024-09-12T03:19:20.373666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:22.912213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382902+0000","last_change":"2024-09-12T03:19:20.382348+0000","last_active":"2024-09-12T03:19:20.382902+0000","last_peered":"2024-09-12T03:19:20.382902+0000","last_clean":"2024-09-12T03:19:20.382902+0000","last_became_active":"2024-09-12T03:19:20.375142+0000","last_became_peered":"2024-09-12T03:19:20.375142+0000","last_unstale":"2024-09-12T03:19:20.382902+0000","last_undegraded":"2024-09-12T03:19:20.382902+0000","last_fullsized":"2024-09-12T03:19:20.382902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:04.573599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375046+0000","last_change":"2024-09-12T03:19:09.333194+0000","last_active":"2024-09-12T03:19:20.375046+0000","last_peered":"2024-09-12T03:19:20.375046+0000","last_clean":"2024-09-12T03:19:20.375046+0000","last_became_active":"2024-09-12T03:19:09.332865+0000","last_became_peered":"2024-09-12T03:19:09.332865+0000","last_unstale":"2024-09-12T03:19:20.375046+0000","last_undegraded":"2024-09-12T03:19:20.375046+0000","last_fullsized":"2024-09-12T03:19:20.375046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:57.096138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386176+0000","last_change":"2024-09-12T03:19:07.209289+0000","last_active":"2024-09-12T03:19:20.386176+0000","last_peered":"2024-09-12T03:19:20.386176+0000","last_clean":"2024-09-12T03:19:20.386176+0000","last_became_active":"2024-09-12T03:19:07.208863+0000","last_became_peered":"2024-09-12T03:19:07.208863+0000","last_unstale":"2024-09-12T03:19:20.386176+0000","last_undegraded":"2024-09-12T03:19:20.386176+0000","last_fullsized":"2024-09-12T03:19:20.386176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:26.168148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372400+0000","last_change":"2024-09-12T03:19:09.343620+0000","last_active":"2024-09-12T03:19:20.372400+0000","last_peered":"2024-09-12T03:19:20.372400+0000","last_clean":"2024-09-12T03:19:20.372400+0000","last_became_active":"2024-09-12T03:19:09.343299+0000","last_became_peered":"2024-09-12T03:19:09.343299+0000","last_unstale":"2024-09-12T03:19:20.372400+0000","last_undegraded":"2024-09-12T03:19:20.372400+0000","last_fullsized":"2024-09-12T03:19:20.372400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:34.268886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:30.397086+0000","last_change":"2024-09-12T03:19:07.206160+0000","last_active":"2024-09-12T03:19:30.397086+0000","last_peered":"2024-09-12T03:19:30.397086+0000","last_clean":"2024-09-12T03:19:30.397086+0000","last_became_active":"2024-09-12T03:19:07.205509+0000","last_became_peered":"2024-09-12T03:19:07.205509+0000","last_unstale":"2024-09-12T03:19:30.397086+0000","last_undegraded":"2024-09-12T03:19:30.397086+0000","last_fullsized":"2024-09-12T03:19:30.397086+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:35.362831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372068+0000","last_change":"2024-09-12T03:19:09.337595+0000","last_active":"2024-09-12T03:19:20.372068+0000","last_peered":"2024-09-12T03:19:20.372068+0000","last_clean":"2024-09-12T03:19:20.372068+0000","last_became_active":"2024-09-12T03:19:09.337281+0000","last_became_peered":"2024-09-12T03:19:09.337281+0000","last_unstale":"2024-09-12T03:19:20.372068+0000","last_undegraded":"2024-09-12T03:19:20.372068+0000","last_fullsized":"2024-09-12T03:19:20.372068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:35.761679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373995+0000","last_change":"2024-09-12T03:19:07.389336+0000","last_active":"2024-09-12T03:19:20.373995+0000","last_peered":"2024-09-12T03:19:20.373995+0000","last_clean":"2024-09-12T03:19:20.373995+0000","last_became_active":"2024-09-12T03:19:07.388649+0000","last_became_peered":"2024-09-12T03:19:07.388649+0000","last_unstale":"2024-09-12T03:19:20.373995+0000","last_undegraded":"2024-09-12T03:19:20.373995+0000","last_fullsized":"2024-09-12T03:19:20.373995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:33.312700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386030+0000","last_change":"2024-09-12T03:19:09.340674+0000","last_active":"2024-09-12T03:19:20.386030+0000","last_peered":"2024-09-12T03:19:20.386030+0000","last_clean":"2024-09-12T03:19:20.386030+0000","last_became_active":"2024-09-12T03:19:09.340144+0000","last_became_peered":"2024-09-12T03:19:09.340144+0000","last_unstale":"2024-09-12T03:19:20.386030+0000","last_undegraded":"2024-09-12T03:19:20.386030+0000","last_fullsized":"2024-09-12T03:19:20.386030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:21:15.605086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373837+0000","last_change":"2024-09-12T03:19:07.206513+0000","last_active":"2024-09-12T03:19:20.373837+0000","last_peered":"2024-09-12T03:19:20.373837+0000","last_clean":"2024-09-12T03:19:20.373837+0000","last_became_active":"2024-09-12T03:19:07.205974+0000","last_became_peered":"2024-09-12T03:19:07.205974+0000","last_unstale":"2024-09-12T03:19:20.373837+0000","last_undegraded":"2024-09-12T03:19:20.373837+0000","last_fullsized":"2024-09-12T03:19:20.373837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:43:10.002123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.371052+0000","last_change":"2024-09-12T03:19:09.342200+0000","last_active":"2024-09-12T03:19:20.371052+0000","last_peered":"2024-09-12T03:19:20.371052+0000","last_clean":"2024-09-12T03:19:20.371052+0000","last_became_active":"2024-09-12T03:19:09.341901+0000","last_became_peered":"2024-09-12T03:19:09.341901+0000","last_unstale":"2024-09-12T03:19:20.371052+0000","last_undegraded":"2024-09-12T03:19:20.371052+0000","last_fullsized":"2024-09-12T03:19:20.371052+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:27.486214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375373+0000","last_change":"2024-09-12T03:19:07.207514+0000","last_active":"2024-09-12T03:19:20.375373+0000","last_peered":"2024-09-12T03:19:20.375373+0000","last_clean":"2024-09-12T03:19:20.375373+0000","last_became_active":"2024-09-12T03:19:07.205925+0000","last_became_peered":"2024-09-12T03:19:07.205925+0000","last_unstale":"2024-09-12T03:19:20.375373+0000","last_undegraded":"2024-09-12T03:19:20.375373+0000","last_fullsized":"2024-09-12T03:19:20.375373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:20.244703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372696+0000","last_change":"2024-09-12T03:19:09.337837+0000","last_active":"2024-09-12T03:19:20.372696+0000","last_peered":"2024-09-12T03:19:20.372696+0000","last_clean":"2024-09-12T03:19:20.372696+0000","last_became_active":"2024-09-12T03:19:09.337502+0000","last_became_peered":"2024-09-12T03:19:09.337502+0000","last_unstale":"2024-09-12T03:19:20.372696+0000","last_undegraded":"2024-09-12T03:19:20.372696+0000","last_fullsized":"2024-09-12T03:19:20.372696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:30.167139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386356+0000","last_change":"2024-09-12T03:19:07.209130+0000","last_active":"2024-09-12T03:19:20.386356+0000","last_peered":"2024-09-12T03:19:20.386356+0000","last_clean":"2024-09-12T03:19:20.386356+0000","last_became_active":"2024-09-12T03:19:07.208753+0000","last_became_peered":"2024-09-12T03:19:07.208753+0000","last_unstale":"2024-09-12T03:19:20.386356+0000","last_undegraded":"2024-09-12T03:19:20.386356+0000","last_fullsized":"2024-09-12T03:19:20.386356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:34.719292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372500+0000","last_change":"2024-09-12T03:19:09.336382+0000","last_active":"2024-09-12T03:19:20.372500+0000","last_peered":"2024-09-12T03:19:20.372500+0000","last_clean":"2024-09-12T03:19:20.372500+0000","last_became_active":"2024-09-12T03:19:09.336015+0000","last_became_peered":"2024-09-12T03:19:09.336015+0000","last_unstale":"2024-09-12T03:19:20.372500+0000","last_undegraded":"2024-09-12T03:19:20.372500+0000","last_fullsized":"2024-09-12T03:19:20.372500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:19.430849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375044+0000","last_change":"2024-09-12T03:19:07.528567+0000","last_active":"2024-09-12T03:19:20.375044+0000","last_peered":"2024-09-12T03:19:20.375044+0000","last_clean":"2024-09-12T03:19:20.375044+0000","last_became_active":"2024-09-12T03:19:07.528123+0000","last_became_peered":"2024-09-12T03:19:07.528123+0000","last_unstale":"2024-09-12T03:19:20.375044+0000","last_undegraded":"2024-09-12T03:19:20.375044+0000","last_fullsized":"2024-09-12T03:19:20.375044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:34.416714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372773+0000","last_change":"2024-09-12T03:19:09.338836+0000","last_active":"2024-09-12T03:19:20.372773+0000","last_peered":"2024-09-12T03:19:20.372773+0000","last_clean":"2024-09-12T03:19:20.372773+0000","last_became_active":"2024-09-12T03:19:09.337571+0000","last_became_peered":"2024-09-12T03:19:09.337571+0000","last_unstale":"2024-09-12T03:19:20.372773+0000","last_undegraded":"2024-09-12T03:19:20.372773+0000","last_fullsized":"2024-09-12T03:19:20.372773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:39.039878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":329,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372694+0000","last_change":"2024-09-12T03:18:01.051311+0000","last_active":"2024-09-12T03:19:20.372694+0000","last_peered":"2024-09-12T03:19:20.372694+0000","last_clean":"2024-09-12T03:19:20.372694+0000","last_became_active":"2024-09-12T03:18:01.050849+0000","last_became_peered":"2024-09-12T03:18:01.050849+0000","last_unstale":"2024-09-12T03:19:20.372694+0000","last_undegraded":"2024-09-12T03:19:20.372694+0000","last_fullsized":"2024-09-12T03:19:20.372694+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:14:38.096222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:14:38.096222+0000","last_clean_scrub_stamp":"2024-09-12T03:14:38.096222+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:33.067603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382035+0000","last_change":"2024-09-12T03:19:07.206045+0000","last_active":"2024-09-12T03:19:20.382035+0000","last_peered":"2024-09-12T03:19:20.382035+0000","last_clean":"2024-09-12T03:19:20.382035+0000","last_became_active":"2024-09-12T03:19:07.205594+0000","last_became_peered":"2024-09-12T03:19:07.205594+0000","last_unstale":"2024-09-12T03:19:20.382035+0000","last_undegraded":"2024-09-12T03:19:20.382035+0000","last_fullsized":"2024-09-12T03:19:20.382035+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:12.811906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.387690+0000","last_change":"2024-09-12T03:19:20.387274+0000","last_active":"2024-09-12T03:19:20.387690+0000","last_peered":"2024-09-12T03:19:20.387690+0000","last_clean":"2024-09-12T03:19:20.387690+0000","last_became_active":"2024-09-12T03:19:20.386878+0000","last_became_peered":"2024-09-12T03:19:20.386878+0000","last_unstale":"2024-09-12T03:19:20.387690+0000","last_undegraded":"2024-09-12T03:19:20.387690+0000","last_fullsized":"2024-09-12T03:19:20.387690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:26.674032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372302+0000","last_change":"2024-09-12T03:19:07.206212+0000","last_active":"2024-09-12T03:19:20.372302+0000","last_peered":"2024-09-12T03:19:20.372302+0000","last_clean":"2024-09-12T03:19:20.372302+0000","last_became_active":"2024-09-12T03:19:07.205899+0000","last_became_peered":"2024-09-12T03:19:07.205899+0000","last_unstale":"2024-09-12T03:19:20.372302+0000","last_undegraded":"2024-09-12T03:19:20.372302+0000","last_fullsized":"2024-09-12T03:19:20.372302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:02.169436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.377494+0000","last_change":"2024-09-12T03:19:20.377494+0000","last_active":"2024-09-12T03:19:20.377494+0000","last_peered":"2024-09-12T03:19:20.377494+0000","last_clean":"2024-09-12T03:19:20.377494+0000","last_became_active":"2024-09-12T03:19:20.377091+0000","last_became_peered":"2024-09-12T03:19:20.377091+0000","last_unstale":"2024-09-12T03:19:20.377494+0000","last_undegraded":"2024-09-12T03:19:20.377494+0000","last_fullsized":"2024-09-12T03:19:20.377494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:30.926853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:31.397836+0000","last_change":"2024-09-12T03:19:07.388486+0000","last_active":"2024-09-12T03:19:31.397836+0000","last_peered":"2024-09-12T03:19:31.397836+0000","last_clean":"2024-09-12T03:19:31.397836+0000","last_became_active":"2024-09-12T03:19:07.388125+0000","last_became_peered":"2024-09-12T03:19:07.388125+0000","last_unstale":"2024-09-12T03:19:31.397836+0000","last_undegraded":"2024-09-12T03:19:31.397836+0000","last_fullsized":"2024-09-12T03:19:31.397836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:46.218601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.385912+0000","last_change":"2024-09-12T03:19:09.344308+0000","last_active":"2024-09-12T03:19:20.385912+0000","last_peered":"2024-09-12T03:19:20.385912+0000","last_clean":"2024-09-12T03:19:20.385912+0000","last_became_active":"2024-09-12T03:19:09.343987+0000","last_became_peered":"2024-09-12T03:19:09.343987+0000","last_unstale":"2024-09-12T03:19:20.385912+0000","last_undegraded":"2024-09-12T03:19:20.385912+0000","last_fullsized":"2024-09-12T03:19:20.385912+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:16:11.616555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372560+0000","last_change":"2024-09-12T03:19:07.208619+0000","last_active":"2024-09-12T03:19:20.372560+0000","last_peered":"2024-09-12T03:19:20.372560+0000","last_clean":"2024-09-12T03:19:20.372560+0000","last_became_active":"2024-09-12T03:19:07.208254+0000","last_became_peered":"2024-09-12T03:19:07.208254+0000","last_unstale":"2024-09-12T03:19:20.372560+0000","last_undegraded":"2024-09-12T03:19:20.372560+0000","last_fullsized":"2024-09-12T03:19:20.372560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:56.177948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374096+0000","last_change":"2024-09-12T03:19:09.341837+0000","last_active":"2024-09-12T03:19:20.374096+0000","last_peered":"2024-09-12T03:19:20.374096+0000","last_clean":"2024-09-12T03:19:20.374096+0000","last_became_active":"2024-09-12T03:19:09.341069+0000","last_became_peered":"2024-09-12T03:19:09.341069+0000","last_unstale":"2024-09-12T03:19:20.374096+0000","last_undegraded":"2024-09-12T03:19:20.374096+0000","last_fullsized":"2024-09-12T03:19:20.374096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:04.238986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372460+0000","last_change":"2024-09-12T03:19:07.388870+0000","last_active":"2024-09-12T03:19:20.372460+0000","last_peered":"2024-09-12T03:19:20.372460+0000","last_clean":"2024-09-12T03:19:20.372460+0000","last_became_active":"2024-09-12T03:19:07.388555+0000","last_became_peered":"2024-09-12T03:19:07.388555+0000","last_unstale":"2024-09-12T03:19:20.372460+0000","last_undegraded":"2024-09-12T03:19:20.372460+0000","last_fullsized":"2024-09-12T03:19:20.372460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:29.789454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374271+0000","last_change":"2024-09-12T03:19:09.334714+0000","last_active":"2024-09-12T03:19:20.374271+0000","last_peered":"2024-09-12T03:19:20.374271+0000","last_clean":"2024-09-12T03:19:20.374271+0000","last_became_active":"2024-09-12T03:19:09.334413+0000","last_became_peered":"2024-09-12T03:19:09.334413+0000","last_unstale":"2024-09-12T03:19:20.374271+0000","last_undegraded":"2024-09-12T03:19:20.374271+0000","last_fullsized":"2024-09-12T03:19:20.374271+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:52.222340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373797+0000","last_change":"2024-09-12T03:19:07.389055+0000","last_active":"2024-09-12T03:19:20.373797+0000","last_peered":"2024-09-12T03:19:20.373797+0000","last_clean":"2024-09-12T03:19:20.373797+0000","last_became_active":"2024-09-12T03:19:07.388673+0000","last_became_peered":"2024-09-12T03:19:07.388673+0000","last_unstale":"2024-09-12T03:19:20.373797+0000","last_undegraded":"2024-09-12T03:19:20.373797+0000","last_fullsized":"2024-09-12T03:19:20.373797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:20.974533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375304+0000","last_change":"2024-09-12T03:19:09.340105+0000","last_active":"2024-09-12T03:19:20.375304+0000","last_peered":"2024-09-12T03:19:20.375304+0000","last_clean":"2024-09-12T03:19:20.375304+0000","last_became_active":"2024-09-12T03:19:09.339642+0000","last_became_peered":"2024-09-12T03:19:09.339642+0000","last_unstale":"2024-09-12T03:19:20.375304+0000","last_undegraded":"2024-09-12T03:19:20.375304+0000","last_fullsized":"2024-09-12T03:19:20.375304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:01.624236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.330704+0000","last_change":"2024-09-12T03:19:07.207232+0000","last_active":"2024-09-12T03:19:26.330704+0000","last_peered":"2024-09-12T03:19:26.330704+0000","last_clean":"2024-09-12T03:19:26.330704+0000","last_became_active":"2024-09-12T03:19:07.206921+0000","last_became_peered":"2024-09-12T03:19:07.206921+0000","last_unstale":"2024-09-12T03:19:26.330704+0000","last_undegraded":"2024-09-12T03:19:26.330704+0000","last_fullsized":"2024-09-12T03:19:26.330704+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:25.457853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372958+0000","last_change":"2024-09-12T03:19:09.330442+0000","last_active":"2024-09-12T03:19:20.372958+0000","last_peered":"2024-09-12T03:19:20.372958+0000","last_clean":"2024-09-12T03:19:20.372958+0000","last_became_active":"2024-09-12T03:19:09.330081+0000","last_became_peered":"2024-09-12T03:19:09.330081+0000","last_unstale":"2024-09-12T03:19:20.372958+0000","last_undegraded":"2024-09-12T03:19:20.372958+0000","last_fullsized":"2024-09-12T03:19:20.372958+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:56.152562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382392+0000","last_change":"2024-09-12T03:19:07.528646+0000","last_active":"2024-09-12T03:19:20.382392+0000","last_peered":"2024-09-12T03:19:20.382392+0000","last_clean":"2024-09-12T03:19:20.382392+0000","last_became_active":"2024-09-12T03:19:07.528334+0000","last_became_peered":"2024-09-12T03:19:07.528334+0000","last_unstale":"2024-09-12T03:19:20.382392+0000","last_undegraded":"2024-09-12T03:19:20.382392+0000","last_fullsized":"2024-09-12T03:19:20.382392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:40.425579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408541+0000","last_change":"2024-09-12T03:19:09.331993+0000","last_active":"2024-09-12T03:19:20.408541+0000","last_peered":"2024-09-12T03:19:20.408541+0000","last_clean":"2024-09-12T03:19:20.408541+0000","last_became_active":"2024-09-12T03:19:09.331698+0000","last_became_peered":"2024-09-12T03:19:09.331698+0000","last_unstale":"2024-09-12T03:19:20.408541+0000","last_undegraded":"2024-09-12T03:19:20.408541+0000","last_fullsized":"2024-09-12T03:19:20.408541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:03:54.182833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370179+0000","last_change":"2024-09-12T03:19:07.388297+0000","last_active":"2024-09-12T03:19:20.370179+0000","last_peered":"2024-09-12T03:19:20.370179+0000","last_clean":"2024-09-12T03:19:20.370179+0000","last_became_active":"2024-09-12T03:19:07.388038+0000","last_became_peered":"2024-09-12T03:19:07.388038+0000","last_unstale":"2024-09-12T03:19:20.370179+0000","last_undegraded":"2024-09-12T03:19:20.370179+0000","last_fullsized":"2024-09-12T03:19:20.370179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:45.874195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373463+0000","last_change":"2024-09-12T03:19:09.331525+0000","last_active":"2024-09-12T03:19:20.373463+0000","last_peered":"2024-09-12T03:19:20.373463+0000","last_clean":"2024-09-12T03:19:20.373463+0000","last_became_active":"2024-09-12T03:19:09.330903+0000","last_became_peered":"2024-09-12T03:19:09.330903+0000","last_unstale":"2024-09-12T03:19:20.373463+0000","last_undegraded":"2024-09-12T03:19:20.373463+0000","last_fullsized":"2024-09-12T03:19:20.373463+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:02.455132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372235+0000","last_change":"2024-09-12T03:19:07.205181+0000","last_active":"2024-09-12T03:19:20.372235+0000","last_peered":"2024-09-12T03:19:20.372235+0000","last_clean":"2024-09-12T03:19:20.372235+0000","last_became_active":"2024-09-12T03:19:07.204726+0000","last_became_peered":"2024-09-12T03:19:07.204726+0000","last_unstale":"2024-09-12T03:19:20.372235+0000","last_undegraded":"2024-09-12T03:19:20.372235+0000","last_fullsized":"2024-09-12T03:19:20.372235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:26.112800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370908+0000","last_change":"2024-09-12T03:19:09.345338+0000","last_active":"2024-09-12T03:19:20.370908+0000","last_peered":"2024-09-12T03:19:20.370908+0000","last_clean":"2024-09-12T03:19:20.370908+0000","last_became_active":"2024-09-12T03:19:09.345023+0000","last_became_peered":"2024-09-12T03:19:09.345023+0000","last_unstale":"2024-09-12T03:19:20.370908+0000","last_undegraded":"2024-09-12T03:19:20.370908+0000","last_fullsized":"2024-09-12T03:19:20.370908+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:39.184065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374938+0000","last_change":"2024-09-12T03:19:07.211954+0000","last_active":"2024-09-12T03:19:20.374938+0000","last_peered":"2024-09-12T03:19:20.374938+0000","last_clean":"2024-09-12T03:19:20.374938+0000","last_became_active":"2024-09-12T03:19:07.211509+0000","last_became_peered":"2024-09-12T03:19:07.211509+0000","last_unstale":"2024-09-12T03:19:20.374938+0000","last_undegraded":"2024-09-12T03:19:20.374938+0000","last_fullsized":"2024-09-12T03:19:20.374938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:27.461924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373018+0000","last_change":"2024-09-12T03:19:09.331168+0000","last_active":"2024-09-12T03:19:20.373018+0000","last_peered":"2024-09-12T03:19:20.373018+0000","last_clean":"2024-09-12T03:19:20.373018+0000","last_became_active":"2024-09-12T03:19:09.329592+0000","last_became_peered":"2024-09-12T03:19:09.329592+0000","last_unstale":"2024-09-12T03:19:20.373018+0000","last_undegraded":"2024-09-12T03:19:20.373018+0000","last_fullsized":"2024-09-12T03:19:20.373018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:46.127861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373572+0000","last_change":"2024-09-12T03:19:07.207802+0000","last_active":"2024-09-12T03:19:20.373572+0000","last_peered":"2024-09-12T03:19:20.373572+0000","last_clean":"2024-09-12T03:19:20.373572+0000","last_became_active":"2024-09-12T03:19:07.207428+0000","last_became_peered":"2024-09-12T03:19:07.207428+0000","last_unstale":"2024-09-12T03:19:20.373572+0000","last_undegraded":"2024-09-12T03:19:20.373572+0000","last_fullsized":"2024-09-12T03:19:20.373572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:45.135019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372336+0000","last_change":"2024-09-12T03:19:09.336094+0000","last_active":"2024-09-12T03:19:20.372336+0000","last_peered":"2024-09-12T03:19:20.372336+0000","last_clean":"2024-09-12T03:19:20.372336+0000","last_became_active":"2024-09-12T03:19:09.335779+0000","last_became_peered":"2024-09-12T03:19:09.335779+0000","last_unstale":"2024-09-12T03:19:20.372336+0000","last_undegraded":"2024-09-12T03:19:20.372336+0000","last_fullsized":"2024-09-12T03:19:20.372336+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:37.019082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386131+0000","last_change":"2024-09-12T03:19:09.344537+0000","last_active":"2024-09-12T03:19:20.386131+0000","last_peered":"2024-09-12T03:19:20.386131+0000","last_clean":"2024-09-12T03:19:20.386131+0000","last_became_active":"2024-09-12T03:19:09.343879+0000","last_became_peered":"2024-09-12T03:19:09.343879+0000","last_unstale":"2024-09-12T03:19:20.386131+0000","last_undegraded":"2024-09-12T03:19:20.386131+0000","last_fullsized":"2024-09-12T03:19:20.386131+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:40.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373973+0000","last_change":"2024-09-12T03:19:07.208040+0000","last_active":"2024-09-12T03:19:20.373973+0000","last_peered":"2024-09-12T03:19:20.373973+0000","last_clean":"2024-09-12T03:19:20.373973+0000","last_became_active":"2024-09-12T03:19:07.207658+0000","last_became_peered":"2024-09-12T03:19:07.207658+0000","last_unstale":"2024-09-12T03:19:20.373973+0000","last_undegraded":"2024-09-12T03:19:20.373973+0000","last_fullsized":"2024-09-12T03:19:20.373973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:48.620707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370231+0000","last_change":"2024-09-12T03:19:09.333837+0000","last_active":"2024-09-12T03:19:20.370231+0000","last_peered":"2024-09-12T03:19:20.370231+0000","last_clean":"2024-09-12T03:19:20.370231+0000","last_became_active":"2024-09-12T03:19:09.333194+0000","last_became_peered":"2024-09-12T03:19:09.333194+0000","last_unstale":"2024-09-12T03:19:20.370231+0000","last_undegraded":"2024-09-12T03:19:20.370231+0000","last_fullsized":"2024-09-12T03:19:20.370231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:18.766181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.385455+0000","last_change":"2024-09-12T03:19:20.384978+0000","last_active":"2024-09-12T03:19:20.385455+0000","last_peered":"2024-09-12T03:19:20.385455+0000","last_clean":"2024-09-12T03:19:20.385455+0000","last_became_active":"2024-09-12T03:19:20.376883+0000","last_became_peered":"2024-09-12T03:19:20.376883+0000","last_unstale":"2024-09-12T03:19:20.385455+0000","last_undegraded":"2024-09-12T03:19:20.385455+0000","last_fullsized":"2024-09-12T03:19:20.385455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:32.542334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408633+0000","last_change":"2024-09-12T03:19:09.334684+0000","last_active":"2024-09-12T03:19:20.408633+0000","last_peered":"2024-09-12T03:19:20.408633+0000","last_clean":"2024-09-12T03:19:20.408633+0000","last_became_active":"2024-09-12T03:19:09.334281+0000","last_became_peered":"2024-09-12T03:19:09.334281+0000","last_unstale":"2024-09-12T03:19:20.408633+0000","last_undegraded":"2024-09-12T03:19:20.408633+0000","last_fullsized":"2024-09-12T03:19:20.408633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:44.837067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372829+0000","last_change":"2024-09-12T03:19:07.388555+0000","last_active":"2024-09-12T03:19:20.372829+0000","last_peered":"2024-09-12T03:19:20.372829+0000","last_clean":"2024-09-12T03:19:20.372829+0000","last_became_active":"2024-09-12T03:19:07.388206+0000","last_became_peered":"2024-09-12T03:19:07.388206+0000","last_unstale":"2024-09-12T03:19:20.372829+0000","last_undegraded":"2024-09-12T03:19:20.372829+0000","last_fullsized":"2024-09-12T03:19:20.372829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:33.628294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372626+0000","last_change":"2024-09-12T03:19:09.339108+0000","last_active":"2024-09-12T03:19:20.372626+0000","last_peered":"2024-09-12T03:19:20.372626+0000","last_clean":"2024-09-12T03:19:20.372626+0000","last_became_active":"2024-09-12T03:19:09.338819+0000","last_became_peered":"2024-09-12T03:19:09.338819+0000","last_unstale":"2024-09-12T03:19:20.372626+0000","last_undegraded":"2024-09-12T03:19:20.372626+0000","last_fullsized":"2024-09-12T03:19:20.372626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:32.864900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372798+0000","last_change":"2024-09-12T03:19:07.207197+0000","last_active":"2024-09-12T03:19:20.372798+0000","last_peered":"2024-09-12T03:19:20.372798+0000","last_clean":"2024-09-12T03:19:20.372798+0000","last_became_active":"2024-09-12T03:19:07.206879+0000","last_became_peered":"2024-09-12T03:19:07.206879+0000","last_unstale":"2024-09-12T03:19:20.372798+0000","last_undegraded":"2024-09-12T03:19:20.372798+0000","last_fullsized":"2024-09-12T03:19:20.372798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:00:46.269636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54115,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":103,"num_read_kb":110,"num_write":151,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":153298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131,"ondisk_log_size":131,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579921,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":892928,"data_stored":433387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36884,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708268,"statfs":{"total":95995035648,"available":95957266432,"internally_reserved":0,"allocated":1576960,"data_stored":1074946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972377,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":433589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201312,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":433509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430245,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1499136,"data_stored":1025441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593770,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":432978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822702,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1511424,"data_stored":1025125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953524,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":897024,"data_stored":433415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":433289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":475545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40416,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704736,"statfs":{"total":95995035648,"available":95953649664,"internally_reserved":0,"allocated":933888,"data_stored":473836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":913408,"data_stored":434249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":41065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":51821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":51690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:19:34.908 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:19:35.163 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:35.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:35 smithi118 ceph-mon[35048]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:35.528 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:35 smithi006 ceph-mon[30991]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:35.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.612+0000 7f9579ea9640 1 -- 172.21.15.6:0/3456651867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9574103660 msgr2=0x7f957410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:35.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.612+0000 7f9579ea9640 1 --2- 172.21.15.6:0/3456651867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9574103660 0x7f957410bb50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9568009900 tx=0x7f956802a480 comp rx=0 tx=0).stop 2024-09-12T03:19:35.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.613+0000 7f9579ea9640 1 -- 172.21.15.6:0/3456651867 shutdown_connections 2024-09-12T03:19:35.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.613+0000 7f9579ea9640 1 --2- 172.21.15.6:0/3456651867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f957410c170 0x7f957410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.613+0000 7f9579ea9640 1 --2- 172.21.15.6:0/3456651867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9574103660 0x7f957410bb50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.613+0000 7f9579ea9640 1 --2- 172.21.15.6:0/3456651867 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9574102d40 0x7f9574103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.613+0000 7f9579ea9640 1 -- 172.21.15.6:0/3456651867 >> 172.21.15.6:0/3456651867 conn(0x7f95740fc7a0 msgr2=0x7f95740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:35.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.613+0000 7f9579ea9640 1 -- 172.21.15.6:0/3456651867 shutdown_connections 2024-09-12T03:19:35.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.613+0000 7f9579ea9640 1 -- 172.21.15.6:0/3456651867 wait complete. 2024-09-12T03:19:35.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.613+0000 7f9579ea9640 1 Processor -- start 2024-09-12T03:19:35.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:35 smithi161 ceph-mon[35485]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:35.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.614+0000 7f9579ea9640 1 -- start start 2024-09-12T03:19:35.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.614+0000 7f9579ea9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9574102d40 0x7f95741009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:35.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.614+0000 7f9579ea9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9574103660 0x7f9574100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:35.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.614+0000 7f9579ea9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f957410c170 0x7f9574101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:35.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.614+0000 7f9579ea9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9574111fc0 con 0x7f9574102d40 2024-09-12T03:19:35.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.614+0000 7f9579ea9640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f9574111e40 con 0x7f9574103660 2024-09-12T03:19:35.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.614+0000 7f9579ea9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9574112140 con 0x7f957410c170 2024-09-12T03:19:35.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.614+0000 7f95737fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9574102d40 0x7f95741009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:35.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.614+0000 7f95737fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9574102d40 0x7f95741009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59308/0 (socket says 172.21.15.6:59308) 2024-09-12T03:19:35.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.614+0000 7f95737fe640 1 -- 172.21.15.6:0/792291768 learned_addr learned my addr 172.21.15.6:0/792291768 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:35.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.614+0000 7f9572ffd640 1 --2- 172.21.15.6:0/792291768 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9574103660 0x7f9574100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:35.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.614+0000 7f9573fff640 1 --2- 172.21.15.6:0/792291768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f957410c170 0x7f9574101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:35.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.614+0000 7f95737fe640 1 -- 172.21.15.6:0/792291768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f957410c170 msgr2=0x7f9574101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:35.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.614+0000 7f95737fe640 1 --2- 172.21.15.6:0/792291768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f957410c170 0x7f9574101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.614+0000 7f95737fe640 1 -- 172.21.15.6:0/792291768 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9574103660 msgr2=0x7f9574100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:35.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.614+0000 7f95737fe640 1 --2- 172.21.15.6:0/792291768 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9574103660 0x7f9574100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.614+0000 7f95737fe640 1 -- 172.21.15.6:0/792291768 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95741a6910 con 0x7f9574102d40 2024-09-12T03:19:35.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.614+0000 7f9573fff640 1 --2- 172.21.15.6:0/792291768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f957410c170 0x7f9574101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:35.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.615+0000 7f95737fe640 1 --2- 172.21.15.6:0/792291768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9574102d40 0x7f95741009a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f956400c920 tx=0x7f956400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:35.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.615+0000 7f9570ff9640 1 -- 172.21.15.6:0/792291768 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564007c70 con 0x7f9574102d40 2024-09-12T03:19:35.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.615+0000 7f9570ff9640 1 -- 172.21.15.6:0/792291768 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9564007e10 con 0x7f9574102d40 2024-09-12T03:19:35.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.615+0000 7f9579ea9640 1 -- 172.21.15.6:0/792291768 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95741a6c00 con 0x7f9574102d40 2024-09-12T03:19:35.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.615+0000 7f9570ff9640 1 -- 172.21.15.6:0/792291768 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564018890 con 0x7f9574102d40 2024-09-12T03:19:35.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.615+0000 7f9579ea9640 1 -- 172.21.15.6:0/792291768 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95741a7160 con 0x7f9574102d40 2024-09-12T03:19:35.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.616+0000 7f9579ea9640 1 -- 172.21.15.6:0/792291768 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9574103120 con 0x7f9574102d40 2024-09-12T03:19:35.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.617+0000 7f9570ff9640 1 -- 172.21.15.6:0/792291768 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9564018a70 con 0x7f9574102d40 2024-09-12T03:19:35.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.618+0000 7f9570ff9640 1 --2- 172.21.15.6:0/792291768 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f955c0a2c90 0x7f955c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:35.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.618+0000 7f9572ffd640 1 --2- 172.21.15.6:0/792291768 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f955c0a2c90 0x7f955c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:35.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.618+0000 7f9570ff9640 1 -- 172.21.15.6:0/792291768 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f95640cf260 con 0x7f9574102d40 2024-09-12T03:19:35.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.618+0000 7f9572ffd640 1 --2- 172.21.15.6:0/792291768 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f955c0a2c90 0x7f955c0a5150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9568000c00 tx=0x7f9568005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:35.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.622+0000 7f9570ff9640 1 -- 172.21.15.6:0/792291768 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9564014030 con 0x7f9574102d40 2024-09-12T03:19:35.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.793+0000 7f9579ea9640 1 -- 172.21.15.6:0/792291768 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9574107250 con 0x7f955c0a2c90 2024-09-12T03:19:35.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.800+0000 7f9570ff9640 1 -- 172.21.15.6:0/792291768 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258718 (secure 0 0 0) 0x7f9568004270 con 0x7f955c0a2c90 2024-09-12T03:19:35.806 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:19:35.807 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-09-12T03:19:35.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.803+0000 7f9579ea9640 1 -- 172.21.15.6:0/792291768 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f955c0a2c90 msgr2=0x7f955c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:35.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.803+0000 7f9579ea9640 1 --2- 172.21.15.6:0/792291768 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f955c0a2c90 0x7f955c0a5150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9568000c00 tx=0x7f9568005b50 comp rx=0 tx=0).stop 2024-09-12T03:19:35.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.803+0000 7f9579ea9640 1 -- 172.21.15.6:0/792291768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9574102d40 msgr2=0x7f95741009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:35.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.803+0000 7f9579ea9640 1 --2- 172.21.15.6:0/792291768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9574102d40 0x7f95741009a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f956400c920 tx=0x7f956400cde0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.803+0000 7f9579ea9640 1 -- 172.21.15.6:0/792291768 shutdown_connections 2024-09-12T03:19:35.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.803+0000 7f9579ea9640 1 --2- 172.21.15.6:0/792291768 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f955c0a2c90 0x7f955c0a5150 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.803+0000 7f9579ea9640 1 --2- 172.21.15.6:0/792291768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f957410c170 0x7f9574101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.803+0000 7f9579ea9640 1 --2- 172.21.15.6:0/792291768 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9574103660 0x7f9574100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.803+0000 7f9579ea9640 1 --2- 172.21.15.6:0/792291768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9574102d40 0x7f95741009a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.803+0000 7f9579ea9640 1 -- 172.21.15.6:0/792291768 >> 172.21.15.6:0/792291768 conn(0x7f95740fc7a0 msgr2=0x7f95740fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:35.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.803+0000 7f9579ea9640 1 -- 172.21.15.6:0/792291768 shutdown_connections 2024-09-12T03:19:35.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:35.803+0000 7f9579ea9640 1 -- 172.21.15.6:0/792291768 wait complete. 2024-09-12T03:19:36.583 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:36 smithi006 ceph-mon[30991]: pgmap v290: 97 pgs: 97 active+clean; 630 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-09-12T03:19:36.583 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:36 smithi006 ceph-mon[30991]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:36.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:36 smithi161 ceph-mon[35485]: pgmap v290: 97 pgs: 97 active+clean; 630 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-09-12T03:19:36.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:36 smithi161 ceph-mon[35485]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:36.617 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-09-12T03:19:35.242353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644717,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":229,"num_read_kb":219,"num_write":342,"num_write_kb":3037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382560,"kb_used_data":12492,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559264,"statfs":{"total":1151940427776,"available":1151548686336,"internally_reserved":0,"allocated":12791808,"data_stored":7109309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42525,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":81,"num_write":33,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002898"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.371249+0000","last_change":"2024-09-12T03:19:09.343829+0000","last_active":"2024-09-12T03:19:20.371249+0000","last_peered":"2024-09-12T03:19:20.371249+0000","last_clean":"2024-09-12T03:19:20.371249+0000","last_became_active":"2024-09-12T03:19:09.343228+0000","last_became_peered":"2024-09-12T03:19:09.343228+0000","last_unstale":"2024-09-12T03:19:20.371249+0000","last_undegraded":"2024-09-12T03:19:20.371249+0000","last_fullsized":"2024-09-12T03:19:20.371249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:08.720934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408528+0000","last_change":"2024-09-12T03:19:09.331829+0000","last_active":"2024-09-12T03:19:20.408528+0000","last_peered":"2024-09-12T03:19:20.408528+0000","last_clean":"2024-09-12T03:19:20.408528+0000","last_became_active":"2024-09-12T03:19:09.331276+0000","last_became_peered":"2024-09-12T03:19:09.331276+0000","last_unstale":"2024-09-12T03:19:20.408528+0000","last_undegraded":"2024-09-12T03:19:20.408528+0000","last_fullsized":"2024-09-12T03:19:20.408528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:48.672085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408509+0000","last_change":"2024-09-12T03:19:09.338120+0000","last_active":"2024-09-12T03:19:20.408509+0000","last_peered":"2024-09-12T03:19:20.408509+0000","last_clean":"2024-09-12T03:19:20.408509+0000","last_became_active":"2024-09-12T03:19:09.337773+0000","last_became_peered":"2024-09-12T03:19:09.337773+0000","last_unstale":"2024-09-12T03:19:20.408509+0000","last_undegraded":"2024-09-12T03:19:20.408509+0000","last_fullsized":"2024-09-12T03:19:20.408509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:13.716047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382087+0000","last_change":"2024-09-12T03:19:09.339404+0000","last_active":"2024-09-12T03:19:20.382087+0000","last_peered":"2024-09-12T03:19:20.382087+0000","last_clean":"2024-09-12T03:19:20.382087+0000","last_became_active":"2024-09-12T03:19:09.339105+0000","last_became_peered":"2024-09-12T03:19:09.339105+0000","last_unstale":"2024-09-12T03:19:20.382087+0000","last_undegraded":"2024-09-12T03:19:20.382087+0000","last_fullsized":"2024-09-12T03:19:20.382087+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:37.930963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373401+0000","last_change":"2024-09-12T03:19:09.326285+0000","last_active":"2024-09-12T03:19:20.373401+0000","last_peered":"2024-09-12T03:19:20.373401+0000","last_clean":"2024-09-12T03:19:20.373401+0000","last_became_active":"2024-09-12T03:19:09.325852+0000","last_became_peered":"2024-09-12T03:19:09.325852+0000","last_unstale":"2024-09-12T03:19:20.373401+0000","last_undegraded":"2024-09-12T03:19:20.373401+0000","last_fullsized":"2024-09-12T03:19:20.373401+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:44:25.387014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.376197+0000","last_change":"2024-09-12T03:19:20.376197+0000","last_active":"2024-09-12T03:19:20.376197+0000","last_peered":"2024-09-12T03:19:20.376197+0000","last_clean":"2024-09-12T03:19:20.376197+0000","last_became_active":"2024-09-12T03:19:20.375744+0000","last_became_peered":"2024-09-12T03:19:20.375744+0000","last_unstale":"2024-09-12T03:19:20.376197+0000","last_undegraded":"2024-09-12T03:19:20.376197+0000","last_fullsized":"2024-09-12T03:19:20.376197+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:08.663121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372152+0000","last_change":"2024-09-12T03:19:09.335284+0000","last_active":"2024-09-12T03:19:20.372152+0000","last_peered":"2024-09-12T03:19:20.372152+0000","last_clean":"2024-09-12T03:19:20.372152+0000","last_became_active":"2024-09-12T03:19:09.334948+0000","last_became_peered":"2024-09-12T03:19:09.334948+0000","last_unstale":"2024-09-12T03:19:20.372152+0000","last_undegraded":"2024-09-12T03:19:20.372152+0000","last_fullsized":"2024-09-12T03:19:20.372152+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:32.778433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372122+0000","last_change":"2024-09-12T03:19:09.338398+0000","last_active":"2024-09-12T03:19:20.372122+0000","last_peered":"2024-09-12T03:19:20.372122+0000","last_clean":"2024-09-12T03:19:20.372122+0000","last_became_active":"2024-09-12T03:19:09.337750+0000","last_became_peered":"2024-09-12T03:19:09.337750+0000","last_unstale":"2024-09-12T03:19:20.372122+0000","last_undegraded":"2024-09-12T03:19:20.372122+0000","last_fullsized":"2024-09-12T03:19:20.372122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:07.542322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370699+0000","last_change":"2024-09-12T03:19:09.340703+0000","last_active":"2024-09-12T03:19:20.370699+0000","last_peered":"2024-09-12T03:19:20.370699+0000","last_clean":"2024-09-12T03:19:20.370699+0000","last_became_active":"2024-09-12T03:19:09.340386+0000","last_became_peered":"2024-09-12T03:19:09.340386+0000","last_unstale":"2024-09-12T03:19:20.370699+0000","last_undegraded":"2024-09-12T03:19:20.370699+0000","last_fullsized":"2024-09-12T03:19:20.370699+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:19.990894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374331+0000","last_change":"2024-09-12T03:19:09.341489+0000","last_active":"2024-09-12T03:19:20.374331+0000","last_peered":"2024-09-12T03:19:20.374331+0000","last_clean":"2024-09-12T03:19:20.374331+0000","last_became_active":"2024-09-12T03:19:09.341183+0000","last_became_peered":"2024-09-12T03:19:09.341183+0000","last_unstale":"2024-09-12T03:19:20.374331+0000","last_undegraded":"2024-09-12T03:19:20.374331+0000","last_fullsized":"2024-09-12T03:19:20.374331+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:05.688402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372589+0000","last_change":"2024-09-12T03:19:09.339289+0000","last_active":"2024-09-12T03:19:20.372589+0000","last_peered":"2024-09-12T03:19:20.372589+0000","last_clean":"2024-09-12T03:19:20.372589+0000","last_became_active":"2024-09-12T03:19:09.338466+0000","last_became_peered":"2024-09-12T03:19:09.338466+0000","last_unstale":"2024-09-12T03:19:20.372589+0000","last_undegraded":"2024-09-12T03:19:20.372589+0000","last_fullsized":"2024-09-12T03:19:20.372589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:47:13.070057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382092+0000","last_change":"2024-09-12T03:19:09.337754+0000","last_active":"2024-09-12T03:19:20.382092+0000","last_peered":"2024-09-12T03:19:20.382092+0000","last_clean":"2024-09-12T03:19:20.382092+0000","last_became_active":"2024-09-12T03:19:09.332789+0000","last_became_peered":"2024-09-12T03:19:09.332789+0000","last_unstale":"2024-09-12T03:19:20.382092+0000","last_undegraded":"2024-09-12T03:19:20.382092+0000","last_fullsized":"2024-09-12T03:19:20.382092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:34.622136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375297+0000","last_change":"2024-09-12T03:19:09.333577+0000","last_active":"2024-09-12T03:19:20.375297+0000","last_peered":"2024-09-12T03:19:20.375297+0000","last_clean":"2024-09-12T03:19:20.375297+0000","last_became_active":"2024-09-12T03:19:09.333234+0000","last_became_peered":"2024-09-12T03:19:09.333234+0000","last_unstale":"2024-09-12T03:19:20.375297+0000","last_undegraded":"2024-09-12T03:19:20.375297+0000","last_fullsized":"2024-09-12T03:19:20.375297+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:22.604640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408267+0000","last_change":"2024-09-12T03:19:09.331607+0000","last_active":"2024-09-12T03:19:20.408267+0000","last_peered":"2024-09-12T03:19:20.408267+0000","last_clean":"2024-09-12T03:19:20.408267+0000","last_became_active":"2024-09-12T03:19:09.331253+0000","last_became_peered":"2024-09-12T03:19:09.331253+0000","last_unstale":"2024-09-12T03:19:20.408267+0000","last_undegraded":"2024-09-12T03:19:20.408267+0000","last_fullsized":"2024-09-12T03:19:20.408267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:11.969265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408245+0000","last_change":"2024-09-12T03:19:09.340768+0000","last_active":"2024-09-12T03:19:20.408245+0000","last_peered":"2024-09-12T03:19:20.408245+0000","last_clean":"2024-09-12T03:19:20.408245+0000","last_became_active":"2024-09-12T03:19:09.340437+0000","last_became_peered":"2024-09-12T03:19:09.340437+0000","last_unstale":"2024-09-12T03:19:20.408245+0000","last_undegraded":"2024-09-12T03:19:20.408245+0000","last_fullsized":"2024-09-12T03:19:20.408245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:32.645691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370081+0000","last_change":"2024-09-12T03:19:09.333829+0000","last_active":"2024-09-12T03:19:20.370081+0000","last_peered":"2024-09-12T03:19:20.370081+0000","last_clean":"2024-09-12T03:19:20.370081+0000","last_became_active":"2024-09-12T03:19:09.333462+0000","last_became_peered":"2024-09-12T03:19:09.333462+0000","last_unstale":"2024-09-12T03:19:20.370081+0000","last_undegraded":"2024-09-12T03:19:20.370081+0000","last_fullsized":"2024-09-12T03:19:20.370081+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:18:57.586836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.371273+0000","last_change":"2024-09-12T03:19:09.344068+0000","last_active":"2024-09-12T03:19:20.371273+0000","last_peered":"2024-09-12T03:19:20.371273+0000","last_clean":"2024-09-12T03:19:20.371273+0000","last_became_active":"2024-09-12T03:19:09.343751+0000","last_became_peered":"2024-09-12T03:19:09.343751+0000","last_unstale":"2024-09-12T03:19:20.371273+0000","last_undegraded":"2024-09-12T03:19:20.371273+0000","last_fullsized":"2024-09-12T03:19:20.371273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:52.425547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372644+0000","last_change":"2024-09-12T03:19:09.330828+0000","last_active":"2024-09-12T03:19:20.372644+0000","last_peered":"2024-09-12T03:19:20.372644+0000","last_clean":"2024-09-12T03:19:20.372644+0000","last_became_active":"2024-09-12T03:19:09.330497+0000","last_became_peered":"2024-09-12T03:19:09.330497+0000","last_unstale":"2024-09-12T03:19:20.372644+0000","last_undegraded":"2024-09-12T03:19:20.372644+0000","last_fullsized":"2024-09-12T03:19:20.372644+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:21:20.064177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375155+0000","last_change":"2024-09-12T03:19:09.344403+0000","last_active":"2024-09-12T03:19:20.375155+0000","last_peered":"2024-09-12T03:19:20.375155+0000","last_clean":"2024-09-12T03:19:20.375155+0000","last_became_active":"2024-09-12T03:19:09.343946+0000","last_became_peered":"2024-09-12T03:19:09.343946+0000","last_unstale":"2024-09-12T03:19:20.375155+0000","last_undegraded":"2024-09-12T03:19:20.375155+0000","last_fullsized":"2024-09-12T03:19:20.375155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:48.386117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370884+0000","last_change":"2024-09-12T03:19:09.341481+0000","last_active":"2024-09-12T03:19:20.370884+0000","last_peered":"2024-09-12T03:19:20.370884+0000","last_clean":"2024-09-12T03:19:20.370884+0000","last_became_active":"2024-09-12T03:19:09.341142+0000","last_became_peered":"2024-09-12T03:19:09.341142+0000","last_unstale":"2024-09-12T03:19:20.370884+0000","last_undegraded":"2024-09-12T03:19:20.370884+0000","last_fullsized":"2024-09-12T03:19:20.370884+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:42.371221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375324+0000","last_change":"2024-09-12T03:19:09.330621+0000","last_active":"2024-09-12T03:19:20.375324+0000","last_peered":"2024-09-12T03:19:20.375324+0000","last_clean":"2024-09-12T03:19:20.375324+0000","last_became_active":"2024-09-12T03:19:09.330169+0000","last_became_peered":"2024-09-12T03:19:09.330169+0000","last_unstale":"2024-09-12T03:19:20.375324+0000","last_undegraded":"2024-09-12T03:19:20.375324+0000","last_fullsized":"2024-09-12T03:19:20.375324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:16.634141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370190+0000","last_change":"2024-09-12T03:19:09.332817+0000","last_active":"2024-09-12T03:19:20.370190+0000","last_peered":"2024-09-12T03:19:20.370190+0000","last_clean":"2024-09-12T03:19:20.370190+0000","last_became_active":"2024-09-12T03:19:09.332236+0000","last_became_peered":"2024-09-12T03:19:09.332236+0000","last_unstale":"2024-09-12T03:19:20.370190+0000","last_undegraded":"2024-09-12T03:19:20.370190+0000","last_fullsized":"2024-09-12T03:19:20.370190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:28.024294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372100+0000","last_change":"2024-09-12T03:19:09.337910+0000","last_active":"2024-09-12T03:19:20.372100+0000","last_peered":"2024-09-12T03:19:20.372100+0000","last_clean":"2024-09-12T03:19:20.372100+0000","last_became_active":"2024-09-12T03:19:09.337176+0000","last_became_peered":"2024-09-12T03:19:09.337176+0000","last_unstale":"2024-09-12T03:19:20.372100+0000","last_undegraded":"2024-09-12T03:19:20.372100+0000","last_fullsized":"2024-09-12T03:19:20.372100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:53.795898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.385889+0000","last_change":"2024-09-12T03:19:09.337902+0000","last_active":"2024-09-12T03:19:20.385889+0000","last_peered":"2024-09-12T03:19:20.385889+0000","last_clean":"2024-09-12T03:19:20.385889+0000","last_became_active":"2024-09-12T03:19:09.337567+0000","last_became_peered":"2024-09-12T03:19:09.337567+0000","last_unstale":"2024-09-12T03:19:20.385889+0000","last_undegraded":"2024-09-12T03:19:20.385889+0000","last_fullsized":"2024-09-12T03:19:20.385889+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:08.471313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372957+0000","last_change":"2024-09-12T03:19:09.339850+0000","last_active":"2024-09-12T03:19:20.372957+0000","last_peered":"2024-09-12T03:19:20.372957+0000","last_clean":"2024-09-12T03:19:20.372957+0000","last_became_active":"2024-09-12T03:19:09.339563+0000","last_became_peered":"2024-09-12T03:19:09.339563+0000","last_unstale":"2024-09-12T03:19:20.372957+0000","last_undegraded":"2024-09-12T03:19:20.372957+0000","last_fullsized":"2024-09-12T03:19:20.372957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:27.473351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373965+0000","last_change":"2024-09-12T03:19:09.330555+0000","last_active":"2024-09-12T03:19:20.373965+0000","last_peered":"2024-09-12T03:19:20.373965+0000","last_clean":"2024-09-12T03:19:20.373965+0000","last_became_active":"2024-09-12T03:19:09.330183+0000","last_became_peered":"2024-09-12T03:19:09.330183+0000","last_unstale":"2024-09-12T03:19:20.373965+0000","last_undegraded":"2024-09-12T03:19:20.373965+0000","last_fullsized":"2024-09-12T03:19:20.373965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:39.334116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.371187+0000","last_change":"2024-09-12T03:19:09.338601+0000","last_active":"2024-09-12T03:19:20.371187+0000","last_peered":"2024-09-12T03:19:20.371187+0000","last_clean":"2024-09-12T03:19:20.371187+0000","last_became_active":"2024-09-12T03:19:09.337903+0000","last_became_peered":"2024-09-12T03:19:09.337903+0000","last_unstale":"2024-09-12T03:19:20.371187+0000","last_undegraded":"2024-09-12T03:19:20.371187+0000","last_fullsized":"2024-09-12T03:19:20.371187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:17.810139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375192+0000","last_change":"2024-09-12T03:19:09.326629+0000","last_active":"2024-09-12T03:19:20.375192+0000","last_peered":"2024-09-12T03:19:20.375192+0000","last_clean":"2024-09-12T03:19:20.375192+0000","last_became_active":"2024-09-12T03:19:09.326333+0000","last_became_peered":"2024-09-12T03:19:09.326333+0000","last_unstale":"2024-09-12T03:19:20.375192+0000","last_undegraded":"2024-09-12T03:19:20.375192+0000","last_fullsized":"2024-09-12T03:19:20.375192+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:20.807593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408435+0000","last_change":"2024-09-12T03:19:09.339919+0000","last_active":"2024-09-12T03:19:20.408435+0000","last_peered":"2024-09-12T03:19:20.408435+0000","last_clean":"2024-09-12T03:19:20.408435+0000","last_became_active":"2024-09-12T03:19:09.339618+0000","last_became_peered":"2024-09-12T03:19:09.339618+0000","last_unstale":"2024-09-12T03:19:20.408435+0000","last_undegraded":"2024-09-12T03:19:20.408435+0000","last_fullsized":"2024-09-12T03:19:20.408435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:11:03.740079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374805+0000","last_change":"2024-09-12T03:19:09.336148+0000","last_active":"2024-09-12T03:19:20.374805+0000","last_peered":"2024-09-12T03:19:20.374805+0000","last_clean":"2024-09-12T03:19:20.374805+0000","last_became_active":"2024-09-12T03:19:09.335779+0000","last_became_peered":"2024-09-12T03:19:09.335779+0000","last_unstale":"2024-09-12T03:19:20.374805+0000","last_undegraded":"2024-09-12T03:19:20.374805+0000","last_fullsized":"2024-09-12T03:19:20.374805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:18.234320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382536+0000","last_change":"2024-09-12T03:19:20.382536+0000","last_active":"2024-09-12T03:19:20.382536+0000","last_peered":"2024-09-12T03:19:20.382536+0000","last_clean":"2024-09-12T03:19:20.382536+0000","last_became_active":"2024-09-12T03:19:20.382118+0000","last_became_peered":"2024-09-12T03:19:20.382118+0000","last_unstale":"2024-09-12T03:19:20.382536+0000","last_undegraded":"2024-09-12T03:19:20.382536+0000","last_fullsized":"2024-09-12T03:19:20.382536+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:55.836646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370826+0000","last_change":"2024-09-12T03:19:09.345064+0000","last_active":"2024-09-12T03:19:20.370826+0000","last_peered":"2024-09-12T03:19:20.370826+0000","last_clean":"2024-09-12T03:19:20.370826+0000","last_became_active":"2024-09-12T03:19:09.344628+0000","last_became_peered":"2024-09-12T03:19:09.344628+0000","last_unstale":"2024-09-12T03:19:20.370826+0000","last_undegraded":"2024-09-12T03:19:20.370826+0000","last_fullsized":"2024-09-12T03:19:20.370826+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:50.011751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373663+0000","last_change":"2024-09-12T03:19:09.338052+0000","last_active":"2024-09-12T03:19:20.373663+0000","last_peered":"2024-09-12T03:19:20.373663+0000","last_clean":"2024-09-12T03:19:20.373663+0000","last_became_active":"2024-09-12T03:19:09.337710+0000","last_became_peered":"2024-09-12T03:19:09.337710+0000","last_unstale":"2024-09-12T03:19:20.373663+0000","last_undegraded":"2024-09-12T03:19:20.373663+0000","last_fullsized":"2024-09-12T03:19:20.373663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:36.521269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:29.397897+0000","last_change":"2024-09-12T03:19:07.208341+0000","last_active":"2024-09-12T03:19:29.397897+0000","last_peered":"2024-09-12T03:19:29.397897+0000","last_clean":"2024-09-12T03:19:29.397897+0000","last_became_active":"2024-09-12T03:19:07.207385+0000","last_became_peered":"2024-09-12T03:19:07.207385+0000","last_unstale":"2024-09-12T03:19:29.397897+0000","last_undegraded":"2024-09-12T03:19:29.397897+0000","last_fullsized":"2024-09-12T03:19:29.397897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:18.476795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386347+0000","last_change":"2024-09-12T03:19:09.339997+0000","last_active":"2024-09-12T03:19:20.386347+0000","last_peered":"2024-09-12T03:19:20.386347+0000","last_clean":"2024-09-12T03:19:20.386347+0000","last_became_active":"2024-09-12T03:19:09.339261+0000","last_became_peered":"2024-09-12T03:19:09.339261+0000","last_unstale":"2024-09-12T03:19:20.386347+0000","last_undegraded":"2024-09-12T03:19:20.386347+0000","last_fullsized":"2024-09-12T03:19:20.386347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:07.670345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374030+0000","last_change":"2024-09-12T03:19:07.200222+0000","last_active":"2024-09-12T03:19:20.374030+0000","last_peered":"2024-09-12T03:19:20.374030+0000","last_clean":"2024-09-12T03:19:20.374030+0000","last_became_active":"2024-09-12T03:19:07.199819+0000","last_became_peered":"2024-09-12T03:19:07.199819+0000","last_unstale":"2024-09-12T03:19:20.374030+0000","last_undegraded":"2024-09-12T03:19:20.374030+0000","last_fullsized":"2024-09-12T03:19:20.374030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:39:28.933609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375686+0000","last_change":"2024-09-12T03:19:07.207005+0000","last_active":"2024-09-12T03:19:20.375686+0000","last_peered":"2024-09-12T03:19:20.375686+0000","last_clean":"2024-09-12T03:19:20.375686+0000","last_became_active":"2024-09-12T03:19:07.206685+0000","last_became_peered":"2024-09-12T03:19:07.206685+0000","last_unstale":"2024-09-12T03:19:20.375686+0000","last_undegraded":"2024-09-12T03:19:20.375686+0000","last_fullsized":"2024-09-12T03:19:20.375686+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:31.111706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372028+0000","last_change":"2024-09-12T03:19:09.335871+0000","last_active":"2024-09-12T03:19:20.372028+0000","last_peered":"2024-09-12T03:19:20.372028+0000","last_clean":"2024-09-12T03:19:20.372028+0000","last_became_active":"2024-09-12T03:19:09.335568+0000","last_became_peered":"2024-09-12T03:19:09.335568+0000","last_unstale":"2024-09-12T03:19:20.372028+0000","last_undegraded":"2024-09-12T03:19:20.372028+0000","last_fullsized":"2024-09-12T03:19:20.372028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:52:34.471512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386049+0000","last_change":"2024-09-12T03:19:07.208320+0000","last_active":"2024-09-12T03:19:20.386049+0000","last_peered":"2024-09-12T03:19:20.386049+0000","last_clean":"2024-09-12T03:19:20.386049+0000","last_became_active":"2024-09-12T03:19:07.207873+0000","last_became_peered":"2024-09-12T03:19:07.207873+0000","last_unstale":"2024-09-12T03:19:20.386049+0000","last_undegraded":"2024-09-12T03:19:20.386049+0000","last_fullsized":"2024-09-12T03:19:20.386049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:00.128595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372467+0000","last_change":"2024-09-12T03:19:09.331930+0000","last_active":"2024-09-12T03:19:20.372467+0000","last_peered":"2024-09-12T03:19:20.372467+0000","last_clean":"2024-09-12T03:19:20.372467+0000","last_became_active":"2024-09-12T03:19:09.331558+0000","last_became_peered":"2024-09-12T03:19:09.331558+0000","last_unstale":"2024-09-12T03:19:20.372467+0000","last_undegraded":"2024-09-12T03:19:20.372467+0000","last_fullsized":"2024-09-12T03:19:20.372467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:00:27.935717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:31.399745+0000","last_change":"2024-09-12T03:19:07.205473+0000","last_active":"2024-09-12T03:19:31.399745+0000","last_peered":"2024-09-12T03:19:31.399745+0000","last_clean":"2024-09-12T03:19:31.399745+0000","last_became_active":"2024-09-12T03:19:07.205134+0000","last_became_peered":"2024-09-12T03:19:07.205134+0000","last_unstale":"2024-09-12T03:19:31.399745+0000","last_undegraded":"2024-09-12T03:19:31.399745+0000","last_fullsized":"2024-09-12T03:19:31.399745+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:23.443957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370511+0000","last_change":"2024-09-12T03:19:09.333868+0000","last_active":"2024-09-12T03:19:20.370511+0000","last_peered":"2024-09-12T03:19:20.370511+0000","last_clean":"2024-09-12T03:19:20.370511+0000","last_became_active":"2024-09-12T03:19:09.333484+0000","last_became_peered":"2024-09-12T03:19:09.333484+0000","last_unstale":"2024-09-12T03:19:20.370511+0000","last_undegraded":"2024-09-12T03:19:20.370511+0000","last_fullsized":"2024-09-12T03:19:20.370511+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:21:10.226821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386303+0000","last_change":"2024-09-12T03:19:20.386303+0000","last_active":"2024-09-12T03:19:20.386303+0000","last_peered":"2024-09-12T03:19:20.386303+0000","last_clean":"2024-09-12T03:19:20.386303+0000","last_became_active":"2024-09-12T03:19:20.383198+0000","last_became_peered":"2024-09-12T03:19:20.383198+0000","last_unstale":"2024-09-12T03:19:20.386303+0000","last_undegraded":"2024-09-12T03:19:20.386303+0000","last_fullsized":"2024-09-12T03:19:20.386303+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:42.224849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408655+0000","last_change":"2024-09-12T03:19:09.340849+0000","last_active":"2024-09-12T03:19:20.408655+0000","last_peered":"2024-09-12T03:19:20.408655+0000","last_clean":"2024-09-12T03:19:20.408655+0000","last_became_active":"2024-09-12T03:19:09.340517+0000","last_became_peered":"2024-09-12T03:19:09.340517+0000","last_unstale":"2024-09-12T03:19:20.408655+0000","last_undegraded":"2024-09-12T03:19:20.408655+0000","last_fullsized":"2024-09-12T03:19:20.408655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:42.018269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386240+0000","last_change":"2024-09-12T03:19:07.389147+0000","last_active":"2024-09-12T03:19:20.386240+0000","last_peered":"2024-09-12T03:19:20.386240+0000","last_clean":"2024-09-12T03:19:20.386240+0000","last_became_active":"2024-09-12T03:19:07.388744+0000","last_became_peered":"2024-09-12T03:19:07.388744+0000","last_unstale":"2024-09-12T03:19:20.386240+0000","last_undegraded":"2024-09-12T03:19:20.386240+0000","last_fullsized":"2024-09-12T03:19:20.386240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:16.700644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370280+0000","last_change":"2024-09-12T03:19:09.329243+0000","last_active":"2024-09-12T03:19:20.370280+0000","last_peered":"2024-09-12T03:19:20.370280+0000","last_clean":"2024-09-12T03:19:20.370280+0000","last_became_active":"2024-09-12T03:19:09.328610+0000","last_became_peered":"2024-09-12T03:19:09.328610+0000","last_unstale":"2024-09-12T03:19:20.370280+0000","last_undegraded":"2024-09-12T03:19:20.370280+0000","last_fullsized":"2024-09-12T03:19:20.370280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:49.452942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'25","reported_seq":80,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:32.401687+0000","last_change":"2024-09-12T03:19:07.207804+0000","last_active":"2024-09-12T03:19:32.401687+0000","last_peered":"2024-09-12T03:19:32.401687+0000","last_clean":"2024-09-12T03:19:32.401687+0000","last_became_active":"2024-09-12T03:19:07.207409+0000","last_became_peered":"2024-09-12T03:19:07.207409+0000","last_unstale":"2024-09-12T03:19:32.401687+0000","last_undegraded":"2024-09-12T03:19:32.401687+0000","last_fullsized":"2024-09-12T03:19:32.401687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:10.732423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50112,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":64,"num_write":26,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373666+0000","last_change":"2024-09-12T03:19:09.333102+0000","last_active":"2024-09-12T03:19:20.373666+0000","last_peered":"2024-09-12T03:19:20.373666+0000","last_clean":"2024-09-12T03:19:20.373666+0000","last_became_active":"2024-09-12T03:19:09.332712+0000","last_became_peered":"2024-09-12T03:19:09.332712+0000","last_unstale":"2024-09-12T03:19:20.373666+0000","last_undegraded":"2024-09-12T03:19:20.373666+0000","last_fullsized":"2024-09-12T03:19:20.373666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:22.912213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382902+0000","last_change":"2024-09-12T03:19:20.382348+0000","last_active":"2024-09-12T03:19:20.382902+0000","last_peered":"2024-09-12T03:19:20.382902+0000","last_clean":"2024-09-12T03:19:20.382902+0000","last_became_active":"2024-09-12T03:19:20.375142+0000","last_became_peered":"2024-09-12T03:19:20.375142+0000","last_unstale":"2024-09-12T03:19:20.382902+0000","last_undegraded":"2024-09-12T03:19:20.382902+0000","last_fullsized":"2024-09-12T03:19:20.382902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:04.573599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375046+0000","last_change":"2024-09-12T03:19:09.333194+0000","last_active":"2024-09-12T03:19:20.375046+0000","last_peered":"2024-09-12T03:19:20.375046+0000","last_clean":"2024-09-12T03:19:20.375046+0000","last_became_active":"2024-09-12T03:19:09.332865+0000","last_became_peered":"2024-09-12T03:19:09.332865+0000","last_unstale":"2024-09-12T03:19:20.375046+0000","last_undegraded":"2024-09-12T03:19:20.375046+0000","last_fullsized":"2024-09-12T03:19:20.375046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:57.096138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386176+0000","last_change":"2024-09-12T03:19:07.209289+0000","last_active":"2024-09-12T03:19:20.386176+0000","last_peered":"2024-09-12T03:19:20.386176+0000","last_clean":"2024-09-12T03:19:20.386176+0000","last_became_active":"2024-09-12T03:19:07.208863+0000","last_became_peered":"2024-09-12T03:19:07.208863+0000","last_unstale":"2024-09-12T03:19:20.386176+0000","last_undegraded":"2024-09-12T03:19:20.386176+0000","last_fullsized":"2024-09-12T03:19:20.386176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:26.168148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372400+0000","last_change":"2024-09-12T03:19:09.343620+0000","last_active":"2024-09-12T03:19:20.372400+0000","last_peered":"2024-09-12T03:19:20.372400+0000","last_clean":"2024-09-12T03:19:20.372400+0000","last_became_active":"2024-09-12T03:19:09.343299+0000","last_became_peered":"2024-09-12T03:19:09.343299+0000","last_unstale":"2024-09-12T03:19:20.372400+0000","last_undegraded":"2024-09-12T03:19:20.372400+0000","last_fullsized":"2024-09-12T03:19:20.372400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:34.268886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:30.397086+0000","last_change":"2024-09-12T03:19:07.206160+0000","last_active":"2024-09-12T03:19:30.397086+0000","last_peered":"2024-09-12T03:19:30.397086+0000","last_clean":"2024-09-12T03:19:30.397086+0000","last_became_active":"2024-09-12T03:19:07.205509+0000","last_became_peered":"2024-09-12T03:19:07.205509+0000","last_unstale":"2024-09-12T03:19:30.397086+0000","last_undegraded":"2024-09-12T03:19:30.397086+0000","last_fullsized":"2024-09-12T03:19:30.397086+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:35.362831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372068+0000","last_change":"2024-09-12T03:19:09.337595+0000","last_active":"2024-09-12T03:19:20.372068+0000","last_peered":"2024-09-12T03:19:20.372068+0000","last_clean":"2024-09-12T03:19:20.372068+0000","last_became_active":"2024-09-12T03:19:09.337281+0000","last_became_peered":"2024-09-12T03:19:09.337281+0000","last_unstale":"2024-09-12T03:19:20.372068+0000","last_undegraded":"2024-09-12T03:19:20.372068+0000","last_fullsized":"2024-09-12T03:19:20.372068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:35.761679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373995+0000","last_change":"2024-09-12T03:19:07.389336+0000","last_active":"2024-09-12T03:19:20.373995+0000","last_peered":"2024-09-12T03:19:20.373995+0000","last_clean":"2024-09-12T03:19:20.373995+0000","last_became_active":"2024-09-12T03:19:07.388649+0000","last_became_peered":"2024-09-12T03:19:07.388649+0000","last_unstale":"2024-09-12T03:19:20.373995+0000","last_undegraded":"2024-09-12T03:19:20.373995+0000","last_fullsized":"2024-09-12T03:19:20.373995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:33.312700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386030+0000","last_change":"2024-09-12T03:19:09.340674+0000","last_active":"2024-09-12T03:19:20.386030+0000","last_peered":"2024-09-12T03:19:20.386030+0000","last_clean":"2024-09-12T03:19:20.386030+0000","last_became_active":"2024-09-12T03:19:09.340144+0000","last_became_peered":"2024-09-12T03:19:09.340144+0000","last_unstale":"2024-09-12T03:19:20.386030+0000","last_undegraded":"2024-09-12T03:19:20.386030+0000","last_fullsized":"2024-09-12T03:19:20.386030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:21:15.605086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373837+0000","last_change":"2024-09-12T03:19:07.206513+0000","last_active":"2024-09-12T03:19:20.373837+0000","last_peered":"2024-09-12T03:19:20.373837+0000","last_clean":"2024-09-12T03:19:20.373837+0000","last_became_active":"2024-09-12T03:19:07.205974+0000","last_became_peered":"2024-09-12T03:19:07.205974+0000","last_unstale":"2024-09-12T03:19:20.373837+0000","last_undegraded":"2024-09-12T03:19:20.373837+0000","last_fullsized":"2024-09-12T03:19:20.373837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:43:10.002123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.371052+0000","last_change":"2024-09-12T03:19:09.342200+0000","last_active":"2024-09-12T03:19:20.371052+0000","last_peered":"2024-09-12T03:19:20.371052+0000","last_clean":"2024-09-12T03:19:20.371052+0000","last_became_active":"2024-09-12T03:19:09.341901+0000","last_became_peered":"2024-09-12T03:19:09.341901+0000","last_unstale":"2024-09-12T03:19:20.371052+0000","last_undegraded":"2024-09-12T03:19:20.371052+0000","last_fullsized":"2024-09-12T03:19:20.371052+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:27.486214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375373+0000","last_change":"2024-09-12T03:19:07.207514+0000","last_active":"2024-09-12T03:19:20.375373+0000","last_peered":"2024-09-12T03:19:20.375373+0000","last_clean":"2024-09-12T03:19:20.375373+0000","last_became_active":"2024-09-12T03:19:07.205925+0000","last_became_peered":"2024-09-12T03:19:07.205925+0000","last_unstale":"2024-09-12T03:19:20.375373+0000","last_undegraded":"2024-09-12T03:19:20.375373+0000","last_fullsized":"2024-09-12T03:19:20.375373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:20.244703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372696+0000","last_change":"2024-09-12T03:19:09.337837+0000","last_active":"2024-09-12T03:19:20.372696+0000","last_peered":"2024-09-12T03:19:20.372696+0000","last_clean":"2024-09-12T03:19:20.372696+0000","last_became_active":"2024-09-12T03:19:09.337502+0000","last_became_peered":"2024-09-12T03:19:09.337502+0000","last_unstale":"2024-09-12T03:19:20.372696+0000","last_undegraded":"2024-09-12T03:19:20.372696+0000","last_fullsized":"2024-09-12T03:19:20.372696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:30.167139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386356+0000","last_change":"2024-09-12T03:19:07.209130+0000","last_active":"2024-09-12T03:19:20.386356+0000","last_peered":"2024-09-12T03:19:20.386356+0000","last_clean":"2024-09-12T03:19:20.386356+0000","last_became_active":"2024-09-12T03:19:07.208753+0000","last_became_peered":"2024-09-12T03:19:07.208753+0000","last_unstale":"2024-09-12T03:19:20.386356+0000","last_undegraded":"2024-09-12T03:19:20.386356+0000","last_fullsized":"2024-09-12T03:19:20.386356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:34.719292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372500+0000","last_change":"2024-09-12T03:19:09.336382+0000","last_active":"2024-09-12T03:19:20.372500+0000","last_peered":"2024-09-12T03:19:20.372500+0000","last_clean":"2024-09-12T03:19:20.372500+0000","last_became_active":"2024-09-12T03:19:09.336015+0000","last_became_peered":"2024-09-12T03:19:09.336015+0000","last_unstale":"2024-09-12T03:19:20.372500+0000","last_undegraded":"2024-09-12T03:19:20.372500+0000","last_fullsized":"2024-09-12T03:19:20.372500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:19.430849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375044+0000","last_change":"2024-09-12T03:19:07.528567+0000","last_active":"2024-09-12T03:19:20.375044+0000","last_peered":"2024-09-12T03:19:20.375044+0000","last_clean":"2024-09-12T03:19:20.375044+0000","last_became_active":"2024-09-12T03:19:07.528123+0000","last_became_peered":"2024-09-12T03:19:07.528123+0000","last_unstale":"2024-09-12T03:19:20.375044+0000","last_undegraded":"2024-09-12T03:19:20.375044+0000","last_fullsized":"2024-09-12T03:19:20.375044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:34.416714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372773+0000","last_change":"2024-09-12T03:19:09.338836+0000","last_active":"2024-09-12T03:19:20.372773+0000","last_peered":"2024-09-12T03:19:20.372773+0000","last_clean":"2024-09-12T03:19:20.372773+0000","last_became_active":"2024-09-12T03:19:09.337571+0000","last_became_peered":"2024-09-12T03:19:09.337571+0000","last_unstale":"2024-09-12T03:19:20.372773+0000","last_undegraded":"2024-09-12T03:19:20.372773+0000","last_fullsized":"2024-09-12T03:19:20.372773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:39.039878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":329,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372694+0000","last_change":"2024-09-12T03:18:01.051311+0000","last_active":"2024-09-12T03:19:20.372694+0000","last_peered":"2024-09-12T03:19:20.372694+0000","last_clean":"2024-09-12T03:19:20.372694+0000","last_became_active":"2024-09-12T03:18:01.050849+0000","last_became_peered":"2024-09-12T03:18:01.050849+0000","last_unstale":"2024-09-12T03:19:20.372694+0000","last_undegraded":"2024-09-12T03:19:20.372694+0000","last_fullsized":"2024-09-12T03:19:20.372694+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:14:38.096222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:14:38.096222+0000","last_clean_scrub_stamp":"2024-09-12T03:14:38.096222+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:33.067603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382035+0000","last_change":"2024-09-12T03:19:07.206045+0000","last_active":"2024-09-12T03:19:20.382035+0000","last_peered":"2024-09-12T03:19:20.382035+0000","last_clean":"2024-09-12T03:19:20.382035+0000","last_became_active":"2024-09-12T03:19:07.205594+0000","last_became_peered":"2024-09-12T03:19:07.205594+0000","last_unstale":"2024-09-12T03:19:20.382035+0000","last_undegraded":"2024-09-12T03:19:20.382035+0000","last_fullsized":"2024-09-12T03:19:20.382035+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:12.811906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.387690+0000","last_change":"2024-09-12T03:19:20.387274+0000","last_active":"2024-09-12T03:19:20.387690+0000","last_peered":"2024-09-12T03:19:20.387690+0000","last_clean":"2024-09-12T03:19:20.387690+0000","last_became_active":"2024-09-12T03:19:20.386878+0000","last_became_peered":"2024-09-12T03:19:20.386878+0000","last_unstale":"2024-09-12T03:19:20.387690+0000","last_undegraded":"2024-09-12T03:19:20.387690+0000","last_fullsized":"2024-09-12T03:19:20.387690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:26.674032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372302+0000","last_change":"2024-09-12T03:19:07.206212+0000","last_active":"2024-09-12T03:19:20.372302+0000","last_peered":"2024-09-12T03:19:20.372302+0000","last_clean":"2024-09-12T03:19:20.372302+0000","last_became_active":"2024-09-12T03:19:07.205899+0000","last_became_peered":"2024-09-12T03:19:07.205899+0000","last_unstale":"2024-09-12T03:19:20.372302+0000","last_undegraded":"2024-09-12T03:19:20.372302+0000","last_fullsized":"2024-09-12T03:19:20.372302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:02.169436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.377494+0000","last_change":"2024-09-12T03:19:20.377494+0000","last_active":"2024-09-12T03:19:20.377494+0000","last_peered":"2024-09-12T03:19:20.377494+0000","last_clean":"2024-09-12T03:19:20.377494+0000","last_became_active":"2024-09-12T03:19:20.377091+0000","last_became_peered":"2024-09-12T03:19:20.377091+0000","last_unstale":"2024-09-12T03:19:20.377494+0000","last_undegraded":"2024-09-12T03:19:20.377494+0000","last_fullsized":"2024-09-12T03:19:20.377494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:30.926853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:31.397836+0000","last_change":"2024-09-12T03:19:07.388486+0000","last_active":"2024-09-12T03:19:31.397836+0000","last_peered":"2024-09-12T03:19:31.397836+0000","last_clean":"2024-09-12T03:19:31.397836+0000","last_became_active":"2024-09-12T03:19:07.388125+0000","last_became_peered":"2024-09-12T03:19:07.388125+0000","last_unstale":"2024-09-12T03:19:31.397836+0000","last_undegraded":"2024-09-12T03:19:31.397836+0000","last_fullsized":"2024-09-12T03:19:31.397836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:46.218601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.385912+0000","last_change":"2024-09-12T03:19:09.344308+0000","last_active":"2024-09-12T03:19:20.385912+0000","last_peered":"2024-09-12T03:19:20.385912+0000","last_clean":"2024-09-12T03:19:20.385912+0000","last_became_active":"2024-09-12T03:19:09.343987+0000","last_became_peered":"2024-09-12T03:19:09.343987+0000","last_unstale":"2024-09-12T03:19:20.385912+0000","last_undegraded":"2024-09-12T03:19:20.385912+0000","last_fullsized":"2024-09-12T03:19:20.385912+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:16:11.616555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372560+0000","last_change":"2024-09-12T03:19:07.208619+0000","last_active":"2024-09-12T03:19:20.372560+0000","last_peered":"2024-09-12T03:19:20.372560+0000","last_clean":"2024-09-12T03:19:20.372560+0000","last_became_active":"2024-09-12T03:19:07.208254+0000","last_became_peered":"2024-09-12T03:19:07.208254+0000","last_unstale":"2024-09-12T03:19:20.372560+0000","last_undegraded":"2024-09-12T03:19:20.372560+0000","last_fullsized":"2024-09-12T03:19:20.372560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:56.177948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374096+0000","last_change":"2024-09-12T03:19:09.341837+0000","last_active":"2024-09-12T03:19:20.374096+0000","last_peered":"2024-09-12T03:19:20.374096+0000","last_clean":"2024-09-12T03:19:20.374096+0000","last_became_active":"2024-09-12T03:19:09.341069+0000","last_became_peered":"2024-09-12T03:19:09.341069+0000","last_unstale":"2024-09-12T03:19:20.374096+0000","last_undegraded":"2024-09-12T03:19:20.374096+0000","last_fullsized":"2024-09-12T03:19:20.374096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:04.238986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372460+0000","last_change":"2024-09-12T03:19:07.388870+0000","last_active":"2024-09-12T03:19:20.372460+0000","last_peered":"2024-09-12T03:19:20.372460+0000","last_clean":"2024-09-12T03:19:20.372460+0000","last_became_active":"2024-09-12T03:19:07.388555+0000","last_became_peered":"2024-09-12T03:19:07.388555+0000","last_unstale":"2024-09-12T03:19:20.372460+0000","last_undegraded":"2024-09-12T03:19:20.372460+0000","last_fullsized":"2024-09-12T03:19:20.372460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:29.789454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374271+0000","last_change":"2024-09-12T03:19:09.334714+0000","last_active":"2024-09-12T03:19:20.374271+0000","last_peered":"2024-09-12T03:19:20.374271+0000","last_clean":"2024-09-12T03:19:20.374271+0000","last_became_active":"2024-09-12T03:19:09.334413+0000","last_became_peered":"2024-09-12T03:19:09.334413+0000","last_unstale":"2024-09-12T03:19:20.374271+0000","last_undegraded":"2024-09-12T03:19:20.374271+0000","last_fullsized":"2024-09-12T03:19:20.374271+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:52.222340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373797+0000","last_change":"2024-09-12T03:19:07.389055+0000","last_active":"2024-09-12T03:19:20.373797+0000","last_peered":"2024-09-12T03:19:20.373797+0000","last_clean":"2024-09-12T03:19:20.373797+0000","last_became_active":"2024-09-12T03:19:07.388673+0000","last_became_peered":"2024-09-12T03:19:07.388673+0000","last_unstale":"2024-09-12T03:19:20.373797+0000","last_undegraded":"2024-09-12T03:19:20.373797+0000","last_fullsized":"2024-09-12T03:19:20.373797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:20.974533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375304+0000","last_change":"2024-09-12T03:19:09.340105+0000","last_active":"2024-09-12T03:19:20.375304+0000","last_peered":"2024-09-12T03:19:20.375304+0000","last_clean":"2024-09-12T03:19:20.375304+0000","last_became_active":"2024-09-12T03:19:09.339642+0000","last_became_peered":"2024-09-12T03:19:09.339642+0000","last_unstale":"2024-09-12T03:19:20.375304+0000","last_undegraded":"2024-09-12T03:19:20.375304+0000","last_fullsized":"2024-09-12T03:19:20.375304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:01.624236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'11","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:32.332064+0000","last_change":"2024-09-12T03:19:07.207232+0000","last_active":"2024-09-12T03:19:32.332064+0000","last_peered":"2024-09-12T03:19:32.332064+0000","last_clean":"2024-09-12T03:19:32.332064+0000","last_became_active":"2024-09-12T03:19:07.206921+0000","last_became_peered":"2024-09-12T03:19:07.206921+0000","last_unstale":"2024-09-12T03:19:32.332064+0000","last_undegraded":"2024-09-12T03:19:32.332064+0000","last_fullsized":"2024-09-12T03:19:32.332064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:25.457853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372958+0000","last_change":"2024-09-12T03:19:09.330442+0000","last_active":"2024-09-12T03:19:20.372958+0000","last_peered":"2024-09-12T03:19:20.372958+0000","last_clean":"2024-09-12T03:19:20.372958+0000","last_became_active":"2024-09-12T03:19:09.330081+0000","last_became_peered":"2024-09-12T03:19:09.330081+0000","last_unstale":"2024-09-12T03:19:20.372958+0000","last_undegraded":"2024-09-12T03:19:20.372958+0000","last_fullsized":"2024-09-12T03:19:20.372958+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:56.152562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382392+0000","last_change":"2024-09-12T03:19:07.528646+0000","last_active":"2024-09-12T03:19:20.382392+0000","last_peered":"2024-09-12T03:19:20.382392+0000","last_clean":"2024-09-12T03:19:20.382392+0000","last_became_active":"2024-09-12T03:19:07.528334+0000","last_became_peered":"2024-09-12T03:19:07.528334+0000","last_unstale":"2024-09-12T03:19:20.382392+0000","last_undegraded":"2024-09-12T03:19:20.382392+0000","last_fullsized":"2024-09-12T03:19:20.382392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:40.425579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408541+0000","last_change":"2024-09-12T03:19:09.331993+0000","last_active":"2024-09-12T03:19:20.408541+0000","last_peered":"2024-09-12T03:19:20.408541+0000","last_clean":"2024-09-12T03:19:20.408541+0000","last_became_active":"2024-09-12T03:19:09.331698+0000","last_became_peered":"2024-09-12T03:19:09.331698+0000","last_unstale":"2024-09-12T03:19:20.408541+0000","last_undegraded":"2024-09-12T03:19:20.408541+0000","last_fullsized":"2024-09-12T03:19:20.408541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:03:54.182833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370179+0000","last_change":"2024-09-12T03:19:07.388297+0000","last_active":"2024-09-12T03:19:20.370179+0000","last_peered":"2024-09-12T03:19:20.370179+0000","last_clean":"2024-09-12T03:19:20.370179+0000","last_became_active":"2024-09-12T03:19:07.388038+0000","last_became_peered":"2024-09-12T03:19:07.388038+0000","last_unstale":"2024-09-12T03:19:20.370179+0000","last_undegraded":"2024-09-12T03:19:20.370179+0000","last_fullsized":"2024-09-12T03:19:20.370179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:45.874195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373463+0000","last_change":"2024-09-12T03:19:09.331525+0000","last_active":"2024-09-12T03:19:20.373463+0000","last_peered":"2024-09-12T03:19:20.373463+0000","last_clean":"2024-09-12T03:19:20.373463+0000","last_became_active":"2024-09-12T03:19:09.330903+0000","last_became_peered":"2024-09-12T03:19:09.330903+0000","last_unstale":"2024-09-12T03:19:20.373463+0000","last_undegraded":"2024-09-12T03:19:20.373463+0000","last_fullsized":"2024-09-12T03:19:20.373463+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:02.455132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372235+0000","last_change":"2024-09-12T03:19:07.205181+0000","last_active":"2024-09-12T03:19:20.372235+0000","last_peered":"2024-09-12T03:19:20.372235+0000","last_clean":"2024-09-12T03:19:20.372235+0000","last_became_active":"2024-09-12T03:19:07.204726+0000","last_became_peered":"2024-09-12T03:19:07.204726+0000","last_unstale":"2024-09-12T03:19:20.372235+0000","last_undegraded":"2024-09-12T03:19:20.372235+0000","last_fullsized":"2024-09-12T03:19:20.372235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:26.112800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370908+0000","last_change":"2024-09-12T03:19:09.345338+0000","last_active":"2024-09-12T03:19:20.370908+0000","last_peered":"2024-09-12T03:19:20.370908+0000","last_clean":"2024-09-12T03:19:20.370908+0000","last_became_active":"2024-09-12T03:19:09.345023+0000","last_became_peered":"2024-09-12T03:19:09.345023+0000","last_unstale":"2024-09-12T03:19:20.370908+0000","last_undegraded":"2024-09-12T03:19:20.370908+0000","last_fullsized":"2024-09-12T03:19:20.370908+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:39.184065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374938+0000","last_change":"2024-09-12T03:19:07.211954+0000","last_active":"2024-09-12T03:19:20.374938+0000","last_peered":"2024-09-12T03:19:20.374938+0000","last_clean":"2024-09-12T03:19:20.374938+0000","last_became_active":"2024-09-12T03:19:07.211509+0000","last_became_peered":"2024-09-12T03:19:07.211509+0000","last_unstale":"2024-09-12T03:19:20.374938+0000","last_undegraded":"2024-09-12T03:19:20.374938+0000","last_fullsized":"2024-09-12T03:19:20.374938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:27.461924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373018+0000","last_change":"2024-09-12T03:19:09.331168+0000","last_active":"2024-09-12T03:19:20.373018+0000","last_peered":"2024-09-12T03:19:20.373018+0000","last_clean":"2024-09-12T03:19:20.373018+0000","last_became_active":"2024-09-12T03:19:09.329592+0000","last_became_peered":"2024-09-12T03:19:09.329592+0000","last_unstale":"2024-09-12T03:19:20.373018+0000","last_undegraded":"2024-09-12T03:19:20.373018+0000","last_fullsized":"2024-09-12T03:19:20.373018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:46.127861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373572+0000","last_change":"2024-09-12T03:19:07.207802+0000","last_active":"2024-09-12T03:19:20.373572+0000","last_peered":"2024-09-12T03:19:20.373572+0000","last_clean":"2024-09-12T03:19:20.373572+0000","last_became_active":"2024-09-12T03:19:07.207428+0000","last_became_peered":"2024-09-12T03:19:07.207428+0000","last_unstale":"2024-09-12T03:19:20.373572+0000","last_undegraded":"2024-09-12T03:19:20.373572+0000","last_fullsized":"2024-09-12T03:19:20.373572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:45.135019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372336+0000","last_change":"2024-09-12T03:19:09.336094+0000","last_active":"2024-09-12T03:19:20.372336+0000","last_peered":"2024-09-12T03:19:20.372336+0000","last_clean":"2024-09-12T03:19:20.372336+0000","last_became_active":"2024-09-12T03:19:09.335779+0000","last_became_peered":"2024-09-12T03:19:09.335779+0000","last_unstale":"2024-09-12T03:19:20.372336+0000","last_undegraded":"2024-09-12T03:19:20.372336+0000","last_fullsized":"2024-09-12T03:19:20.372336+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:37.019082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386131+0000","last_change":"2024-09-12T03:19:09.344537+0000","last_active":"2024-09-12T03:19:20.386131+0000","last_peered":"2024-09-12T03:19:20.386131+0000","last_clean":"2024-09-12T03:19:20.386131+0000","last_became_active":"2024-09-12T03:19:09.343879+0000","last_became_peered":"2024-09-12T03:19:09.343879+0000","last_unstale":"2024-09-12T03:19:20.386131+0000","last_undegraded":"2024-09-12T03:19:20.386131+0000","last_fullsized":"2024-09-12T03:19:20.386131+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:40.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373973+0000","last_change":"2024-09-12T03:19:07.208040+0000","last_active":"2024-09-12T03:19:20.373973+0000","last_peered":"2024-09-12T03:19:20.373973+0000","last_clean":"2024-09-12T03:19:20.373973+0000","last_became_active":"2024-09-12T03:19:07.207658+0000","last_became_peered":"2024-09-12T03:19:07.207658+0000","last_unstale":"2024-09-12T03:19:20.373973+0000","last_undegraded":"2024-09-12T03:19:20.373973+0000","last_fullsized":"2024-09-12T03:19:20.373973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:48.620707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370231+0000","last_change":"2024-09-12T03:19:09.333837+0000","last_active":"2024-09-12T03:19:20.370231+0000","last_peered":"2024-09-12T03:19:20.370231+0000","last_clean":"2024-09-12T03:19:20.370231+0000","last_became_active":"2024-09-12T03:19:09.333194+0000","last_became_peered":"2024-09-12T03:19:09.333194+0000","last_unstale":"2024-09-12T03:19:20.370231+0000","last_undegraded":"2024-09-12T03:19:20.370231+0000","last_fullsized":"2024-09-12T03:19:20.370231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:18.766181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.385455+0000","last_change":"2024-09-12T03:19:20.384978+0000","last_active":"2024-09-12T03:19:20.385455+0000","last_peered":"2024-09-12T03:19:20.385455+0000","last_clean":"2024-09-12T03:19:20.385455+0000","last_became_active":"2024-09-12T03:19:20.376883+0000","last_became_peered":"2024-09-12T03:19:20.376883+0000","last_unstale":"2024-09-12T03:19:20.385455+0000","last_undegraded":"2024-09-12T03:19:20.385455+0000","last_fullsized":"2024-09-12T03:19:20.385455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:32.542334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408633+0000","last_change":"2024-09-12T03:19:09.334684+0000","last_active":"2024-09-12T03:19:20.408633+0000","last_peered":"2024-09-12T03:19:20.408633+0000","last_clean":"2024-09-12T03:19:20.408633+0000","last_became_active":"2024-09-12T03:19:09.334281+0000","last_became_peered":"2024-09-12T03:19:09.334281+0000","last_unstale":"2024-09-12T03:19:20.408633+0000","last_undegraded":"2024-09-12T03:19:20.408633+0000","last_fullsized":"2024-09-12T03:19:20.408633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:44.837067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372829+0000","last_change":"2024-09-12T03:19:07.388555+0000","last_active":"2024-09-12T03:19:20.372829+0000","last_peered":"2024-09-12T03:19:20.372829+0000","last_clean":"2024-09-12T03:19:20.372829+0000","last_became_active":"2024-09-12T03:19:07.388206+0000","last_became_peered":"2024-09-12T03:19:07.388206+0000","last_unstale":"2024-09-12T03:19:20.372829+0000","last_undegraded":"2024-09-12T03:19:20.372829+0000","last_fullsized":"2024-09-12T03:19:20.372829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:33.628294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372626+0000","last_change":"2024-09-12T03:19:09.339108+0000","last_active":"2024-09-12T03:19:20.372626+0000","last_peered":"2024-09-12T03:19:20.372626+0000","last_clean":"2024-09-12T03:19:20.372626+0000","last_became_active":"2024-09-12T03:19:09.338819+0000","last_became_peered":"2024-09-12T03:19:09.338819+0000","last_unstale":"2024-09-12T03:19:20.372626+0000","last_undegraded":"2024-09-12T03:19:20.372626+0000","last_fullsized":"2024-09-12T03:19:20.372626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:32.864900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372798+0000","last_change":"2024-09-12T03:19:07.207197+0000","last_active":"2024-09-12T03:19:20.372798+0000","last_peered":"2024-09-12T03:19:20.372798+0000","last_clean":"2024-09-12T03:19:20.372798+0000","last_became_active":"2024-09-12T03:19:07.206879+0000","last_became_peered":"2024-09-12T03:19:07.206879+0000","last_unstale":"2024-09-12T03:19:20.372798+0000","last_undegraded":"2024-09-12T03:19:20.372798+0000","last_fullsized":"2024-09-12T03:19:20.372798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:00:46.269636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54115,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":103,"num_read_kb":110,"num_write":159,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":153298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579921,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":892928,"data_stored":433387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36884,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708268,"statfs":{"total":95995035648,"available":95957266432,"internally_reserved":0,"allocated":1576960,"data_stored":1074946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972378,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":433589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201312,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":433509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430245,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1499136,"data_stored":1025441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593770,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":432978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822702,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1511424,"data_stored":1025125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953524,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":897024,"data_stored":433415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":433289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":475545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40416,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704736,"statfs":{"total":95995035648,"available":95953649664,"internally_reserved":0,"allocated":933888,"data_stored":473836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":913408,"data_stored":434249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":41065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":51821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":51690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:19:36.625 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T03:19:36.625 INFO:tasks.cephadm:Setting up client nodes... 2024-09-12T03:19:36.626 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-12T03:19:36.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:36 smithi118 ceph-mon[35048]: pgmap v290: 97 pgs: 97 active+clean; 630 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-09-12T03:19:36.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:36 smithi118 ceph-mon[35048]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:36.889 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:37.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.310+0000 7fd56fbc9640 1 -- 172.21.15.6:0/3596415556 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5681026d0 msgr2=0x7fd568108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.310+0000 7fd56fbc9640 1 --2- 172.21.15.6:0/3596415556 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5681026d0 0x7fd568108f60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd560009ca0 tx=0x7fd56002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.310+0000 7fd56fbc9640 1 -- 172.21.15.6:0/3596415556 shutdown_connections 2024-09-12T03:19:37.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.310+0000 7fd56fbc9640 1 --2- 172.21.15.6:0/3596415556 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5681026d0 0x7fd568108f60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.310+0000 7fd56fbc9640 1 --2- 172.21.15.6:0/3596415556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd568101d10 0x7fd568102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.310+0000 7fd56fbc9640 1 --2- 172.21.15.6:0/3596415556 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd568100b10 0x7fd568100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.310+0000 7fd56fbc9640 1 -- 172.21.15.6:0/3596415556 >> 172.21.15.6:0/3596415556 conn(0x7fd5680fc2a0 msgr2=0x7fd5680fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:37.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.310+0000 7fd56fbc9640 1 -- 172.21.15.6:0/3596415556 shutdown_connections 2024-09-12T03:19:37.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.310+0000 7fd56fbc9640 1 -- 172.21.15.6:0/3596415556 wait complete. 2024-09-12T03:19:37.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.311+0000 7fd56fbc9640 1 Processor -- start 2024-09-12T03:19:37.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.311+0000 7fd56fbc9640 1 -- start start 2024-09-12T03:19:37.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.311+0000 7fd56fbc9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd568100b10 0x7fd568104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.311+0000 7fd56fbc9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd568101d10 0x7fd5681030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.311+0000 7fd56fbc9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5681026d0 0x7fd568103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.311+0000 7fd56fbc9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd56810c790 con 0x7fd568100b10 2024-09-12T03:19:37.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.311+0000 7fd56fbc9640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fd56810c610 con 0x7fd5681026d0 2024-09-12T03:19:37.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.311+0000 7fd56fbc9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd56810c910 con 0x7fd568101d10 2024-09-12T03:19:37.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.311+0000 7fd56d965640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd568100b10 0x7fd568104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.311+0000 7fd56d965640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd568100b10 0x7fd568104aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59326/0 (socket says 172.21.15.6:59326) 2024-09-12T03:19:37.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.311+0000 7fd56d965640 1 -- 172.21.15.6:0/1298217717 learned_addr learned my addr 172.21.15.6:0/1298217717 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:37.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.312+0000 7fd56d164640 1 --2- 172.21.15.6:0/1298217717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd568101d10 0x7fd5681030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.312+0000 7fd56e166640 1 --2- 172.21.15.6:0/1298217717 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5681026d0 0x7fd568103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.312+0000 7fd56d965640 1 -- 172.21.15.6:0/1298217717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd568101d10 msgr2=0x7fd5681030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.312+0000 7fd56d965640 1 --2- 172.21.15.6:0/1298217717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd568101d10 0x7fd5681030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.312+0000 7fd56d965640 1 -- 172.21.15.6:0/1298217717 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5681026d0 msgr2=0x7fd568103630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.312+0000 7fd56d965640 1 --2- 172.21.15.6:0/1298217717 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5681026d0 0x7fd568103630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.312+0000 7fd56d965640 1 -- 172.21.15.6:0/1298217717 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd568103ef0 con 0x7fd568100b10 2024-09-12T03:19:37.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.312+0000 7fd56e166640 1 --2- 172.21.15.6:0/1298217717 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5681026d0 0x7fd568103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:37.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.312+0000 7fd56d965640 1 --2- 172.21.15.6:0/1298217717 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd568100b10 0x7fd568104aa0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd56400c970 tx=0x7fd56400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:37.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.312+0000 7fd55effd640 1 -- 172.21.15.6:0/1298217717 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564007c10 con 0x7fd568100b10 2024-09-12T03:19:37.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.312+0000 7fd56fbc9640 1 -- 172.21.15.6:0/1298217717 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5681a1fe0 con 0x7fd568100b10 2024-09-12T03:19:37.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.312+0000 7fd55effd640 1 -- 172.21.15.6:0/1298217717 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd564007db0 con 0x7fd568100b10 2024-09-12T03:19:37.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.312+0000 7fd55effd640 1 -- 172.21.15.6:0/1298217717 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56400f8a0 con 0x7fd568100b10 2024-09-12T03:19:37.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.312+0000 7fd56fbc9640 1 -- 172.21.15.6:0/1298217717 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5681a2490 con 0x7fd568100b10 2024-09-12T03:19:37.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.313+0000 7fd56fbc9640 1 -- 172.21.15.6:0/1298217717 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5681a2720 con 0x7fd568100b10 2024-09-12T03:19:37.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.315+0000 7fd55effd640 1 -- 172.21.15.6:0/1298217717 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fd56400fa40 con 0x7fd568100b10 2024-09-12T03:19:37.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.315+0000 7fd55effd640 1 --2- 172.21.15.6:0/1298217717 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd5540a2c10 0x7fd5540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.315+0000 7fd56d164640 1 --2- 172.21.15.6:0/1298217717 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd5540a2c10 0x7fd5540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.315+0000 7fd55effd640 1 -- 172.21.15.6:0/1298217717 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd5640cf7f0 con 0x7fd568100b10 2024-09-12T03:19:37.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.316+0000 7fd56d164640 1 --2- 172.21.15.6:0/1298217717 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd5540a2c10 0x7fd5540a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd558005fd0 tx=0x7fd558005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:37.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.318+0000 7fd55effd640 1 -- 172.21.15.6:0/1298217717 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd56409c740 con 0x7fd568100b10 2024-09-12T03:19:37.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.561+0000 7fd56fbc9640 1 -- 172.21.15.6:0/1298217717 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fd5681a2990 con 0x7fd568100b10 2024-09-12T03:19:37.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.563+0000 7fd55effd640 1 -- 172.21.15.6:0/1298217717 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fd56409c040 con 0x7fd568100b10 2024-09-12T03:19:37.563 INFO:teuthology.orchestra.run.smithi006.stdout:[client.0] 2024-09-12T03:19:37.564 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQDJXeJmQiSNIRAA3FrraNL4G2Pg5nai8ZNgXg== 2024-09-12T03:19:37.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.565+0000 7fd56fbc9640 1 -- 172.21.15.6:0/1298217717 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd5540a2c10 msgr2=0x7fd5540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.565+0000 7fd56fbc9640 1 --2- 172.21.15.6:0/1298217717 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd5540a2c10 0x7fd5540a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd558005fd0 tx=0x7fd558005e20 comp rx=0 tx=0).stop 2024-09-12T03:19:37.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.565+0000 7fd56fbc9640 1 -- 172.21.15.6:0/1298217717 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd568100b10 msgr2=0x7fd568104aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.565+0000 7fd56fbc9640 1 --2- 172.21.15.6:0/1298217717 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd568100b10 0x7fd568104aa0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd56400c970 tx=0x7fd56400ce30 comp rx=0 tx=0).stop 2024-09-12T03:19:37.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.566+0000 7fd56fbc9640 1 -- 172.21.15.6:0/1298217717 shutdown_connections 2024-09-12T03:19:37.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.566+0000 7fd56fbc9640 1 --2- 172.21.15.6:0/1298217717 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd5540a2c10 0x7fd5540a50d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.566+0000 7fd56fbc9640 1 --2- 172.21.15.6:0/1298217717 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5681026d0 0x7fd568103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.566+0000 7fd56fbc9640 1 --2- 172.21.15.6:0/1298217717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd568101d10 0x7fd5681030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.566+0000 7fd56fbc9640 1 --2- 172.21.15.6:0/1298217717 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd568100b10 0x7fd568104aa0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.566+0000 7fd56fbc9640 1 -- 172.21.15.6:0/1298217717 >> 172.21.15.6:0/1298217717 conn(0x7fd5680fc2a0 msgr2=0x7fd5680fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:37.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.566+0000 7fd56fbc9640 1 -- 172.21.15.6:0/1298217717 shutdown_connections 2024-09-12T03:19:37.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:37.566+0000 7fd56fbc9640 1 -- 172.21.15.6:0/1298217717 wait complete. 2024-09-12T03:19:38.120 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-09-12T03:19:38.120 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-12T03:19:38.120 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-12T03:19:38.160 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-12T03:19:38.160 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-12T03:19:38.161 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-12T03:19:38.457 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:38.573 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:38 smithi006 ceph-mon[30991]: pgmap v291: 97 pgs: 97 active+clean; 630 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-12T03:19:38.573 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:38 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1298217717' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T03:19:38.573 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:38 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1298217717' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T03:19:38.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:38 smithi161 ceph-mon[35485]: pgmap v291: 97 pgs: 97 active+clean; 630 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-12T03:19:38.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:38 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1298217717' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T03:19:38.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:38 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1298217717' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T03:19:38.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:38 smithi118 ceph-mon[35048]: pgmap v291: 97 pgs: 97 active+clean; 630 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-12T03:19:38.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:38 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1298217717' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T03:19:38.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:38 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1298217717' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T03:19:38.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.915+0000 7fe564903640 1 -- 172.21.15.6:0/2866171463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe560101de0 msgr2=0x7fe5601021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:38.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.915+0000 7fe564903640 1 --2- 172.21.15.6:0/2866171463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe560101de0 0x7fe5601021c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe550009900 tx=0x7fe55002a410 comp rx=0 tx=0).stop 2024-09-12T03:19:38.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.915+0000 7fe564903640 1 -- 172.21.15.6:0/2866171463 shutdown_connections 2024-09-12T03:19:38.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.915+0000 7fe564903640 1 --2- 172.21.15.6:0/2866171463 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe560106f10 0x7fe56010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.915+0000 7fe564903640 1 --2- 172.21.15.6:0/2866171463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe560102790 0x7fe5601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.915+0000 7fe564903640 1 --2- 172.21.15.6:0/2866171463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe560101de0 0x7fe5601021c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.915+0000 7fe564903640 1 -- 172.21.15.6:0/2866171463 >> 172.21.15.6:0/2866171463 conn(0x7fe5600fd8c0 msgr2=0x7fe5600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:38.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.915+0000 7fe564903640 1 -- 172.21.15.6:0/2866171463 shutdown_connections 2024-09-12T03:19:38.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.915+0000 7fe564903640 1 -- 172.21.15.6:0/2866171463 wait complete. 2024-09-12T03:19:38.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.916+0000 7fe564903640 1 Processor -- start 2024-09-12T03:19:38.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.916+0000 7fe564903640 1 -- start start 2024-09-12T03:19:38.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.916+0000 7fe564903640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe560101de0 0x7fe56019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:38.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.916+0000 7fe564903640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe560102790 0x7fe56019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:38.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.916+0000 7fe564903640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe560106f10 0x7fe56019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:38.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.916+0000 7fe564903640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe560111f70 con 0x7fe560101de0 2024-09-12T03:19:38.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.916+0000 7fe564903640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe560111df0 con 0x7fe560106f10 2024-09-12T03:19:38.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.916+0000 7fe564903640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe5601120f0 con 0x7fe560102790 2024-09-12T03:19:38.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.916+0000 7fe55e59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe560101de0 0x7fe56019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:38.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.917+0000 7fe55e59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe560101de0 0x7fe56019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59342/0 (socket says 172.21.15.6:59342) 2024-09-12T03:19:38.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.917+0000 7fe55e59c640 1 -- 172.21.15.6:0/3745004181 learned_addr learned my addr 172.21.15.6:0/3745004181 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:38.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.917+0000 7fe55dd9b640 1 --2- 172.21.15.6:0/3745004181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe560102790 0x7fe56019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:38.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.917+0000 7fe55ed9d640 1 --2- 172.21.15.6:0/3745004181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe560106f10 0x7fe56019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:38.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.917+0000 7fe55e59c640 1 -- 172.21.15.6:0/3745004181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe560102790 msgr2=0x7fe56019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:38.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.917+0000 7fe55e59c640 1 --2- 172.21.15.6:0/3745004181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe560102790 0x7fe56019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.917+0000 7fe55e59c640 1 -- 172.21.15.6:0/3745004181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe560106f10 msgr2=0x7fe56019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:38.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.917+0000 7fe55e59c640 1 --2- 172.21.15.6:0/3745004181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe560106f10 0x7fe56019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.917+0000 7fe55e59c640 1 -- 172.21.15.6:0/3745004181 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56019f540 con 0x7fe560101de0 2024-09-12T03:19:38.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.917+0000 7fe55e59c640 1 --2- 172.21.15.6:0/3745004181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe560101de0 0x7fe56019a590 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe550005ab0 tx=0x7fe5500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:38.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.917+0000 7fe54f7fe640 1 -- 172.21.15.6:0/3745004181 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55003f030 con 0x7fe560101de0 2024-09-12T03:19:38.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.917+0000 7fe54f7fe640 1 -- 172.21.15.6:0/3745004181 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe550003ce0 con 0x7fe560101de0 2024-09-12T03:19:38.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.917+0000 7fe54f7fe640 1 -- 172.21.15.6:0/3745004181 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe550035070 con 0x7fe560101de0 2024-09-12T03:19:38.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.917+0000 7fe564903640 1 -- 172.21.15.6:0/3745004181 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56019f7d0 con 0x7fe560101de0 2024-09-12T03:19:38.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.917+0000 7fe564903640 1 -- 172.21.15.6:0/3745004181 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5601a7110 con 0x7fe560101de0 2024-09-12T03:19:38.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.918+0000 7fe564903640 1 -- 172.21.15.6:0/3745004181 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5601021c0 con 0x7fe560101de0 2024-09-12T03:19:38.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.919+0000 7fe54f7fe640 1 -- 172.21.15.6:0/3745004181 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe550042030 con 0x7fe560101de0 2024-09-12T03:19:38.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.920+0000 7fe54f7fe640 1 --2- 172.21.15.6:0/3745004181 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5480a2c10 0x7fe5480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:38.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.920+0000 7fe55dd9b640 1 --2- 172.21.15.6:0/3745004181 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5480a2c10 0x7fe5480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:38.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.920+0000 7fe54f7fe640 1 -- 172.21.15.6:0/3745004181 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe5500efad0 con 0x7fe560101de0 2024-09-12T03:19:38.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.921+0000 7fe55dd9b640 1 --2- 172.21.15.6:0/3745004181 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5480a2c10 0x7fe5480a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe5600fdf70 tx=0x7fe55400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:38.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:38.924+0000 7fe54f7fe640 1 -- 172.21.15.6:0/3745004181 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe5500bc900 con 0x7fe560101de0 2024-09-12T03:19:39.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:39.148+0000 7fe564903640 1 -- 172.21.15.6:0/3745004181 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fe5601a72e0 con 0x7fe560101de0 2024-09-12T03:19:39.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:39.152+0000 7fe54f7fe640 1 -- 172.21.15.6:0/3745004181 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263182 (secure 0 0 0) 0x7fe5500bc200 con 0x7fe560101de0 2024-09-12T03:19:39.156 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:19:39.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:39.155+0000 7fe564903640 1 -- 172.21.15.6:0/3745004181 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5480a2c10 msgr2=0x7fe5480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:39.155+0000 7fe564903640 1 --2- 172.21.15.6:0/3745004181 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5480a2c10 0x7fe5480a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe5600fdf70 tx=0x7fe55400a2d0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:39.155+0000 7fe564903640 1 -- 172.21.15.6:0/3745004181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe560101de0 msgr2=0x7fe56019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:39.155+0000 7fe564903640 1 --2- 172.21.15.6:0/3745004181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe560101de0 0x7fe56019a590 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe550005ab0 tx=0x7fe5500042f0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:39.155+0000 7fe564903640 1 -- 172.21.15.6:0/3745004181 shutdown_connections 2024-09-12T03:19:39.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:39.155+0000 7fe564903640 1 --2- 172.21.15.6:0/3745004181 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5480a2c10 0x7fe5480a50d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:39.155+0000 7fe564903640 1 --2- 172.21.15.6:0/3745004181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe560106f10 0x7fe56019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:39.155+0000 7fe564903640 1 --2- 172.21.15.6:0/3745004181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe560102790 0x7fe56019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:39.155+0000 7fe564903640 1 --2- 172.21.15.6:0/3745004181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe560101de0 0x7fe56019a590 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:39.155+0000 7fe564903640 1 -- 172.21.15.6:0/3745004181 >> 172.21.15.6:0/3745004181 conn(0x7fe5600fd8c0 msgr2=0x7fe5600fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:39.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:39.155+0000 7fe564903640 1 -- 172.21.15.6:0/3745004181 shutdown_connections 2024-09-12T03:19:39.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:39.156+0000 7fe564903640 1 -- 172.21.15.6:0/3745004181 wait complete. 2024-09-12T03:19:39.575 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:39 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3745004181' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T03:19:39.733 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:39 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3745004181' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T03:19:39.733 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:39 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3745004181' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T03:19:39.790 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":5477767},{"type":"v1","addr":"172.21.15.6:6801","nonce":5477767}]},"active_addr":"172.21.15.6:6801/5477767","active_change":"2024-09-12T03:12:19.155375+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24115,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.6:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":3076856827}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":933258147}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":2549978381}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":1720356888}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":3910648088}]}]} 2024-09-12T03:19:39.796 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-12T03:19:39.796 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-12T03:19:39.797 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:19:40.051 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:40.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:40 smithi006 ceph-mon[30991]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2024-09-12T03:19:40.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.539+0000 7f39f109e640 1 -- 172.21.15.6:0/3246825913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39ec101de0 msgr2=0x7f39ec1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:40.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.539+0000 7f39f109e640 1 --2- 172.21.15.6:0/3246825913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39ec101de0 0x7f39ec1021c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f39dc009900 tx=0x7f39dc02a410 comp rx=0 tx=0).stop 2024-09-12T03:19:40.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.539+0000 7f39f109e640 1 -- 172.21.15.6:0/3246825913 shutdown_connections 2024-09-12T03:19:40.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.539+0000 7f39f109e640 1 --2- 172.21.15.6:0/3246825913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39ec106f10 0x7f39ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.539+0000 7f39f109e640 1 --2- 172.21.15.6:0/3246825913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39ec102790 0x7f39ec1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.539+0000 7f39f109e640 1 --2- 172.21.15.6:0/3246825913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39ec101de0 0x7f39ec1021c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.539+0000 7f39f109e640 1 -- 172.21.15.6:0/3246825913 >> 172.21.15.6:0/3246825913 conn(0x7f39ec0fd8c0 msgr2=0x7f39ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:40.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.539+0000 7f39f109e640 1 -- 172.21.15.6:0/3246825913 shutdown_connections 2024-09-12T03:19:40.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.539+0000 7f39f109e640 1 -- 172.21.15.6:0/3246825913 wait complete. 2024-09-12T03:19:40.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.540+0000 7f39f109e640 1 Processor -- start 2024-09-12T03:19:40.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.540+0000 7f39f109e640 1 -- start start 2024-09-12T03:19:40.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.540+0000 7f39f109e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39ec101de0 0x7f39ec1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.540+0000 7f39f109e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39ec102790 0x7f39ec198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.540+0000 7f39f109e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39ec106f10 0x7f39ec19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.540+0000 7f39f109e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec111f70 con 0x7f39ec101de0 2024-09-12T03:19:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.540+0000 7f39f109e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec111df0 con 0x7f39ec102790 2024-09-12T03:19:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.540+0000 7f39f109e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec1120f0 con 0x7f39ec106f10 2024-09-12T03:19:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.540+0000 7f39ead9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39ec101de0 0x7f39ec1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.540+0000 7f39ead9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39ec101de0 0x7f39ec1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48110/0 (socket says 172.21.15.6:48110) 2024-09-12T03:19:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.540+0000 7f39ead9d640 1 -- 172.21.15.6:0/77919421 learned_addr learned my addr 172.21.15.6:0/77919421 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.540+0000 7f39e3fff640 1 --2- 172.21.15.6:0/77919421 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39ec102790 0x7f39ec198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.540+0000 7f39eb59e640 1 --2- 172.21.15.6:0/77919421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39ec106f10 0x7f39ec19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.541+0000 7f39ead9d640 1 -- 172.21.15.6:0/77919421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39ec106f10 msgr2=0x7f39ec19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.541+0000 7f39ead9d640 1 --2- 172.21.15.6:0/77919421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39ec106f10 0x7f39ec19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.541+0000 7f39ead9d640 1 -- 172.21.15.6:0/77919421 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39ec102790 msgr2=0x7f39ec198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.541+0000 7f39ead9d640 1 --2- 172.21.15.6:0/77919421 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39ec102790 0x7f39ec198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.541+0000 7f39ead9d640 1 -- 172.21.15.6:0/77919421 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39ec19d480 con 0x7f39ec101de0 2024-09-12T03:19:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.541+0000 7f39eb59e640 1 --2- 172.21.15.6:0/77919421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39ec106f10 0x7f39ec19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.541+0000 7f39ead9d640 1 --2- 172.21.15.6:0/77919421 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39ec101de0 0x7f39ec1984d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f39dc006010 tx=0x7f39dc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.541+0000 7f39e8d99640 1 -- 172.21.15.6:0/77919421 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc03f030 con 0x7f39ec101de0 2024-09-12T03:19:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.541+0000 7f39f109e640 1 -- 172.21.15.6:0/77919421 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39ec19d710 con 0x7f39ec101de0 2024-09-12T03:19:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.541+0000 7f39e8d99640 1 -- 172.21.15.6:0/77919421 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39dc003ce0 con 0x7f39ec101de0 2024-09-12T03:19:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.541+0000 7f39e8d99640 1 -- 172.21.15.6:0/77919421 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc035070 con 0x7f39ec101de0 2024-09-12T03:19:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.541+0000 7f39f109e640 1 -- 172.21.15.6:0/77919421 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39ec1a5050 con 0x7f39ec101de0 2024-09-12T03:19:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.542+0000 7f39f109e640 1 -- 172.21.15.6:0/77919421 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39ec1021c0 con 0x7f39ec101de0 2024-09-12T03:19:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.543+0000 7f39e8d99640 1 -- 172.21.15.6:0/77919421 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f39dc042030 con 0x7f39ec101de0 2024-09-12T03:19:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.544+0000 7f39e8d99640 1 --2- 172.21.15.6:0/77919421 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f39d80a2c10 0x7f39d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.544+0000 7f39e3fff640 1 --2- 172.21.15.6:0/77919421 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f39d80a2c10 0x7f39d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.544+0000 7f39e8d99640 1 -- 172.21.15.6:0/77919421 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f39dc0ef700 con 0x7f39ec101de0 2024-09-12T03:19:40.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.545+0000 7f39e3fff640 1 --2- 172.21.15.6:0/77919421 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f39d80a2c10 0x7f39d80a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f39ec0ff840 tx=0x7f39d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:40.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.548+0000 7f39e8d99640 1 -- 172.21.15.6:0/77919421 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39dc0bc5f0 con 0x7f39ec101de0 2024-09-12T03:19:40.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:40 smithi161 ceph-mon[35485]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2024-09-12T03:19:40.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:40 smithi118 ceph-mon[35048]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2024-09-12T03:19:40.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.726+0000 7f39f109e640 1 -- 172.21.15.6:0/77919421 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39ec10a0f0 con 0x7f39ec101de0 2024-09-12T03:19:40.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.728+0000 7f39e8d99640 1 -- 172.21.15.6:0/77919421 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22607 (secure 0 0 0) 0x7f39dc0bbef0 con 0x7f39ec101de0 2024-09-12T03:19:40.729 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:19:40.729 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":82,"fsid":"89d99938-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:11:29.530545+0000","modified":"2024-09-12T03:19:20.366710+0000","last_up_change":"2024-09-12T03:18:21.292572+0000","last_in_change":"2024-09-12T03:17:58.013800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:14:37.662010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:06.143398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:08.305279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f55d68b-95a0-4ee2-a46f-bc422b38471e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6803","nonce":726729286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6805","nonce":726729286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6809","nonce":726729286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6807","nonce":726729286}]},"public_addr":"172.21.15.6:6803/726729286","cluster_addr":"172.21.15.6:6805/726729286","heartbeat_back_addr":"172.21.15.6:6809/726729286","heartbeat_front_addr":"172.21.15.6:6807/726729286","state":["exists","up"]},{"osd":1,"uuid":"83afdb01-c731-4de1-96c6-3f798549b1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6801","nonce":3377555415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6803","nonce":3377555415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6807","nonce":3377555415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6805","nonce":3377555415}]},"public_addr":"172.21.15.118:6801/3377555415","cluster_addr":"172.21.15.118:6803/3377555415","heartbeat_back_addr":"172.21.15.118:6807/3377555415","heartbeat_front_addr":"172.21.15.118:6805/3377555415","state":["exists","up"]},{"osd":2,"uuid":"aed718b4-b849-403a-b81c-3a140b5c380c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6801","nonce":392541009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6803","nonce":392541009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6807","nonce":392541009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6805","nonce":392541009}]},"public_addr":"172.21.15.161:6801/392541009","cluster_addr":"172.21.15.161:6803/392541009","heartbeat_back_addr":"172.21.15.161:6807/392541009","heartbeat_front_addr":"172.21.15.161:6805/392541009","state":["exists","up"]},{"osd":3,"uuid":"25ced4c1-bf31-402d-8397-d42fd57f615f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6811","nonce":1892039151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6813","nonce":1892039151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6817","nonce":1892039151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6815","nonce":1892039151}]},"public_addr":"172.21.15.6:6811/1892039151","cluster_addr":"172.21.15.6:6813/1892039151","heartbeat_back_addr":"172.21.15.6:6817/1892039151","heartbeat_front_addr":"172.21.15.6:6815/1892039151","state":["exists","up"]},{"osd":4,"uuid":"1d884604-920b-4c35-b41e-d629ad9f9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6809","nonce":150177024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6811","nonce":150177024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6815","nonce":150177024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6813","nonce":150177024}]},"public_addr":"172.21.15.118:6809/150177024","cluster_addr":"172.21.15.118:6811/150177024","heartbeat_back_addr":"172.21.15.118:6815/150177024","heartbeat_front_addr":"172.21.15.118:6813/150177024","state":["exists","up"]},{"osd":5,"uuid":"1d40c0ef-99f6-426f-90c7-27579edc66af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6809","nonce":3758167854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6811","nonce":3758167854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6815","nonce":3758167854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6813","nonce":3758167854}]},"public_addr":"172.21.15.161:6809/3758167854","cluster_addr":"172.21.15.161:6811/3758167854","heartbeat_back_addr":"172.21.15.161:6815/3758167854","heartbeat_front_addr":"172.21.15.161:6813/3758167854","state":["exists","up"]},{"osd":6,"uuid":"9d9f9510-b107-40c2-9a98-9012715944b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6819","nonce":2129224874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6821","nonce":2129224874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6825","nonce":2129224874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6823","nonce":2129224874}]},"public_addr":"172.21.15.6:6819/2129224874","cluster_addr":"172.21.15.6:6821/2129224874","heartbeat_back_addr":"172.21.15.6:6825/2129224874","heartbeat_front_addr":"172.21.15.6:6823/2129224874","state":["exists","up"]},{"osd":7,"uuid":"84635ae1-3240-4954-af14-76bb3aa162e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6817","nonce":269980353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6819","nonce":269980353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6823","nonce":269980353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6821","nonce":269980353}]},"public_addr":"172.21.15.118:6817/269980353","cluster_addr":"172.21.15.118:6819/269980353","heartbeat_back_addr":"172.21.15.118:6823/269980353","heartbeat_front_addr":"172.21.15.118:6821/269980353","state":["exists","up"]},{"osd":8,"uuid":"6fd58e55-1492-4595-8647-823ea3e3c5d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6817","nonce":1783416134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6819","nonce":1783416134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6823","nonce":1783416134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6821","nonce":1783416134}]},"public_addr":"172.21.15.161:6817/1783416134","cluster_addr":"172.21.15.161:6819/1783416134","heartbeat_back_addr":"172.21.15.161:6823/1783416134","heartbeat_front_addr":"172.21.15.161:6821/1783416134","state":["exists","up"]},{"osd":9,"uuid":"90508a6d-a49c-43c6-a600-5bbccd8702ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6827","nonce":2987332365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6829","nonce":2987332365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6833","nonce":2987332365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6831","nonce":2987332365}]},"public_addr":"172.21.15.6:6827/2987332365","cluster_addr":"172.21.15.6:6829/2987332365","heartbeat_back_addr":"172.21.15.6:6833/2987332365","heartbeat_front_addr":"172.21.15.6:6831/2987332365","state":["exists","up"]},{"osd":10,"uuid":"911e86d1-669a-4935-82c0-7da51d141df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6825","nonce":2167146341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6827","nonce":2167146341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6831","nonce":2167146341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6829","nonce":2167146341}]},"public_addr":"172.21.15.118:6825/2167146341","cluster_addr":"172.21.15.118:6827/2167146341","heartbeat_back_addr":"172.21.15.118:6831/2167146341","heartbeat_front_addr":"172.21.15.118:6829/2167146341","state":["exists","up"]},{"osd":11,"uuid":"5a15c6a7-ff43-473c-bb48-0c129c316df5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6825","nonce":2889795796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6827","nonce":2889795796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6831","nonce":2889795796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6829","nonce":2889795796}]},"public_addr":"172.21.15.161:6825/2889795796","cluster_addr":"172.21.15.161:6827/2889795796","heartbeat_back_addr":"172.21.15.161:6831/2889795796","heartbeat_front_addr":"172.21.15.161:6829/2889795796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:42.912205+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:08.410669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:34.299134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:59.093181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:24.116635+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:49.566433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:15.719320+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:40.541017+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:05.178178+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:31.163877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:55.405110+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:19.417156+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6800/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/2620491916":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/4037656392":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/1377116573":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/3062800125":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:6800/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/4145338968":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/825701128":"2024-09-13T03:11:55.244257+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:40.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.731+0000 7f39f109e640 1 -- 172.21.15.6:0/77919421 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f39d80a2c10 msgr2=0x7f39d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:40.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.731+0000 7f39f109e640 1 --2- 172.21.15.6:0/77919421 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f39d80a2c10 0x7f39d80a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f39ec0ff840 tx=0x7f39d4005f90 comp rx=0 tx=0).stop 2024-09-12T03:19:40.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.731+0000 7f39f109e640 1 -- 172.21.15.6:0/77919421 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39ec101de0 msgr2=0x7f39ec1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:40.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.731+0000 7f39f109e640 1 --2- 172.21.15.6:0/77919421 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39ec101de0 0x7f39ec1984d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f39dc006010 tx=0x7f39dc0042f0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.731+0000 7f39f109e640 1 -- 172.21.15.6:0/77919421 shutdown_connections 2024-09-12T03:19:40.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.731+0000 7f39f109e640 1 --2- 172.21.15.6:0/77919421 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f39d80a2c10 0x7f39d80a50d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.731+0000 7f39f109e640 1 --2- 172.21.15.6:0/77919421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39ec106f10 0x7f39ec19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.731+0000 7f39f109e640 1 --2- 172.21.15.6:0/77919421 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39ec102790 0x7f39ec198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.731+0000 7f39f109e640 1 --2- 172.21.15.6:0/77919421 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39ec101de0 0x7f39ec1984d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.731+0000 7f39f109e640 1 -- 172.21.15.6:0/77919421 >> 172.21.15.6:0/77919421 conn(0x7f39ec0fd8c0 msgr2=0x7f39ec0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:40.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.732+0000 7f39f109e640 1 -- 172.21.15.6:0/77919421 shutdown_connections 2024-09-12T03:19:40.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:40.732+0000 7f39f109e640 1 -- 172.21.15.6:0/77919421 wait complete. 2024-09-12T03:19:41.346 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:41 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/77919421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:41.374 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-12T03:19:41.375 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:19:41.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:41 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/77919421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:41.630 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:41.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:41 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/77919421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:42.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.037+0000 7fe7e062a640 1 -- 172.21.15.6:0/3056303705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7d8101de0 msgr2=0x7fe7d81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.037+0000 7fe7e062a640 1 --2- 172.21.15.6:0/3056303705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7d8101de0 0x7fe7d81021c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009900 tx=0x7fe7d402a410 comp rx=0 tx=0).stop 2024-09-12T03:19:42.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.037+0000 7fe7e062a640 1 -- 172.21.15.6:0/3056303705 shutdown_connections 2024-09-12T03:19:42.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.037+0000 7fe7e062a640 1 --2- 172.21.15.6:0/3056303705 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7d8106f10 0x7fe7d810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.037+0000 7fe7e062a640 1 --2- 172.21.15.6:0/3056303705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7d8102790 0x7fe7d81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.037+0000 7fe7e062a640 1 --2- 172.21.15.6:0/3056303705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7d8101de0 0x7fe7d81021c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.037+0000 7fe7e062a640 1 -- 172.21.15.6:0/3056303705 >> 172.21.15.6:0/3056303705 conn(0x7fe7d80fd8c0 msgr2=0x7fe7d80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:42.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.037+0000 7fe7e062a640 1 -- 172.21.15.6:0/3056303705 shutdown_connections 2024-09-12T03:19:42.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.037+0000 7fe7e062a640 1 -- 172.21.15.6:0/3056303705 wait complete. 2024-09-12T03:19:42.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.038+0000 7fe7e062a640 1 Processor -- start 2024-09-12T03:19:42.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.038+0000 7fe7e062a640 1 -- start start 2024-09-12T03:19:42.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.038+0000 7fe7e062a640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7d8101de0 0x7fe7d81a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.038+0000 7fe7e062a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7d8102790 0x7fe7d81a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.038+0000 7fe7e062a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7d8106f10 0x7fe7d819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.038+0000 7fe7e062a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d8112030 con 0x7fe7d8106f10 2024-09-12T03:19:42.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.038+0000 7fe7e062a640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d8111eb0 con 0x7fe7d8101de0 2024-09-12T03:19:42.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.038+0000 7fe7e062a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d81121b0 con 0x7fe7d8102790 2024-09-12T03:19:42.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.038+0000 7fe7debc7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7d8106f10 0x7fe7d819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.038+0000 7fe7debc7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7d8106f10 0x7fe7d819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48120/0 (socket says 172.21.15.6:48120) 2024-09-12T03:19:42.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.038+0000 7fe7debc7640 1 -- 172.21.15.6:0/1401691855 learned_addr learned my addr 172.21.15.6:0/1401691855 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:42.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.039+0000 7fe7de3c6640 1 --2- 172.21.15.6:0/1401691855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7d8101de0 0x7fe7d81a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.039+0000 7fe7debc7640 1 -- 172.21.15.6:0/1401691855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7d8102790 msgr2=0x7fe7d81a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.039+0000 7fe7debc7640 1 --2- 172.21.15.6:0/1401691855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7d8102790 0x7fe7d81a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.039+0000 7fe7debc7640 1 -- 172.21.15.6:0/1401691855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7d8101de0 msgr2=0x7fe7d81a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.039+0000 7fe7debc7640 1 --2- 172.21.15.6:0/1401691855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7d8101de0 0x7fe7d81a0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.039+0000 7fe7debc7640 1 -- 172.21.15.6:0/1401691855 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d819ad30 con 0x7fe7d8106f10 2024-09-12T03:19:42.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.039+0000 7fe7debc7640 1 --2- 172.21.15.6:0/1401691855 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7d8106f10 0x7fe7d819a600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000b970 tx=0x7fe7d000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:42.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.039+0000 7fe7cf7fe640 1 -- 172.21.15.6:0/1401691855 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0019070 con 0x7fe7d8106f10 2024-09-12T03:19:42.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.039+0000 7fe7e062a640 1 -- 172.21.15.6:0/1401691855 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d819b020 con 0x7fe7d8106f10 2024-09-12T03:19:42.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.039+0000 7fe7cf7fe640 1 -- 172.21.15.6:0/1401691855 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7d000cb40 con 0x7fe7d8106f10 2024-09-12T03:19:42.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.039+0000 7fe7cf7fe640 1 -- 172.21.15.6:0/1401691855 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0007650 con 0x7fe7d8106f10 2024-09-12T03:19:42.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.039+0000 7fe7e062a640 1 -- 172.21.15.6:0/1401691855 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d81a7310 con 0x7fe7d8106f10 2024-09-12T03:19:42.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.040+0000 7fe7e062a640 1 -- 172.21.15.6:0/1401691855 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d8102240 con 0x7fe7d8106f10 2024-09-12T03:19:42.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.041+0000 7fe7cf7fe640 1 -- 172.21.15.6:0/1401691855 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe7d001c070 con 0x7fe7d8106f10 2024-09-12T03:19:42.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.042+0000 7fe7cf7fe640 1 --2- 172.21.15.6:0/1401691855 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe7c40a2c10 0x7fe7c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.042+0000 7fe7de3c6640 1 --2- 172.21.15.6:0/1401691855 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe7c40a2c10 0x7fe7c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.042+0000 7fe7cf7fe640 1 -- 172.21.15.6:0/1401691855 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe7d00cec40 con 0x7fe7d8106f10 2024-09-12T03:19:42.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.042+0000 7fe7de3c6640 1 --2- 172.21.15.6:0/1401691855 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe7c40a2c10 0x7fe7c40a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40096c0 tx=0x7fe7d4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:42.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.045+0000 7fe7cf7fe640 1 -- 172.21.15.6:0/1401691855 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe7d009bb20 con 0x7fe7d8106f10 2024-09-12T03:19:42.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.223+0000 7fe7e062a640 1 -- 172.21.15.6:0/1401691855 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7d819bb30 con 0x7fe7d8106f10 2024-09-12T03:19:42.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.224+0000 7fe7cf7fe640 1 -- 172.21.15.6:0/1401691855 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22607 (secure 0 0 0) 0x7fe7d819bb30 con 0x7fe7d8106f10 2024-09-12T03:19:42.225 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:19:42.225 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":82,"fsid":"89d99938-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:11:29.530545+0000","modified":"2024-09-12T03:19:20.366710+0000","last_up_change":"2024-09-12T03:18:21.292572+0000","last_in_change":"2024-09-12T03:17:58.013800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:14:37.662010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:06.143398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:08.305279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f55d68b-95a0-4ee2-a46f-bc422b38471e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6803","nonce":726729286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6805","nonce":726729286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6809","nonce":726729286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6807","nonce":726729286}]},"public_addr":"172.21.15.6:6803/726729286","cluster_addr":"172.21.15.6:6805/726729286","heartbeat_back_addr":"172.21.15.6:6809/726729286","heartbeat_front_addr":"172.21.15.6:6807/726729286","state":["exists","up"]},{"osd":1,"uuid":"83afdb01-c731-4de1-96c6-3f798549b1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6801","nonce":3377555415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6803","nonce":3377555415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6807","nonce":3377555415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6805","nonce":3377555415}]},"public_addr":"172.21.15.118:6801/3377555415","cluster_addr":"172.21.15.118:6803/3377555415","heartbeat_back_addr":"172.21.15.118:6807/3377555415","heartbeat_front_addr":"172.21.15.118:6805/3377555415","state":["exists","up"]},{"osd":2,"uuid":"aed718b4-b849-403a-b81c-3a140b5c380c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6801","nonce":392541009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6803","nonce":392541009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6807","nonce":392541009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6805","nonce":392541009}]},"public_addr":"172.21.15.161:6801/392541009","cluster_addr":"172.21.15.161:6803/392541009","heartbeat_back_addr":"172.21.15.161:6807/392541009","heartbeat_front_addr":"172.21.15.161:6805/392541009","state":["exists","up"]},{"osd":3,"uuid":"25ced4c1-bf31-402d-8397-d42fd57f615f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6811","nonce":1892039151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6813","nonce":1892039151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6817","nonce":1892039151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6815","nonce":1892039151}]},"public_addr":"172.21.15.6:6811/1892039151","cluster_addr":"172.21.15.6:6813/1892039151","heartbeat_back_addr":"172.21.15.6:6817/1892039151","heartbeat_front_addr":"172.21.15.6:6815/1892039151","state":["exists","up"]},{"osd":4,"uuid":"1d884604-920b-4c35-b41e-d629ad9f9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6809","nonce":150177024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6811","nonce":150177024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6815","nonce":150177024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6813","nonce":150177024}]},"public_addr":"172.21.15.118:6809/150177024","cluster_addr":"172.21.15.118:6811/150177024","heartbeat_back_addr":"172.21.15.118:6815/150177024","heartbeat_front_addr":"172.21.15.118:6813/150177024","state":["exists","up"]},{"osd":5,"uuid":"1d40c0ef-99f6-426f-90c7-27579edc66af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6809","nonce":3758167854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6811","nonce":3758167854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6815","nonce":3758167854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6813","nonce":3758167854}]},"public_addr":"172.21.15.161:6809/3758167854","cluster_addr":"172.21.15.161:6811/3758167854","heartbeat_back_addr":"172.21.15.161:6815/3758167854","heartbeat_front_addr":"172.21.15.161:6813/3758167854","state":["exists","up"]},{"osd":6,"uuid":"9d9f9510-b107-40c2-9a98-9012715944b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6819","nonce":2129224874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6821","nonce":2129224874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6825","nonce":2129224874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6823","nonce":2129224874}]},"public_addr":"172.21.15.6:6819/2129224874","cluster_addr":"172.21.15.6:6821/2129224874","heartbeat_back_addr":"172.21.15.6:6825/2129224874","heartbeat_front_addr":"172.21.15.6:6823/2129224874","state":["exists","up"]},{"osd":7,"uuid":"84635ae1-3240-4954-af14-76bb3aa162e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6817","nonce":269980353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6819","nonce":269980353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6823","nonce":269980353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6821","nonce":269980353}]},"public_addr":"172.21.15.118:6817/269980353","cluster_addr":"172.21.15.118:6819/269980353","heartbeat_back_addr":"172.21.15.118:6823/269980353","heartbeat_front_addr":"172.21.15.118:6821/269980353","state":["exists","up"]},{"osd":8,"uuid":"6fd58e55-1492-4595-8647-823ea3e3c5d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6817","nonce":1783416134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6819","nonce":1783416134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6823","nonce":1783416134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6821","nonce":1783416134}]},"public_addr":"172.21.15.161:6817/1783416134","cluster_addr":"172.21.15.161:6819/1783416134","heartbeat_back_addr":"172.21.15.161:6823/1783416134","heartbeat_front_addr":"172.21.15.161:6821/1783416134","state":["exists","up"]},{"osd":9,"uuid":"90508a6d-a49c-43c6-a600-5bbccd8702ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6827","nonce":2987332365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6829","nonce":2987332365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6833","nonce":2987332365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6831","nonce":2987332365}]},"public_addr":"172.21.15.6:6827/2987332365","cluster_addr":"172.21.15.6:6829/2987332365","heartbeat_back_addr":"172.21.15.6:6833/2987332365","heartbeat_front_addr":"172.21.15.6:6831/2987332365","state":["exists","up"]},{"osd":10,"uuid":"911e86d1-669a-4935-82c0-7da51d141df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6825","nonce":2167146341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6827","nonce":2167146341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6831","nonce":2167146341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6829","nonce":2167146341}]},"public_addr":"172.21.15.118:6825/2167146341","cluster_addr":"172.21.15.118:6827/2167146341","heartbeat_back_addr":"172.21.15.118:6831/2167146341","heartbeat_front_addr":"172.21.15.118:6829/2167146341","state":["exists","up"]},{"osd":11,"uuid":"5a15c6a7-ff43-473c-bb48-0c129c316df5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6825","nonce":2889795796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6827","nonce":2889795796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6831","nonce":2889795796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6829","nonce":2889795796}]},"public_addr":"172.21.15.161:6825/2889795796","cluster_addr":"172.21.15.161:6827/2889795796","heartbeat_back_addr":"172.21.15.161:6831/2889795796","heartbeat_front_addr":"172.21.15.161:6829/2889795796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:42.912205+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:08.410669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:34.299134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:59.093181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:24.116635+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:49.566433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:15.719320+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:40.541017+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:05.178178+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:31.163877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:55.405110+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:19.417156+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6800/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/2620491916":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/4037656392":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/1377116573":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/3062800125":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:6800/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/4145338968":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/825701128":"2024-09-13T03:11:55.244257+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:42.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.226+0000 7fe7e062a640 1 -- 172.21.15.6:0/1401691855 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe7c40a2c10 msgr2=0x7fe7c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.226+0000 7fe7e062a640 1 --2- 172.21.15.6:0/1401691855 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe7c40a2c10 0x7fe7c40a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40096c0 tx=0x7fe7d4000bc0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.226+0000 7fe7e062a640 1 -- 172.21.15.6:0/1401691855 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7d8106f10 msgr2=0x7fe7d819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.226+0000 7fe7e062a640 1 --2- 172.21.15.6:0/1401691855 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7d8106f10 0x7fe7d819a600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000b970 tx=0x7fe7d000be30 comp rx=0 tx=0).stop 2024-09-12T03:19:42.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.226+0000 7fe7e062a640 1 -- 172.21.15.6:0/1401691855 shutdown_connections 2024-09-12T03:19:42.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.226+0000 7fe7e062a640 1 --2- 172.21.15.6:0/1401691855 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe7c40a2c10 0x7fe7c40a50d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.226+0000 7fe7e062a640 1 --2- 172.21.15.6:0/1401691855 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7d8106f10 0x7fe7d819a600 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.226+0000 7fe7e062a640 1 --2- 172.21.15.6:0/1401691855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7d8102790 0x7fe7d81a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.226+0000 7fe7e062a640 1 --2- 172.21.15.6:0/1401691855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7d8101de0 0x7fe7d81a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.226+0000 7fe7e062a640 1 -- 172.21.15.6:0/1401691855 >> 172.21.15.6:0/1401691855 conn(0x7fe7d80fd8c0 msgr2=0x7fe7d80fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:42.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.226+0000 7fe7e062a640 1 -- 172.21.15.6:0/1401691855 shutdown_connections 2024-09-12T03:19:42.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:42.227+0000 7fe7e062a640 1 -- 172.21.15.6:0/1401691855 wait complete. 2024-09-12T03:19:42.592 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:42 smithi006 ceph-mon[30991]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-12T03:19:42.592 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:42 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1401691855' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:42.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:42 smithi161 ceph-mon[35485]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-12T03:19:42.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:42 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1401691855' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:42.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:42 smithi118 ceph-mon[35048]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-12T03:19:42.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:42 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1401691855' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:42.982 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-12T03:19:42.982 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-12T03:19:42.983 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-12T03:19:42.983 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-12T03:19:42.983 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-12T03:19:42.984 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-12T03:19:42.984 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-12T03:19:42.985 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-12T03:19:42.985 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-12T03:19:42.985 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-12T03:19:42.986 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-12T03:19:42.987 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-12T03:19:43.644 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:43.650 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:43.973 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:43.974 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:43.974 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:43.975 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:43.975 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:43.980 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:43.995 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:44.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:44 smithi006 ceph-mon[30991]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-12T03:19:44.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:44 smithi161 ceph-mon[35485]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-12T03:19:44.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:44 smithi118 ceph-mon[35048]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-12T03:19:45.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.368+0000 7f20fd905640 1 -- 172.21.15.6:0/3716208121 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20f8100e20 msgr2=0x7f20f8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.368+0000 7f20fd905640 1 --2- 172.21.15.6:0/3716208121 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20f8100e20 0x7f20f8101200 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009ca0 tx=0x7f20e802a950 comp rx=0 tx=0).stop 2024-09-12T03:19:45.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.368+0000 7f20fd905640 1 -- 172.21.15.6:0/3716208121 shutdown_connections 2024-09-12T03:19:45.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.368+0000 7f20fd905640 1 --2- 172.21.15.6:0/3716208121 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20f8102ef0 0x7f20f810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.368+0000 7f20fd905640 1 --2- 172.21.15.6:0/3716208121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20f8102570 0x7f20f81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.368+0000 7f20fd905640 1 --2- 172.21.15.6:0/3716208121 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20f8100e20 0x7f20f8101200 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.368+0000 7f20fd905640 1 -- 172.21.15.6:0/3716208121 >> 172.21.15.6:0/3716208121 conn(0x7f20f80fc7d0 msgr2=0x7f20f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:45.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.368+0000 7f20fd905640 1 -- 172.21.15.6:0/3716208121 shutdown_connections 2024-09-12T03:19:45.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.368+0000 7f20fd905640 1 -- 172.21.15.6:0/3716208121 wait complete. 2024-09-12T03:19:45.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.369+0000 7f20fd905640 1 Processor -- start 2024-09-12T03:19:45.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.369+0000 7f20fd905640 1 -- start start 2024-09-12T03:19:45.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.369+0000 7f20fd905640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20f8100e20 0x7f20f8103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.369+0000 7f20fd905640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20f8102570 0x7f20f8103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.370+0000 7f20fd905640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20f8102ef0 0x7f20f81041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.370+0000 7f20fd905640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f20f8110fd0 con 0x7f20f8102570 2024-09-12T03:19:45.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.370+0000 7f20fd905640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f20f8110e50 con 0x7f20f8100e20 2024-09-12T03:19:45.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.370+0000 7f20fd905640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f20f8111150 con 0x7f20f8102ef0 2024-09-12T03:19:45.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.370+0000 7f20f67fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20f8102570 0x7f20f8103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.370+0000 7f20f67fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20f8102570 0x7f20f8103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48136/0 (socket says 172.21.15.6:48136) 2024-09-12T03:19:45.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.370+0000 7f20f67fc640 1 -- 172.21.15.6:0/1609673188 learned_addr learned my addr 172.21.15.6:0/1609673188 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:45.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.370+0000 7f20f6ffd640 1 --2- 172.21.15.6:0/1609673188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20f8100e20 0x7f20f8103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.370+0000 7f20f67fc640 1 -- 172.21.15.6:0/1609673188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20f8102ef0 msgr2=0x7f20f81041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.370+0000 7f20f67fc640 1 --2- 172.21.15.6:0/1609673188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20f8102ef0 0x7f20f81041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.370+0000 7f20f67fc640 1 -- 172.21.15.6:0/1609673188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20f8100e20 msgr2=0x7f20f8103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.370+0000 7f20f67fc640 1 --2- 172.21.15.6:0/1609673188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20f8100e20 0x7f20f8103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.370+0000 7f20f67fc640 1 -- 172.21.15.6:0/1609673188 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f81a6920 con 0x7f20f8102570 2024-09-12T03:19:45.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.370+0000 7f20f67fc640 1 --2- 172.21.15.6:0/1609673188 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20f8102570 0x7f20f8103c90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00d950 tx=0x7f20ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:45.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.370+0000 7f20fc903640 1 -- 172.21.15.6:0/1609673188 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec017070 con 0x7f20f8102570 2024-09-12T03:19:45.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.370+0000 7f20fc903640 1 -- 172.21.15.6:0/1609673188 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20ec009c30 con 0x7f20f8102570 2024-09-12T03:19:45.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.370+0000 7f20fd905640 1 -- 172.21.15.6:0/1609673188 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f81a6c10 con 0x7f20f8102570 2024-09-12T03:19:45.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.370+0000 7f20fc903640 1 -- 172.21.15.6:0/1609673188 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec00f6b0 con 0x7f20f8102570 2024-09-12T03:19:45.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.370+0000 7f20fd905640 1 -- 172.21.15.6:0/1609673188 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f81a7150 con 0x7f20f8102570 2024-09-12T03:19:45.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.371+0000 7f20fd905640 1 -- 172.21.15.6:0/1609673188 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f20f8106bf0 con 0x7f20f8102570 2024-09-12T03:19:45.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.372+0000 7f20fc903640 1 -- 172.21.15.6:0/1609673188 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f20ec0109d0 con 0x7f20f8102570 2024-09-12T03:19:45.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.373+0000 7f20fc903640 1 --2- 172.21.15.6:0/1609673188 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f20e00a2c10 0x7f20e00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.373+0000 7f20f6ffd640 1 --2- 172.21.15.6:0/1609673188 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f20e00a2c10 0x7f20e00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.373+0000 7f20fc903640 1 -- 172.21.15.6:0/1609673188 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f20ec0cf260 con 0x7f20f8102570 2024-09-12T03:19:45.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.373+0000 7f20fc903640 1 --2- 172.21.15.6:0/1609673188 >> [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874] conn(0x7f20e00ac740 0x7f20e00aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.373+0000 7f20fc903640 1 -- 172.21.15.6:0/1609673188 --> [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20ec016ec0 con 0x7f20e00ac740 2024-09-12T03:19:45.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.373+0000 7f20fc903640 1 -- 172.21.15.6:0/1609673188 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f20ec0cf6b0 con 0x7f20f8102570 2024-09-12T03:19:45.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.374+0000 7f20f77fe640 1 --2- 172.21.15.6:0/1609673188 >> [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874] conn(0x7f20e00ac740 0x7f20e00aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.374+0000 7f20f6ffd640 1 --2- 172.21.15.6:0/1609673188 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f20e00a2c10 0x7f20e00a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f20e8005e30 tx=0x7f20e8005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:45.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.374+0000 7f20f77fe640 1 --2- 172.21.15.6:0/1609673188 >> [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874] conn(0x7f20e00ac740 0x7f20e00aebe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:45.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.375+0000 7f20fc903640 1 -- 172.21.15.6:0/1609673188 <== osd.6 v2:172.21.15.6:6818/2129224874 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f20ec016ec0 con 0x7f20e00ac740 2024-09-12T03:19:45.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.388+0000 7f20fd905640 1 -- 172.21.15.6:0/1609673188 --> [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f20f81021b0 con 0x7f20e00ac740 2024-09-12T03:19:45.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.388+0000 7f20fc903640 1 -- 172.21.15.6:0/1609673188 <== osd.6 v2:172.21.15.6:6818/2129224874 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f20f81021b0 con 0x7f20e00ac740 2024-09-12T03:19:45.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.389+0000 7f20fd905640 1 -- 172.21.15.6:0/1609673188 >> [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874] conn(0x7f20e00ac740 msgr2=0x7f20e00aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.389+0000 7f20fd905640 1 --2- 172.21.15.6:0/1609673188 >> [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874] conn(0x7f20e00ac740 0x7f20e00aebe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.389+0000 7f20fd905640 1 -- 172.21.15.6:0/1609673188 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f20e00a2c10 msgr2=0x7f20e00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.389+0000 7f20fd905640 1 --2- 172.21.15.6:0/1609673188 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f20e00a2c10 0x7f20e00a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f20e8005e30 tx=0x7f20e8005d80 comp rx=0 tx=0).stop 2024-09-12T03:19:45.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.389+0000 7f20fd905640 1 -- 172.21.15.6:0/1609673188 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20f8102570 msgr2=0x7f20f8103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.389+0000 7f20fd905640 1 --2- 172.21.15.6:0/1609673188 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20f8102570 0x7f20f8103c90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00d950 tx=0x7f20ec00de10 comp rx=0 tx=0).stop 2024-09-12T03:19:45.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.389+0000 7f20f77fe640 1 -- 172.21.15.6:0/1609673188 reap_dead start 2024-09-12T03:19:45.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.389+0000 7f20fd905640 1 -- 172.21.15.6:0/1609673188 shutdown_connections 2024-09-12T03:19:45.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.389+0000 7f20fd905640 1 -- 172.21.15.6:0/1609673188 >> 172.21.15.6:0/1609673188 conn(0x7f20f80fc7d0 msgr2=0x7f20f806a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:45.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.389+0000 7f20fd905640 1 -- 172.21.15.6:0/1609673188 shutdown_connections 2024-09-12T03:19:45.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:45.389+0000 7f20fd905640 1 -- 172.21.15.6:0/1609673188 wait complete. 2024-09-12T03:19:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.272+0000 7f94311de640 1 -- 172.21.15.6:0/260575136 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c100a80 msgr2=0x7f942c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.272+0000 7f94311de640 1 --2- 172.21.15.6:0/260575136 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c100a80 0x7f942c100ee0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9414009930 tx=0x7f941402a420 comp rx=0 tx=0).stop 2024-09-12T03:19:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.272+0000 7f94311de640 1 -- 172.21.15.6:0/260575136 shutdown_connections 2024-09-12T03:19:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.272+0000 7f94311de640 1 --2- 172.21.15.6:0/260575136 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f942c101420 0x7f942c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.272+0000 7f94311de640 1 --2- 172.21.15.6:0/260575136 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c100a80 0x7f942c100ee0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.272+0000 7f94311de640 1 --2- 172.21.15.6:0/260575136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f942c106a80 0x7f942c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.272+0000 7f94311de640 1 -- 172.21.15.6:0/260575136 >> 172.21.15.6:0/260575136 conn(0x7f942c0fc7a0 msgr2=0x7f942c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.272+0000 7f94311de640 1 -- 172.21.15.6:0/260575136 shutdown_connections 2024-09-12T03:19:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.272+0000 7f94311de640 1 -- 172.21.15.6:0/260575136 wait complete. 2024-09-12T03:19:46.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.273+0000 7f94311de640 1 Processor -- start 2024-09-12T03:19:46.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.273+0000 7f94311de640 1 -- start start 2024-09-12T03:19:46.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.273+0000 7f94311de640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f942c100a80 0x7f942c10e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.273+0000 7f94311de640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f942c101420 0x7f942c10e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.273+0000 7f94311de640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c106a80 0x7f942c10ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.273+0000 7f94311de640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f942c111fa0 con 0x7f942c106a80 2024-09-12T03:19:46.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.273+0000 7f94311de640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f942c111e20 con 0x7f942c101420 2024-09-12T03:19:46.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.273+0000 7f94311de640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f942c112120 con 0x7f942c100a80 2024-09-12T03:19:46.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.273+0000 7f942b59e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c106a80 0x7f942c10ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.273+0000 7f942b59e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c106a80 0x7f942c10ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48152/0 (socket says 172.21.15.6:48152) 2024-09-12T03:19:46.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.274+0000 7f942b59e640 1 -- 172.21.15.6:0/1702625338 learned_addr learned my addr 172.21.15.6:0/1702625338 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:46.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.274+0000 7f9423fff640 1 --2- 172.21.15.6:0/1702625338 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f942c101420 0x7f942c10e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.274+0000 7f942b59e640 1 -- 172.21.15.6:0/1702625338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f942c100a80 msgr2=0x7f942c10e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.274+0000 7f942b59e640 1 --2- 172.21.15.6:0/1702625338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f942c100a80 0x7f942c10e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.274+0000 7f942b59e640 1 -- 172.21.15.6:0/1702625338 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f942c101420 msgr2=0x7f942c10e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.274+0000 7f942b59e640 1 --2- 172.21.15.6:0/1702625338 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f942c101420 0x7f942c10e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.274+0000 7f942b59e640 1 -- 172.21.15.6:0/1702625338 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942c1a69a0 con 0x7f942c106a80 2024-09-12T03:19:46.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.274+0000 7f942b59e640 1 --2- 172.21.15.6:0/1702625338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c106a80 0x7f942c10ee60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9424009fd0 tx=0x7f942400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:46.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.274+0000 7f9428d99640 1 -- 172.21.15.6:0/1702625338 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9424017070 con 0x7f942c106a80 2024-09-12T03:19:46.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.274+0000 7f9428d99640 1 -- 172.21.15.6:0/1702625338 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9424003e50 con 0x7f942c106a80 2024-09-12T03:19:46.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.274+0000 7f94311de640 1 -- 172.21.15.6:0/1702625338 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942c1a6c90 con 0x7f942c106a80 2024-09-12T03:19:46.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.274+0000 7f9428d99640 1 -- 172.21.15.6:0/1702625338 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94240106c0 con 0x7f942c106a80 2024-09-12T03:19:46.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.274+0000 7f94311de640 1 -- 172.21.15.6:0/1702625338 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f942c1a7150 con 0x7f942c106a80 2024-09-12T03:19:46.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.275+0000 7f94311de640 1 -- 172.21.15.6:0/1702625338 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f942c062900 con 0x7f942c106a80 2024-09-12T03:19:46.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.276+0000 7f9428d99640 1 -- 172.21.15.6:0/1702625338 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9424003970 con 0x7f942c106a80 2024-09-12T03:19:46.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.277+0000 7f9428d99640 1 --2- 172.21.15.6:0/1702625338 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f94180a2c10 0x7f94180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.277+0000 7f942ad9d640 1 --2- 172.21.15.6:0/1702625338 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f94180a2c10 0x7f94180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.277+0000 7f9428d99640 1 -- 172.21.15.6:0/1702625338 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f94240cf3c0 con 0x7f942c106a80 2024-09-12T03:19:46.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.277+0000 7f9428d99640 1 --2- 172.21.15.6:0/1702625338 >> [v2:172.21.15.118:6824/2167146341,v1:172.21.15.118:6825/2167146341] conn(0x7f94180ac740 0x7f94180aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.277+0000 7f9428d99640 1 -- 172.21.15.6:0/1702625338 --> [v2:172.21.15.118:6824/2167146341,v1:172.21.15.118:6825/2167146341] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94180af2b0 con 0x7f94180ac740 2024-09-12T03:19:46.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.277+0000 7f9428d99640 1 -- 172.21.15.6:0/1702625338 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f94240cf810 con 0x7f942c106a80 2024-09-12T03:19:46.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.278+0000 7f942ad9d640 1 --2- 172.21.15.6:0/1702625338 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f94180a2c10 0x7f94180a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f941c009ad0 tx=0x7f941c009590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:46.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.278+0000 7f9423fff640 1 --2- 172.21.15.6:0/1702625338 >> [v2:172.21.15.118:6824/2167146341,v1:172.21.15.118:6825/2167146341] conn(0x7f94180ac740 0x7f94180aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.278+0000 7f9423fff640 1 --2- 172.21.15.6:0/1702625338 >> [v2:172.21.15.118:6824/2167146341,v1:172.21.15.118:6825/2167146341] conn(0x7f94180ac740 0x7f94180aebe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:46.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.279+0000 7f9428d99640 1 -- 172.21.15.6:0/1702625338 <== osd.10 v2:172.21.15.118:6824/2167146341 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94180af2b0 con 0x7f94180ac740 2024-09-12T03:19:46.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.293+0000 7f94311de640 1 -- 172.21.15.6:0/1702625338 --> [v2:172.21.15.118:6824/2167146341,v1:172.21.15.118:6825/2167146341] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f942c062d80 con 0x7f94180ac740 2024-09-12T03:19:46.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.293+0000 7f9428d99640 1 -- 172.21.15.6:0/1702625338 <== osd.10 v2:172.21.15.118:6824/2167146341 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f942c062d80 con 0x7f94180ac740 2024-09-12T03:19:46.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.293+0000 7f94311de640 1 -- 172.21.15.6:0/1702625338 >> [v2:172.21.15.118:6824/2167146341,v1:172.21.15.118:6825/2167146341] conn(0x7f94180ac740 msgr2=0x7f94180aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.293+0000 7f94311de640 1 --2- 172.21.15.6:0/1702625338 >> [v2:172.21.15.118:6824/2167146341,v1:172.21.15.118:6825/2167146341] conn(0x7f94180ac740 0x7f94180aebe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.293+0000 7f94311de640 1 -- 172.21.15.6:0/1702625338 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f94180a2c10 msgr2=0x7f94180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.293+0000 7f94311de640 1 --2- 172.21.15.6:0/1702625338 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f94180a2c10 0x7f94180a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f941c009ad0 tx=0x7f941c009590 comp rx=0 tx=0).stop 2024-09-12T03:19:46.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.293+0000 7f94311de640 1 -- 172.21.15.6:0/1702625338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c106a80 msgr2=0x7f942c10ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.293+0000 7f94311de640 1 --2- 172.21.15.6:0/1702625338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c106a80 0x7f942c10ee60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9424009fd0 tx=0x7f942400b260 comp rx=0 tx=0).stop 2024-09-12T03:19:46.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.293+0000 7f942b59e640 1 -- 172.21.15.6:0/1702625338 reap_dead start 2024-09-12T03:19:46.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.294+0000 7f94311de640 1 -- 172.21.15.6:0/1702625338 shutdown_connections 2024-09-12T03:19:46.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.294+0000 7f94311de640 1 -- 172.21.15.6:0/1702625338 >> 172.21.15.6:0/1702625338 conn(0x7f942c0fc7a0 msgr2=0x7f942c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:46.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.294+0000 7f94311de640 1 -- 172.21.15.6:0/1702625338 shutdown_connections 2024-09-12T03:19:46.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.294+0000 7f94311de640 1 -- 172.21.15.6:0/1702625338 wait complete. 2024-09-12T03:19:46.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:46 smithi006 ceph-mon[30991]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:19:46.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:46 smithi161 ceph-mon[35485]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:19:46.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.699+0000 7fa50738c640 1 -- 172.21.15.6:0/1744254932 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa500106f10 msgr2=0x7fa50010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.699+0000 7fa50738c640 1 --2- 172.21.15.6:0/1744254932 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa500106f10 0x7fa50010ea90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009ca0 tx=0x7fa4f802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.699+0000 7fa50738c640 1 -- 172.21.15.6:0/1744254932 shutdown_connections 2024-09-12T03:19:46.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.699+0000 7fa50738c640 1 --2- 172.21.15.6:0/1744254932 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa500106f10 0x7fa50010ea90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.699+0000 7fa50738c640 1 --2- 172.21.15.6:0/1744254932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa500102790 0x7fa5001067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.699+0000 7fa50738c640 1 --2- 172.21.15.6:0/1744254932 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa500101de0 0x7fa5001021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.699+0000 7fa50738c640 1 -- 172.21.15.6:0/1744254932 >> 172.21.15.6:0/1744254932 conn(0x7fa5000fd8c0 msgr2=0x7fa5000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:46.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.699+0000 7fa50738c640 1 -- 172.21.15.6:0/1744254932 shutdown_connections 2024-09-12T03:19:46.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.699+0000 7fa50738c640 1 -- 172.21.15.6:0/1744254932 wait complete. 2024-09-12T03:19:46.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.700+0000 7fa50738c640 1 Processor -- start 2024-09-12T03:19:46.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.700+0000 7fa50738c640 1 -- start start 2024-09-12T03:19:46.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.700+0000 7fa50738c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa500101de0 0x7fa500198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.700+0000 7fa50738c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa500102790 0x7fa500198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.700+0000 7fa50738c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa500106f10 0x7fa50019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.700+0000 7fa50738c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa500111f90 con 0x7fa500102790 2024-09-12T03:19:46.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.700+0000 7fa50738c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa500111e10 con 0x7fa500101de0 2024-09-12T03:19:46.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.700+0000 7fa50738c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa500112110 con 0x7fa500106f10 2024-09-12T03:19:46.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.700+0000 7fa504927640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa500102790 0x7fa500198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.701+0000 7fa504927640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa500102790 0x7fa500198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48166/0 (socket says 172.21.15.6:48166) 2024-09-12T03:19:46.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.701+0000 7fa504927640 1 -- 172.21.15.6:0/2948334277 learned_addr learned my addr 172.21.15.6:0/2948334277 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:46.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.701+0000 7fa505128640 1 --2- 172.21.15.6:0/2948334277 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa500101de0 0x7fa500198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.701+0000 7fa505929640 1 --2- 172.21.15.6:0/2948334277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa500106f10 0x7fa50019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.701+0000 7fa504927640 1 -- 172.21.15.6:0/2948334277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa500106f10 msgr2=0x7fa50019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.701+0000 7fa504927640 1 --2- 172.21.15.6:0/2948334277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa500106f10 0x7fa50019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.701+0000 7fa504927640 1 -- 172.21.15.6:0/2948334277 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa500101de0 msgr2=0x7fa500198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.701+0000 7fa504927640 1 --2- 172.21.15.6:0/2948334277 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa500101de0 0x7fa500198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.701+0000 7fa504927640 1 -- 172.21.15.6:0/2948334277 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50019d400 con 0x7fa500102790 2024-09-12T03:19:46.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.701+0000 7fa505929640 1 --2- 172.21.15.6:0/2948334277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa500106f10 0x7fa50019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:46.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.701+0000 7fa504927640 1 --2- 172.21.15.6:0/2948334277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa500102790 0x7fa500198990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000b4a0 tx=0x7fa4f000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:46.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.701+0000 7fa4f67fc640 1 -- 172.21.15.6:0/2948334277 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0014070 con 0x7fa500102790 2024-09-12T03:19:46.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.701+0000 7fa50738c640 1 -- 172.21.15.6:0/2948334277 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa50019d6f0 con 0x7fa500102790 2024-09-12T03:19:46.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.701+0000 7fa4f67fc640 1 -- 172.21.15.6:0/2948334277 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f0010af0 con 0x7fa500102790 2024-09-12T03:19:46.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.701+0000 7fa4f67fc640 1 -- 172.21.15.6:0/2948334277 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0017d10 con 0x7fa500102790 2024-09-12T03:19:46.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.701+0000 7fa50738c640 1 -- 172.21.15.6:0/2948334277 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5001a4fd0 con 0x7fa500102790 2024-09-12T03:19:46.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.702+0000 7fa50738c640 1 -- 172.21.15.6:0/2948334277 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa500062980 con 0x7fa500102790 2024-09-12T03:19:46.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.704+0000 7fa4f67fc640 1 -- 172.21.15.6:0/2948334277 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa4f0017070 con 0x7fa500102790 2024-09-12T03:19:46.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.704+0000 7fa4f67fc640 1 --2- 172.21.15.6:0/2948334277 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa4ec0a2c10 0x7fa4ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.704+0000 7fa4f67fc640 1 -- 172.21.15.6:0/2948334277 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa4f00ce7f0 con 0x7fa500102790 2024-09-12T03:19:46.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.704+0000 7fa505128640 1 --2- 172.21.15.6:0/2948334277 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa4ec0a2c10 0x7fa4ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.704+0000 7fa4f67fc640 1 --2- 172.21.15.6:0/2948334277 >> [v2:172.21.15.161:6800/392541009,v1:172.21.15.161:6801/392541009] conn(0x7fa4ec0ac740 0x7fa4ec0aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.704+0000 7fa4f67fc640 1 -- 172.21.15.6:0/2948334277 --> [v2:172.21.15.161:6800/392541009,v1:172.21.15.161:6801/392541009] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa4f000dec0 con 0x7fa4ec0ac740 2024-09-12T03:19:46.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.704+0000 7fa4f67fc640 1 -- 172.21.15.6:0/2948334277 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fa4f00cec40 con 0x7fa500102790 2024-09-12T03:19:46.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.705+0000 7fa505128640 1 --2- 172.21.15.6:0/2948334277 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa4ec0a2c10 0x7fa4ec0a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00bbe0 tx=0x7fa4fc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:46.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.705+0000 7fa505929640 1 --2- 172.21.15.6:0/2948334277 >> [v2:172.21.15.161:6800/392541009,v1:172.21.15.161:6801/392541009] conn(0x7fa4ec0ac740 0x7fa4ec0aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.705+0000 7fa505929640 1 --2- 172.21.15.6:0/2948334277 >> [v2:172.21.15.161:6800/392541009,v1:172.21.15.161:6801/392541009] conn(0x7fa4ec0ac740 0x7fa4ec0aebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:46.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.706+0000 7fa4f67fc640 1 -- 172.21.15.6:0/2948334277 <== osd.2 v2:172.21.15.161:6800/392541009 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa4f000dec0 con 0x7fa4ec0ac740 2024-09-12T03:19:46.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:46 smithi118 ceph-mon[35048]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:19:46.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.720+0000 7fa50738c640 1 -- 172.21.15.6:0/2948334277 --> [v2:172.21.15.161:6800/392541009,v1:172.21.15.161:6801/392541009] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa500062d80 con 0x7fa4ec0ac740 2024-09-12T03:19:46.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.720+0000 7fa4f67fc640 1 -- 172.21.15.6:0/2948334277 <== osd.2 v2:172.21.15.161:6800/392541009 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa500062d80 con 0x7fa4ec0ac740 2024-09-12T03:19:46.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.720+0000 7fa50738c640 1 -- 172.21.15.6:0/2948334277 >> [v2:172.21.15.161:6800/392541009,v1:172.21.15.161:6801/392541009] conn(0x7fa4ec0ac740 msgr2=0x7fa4ec0aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.720+0000 7fa50738c640 1 --2- 172.21.15.6:0/2948334277 >> [v2:172.21.15.161:6800/392541009,v1:172.21.15.161:6801/392541009] conn(0x7fa4ec0ac740 0x7fa4ec0aebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.721+0000 7fa50738c640 1 -- 172.21.15.6:0/2948334277 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa4ec0a2c10 msgr2=0x7fa4ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.721+0000 7fa50738c640 1 --2- 172.21.15.6:0/2948334277 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa4ec0a2c10 0x7fa4ec0a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00bbe0 tx=0x7fa4fc00b390 comp rx=0 tx=0).stop 2024-09-12T03:19:46.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.721+0000 7fa50738c640 1 -- 172.21.15.6:0/2948334277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa500102790 msgr2=0x7fa500198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.721+0000 7fa50738c640 1 --2- 172.21.15.6:0/2948334277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa500102790 0x7fa500198990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000b4a0 tx=0x7fa4f000b960 comp rx=0 tx=0).stop 2024-09-12T03:19:46.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.721+0000 7fa505929640 1 -- 172.21.15.6:0/2948334277 reap_dead start 2024-09-12T03:19:46.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.721+0000 7fa50738c640 1 -- 172.21.15.6:0/2948334277 shutdown_connections 2024-09-12T03:19:46.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.721+0000 7fa50738c640 1 -- 172.21.15.6:0/2948334277 >> 172.21.15.6:0/2948334277 conn(0x7fa5000fd8c0 msgr2=0x7fa5000fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:46.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.721+0000 7fa50738c640 1 -- 172.21.15.6:0/2948334277 shutdown_connections 2024-09-12T03:19:46.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:46.721+0000 7fa50738c640 1 -- 172.21.15.6:0/2948334277 wait complete. 2024-09-12T03:19:47.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.101+0000 7f61452e4640 1 -- 172.21.15.6:0/2517800121 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6140102790 msgr2=0x7f61401067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.101+0000 7f61452e4640 1 --2- 172.21.15.6:0/2517800121 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6140102790 0x7f61401067e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6134009900 tx=0x7f613402a410 comp rx=0 tx=0).stop 2024-09-12T03:19:47.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.101+0000 7f61452e4640 1 -- 172.21.15.6:0/2517800121 shutdown_connections 2024-09-12T03:19:47.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.101+0000 7f61452e4640 1 --2- 172.21.15.6:0/2517800121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6140106f10 0x7f614010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.101+0000 7f61452e4640 1 --2- 172.21.15.6:0/2517800121 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6140102790 0x7f61401067e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.101+0000 7f61452e4640 1 --2- 172.21.15.6:0/2517800121 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6140101de0 0x7f61401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.101+0000 7f61452e4640 1 -- 172.21.15.6:0/2517800121 >> 172.21.15.6:0/2517800121 conn(0x7f61400fd8c0 msgr2=0x7f61400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:47.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.101+0000 7f61452e4640 1 -- 172.21.15.6:0/2517800121 shutdown_connections 2024-09-12T03:19:47.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.101+0000 7f61452e4640 1 -- 172.21.15.6:0/2517800121 wait complete. 2024-09-12T03:19:47.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.102+0000 7f61452e4640 1 Processor -- start 2024-09-12T03:19:47.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.102+0000 7f61452e4640 1 -- start start 2024-09-12T03:19:47.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.102+0000 7f61452e4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6140101de0 0x7f61401985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.102+0000 7f61452e4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6140102790 0x7f6140198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.102+0000 7f61452e4640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6140106f10 0x7f614019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.102+0000 7f61452e4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6140112010 con 0x7f6140102790 2024-09-12T03:19:47.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.102+0000 7f61452e4640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f6140111e90 con 0x7f6140106f10 2024-09-12T03:19:47.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.102+0000 7f61452e4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6140112190 con 0x7f6140101de0 2024-09-12T03:19:47.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.102+0000 7f613e7fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6140102790 0x7f6140198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.102+0000 7f613e7fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6140102790 0x7f6140198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48176/0 (socket says 172.21.15.6:48176) 2024-09-12T03:19:47.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.102+0000 7f613e7fc640 1 -- 172.21.15.6:0/155529653 learned_addr learned my addr 172.21.15.6:0/155529653 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:47.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.103+0000 7f613f7fe640 1 --2- 172.21.15.6:0/155529653 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6140106f10 0x7f614019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.103+0000 7f613effd640 1 --2- 172.21.15.6:0/155529653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6140101de0 0x7f61401985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.103+0000 7f613e7fc640 1 -- 172.21.15.6:0/155529653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6140101de0 msgr2=0x7f61401985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.103+0000 7f613e7fc640 1 --2- 172.21.15.6:0/155529653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6140101de0 0x7f61401985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.103+0000 7f613e7fc640 1 -- 172.21.15.6:0/155529653 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6140106f10 msgr2=0x7f614019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.103+0000 7f613e7fc640 1 --2- 172.21.15.6:0/155529653 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6140106f10 0x7f614019ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.103+0000 7f613e7fc640 1 -- 172.21.15.6:0/155529653 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614019d560 con 0x7f6140102790 2024-09-12T03:19:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.103+0000 7f613e7fc640 1 --2- 172.21.15.6:0/155529653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6140102790 0x7f6140198af0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6134005ab0 tx=0x7f61340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.103+0000 7f6127fff640 1 -- 172.21.15.6:0/155529653 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613403f030 con 0x7f6140102790 2024-09-12T03:19:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.103+0000 7f6127fff640 1 -- 172.21.15.6:0/155529653 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6134003ce0 con 0x7f6140102790 2024-09-12T03:19:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.103+0000 7f6127fff640 1 -- 172.21.15.6:0/155529653 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134035070 con 0x7f6140102790 2024-09-12T03:19:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.103+0000 7f61452e4640 1 -- 172.21.15.6:0/155529653 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614019d7f0 con 0x7f6140102790 2024-09-12T03:19:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.103+0000 7f61452e4640 1 -- 172.21.15.6:0/155529653 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61401a5130 con 0x7f6140102790 2024-09-12T03:19:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.104+0000 7f61452e4640 1 -- 172.21.15.6:0/155529653 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6140062980 con 0x7f6140102790 2024-09-12T03:19:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.106+0000 7f6127fff640 1 -- 172.21.15.6:0/155529653 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6134042030 con 0x7f6140102790 2024-09-12T03:19:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.106+0000 7f6127fff640 1 --2- 172.21.15.6:0/155529653 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f61280a2c10 0x7f61280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.106+0000 7f6127fff640 1 -- 172.21.15.6:0/155529653 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f61340efde0 con 0x7f6140102790 2024-09-12T03:19:47.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.106+0000 7f613effd640 1 --2- 172.21.15.6:0/155529653 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f61280a2c10 0x7f61280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.106+0000 7f6127fff640 1 --2- 172.21.15.6:0/155529653 >> [v2:172.21.15.118:6800/3377555415,v1:172.21.15.118:6801/3377555415] conn(0x7f61280ac740 0x7f61280aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.106+0000 7f6127fff640 1 -- 172.21.15.6:0/155529653 --> [v2:172.21.15.118:6800/3377555415,v1:172.21.15.118:6801/3377555415] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6134001d30 con 0x7f61280ac740 2024-09-12T03:19:47.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.106+0000 7f6127fff640 1 -- 172.21.15.6:0/155529653 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f61340f0230 con 0x7f6140102790 2024-09-12T03:19:47.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.107+0000 7f613f7fe640 1 --2- 172.21.15.6:0/155529653 >> [v2:172.21.15.118:6800/3377555415,v1:172.21.15.118:6801/3377555415] conn(0x7f61280ac740 0x7f61280aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.107+0000 7f613effd640 1 --2- 172.21.15.6:0/155529653 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f61280a2c10 0x7f61280a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6130009890 tx=0x7f6130008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.107+0000 7f613f7fe640 1 --2- 172.21.15.6:0/155529653 >> [v2:172.21.15.118:6800/3377555415,v1:172.21.15.118:6801/3377555415] conn(0x7f61280ac740 0x7f61280aebe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.108+0000 7f6127fff640 1 -- 172.21.15.6:0/155529653 <== osd.1 v2:172.21.15.118:6800/3377555415 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6134001d30 con 0x7f61280ac740 2024-09-12T03:19:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.121+0000 7f61452e4640 1 -- 172.21.15.6:0/155529653 --> [v2:172.21.15.118:6800/3377555415,v1:172.21.15.118:6801/3377555415] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6140062e00 con 0x7f61280ac740 2024-09-12T03:19:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.122+0000 7f6127fff640 1 -- 172.21.15.6:0/155529653 <== osd.1 v2:172.21.15.118:6800/3377555415 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6140062e00 con 0x7f61280ac740 2024-09-12T03:19:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.122+0000 7f61452e4640 1 -- 172.21.15.6:0/155529653 >> [v2:172.21.15.118:6800/3377555415,v1:172.21.15.118:6801/3377555415] conn(0x7f61280ac740 msgr2=0x7f61280aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.122+0000 7f61452e4640 1 --2- 172.21.15.6:0/155529653 >> [v2:172.21.15.118:6800/3377555415,v1:172.21.15.118:6801/3377555415] conn(0x7f61280ac740 0x7f61280aebe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.122+0000 7f61452e4640 1 -- 172.21.15.6:0/155529653 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f61280a2c10 msgr2=0x7f61280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.122+0000 7f61452e4640 1 --2- 172.21.15.6:0/155529653 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f61280a2c10 0x7f61280a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6130009890 tx=0x7f6130008040 comp rx=0 tx=0).stop 2024-09-12T03:19:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.123+0000 7f61452e4640 1 -- 172.21.15.6:0/155529653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6140102790 msgr2=0x7f6140198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.123+0000 7f61452e4640 1 --2- 172.21.15.6:0/155529653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6140102790 0x7f6140198af0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6134005ab0 tx=0x7f61340042f0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.123+0000 7f613f7fe640 1 -- 172.21.15.6:0/155529653 reap_dead start 2024-09-12T03:19:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.123+0000 7f61452e4640 1 -- 172.21.15.6:0/155529653 shutdown_connections 2024-09-12T03:19:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.123+0000 7f61452e4640 1 -- 172.21.15.6:0/155529653 >> 172.21.15.6:0/155529653 conn(0x7f61400fd8c0 msgr2=0x7f61400fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.123+0000 7f61452e4640 1 -- 172.21.15.6:0/155529653 shutdown_connections 2024-09-12T03:19:47.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.123+0000 7f61452e4640 1 -- 172.21.15.6:0/155529653 wait complete. 2024-09-12T03:19:47.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.562+0000 7f7a05c80640 1 -- 172.21.15.6:0/1929996752 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a00101de0 msgr2=0x7f7a001021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.562+0000 7f7a05c80640 1 --2- 172.21.15.6:0/1929996752 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a00101de0 0x7f7a001021c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f79f0009900 tx=0x7f79f002a410 comp rx=0 tx=0).stop 2024-09-12T03:19:47.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.562+0000 7f7a05c80640 1 -- 172.21.15.6:0/1929996752 shutdown_connections 2024-09-12T03:19:47.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.562+0000 7f7a05c80640 1 --2- 172.21.15.6:0/1929996752 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a00106f10 0x7f7a0010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.562+0000 7f7a05c80640 1 --2- 172.21.15.6:0/1929996752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a00102790 0x7f7a001067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.562+0000 7f7a05c80640 1 --2- 172.21.15.6:0/1929996752 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a00101de0 0x7f7a001021c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.562+0000 7f7a05c80640 1 -- 172.21.15.6:0/1929996752 >> 172.21.15.6:0/1929996752 conn(0x7f7a000fd8c0 msgr2=0x7f7a000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:47.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.562+0000 7f7a05c80640 1 -- 172.21.15.6:0/1929996752 shutdown_connections 2024-09-12T03:19:47.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.562+0000 7f7a05c80640 1 -- 172.21.15.6:0/1929996752 wait complete. 2024-09-12T03:19:47.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.562+0000 7f7a05c80640 1 Processor -- start 2024-09-12T03:19:47.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.562+0000 7f7a05c80640 1 -- start start 2024-09-12T03:19:47.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.563+0000 7f7a05c80640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a00101de0 0x7f7a00198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.563+0000 7f7a05c80640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a00102790 0x7f7a001989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.563+0000 7f7a05c80640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a00106f10 0x7f7a0019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.563+0000 7f7a05c80640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7a00111f70 con 0x7f7a00102790 2024-09-12T03:19:47.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.563+0000 7f7a05c80640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f7a00111df0 con 0x7f7a00101de0 2024-09-12T03:19:47.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.563+0000 7f7a05c80640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7a001120f0 con 0x7f7a00106f10 2024-09-12T03:19:47.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.563+0000 7f79f6ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a00102790 0x7f7a001989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.563+0000 7f79f6ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a00102790 0x7f7a001989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48208/0 (socket says 172.21.15.6:48208) 2024-09-12T03:19:47.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.563+0000 7f79f6ffd640 1 -- 172.21.15.6:0/1247464784 learned_addr learned my addr 172.21.15.6:0/1247464784 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:47.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.563+0000 7f79ff7fe640 1 --2- 172.21.15.6:0/1247464784 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a00101de0 0x7f7a00198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.563+0000 7f79fffff640 1 --2- 172.21.15.6:0/1247464784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a00106f10 0x7f7a0019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.563+0000 7f79f6ffd640 1 -- 172.21.15.6:0/1247464784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a00106f10 msgr2=0x7f7a0019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.563+0000 7f79f6ffd640 1 --2- 172.21.15.6:0/1247464784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a00106f10 0x7f7a0019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.563+0000 7f79f6ffd640 1 -- 172.21.15.6:0/1247464784 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a00101de0 msgr2=0x7f7a00198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.563+0000 7f79f6ffd640 1 --2- 172.21.15.6:0/1247464784 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a00101de0 0x7f7a00198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.563+0000 7f79f6ffd640 1 -- 172.21.15.6:0/1247464784 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a0019d420 con 0x7f7a00102790 2024-09-12T03:19:47.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.563+0000 7f79fffff640 1 --2- 172.21.15.6:0/1247464784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a00106f10 0x7f7a0019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:47.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.563+0000 7f79f6ffd640 1 --2- 172.21.15.6:0/1247464784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a00102790 0x7f7a001989b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f79e800c890 tx=0x7f79e800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:47.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.563+0000 7f79fd7fa640 1 -- 172.21.15.6:0/1247464784 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8004220 con 0x7f7a00102790 2024-09-12T03:19:47.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.564+0000 7f79fd7fa640 1 -- 172.21.15.6:0/1247464784 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79e80043c0 con 0x7f7a00102790 2024-09-12T03:19:47.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.564+0000 7f7a05c80640 1 -- 172.21.15.6:0/1247464784 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a0019d710 con 0x7f7a00102790 2024-09-12T03:19:47.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.564+0000 7f79fd7fa640 1 -- 172.21.15.6:0/1247464784 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e80049e0 con 0x7f7a00102790 2024-09-12T03:19:47.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.564+0000 7f7a05c80640 1 -- 172.21.15.6:0/1247464784 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a001a4ff0 con 0x7f7a00102790 2024-09-12T03:19:47.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.565+0000 7f7a05c80640 1 -- 172.21.15.6:0/1247464784 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7a00062980 con 0x7f7a00102790 2024-09-12T03:19:47.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.566+0000 7f79fd7fa640 1 -- 172.21.15.6:0/1247464784 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f79e80079b0 con 0x7f7a00102790 2024-09-12T03:19:47.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.567+0000 7f79fd7fa640 1 --2- 172.21.15.6:0/1247464784 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f79ec0a2c10 0x7f79ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.567+0000 7f79ff7fe640 1 --2- 172.21.15.6:0/1247464784 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f79ec0a2c10 0x7f79ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.567+0000 7f79fd7fa640 1 -- 172.21.15.6:0/1247464784 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f79e80cf490 con 0x7f7a00102790 2024-09-12T03:19:47.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.567+0000 7f79fd7fa640 1 --2- 172.21.15.6:0/1247464784 >> [v2:172.21.15.161:6824/2889795796,v1:172.21.15.161:6825/2889795796] conn(0x7f79ec0ac740 0x7f79ec0aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.567+0000 7f79fd7fa640 1 -- 172.21.15.6:0/1247464784 --> [v2:172.21.15.161:6824/2889795796,v1:172.21.15.161:6825/2889795796] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f79ec0af2b0 con 0x7f79ec0ac740 2024-09-12T03:19:47.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.567+0000 7f79fd7fa640 1 -- 172.21.15.6:0/1247464784 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f79e80cf8e0 con 0x7f7a00102790 2024-09-12T03:19:47.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.567+0000 7f79ff7fe640 1 --2- 172.21.15.6:0/1247464784 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f79ec0a2c10 0x7f79ec0a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f79f00096c0 tx=0x7f79f0031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:47.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.567+0000 7f79fffff640 1 --2- 172.21.15.6:0/1247464784 >> [v2:172.21.15.161:6824/2889795796,v1:172.21.15.161:6825/2889795796] conn(0x7f79ec0ac740 0x7f79ec0aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.568+0000 7f79fffff640 1 --2- 172.21.15.6:0/1247464784 >> [v2:172.21.15.161:6824/2889795796,v1:172.21.15.161:6825/2889795796] conn(0x7f79ec0ac740 0x7f79ec0aebe0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:47.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.569+0000 7f79fd7fa640 1 -- 172.21.15.6:0/1247464784 <== osd.11 v2:172.21.15.161:6824/2889795796 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f79ec0af2b0 con 0x7f79ec0ac740 2024-09-12T03:19:47.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.583+0000 7f7a05c80640 1 -- 172.21.15.6:0/1247464784 --> [v2:172.21.15.161:6824/2889795796,v1:172.21.15.161:6825/2889795796] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7a00062e00 con 0x7f79ec0ac740 2024-09-12T03:19:47.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.584+0000 7f79fd7fa640 1 -- 172.21.15.6:0/1247464784 <== osd.11 v2:172.21.15.161:6824/2889795796 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7a00062e00 con 0x7f79ec0ac740 2024-09-12T03:19:47.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.584+0000 7f7a05c80640 1 -- 172.21.15.6:0/1247464784 >> [v2:172.21.15.161:6824/2889795796,v1:172.21.15.161:6825/2889795796] conn(0x7f79ec0ac740 msgr2=0x7f79ec0aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.584+0000 7f7a05c80640 1 --2- 172.21.15.6:0/1247464784 >> [v2:172.21.15.161:6824/2889795796,v1:172.21.15.161:6825/2889795796] conn(0x7f79ec0ac740 0x7f79ec0aebe0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.584+0000 7f7a05c80640 1 -- 172.21.15.6:0/1247464784 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f79ec0a2c10 msgr2=0x7f79ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.584+0000 7f7a05c80640 1 --2- 172.21.15.6:0/1247464784 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f79ec0a2c10 0x7f79ec0a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f79f00096c0 tx=0x7f79f0031490 comp rx=0 tx=0).stop 2024-09-12T03:19:47.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.584+0000 7f7a05c80640 1 -- 172.21.15.6:0/1247464784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a00102790 msgr2=0x7f7a001989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.584+0000 7f7a05c80640 1 --2- 172.21.15.6:0/1247464784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a00102790 0x7f7a001989b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f79e800c890 tx=0x7f79e800cd50 comp rx=0 tx=0).stop 2024-09-12T03:19:47.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.584+0000 7f79fffff640 1 -- 172.21.15.6:0/1247464784 reap_dead start 2024-09-12T03:19:47.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.584+0000 7f7a05c80640 1 -- 172.21.15.6:0/1247464784 shutdown_connections 2024-09-12T03:19:47.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.584+0000 7f7a05c80640 1 -- 172.21.15.6:0/1247464784 >> 172.21.15.6:0/1247464784 conn(0x7f7a000fd8c0 msgr2=0x7f7a000fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:47.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.584+0000 7f7a05c80640 1 -- 172.21.15.6:0/1247464784 shutdown_connections 2024-09-12T03:19:47.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.584+0000 7f7a05c80640 1 -- 172.21.15.6:0/1247464784 wait complete. 2024-09-12T03:19:47.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.738+0000 7f47fa6ce640 1 -- 172.21.15.6:0/2038445679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47f4102d30 msgr2=0x7f47f4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.738+0000 7f47fa6ce640 1 --2- 172.21.15.6:0/2038445679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47f4102d30 0x7f47f4103110 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009930 tx=0x7f47e402a420 comp rx=0 tx=0).stop 2024-09-12T03:19:47.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.738+0000 7f47fa6ce640 1 -- 172.21.15.6:0/2038445679 shutdown_connections 2024-09-12T03:19:47.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.738+0000 7f47fa6ce640 1 --2- 172.21.15.6:0/2038445679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47f41082a0 0x7f47f410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.738+0000 7f47fa6ce640 1 --2- 172.21.15.6:0/2038445679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47f41036e0 0x7f47f4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.738+0000 7f47fa6ce640 1 --2- 172.21.15.6:0/2038445679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47f4102d30 0x7f47f4103110 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.738+0000 7f47fa6ce640 1 -- 172.21.15.6:0/2038445679 >> 172.21.15.6:0/2038445679 conn(0x7f47f40fc7d0 msgr2=0x7f47f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:47.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.738+0000 7f47fa6ce640 1 -- 172.21.15.6:0/2038445679 shutdown_connections 2024-09-12T03:19:47.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.739+0000 7f47fa6ce640 1 -- 172.21.15.6:0/2038445679 wait complete. 2024-09-12T03:19:47.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.739+0000 7f47fa6ce640 1 Processor -- start 2024-09-12T03:19:47.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.739+0000 7f47fa6ce640 1 -- start start 2024-09-12T03:19:47.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.740+0000 7f47fa6ce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47f4102d30 0x7f47f4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.740+0000 7f47fa6ce640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47f41036e0 0x7f47f4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.740+0000 7f47fa6ce640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47f41082a0 0x7f47f4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.740+0000 7f47fa6ce640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41120a0 con 0x7f47f41036e0 2024-09-12T03:19:47.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.740+0000 7f47fa6ce640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f47f4111f20 con 0x7f47f41082a0 2024-09-12T03:19:47.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.740+0000 7f47fa6ce640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f47f4112220 con 0x7f47f4102d30 2024-09-12T03:19:47.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.740+0000 7f47f37fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47f41036e0 0x7f47f4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.740+0000 7f47f37fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47f41036e0 0x7f47f4100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48224/0 (socket says 172.21.15.6:48224) 2024-09-12T03:19:47.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.740+0000 7f47f37fe640 1 -- 172.21.15.6:0/3846255560 learned_addr learned my addr 172.21.15.6:0/3846255560 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:47.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.740+0000 7f47f37fe640 1 -- 172.21.15.6:0/3846255560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47f4102d30 msgr2=0x7f47f4100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.740+0000 7f47f8c6b640 1 --2- 172.21.15.6:0/3846255560 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47f41082a0 0x7f47f4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.740+0000 7f47f37fe640 1 --2- 172.21.15.6:0/3846255560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47f4102d30 0x7f47f4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.740+0000 7f47f37fe640 1 -- 172.21.15.6:0/3846255560 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47f41082a0 msgr2=0x7f47f4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.740+0000 7f47f37fe640 1 --2- 172.21.15.6:0/3846255560 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47f41082a0 0x7f47f4101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.740+0000 7f47f37fe640 1 -- 172.21.15.6:0/3846255560 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f41a6a90 con 0x7f47f41036e0 2024-09-12T03:19:47.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.740+0000 7f47f37fe640 1 --2- 172.21.15.6:0/3846255560 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47f41036e0 0x7f47f4100ed0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f47e800d950 tx=0x7f47e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:47.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.741+0000 7f47f17fa640 1 -- 172.21.15.6:0/3846255560 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8017070 con 0x7f47f41036e0 2024-09-12T03:19:47.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.741+0000 7f47f17fa640 1 -- 172.21.15.6:0/3846255560 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e8009c30 con 0x7f47f41036e0 2024-09-12T03:19:47.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.741+0000 7f47fa6ce640 1 -- 172.21.15.6:0/3846255560 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f41a6d80 con 0x7f47f41036e0 2024-09-12T03:19:47.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.741+0000 7f47f17fa640 1 -- 172.21.15.6:0/3846255560 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e80047a0 con 0x7f47f41036e0 2024-09-12T03:19:47.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.741+0000 7f47fa6ce640 1 -- 172.21.15.6:0/3846255560 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f41a7260 con 0x7f47f41036e0 2024-09-12T03:19:47.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.742+0000 7f47fa6ce640 1 -- 172.21.15.6:0/3846255560 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f47f4062900 con 0x7f47f41036e0 2024-09-12T03:19:47.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.743+0000 7f47f17fa640 1 -- 172.21.15.6:0/3846255560 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f47e8004020 con 0x7f47f41036e0 2024-09-12T03:19:47.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.743+0000 7f47f17fa640 1 --2- 172.21.15.6:0/3846255560 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f47dc0a2c10 0x7f47dc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.743+0000 7f47f3fff640 1 --2- 172.21.15.6:0/3846255560 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f47dc0a2c10 0x7f47dc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.744+0000 7f47f17fa640 1 -- 172.21.15.6:0/3846255560 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f47e80ceea0 con 0x7f47f41036e0 2024-09-12T03:19:47.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.744+0000 7f47f17fa640 1 --2- 172.21.15.6:0/3846255560 >> [v2:172.21.15.161:6816/1783416134,v1:172.21.15.161:6817/1783416134] conn(0x7f47dc0ac740 0x7f47dc0aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.744+0000 7f47f17fa640 1 -- 172.21.15.6:0/3846255560 --> [v2:172.21.15.161:6816/1783416134,v1:172.21.15.161:6817/1783416134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f47e800fec0 con 0x7f47dc0ac740 2024-09-12T03:19:47.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.744+0000 7f47f17fa640 1 -- 172.21.15.6:0/3846255560 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f47e80d1560 con 0x7f47f41036e0 2024-09-12T03:19:47.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.744+0000 7f47f3fff640 1 --2- 172.21.15.6:0/3846255560 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f47dc0a2c10 0x7f47dc0a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f47e402a940 tx=0x7f47e4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:47.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.744+0000 7f47f8c6b640 1 --2- 172.21.15.6:0/3846255560 >> [v2:172.21.15.161:6816/1783416134,v1:172.21.15.161:6817/1783416134] conn(0x7f47dc0ac740 0x7f47dc0aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.744+0000 7f47f8c6b640 1 --2- 172.21.15.6:0/3846255560 >> [v2:172.21.15.161:6816/1783416134,v1:172.21.15.161:6817/1783416134] conn(0x7f47dc0ac740 0x7f47dc0aebe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:47.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.745+0000 7f47f17fa640 1 -- 172.21.15.6:0/3846255560 <== osd.8 v2:172.21.15.161:6816/1783416134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47e800fec0 con 0x7f47dc0ac740 2024-09-12T03:19:47.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.759+0000 7f47fa6ce640 1 -- 172.21.15.6:0/3846255560 --> [v2:172.21.15.161:6816/1783416134,v1:172.21.15.161:6817/1783416134] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f47f4062d80 con 0x7f47dc0ac740 2024-09-12T03:19:47.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.760+0000 7f47f17fa640 1 -- 172.21.15.6:0/3846255560 <== osd.8 v2:172.21.15.161:6816/1783416134 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f47f4062d80 con 0x7f47dc0ac740 2024-09-12T03:19:47.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.760+0000 7f47fa6ce640 1 -- 172.21.15.6:0/3846255560 >> [v2:172.21.15.161:6816/1783416134,v1:172.21.15.161:6817/1783416134] conn(0x7f47dc0ac740 msgr2=0x7f47dc0aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.760+0000 7f47fa6ce640 1 --2- 172.21.15.6:0/3846255560 >> [v2:172.21.15.161:6816/1783416134,v1:172.21.15.161:6817/1783416134] conn(0x7f47dc0ac740 0x7f47dc0aebe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.760+0000 7f47fa6ce640 1 -- 172.21.15.6:0/3846255560 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f47dc0a2c10 msgr2=0x7f47dc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.760+0000 7f47fa6ce640 1 --2- 172.21.15.6:0/3846255560 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f47dc0a2c10 0x7f47dc0a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f47e402a940 tx=0x7f47e4000bc0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.760+0000 7f47fa6ce640 1 -- 172.21.15.6:0/3846255560 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47f41036e0 msgr2=0x7f47f4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.760+0000 7f47fa6ce640 1 --2- 172.21.15.6:0/3846255560 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47f41036e0 0x7f47f4100ed0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f47e800d950 tx=0x7f47e800de10 comp rx=0 tx=0).stop 2024-09-12T03:19:47.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.760+0000 7f47f8c6b640 1 -- 172.21.15.6:0/3846255560 reap_dead start 2024-09-12T03:19:47.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.760+0000 7f47fa6ce640 1 -- 172.21.15.6:0/3846255560 shutdown_connections 2024-09-12T03:19:47.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.760+0000 7f47fa6ce640 1 -- 172.21.15.6:0/3846255560 >> 172.21.15.6:0/3846255560 conn(0x7f47f40fc7d0 msgr2=0x7f47f4105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:47.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.760+0000 7f47fa6ce640 1 -- 172.21.15.6:0/3846255560 shutdown_connections 2024-09-12T03:19:47.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:47.761+0000 7f47fa6ce640 1 -- 172.21.15.6:0/3846255560 wait complete. 2024-09-12T03:19:48.334 INFO:teuthology.orchestra.run.smithi006.stdout:77309411394 2024-09-12T03:19:48.334 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-12T03:19:48.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.487+0000 7f36c7e07640 1 -- 172.21.15.6:0/594952170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36c0101e10 msgr2=0x7f36c01021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:48.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.487+0000 7f36c7e07640 1 --2- 172.21.15.6:0/594952170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36c0101e10 0x7f36c01021f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009900 tx=0x7f36bc02a410 comp rx=0 tx=0).stop 2024-09-12T03:19:48.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.487+0000 7f36c7e07640 1 -- 172.21.15.6:0/594952170 shutdown_connections 2024-09-12T03:19:48.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.487+0000 7f36c7e07640 1 --2- 172.21.15.6:0/594952170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36c0106f40 0x7f36c010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.487+0000 7f36c7e07640 1 --2- 172.21.15.6:0/594952170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36c01027c0 0x7f36c0106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.487+0000 7f36c7e07640 1 --2- 172.21.15.6:0/594952170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36c0101e10 0x7f36c01021f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.487+0000 7f36c7e07640 1 -- 172.21.15.6:0/594952170 >> 172.21.15.6:0/594952170 conn(0x7f36c00fd8d0 msgr2=0x7f36c00ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:48.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.487+0000 7f36c7e07640 1 -- 172.21.15.6:0/594952170 shutdown_connections 2024-09-12T03:19:48.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.487+0000 7f36c7e07640 1 -- 172.21.15.6:0/594952170 wait complete. 2024-09-12T03:19:48.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.487+0000 7f36c7e07640 1 Processor -- start 2024-09-12T03:19:48.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.488+0000 7f36c7e07640 1 -- start start 2024-09-12T03:19:48.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.488+0000 7f36c7e07640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36c0101e10 0x7f36c0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:48.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.488+0000 7f36c7e07640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36c01027c0 0x7f36c01989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:48.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.488+0000 7f36c7e07640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36c0106f40 0x7f36c019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:48.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.488+0000 7f36c7e07640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f36c0111fc0 con 0x7f36c0106f40 2024-09-12T03:19:48.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.488+0000 7f36c7e07640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f36c0111e40 con 0x7f36c0101e10 2024-09-12T03:19:48.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.488+0000 7f36c7e07640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f36c0112140 con 0x7f36c01027c0 2024-09-12T03:19:48.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.488+0000 7f36c63a4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36c0106f40 0x7f36c019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:48.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.488+0000 7f36c63a4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36c0106f40 0x7f36c019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48238/0 (socket says 172.21.15.6:48238) 2024-09-12T03:19:48.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.488+0000 7f36c63a4640 1 -- 172.21.15.6:0/1652283270 learned_addr learned my addr 172.21.15.6:0/1652283270 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:48.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.488+0000 7f36c5ba3640 1 --2- 172.21.15.6:0/1652283270 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36c0101e10 0x7f36c0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:48.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.488+0000 7f36c53a2640 1 --2- 172.21.15.6:0/1652283270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36c01027c0 0x7f36c01989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:48.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.488+0000 7f36c63a4640 1 -- 172.21.15.6:0/1652283270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36c01027c0 msgr2=0x7f36c01989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:48.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.488+0000 7f36c63a4640 1 --2- 172.21.15.6:0/1652283270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36c01027c0 0x7f36c01989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.488+0000 7f36c63a4640 1 -- 172.21.15.6:0/1652283270 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36c0101e10 msgr2=0x7f36c0198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:48.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.488+0000 7f36c63a4640 1 --2- 172.21.15.6:0/1652283270 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36c0101e10 0x7f36c0198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.488+0000 7f36c63a4640 1 -- 172.21.15.6:0/1652283270 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c019d430 con 0x7f36c0106f40 2024-09-12T03:19:48.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.488+0000 7f36c63a4640 1 --2- 172.21.15.6:0/1652283270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36c0106f40 0x7f36c019cd50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f36b800cce0 tx=0x7f36b800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:48.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.489+0000 7f36b6ffd640 1 -- 172.21.15.6:0/1652283270 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b800f450 con 0x7f36c0106f40 2024-09-12T03:19:48.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.489+0000 7f36c7e07640 1 -- 172.21.15.6:0/1652283270 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c019d720 con 0x7f36c0106f40 2024-09-12T03:19:48.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.489+0000 7f36b6ffd640 1 -- 172.21.15.6:0/1652283270 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36b800fad0 con 0x7f36c0106f40 2024-09-12T03:19:48.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.489+0000 7f36b6ffd640 1 -- 172.21.15.6:0/1652283270 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8018c10 con 0x7f36c0106f40 2024-09-12T03:19:48.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.489+0000 7f36c7e07640 1 -- 172.21.15.6:0/1652283270 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c01a5000 con 0x7f36c0106f40 2024-09-12T03:19:48.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.490+0000 7f36c7e07640 1 -- 172.21.15.6:0/1652283270 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f36c0062980 con 0x7f36c0106f40 2024-09-12T03:19:48.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.491+0000 7f36b6ffd640 1 -- 172.21.15.6:0/1652283270 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f36b800f5f0 con 0x7f36c0106f40 2024-09-12T03:19:48.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.492+0000 7f36b6ffd640 1 --2- 172.21.15.6:0/1652283270 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f36ac0a2c10 0x7f36ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:48.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.492+0000 7f36b6ffd640 1 -- 172.21.15.6:0/1652283270 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f36b80cfaf0 con 0x7f36c0106f40 2024-09-12T03:19:48.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.492+0000 7f36c5ba3640 1 --2- 172.21.15.6:0/1652283270 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f36ac0a2c10 0x7f36ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:48.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.492+0000 7f36b6ffd640 1 --2- 172.21.15.6:0/1652283270 >> [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151] conn(0x7f36ac0ac740 0x7f36ac0aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:48.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.492+0000 7f36b6ffd640 1 -- 172.21.15.6:0/1652283270 --> [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f36b801ee50 con 0x7f36ac0ac740 2024-09-12T03:19:48.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.492+0000 7f36b6ffd640 1 -- 172.21.15.6:0/1652283270 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f36b80cff40 con 0x7f36c0106f40 2024-09-12T03:19:48.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.492+0000 7f36c53a2640 1 --2- 172.21.15.6:0/1652283270 >> [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151] conn(0x7f36ac0ac740 0x7f36ac0aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:48.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.492+0000 7f36c5ba3640 1 --2- 172.21.15.6:0/1652283270 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f36ac0a2c10 0x7f36ac0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f36bc000c00 tx=0x7f36bc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:48.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.493+0000 7f36c53a2640 1 --2- 172.21.15.6:0/1652283270 >> [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151] conn(0x7f36ac0ac740 0x7f36ac0aebe0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:48.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.493+0000 7f36b6ffd640 1 -- 172.21.15.6:0/1652283270 <== osd.3 v2:172.21.15.6:6810/1892039151 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f36b801ee50 con 0x7f36ac0ac740 2024-09-12T03:19:48.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.507+0000 7f36c7e07640 1 -- 172.21.15.6:0/1652283270 --> [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f36c0062e00 con 0x7f36ac0ac740 2024-09-12T03:19:48.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.507+0000 7f36b6ffd640 1 -- 172.21.15.6:0/1652283270 <== osd.3 v2:172.21.15.6:6810/1892039151 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f36c0062e00 con 0x7f36ac0ac740 2024-09-12T03:19:48.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.508+0000 7f36c7e07640 1 -- 172.21.15.6:0/1652283270 >> [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151] conn(0x7f36ac0ac740 msgr2=0x7f36ac0aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:48.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.508+0000 7f36c7e07640 1 --2- 172.21.15.6:0/1652283270 >> [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151] conn(0x7f36ac0ac740 0x7f36ac0aebe0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.508+0000 7f36c7e07640 1 -- 172.21.15.6:0/1652283270 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f36ac0a2c10 msgr2=0x7f36ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:48.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.508+0000 7f36c7e07640 1 --2- 172.21.15.6:0/1652283270 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f36ac0a2c10 0x7f36ac0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f36bc000c00 tx=0x7f36bc0058d0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.508+0000 7f36c7e07640 1 -- 172.21.15.6:0/1652283270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36c0106f40 msgr2=0x7f36c019cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:48.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.508+0000 7f36c7e07640 1 --2- 172.21.15.6:0/1652283270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36c0106f40 0x7f36c019cd50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f36b800cce0 tx=0x7f36b800a620 comp rx=0 tx=0).stop 2024-09-12T03:19:48.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.508+0000 7f36c63a4640 1 -- 172.21.15.6:0/1652283270 reap_dead start 2024-09-12T03:19:48.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.508+0000 7f36c7e07640 1 -- 172.21.15.6:0/1652283270 shutdown_connections 2024-09-12T03:19:48.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.508+0000 7f36c7e07640 1 -- 172.21.15.6:0/1652283270 >> 172.21.15.6:0/1652283270 conn(0x7f36c00fd8d0 msgr2=0x7f36c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:48.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.508+0000 7f36c7e07640 1 -- 172.21.15.6:0/1652283270 shutdown_connections 2024-09-12T03:19:48.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:48.508+0000 7f36c7e07640 1 -- 172.21.15.6:0/1652283270 wait complete. 2024-09-12T03:19:48.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:48 smithi006 ceph-mon[30991]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T03:19:48.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:48 smithi161 ceph-mon[35485]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T03:19:48.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:48 smithi118 ceph-mon[35048]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T03:19:49.145 INFO:teuthology.orchestra.run.smithi006.stdout:184683593773 2024-09-12T03:19:49.145 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-12T03:19:49.747 INFO:teuthology.orchestra.run.smithi006.stdout:317827579925 2024-09-12T03:19:49.747 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-12T03:19:50.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:50 smithi006 ceph-mon[30991]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T03:19:50.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.407+0000 7f8c793ae640 1 -- 172.21.15.6:0/2653102875 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c74102570 msgr2=0x7f8c741029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:50.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.407+0000 7f8c793ae640 1 --2- 172.21.15.6:0/2653102875 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c74102570 0x7f8c741029b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009900 tx=0x7f8c6802a480 comp rx=0 tx=0).stop 2024-09-12T03:19:50.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.407+0000 7f8c793ae640 1 -- 172.21.15.6:0/2653102875 shutdown_connections 2024-09-12T03:19:50.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.407+0000 7f8c793ae640 1 --2- 172.21.15.6:0/2653102875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c74102ef0 0x7f8c7410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.407+0000 7f8c793ae640 1 --2- 172.21.15.6:0/2653102875 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c74102570 0x7f8c741029b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.407+0000 7f8c793ae640 1 --2- 172.21.15.6:0/2653102875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c74100e20 0x7f8c74101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.407+0000 7f8c793ae640 1 -- 172.21.15.6:0/2653102875 >> 172.21.15.6:0/2653102875 conn(0x7f8c740fc7d0 msgr2=0x7f8c740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:50.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.407+0000 7f8c793ae640 1 -- 172.21.15.6:0/2653102875 shutdown_connections 2024-09-12T03:19:50.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.407+0000 7f8c793ae640 1 -- 172.21.15.6:0/2653102875 wait complete. 2024-09-12T03:19:50.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.408+0000 7f8c793ae640 1 Processor -- start 2024-09-12T03:19:50.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.408+0000 7f8c793ae640 1 -- start start 2024-09-12T03:19:50.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.408+0000 7f8c793ae640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c74100e20 0x7f8c74197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.408+0000 7f8c793ae640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c74102570 0x7f8c741976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.408+0000 7f8c793ae640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c74102ef0 0x7f8c74197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.408+0000 7f8c793ae640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8c74110f80 con 0x7f8c74102570 2024-09-12T03:19:50.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.408+0000 7f8c793ae640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f8c74110e00 con 0x7f8c74102ef0 2024-09-12T03:19:50.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.408+0000 7f8c793ae640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8c74111100 con 0x7f8c74100e20 2024-09-12T03:19:50.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.408+0000 7f8c727fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c74102570 0x7f8c741976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.408+0000 7f8c727fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c74102570 0x7f8c741976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52866/0 (socket says 172.21.15.6:52866) 2024-09-12T03:19:50.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.408+0000 7f8c727fc640 1 -- 172.21.15.6:0/793034406 learned_addr learned my addr 172.21.15.6:0/793034406 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:50.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.408+0000 7f8c72ffd640 1 --2- 172.21.15.6:0/793034406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c74100e20 0x7f8c74197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.409+0000 7f8c737fe640 1 --2- 172.21.15.6:0/793034406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c74102ef0 0x7f8c74197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.409+0000 7f8c727fc640 1 -- 172.21.15.6:0/793034406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c74100e20 msgr2=0x7f8c74197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:50.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.409+0000 7f8c727fc640 1 --2- 172.21.15.6:0/793034406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c74100e20 0x7f8c74197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.409+0000 7f8c727fc640 1 -- 172.21.15.6:0/793034406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c74102ef0 msgr2=0x7f8c74197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:50.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.409+0000 7f8c727fc640 1 --2- 172.21.15.6:0/793034406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c74102ef0 0x7f8c74197c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.409+0000 7f8c727fc640 1 -- 172.21.15.6:0/793034406 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c741a69f0 con 0x7f8c74102570 2024-09-12T03:19:50.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.409+0000 7f8c737fe640 1 --2- 172.21.15.6:0/793034406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c74102ef0 0x7f8c74197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:50.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.409+0000 7f8c727fc640 1 --2- 172.21.15.6:0/793034406 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c74102570 0x7f8c741976c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8c68006010 tx=0x7f8c68003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:50.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.409+0000 7f8c5bfff640 1 -- 172.21.15.6:0/793034406 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6803f070 con 0x7f8c74102570 2024-09-12T03:19:50.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.409+0000 7f8c5bfff640 1 -- 172.21.15.6:0/793034406 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c68031c30 con 0x7f8c74102570 2024-09-12T03:19:50.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.409+0000 7f8c793ae640 1 -- 172.21.15.6:0/793034406 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c741a6c80 con 0x7f8c74102570 2024-09-12T03:19:50.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.409+0000 7f8c5bfff640 1 -- 172.21.15.6:0/793034406 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c68035030 con 0x7f8c74102570 2024-09-12T03:19:50.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.409+0000 7f8c793ae640 1 -- 172.21.15.6:0/793034406 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c741a7160 con 0x7f8c74102570 2024-09-12T03:19:50.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.410+0000 7f8c793ae640 1 -- 172.21.15.6:0/793034406 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8c74062940 con 0x7f8c74102570 2024-09-12T03:19:50.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.412+0000 7f8c5bfff640 1 -- 172.21.15.6:0/793034406 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8c68042070 con 0x7f8c74102570 2024-09-12T03:19:50.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.412+0000 7f8c5bfff640 1 --2- 172.21.15.6:0/793034406 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8c5c0a2c10 0x7f8c5c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.412+0000 7f8c72ffd640 1 --2- 172.21.15.6:0/793034406 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8c5c0a2c10 0x7f8c5c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.412+0000 7f8c5bfff640 1 -- 172.21.15.6:0/793034406 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f8c680f0180 con 0x7f8c74102570 2024-09-12T03:19:50.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.412+0000 7f8c5bfff640 1 --2- 172.21.15.6:0/793034406 >> [v2:172.21.15.118:6808/150177024,v1:172.21.15.118:6809/150177024] conn(0x7f8c5c0ac740 0x7f8c5c0aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.413+0000 7f8c5bfff640 1 -- 172.21.15.6:0/793034406 --> [v2:172.21.15.118:6808/150177024,v1:172.21.15.118:6809/150177024] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8c68046e50 con 0x7f8c5c0ac740 2024-09-12T03:19:50.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.413+0000 7f8c5bfff640 1 -- 172.21.15.6:0/793034406 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f8c680f05d0 con 0x7f8c74102570 2024-09-12T03:19:50.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.413+0000 7f8c72ffd640 1 --2- 172.21.15.6:0/793034406 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8c5c0a2c10 0x7f8c5c0a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8c64005fd0 tx=0x7f8c64009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:50.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.413+0000 7f8c737fe640 1 --2- 172.21.15.6:0/793034406 >> [v2:172.21.15.118:6808/150177024,v1:172.21.15.118:6809/150177024] conn(0x7f8c5c0ac740 0x7f8c5c0aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.413+0000 7f8c737fe640 1 --2- 172.21.15.6:0/793034406 >> [v2:172.21.15.118:6808/150177024,v1:172.21.15.118:6809/150177024] conn(0x7f8c5c0ac740 0x7f8c5c0aebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:50.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.414+0000 7f8c5bfff640 1 -- 172.21.15.6:0/793034406 <== osd.4 v2:172.21.15.118:6808/150177024 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8c68046e50 con 0x7f8c5c0ac740 2024-09-12T03:19:50.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.428+0000 7f8c793ae640 1 -- 172.21.15.6:0/793034406 --> [v2:172.21.15.118:6808/150177024,v1:172.21.15.118:6809/150177024] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8c74062d40 con 0x7f8c5c0ac740 2024-09-12T03:19:50.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.428+0000 7f8c5bfff640 1 -- 172.21.15.6:0/793034406 <== osd.4 v2:172.21.15.118:6808/150177024 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8c74062d40 con 0x7f8c5c0ac740 2024-09-12T03:19:50.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.429+0000 7f8c793ae640 1 -- 172.21.15.6:0/793034406 >> [v2:172.21.15.118:6808/150177024,v1:172.21.15.118:6809/150177024] conn(0x7f8c5c0ac740 msgr2=0x7f8c5c0aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:50.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.429+0000 7f8c793ae640 1 --2- 172.21.15.6:0/793034406 >> [v2:172.21.15.118:6808/150177024,v1:172.21.15.118:6809/150177024] conn(0x7f8c5c0ac740 0x7f8c5c0aebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.429+0000 7f8c793ae640 1 -- 172.21.15.6:0/793034406 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8c5c0a2c10 msgr2=0x7f8c5c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:50.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.429+0000 7f8c793ae640 1 --2- 172.21.15.6:0/793034406 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8c5c0a2c10 0x7f8c5c0a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8c64005fd0 tx=0x7f8c64009500 comp rx=0 tx=0).stop 2024-09-12T03:19:50.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.429+0000 7f8c793ae640 1 -- 172.21.15.6:0/793034406 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c74102570 msgr2=0x7f8c741976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:50.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.429+0000 7f8c793ae640 1 --2- 172.21.15.6:0/793034406 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c74102570 0x7f8c741976c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8c68006010 tx=0x7f8c68003c10 comp rx=0 tx=0).stop 2024-09-12T03:19:50.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.429+0000 7f8c737fe640 1 -- 172.21.15.6:0/793034406 reap_dead start 2024-09-12T03:19:50.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.429+0000 7f8c793ae640 1 -- 172.21.15.6:0/793034406 shutdown_connections 2024-09-12T03:19:50.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.429+0000 7f8c793ae640 1 -- 172.21.15.6:0/793034406 >> 172.21.15.6:0/793034406 conn(0x7f8c740fc7d0 msgr2=0x7f8c74105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:50.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.429+0000 7f8c793ae640 1 -- 172.21.15.6:0/793034406 shutdown_connections 2024-09-12T03:19:50.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:50.429+0000 7f8c793ae640 1 -- 172.21.15.6:0/793034406 wait complete. 2024-09-12T03:19:50.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:50 smithi161 ceph-mon[35485]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T03:19:50.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:50 smithi118 ceph-mon[35048]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T03:19:51.311 INFO:teuthology.orchestra.run.smithi006.stdout:236223201315 2024-09-12T03:19:51.311 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-12T03:19:51.747 INFO:teuthology.orchestra.run.smithi006.stdout:107374182459 2024-09-12T03:19:51.748 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-12T03:19:51.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.932+0000 7f706fbb9640 1 -- 172.21.15.6:0/1730443511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7068100aa0 msgr2=0x7f706810ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:51.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.932+0000 7f706fbb9640 1 --2- 172.21.15.6:0/1730443511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7068100aa0 0x7f706810ea20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7060009ca0 tx=0x7f706002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.932+0000 7f706fbb9640 1 -- 172.21.15.6:0/1730443511 shutdown_connections 2024-09-12T03:19:51.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.932+0000 7f706fbb9640 1 --2- 172.21.15.6:0/1730443511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7068100aa0 0x7f706810ea20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.932+0000 7f706fbb9640 1 --2- 172.21.15.6:0/1730443511 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7068100100 0x7f7068100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.932+0000 7f706fbb9640 1 --2- 172.21.15.6:0/1730443511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7068106b80 0x7f7068106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.932+0000 7f706fbb9640 1 -- 172.21.15.6:0/1730443511 >> 172.21.15.6:0/1730443511 conn(0x7f7068077230 msgr2=0x7f70680fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:51.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.933+0000 7f706fbb9640 1 -- 172.21.15.6:0/1730443511 shutdown_connections 2024-09-12T03:19:51.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.933+0000 7f706fbb9640 1 -- 172.21.15.6:0/1730443511 wait complete. 2024-09-12T03:19:51.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.933+0000 7f706fbb9640 1 Processor -- start 2024-09-12T03:19:51.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.933+0000 7f706fbb9640 1 -- start start 2024-09-12T03:19:51.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.934+0000 7f706fbb9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7068100100 0x7f70681012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:51.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.934+0000 7f706fbb9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7068100aa0 0x7f7068101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:51.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.934+0000 7f706fbb9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7068106b80 0x7f7068101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:51.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.934+0000 7f706fbb9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7068112050 con 0x7f7068100aa0 2024-09-12T03:19:51.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.934+0000 7f706fbb9640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f7068111ed0 con 0x7f7068100100 2024-09-12T03:19:51.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.934+0000 7f706d154640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7068100aa0 0x7f7068101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:51.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.934+0000 7f706d154640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7068100aa0 0x7f7068101820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52876/0 (socket says 172.21.15.6:52876) 2024-09-12T03:19:51.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.934+0000 7f706fbb9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f70681121d0 con 0x7f7068106b80 2024-09-12T03:19:51.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.934+0000 7f706d154640 1 -- 172.21.15.6:0/1962486958 learned_addr learned my addr 172.21.15.6:0/1962486958 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:51.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.934+0000 7f706d955640 1 --2- 172.21.15.6:0/1962486958 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7068100100 0x7f70681012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:51.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.934+0000 7f706e156640 1 --2- 172.21.15.6:0/1962486958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7068106b80 0x7f7068101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:51.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.935+0000 7f706d154640 1 -- 172.21.15.6:0/1962486958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7068106b80 msgr2=0x7f7068101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:51.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.935+0000 7f706d154640 1 --2- 172.21.15.6:0/1962486958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7068106b80 0x7f7068101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.935+0000 7f706d154640 1 -- 172.21.15.6:0/1962486958 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7068100100 msgr2=0x7f70681012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:51.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.935+0000 7f706d154640 1 --2- 172.21.15.6:0/1962486958 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7068100100 0x7f70681012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.935+0000 7f706d154640 1 -- 172.21.15.6:0/1962486958 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70681a6930 con 0x7f7068100aa0 2024-09-12T03:19:51.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.935+0000 7f706d154640 1 --2- 172.21.15.6:0/1962486958 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7068100aa0 0x7f7068101820 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f705800d950 tx=0x7f705800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:51.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.935+0000 7f705effd640 1 -- 172.21.15.6:0/1962486958 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058017030 con 0x7f7068100aa0 2024-09-12T03:19:51.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.935+0000 7f705effd640 1 -- 172.21.15.6:0/1962486958 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7058004020 con 0x7f7068100aa0 2024-09-12T03:19:51.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.935+0000 7f706fbb9640 1 -- 172.21.15.6:0/1962486958 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70681a6c20 con 0x7f7068100aa0 2024-09-12T03:19:51.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.935+0000 7f706fbb9640 1 -- 172.21.15.6:0/1962486958 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70681a71b0 con 0x7f7068100aa0 2024-09-12T03:19:51.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.936+0000 7f705effd640 1 -- 172.21.15.6:0/1962486958 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70580047a0 con 0x7f7068100aa0 2024-09-12T03:19:51.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.936+0000 7f706fbb9640 1 -- 172.21.15.6:0/1962486958 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7068105530 con 0x7f7068100aa0 2024-09-12T03:19:51.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.938+0000 7f705effd640 1 -- 172.21.15.6:0/1962486958 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f70580041d0 con 0x7f7068100aa0 2024-09-12T03:19:51.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.939+0000 7f705effd640 1 --2- 172.21.15.6:0/1962486958 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f70540a2c90 0x7f70540a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:51.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.939+0000 7f706d955640 1 --2- 172.21.15.6:0/1962486958 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f70540a2c90 0x7f70540a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:51.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.939+0000 7f705effd640 1 -- 172.21.15.6:0/1962486958 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f70580cf550 con 0x7f7068100aa0 2024-09-12T03:19:51.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.940+0000 7f705effd640 1 --2- 172.21.15.6:0/1962486958 >> [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365] conn(0x7f70540ac7c0 0x7f70540aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:51.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.940+0000 7f705effd640 1 -- 172.21.15.6:0/1962486958 --> [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f70540af330 con 0x7f70540ac7c0 2024-09-12T03:19:51.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.940+0000 7f705effd640 1 -- 172.21.15.6:0/1962486958 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f70580cf9a0 con 0x7f7068100aa0 2024-09-12T03:19:51.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.940+0000 7f706d955640 1 --2- 172.21.15.6:0/1962486958 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f70540a2c90 0x7f70540a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7064005fd0 tx=0x7f706400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:51.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.940+0000 7f706e156640 1 --2- 172.21.15.6:0/1962486958 >> [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365] conn(0x7f70540ac7c0 0x7f70540aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:51.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.940+0000 7f706e156640 1 --2- 172.21.15.6:0/1962486958 >> [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365] conn(0x7f70540ac7c0 0x7f70540aec60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:51.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.941+0000 7f705effd640 1 -- 172.21.15.6:0/1962486958 <== osd.9 v2:172.21.15.6:6826/2987332365 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f70540af330 con 0x7f70540ac7c0 2024-09-12T03:19:51.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.956+0000 7f706fbb9640 1 -- 172.21.15.6:0/1962486958 --> [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7068103420 con 0x7f70540ac7c0 2024-09-12T03:19:51.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.956+0000 7f705effd640 1 -- 172.21.15.6:0/1962486958 <== osd.9 v2:172.21.15.6:6826/2987332365 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7068103420 con 0x7f70540ac7c0 2024-09-12T03:19:51.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.957+0000 7f706fbb9640 1 -- 172.21.15.6:0/1962486958 >> [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365] conn(0x7f70540ac7c0 msgr2=0x7f70540aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:51.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.957+0000 7f706fbb9640 1 --2- 172.21.15.6:0/1962486958 >> [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365] conn(0x7f70540ac7c0 0x7f70540aec60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.957+0000 7f706fbb9640 1 -- 172.21.15.6:0/1962486958 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f70540a2c90 msgr2=0x7f70540a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:51.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.957+0000 7f706fbb9640 1 --2- 172.21.15.6:0/1962486958 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f70540a2c90 0x7f70540a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7064005fd0 tx=0x7f706400b390 comp rx=0 tx=0).stop 2024-09-12T03:19:51.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.957+0000 7f706fbb9640 1 -- 172.21.15.6:0/1962486958 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7068100aa0 msgr2=0x7f7068101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:51.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.957+0000 7f706fbb9640 1 --2- 172.21.15.6:0/1962486958 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7068100aa0 0x7f7068101820 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f705800d950 tx=0x7f705800de10 comp rx=0 tx=0).stop 2024-09-12T03:19:51.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.957+0000 7f706e156640 1 -- 172.21.15.6:0/1962486958 reap_dead start 2024-09-12T03:19:51.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.957+0000 7f706fbb9640 1 -- 172.21.15.6:0/1962486958 shutdown_connections 2024-09-12T03:19:51.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.957+0000 7f706fbb9640 1 -- 172.21.15.6:0/1962486958 >> 172.21.15.6:0/1962486958 conn(0x7f7068077230 msgr2=0x7f7068104b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:51.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.957+0000 7f706fbb9640 1 -- 172.21.15.6:0/1962486958 shutdown_connections 2024-09-12T03:19:51.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:51.957+0000 7f706fbb9640 1 -- 172.21.15.6:0/1962486958 wait complete. 2024-09-12T03:19:52.455 INFO:teuthology.orchestra.run.smithi006.stdout:55834574919 2024-09-12T03:19:52.456 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-12T03:19:52.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:52 smithi006 ceph-mon[30991]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:19:52.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:52 smithi161 ceph-mon[35485]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:19:52.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:52 smithi118 ceph-mon[35048]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:19:52.860 INFO:teuthology.orchestra.run.smithi006.stdout:137438953528 2024-09-12T03:19:52.861 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-12T03:19:52.874 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:52.875 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:52.875 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:53.290 INFO:teuthology.orchestra.run.smithi006.stdout:287762808857 2024-09-12T03:19:53.290 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-12T03:19:53.903 INFO:teuthology.orchestra.run.smithi006.stdout:266287972382 2024-09-12T03:19:53.903 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-12T03:19:53.937 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:54.452 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:54 smithi006 ceph-mon[30991]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T03:19:54.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:54 smithi161 ceph-mon[35485]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T03:19:54.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.649+0000 7f86e1497640 1 -- 172.21.15.6:0/3049161482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86dc1036e0 msgr2=0x7f86dc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.649+0000 7f86e1497640 1 --2- 172.21.15.6:0/3049161482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86dc1036e0 0x7f86dc107b90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f86d0009930 tx=0x7f86d002a420 comp rx=0 tx=0).stop 2024-09-12T03:19:54.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.649+0000 7f86e1497640 1 -- 172.21.15.6:0/3049161482 shutdown_connections 2024-09-12T03:19:54.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.649+0000 7f86e1497640 1 --2- 172.21.15.6:0/3049161482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86dc1082a0 0x7f86dc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.649+0000 7f86e1497640 1 --2- 172.21.15.6:0/3049161482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86dc1036e0 0x7f86dc107b90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.649+0000 7f86e1497640 1 --2- 172.21.15.6:0/3049161482 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86dc102d30 0x7f86dc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.649+0000 7f86e1497640 1 -- 172.21.15.6:0/3049161482 >> 172.21.15.6:0/3049161482 conn(0x7f86dc0fc7d0 msgr2=0x7f86dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:54.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.649+0000 7f86e1497640 1 -- 172.21.15.6:0/3049161482 shutdown_connections 2024-09-12T03:19:54.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.649+0000 7f86e1497640 1 -- 172.21.15.6:0/3049161482 wait complete. 2024-09-12T03:19:54.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.650+0000 7f86e1497640 1 Processor -- start 2024-09-12T03:19:54.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.650+0000 7f86e1497640 1 -- start start 2024-09-12T03:19:54.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.651+0000 7f86e1497640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86dc102d30 0x7f86dc198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.651+0000 7f86e1497640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86dc1036e0 0x7f86dc198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.651+0000 7f86e1497640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86dc1082a0 0x7f86dc19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.651+0000 7f86e1497640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc1120a0 con 0x7f86dc102d30 2024-09-12T03:19:54.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.651+0000 7f86e1497640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc111f20 con 0x7f86dc1082a0 2024-09-12T03:19:54.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.651+0000 7f86e1497640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc112220 con 0x7f86dc1036e0 2024-09-12T03:19:54.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.651+0000 7f86daffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86dc102d30 0x7f86dc198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.651+0000 7f86daffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86dc102d30 0x7f86dc198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52886/0 (socket says 172.21.15.6:52886) 2024-09-12T03:19:54.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.651+0000 7f86daffd640 1 -- 172.21.15.6:0/2852494233 learned_addr learned my addr 172.21.15.6:0/2852494233 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:54.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.651+0000 7f86da7fc640 1 --2- 172.21.15.6:0/2852494233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86dc1036e0 0x7f86dc198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.651+0000 7f86db7fe640 1 --2- 172.21.15.6:0/2852494233 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86dc1082a0 0x7f86dc19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.651+0000 7f86daffd640 1 -- 172.21.15.6:0/2852494233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86dc1036e0 msgr2=0x7f86dc198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.651+0000 7f86daffd640 1 --2- 172.21.15.6:0/2852494233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86dc1036e0 0x7f86dc198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.651+0000 7f86daffd640 1 -- 172.21.15.6:0/2852494233 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86dc1082a0 msgr2=0x7f86dc19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.651+0000 7f86daffd640 1 --2- 172.21.15.6:0/2852494233 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86dc1082a0 0x7f86dc19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.651+0000 7f86daffd640 1 -- 172.21.15.6:0/2852494233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86dc19d4d0 con 0x7f86dc102d30 2024-09-12T03:19:54.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.651+0000 7f86db7fe640 1 --2- 172.21.15.6:0/2852494233 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86dc1082a0 0x7f86dc19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:54.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.651+0000 7f86daffd640 1 --2- 172.21.15.6:0/2852494233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86dc102d30 0x7f86dc198520 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00c920 tx=0x7f86cc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:54.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.652+0000 7f86c3fff640 1 -- 172.21.15.6:0/2852494233 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc007c70 con 0x7f86dc102d30 2024-09-12T03:19:54.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.652+0000 7f86e1497640 1 -- 172.21.15.6:0/2852494233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86dc19d7c0 con 0x7f86dc102d30 2024-09-12T03:19:54.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.652+0000 7f86c3fff640 1 -- 172.21.15.6:0/2852494233 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86cc007e10 con 0x7f86dc102d30 2024-09-12T03:19:54.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.652+0000 7f86c3fff640 1 -- 172.21.15.6:0/2852494233 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc018890 con 0x7f86dc102d30 2024-09-12T03:19:54.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.652+0000 7f86e1497640 1 -- 172.21.15.6:0/2852494233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86dc1a50a0 con 0x7f86dc102d30 2024-09-12T03:19:54.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.653+0000 7f86e1497640 1 -- 172.21.15.6:0/2852494233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f86dc062940 con 0x7f86dc102d30 2024-09-12T03:19:54.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.655+0000 7f86c3fff640 1 -- 172.21.15.6:0/2852494233 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f86cc018a30 con 0x7f86dc102d30 2024-09-12T03:19:54.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.655+0000 7f86c3fff640 1 --2- 172.21.15.6:0/2852494233 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f86c40a2c10 0x7f86c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.656+0000 7f86da7fc640 1 --2- 172.21.15.6:0/2852494233 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f86c40a2c10 0x7f86c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.656+0000 7f86c3fff640 1 -- 172.21.15.6:0/2852494233 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f86cc0cf720 con 0x7f86dc102d30 2024-09-12T03:19:54.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.656+0000 7f86c3fff640 1 --2- 172.21.15.6:0/2852494233 >> [v2:172.21.15.6:6802/726729286,v1:172.21.15.6:6803/726729286] conn(0x7f86c40ac740 0x7f86c40aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.656+0000 7f86c3fff640 1 -- 172.21.15.6:0/2852494233 --> [v2:172.21.15.6:6802/726729286,v1:172.21.15.6:6803/726729286] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f86c40af2b0 con 0x7f86c40ac740 2024-09-12T03:19:54.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.656+0000 7f86c3fff640 1 -- 172.21.15.6:0/2852494233 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f86cc0cfb70 con 0x7f86dc102d30 2024-09-12T03:19:54.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.656+0000 7f86db7fe640 1 --2- 172.21.15.6:0/2852494233 >> [v2:172.21.15.6:6802/726729286,v1:172.21.15.6:6803/726729286] conn(0x7f86c40ac740 0x7f86c40aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.656+0000 7f86da7fc640 1 --2- 172.21.15.6:0/2852494233 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f86c40a2c10 0x7f86c40a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f86d0005940 tx=0x7f86d00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:54.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.656+0000 7f86db7fe640 1 --2- 172.21.15.6:0/2852494233 >> [v2:172.21.15.6:6802/726729286,v1:172.21.15.6:6803/726729286] conn(0x7f86c40ac740 0x7f86c40aebe0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:54.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.657+0000 7f86c3fff640 1 -- 172.21.15.6:0/2852494233 <== osd.0 v2:172.21.15.6:6802/726729286 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f86c40af2b0 con 0x7f86c40ac740 2024-09-12T03:19:54.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.671+0000 7f86e1497640 1 -- 172.21.15.6:0/2852494233 --> [v2:172.21.15.6:6802/726729286,v1:172.21.15.6:6803/726729286] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f86dc062d40 con 0x7f86c40ac740 2024-09-12T03:19:54.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.671+0000 7f86c3fff640 1 -- 172.21.15.6:0/2852494233 <== osd.0 v2:172.21.15.6:6802/726729286 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f86dc062d40 con 0x7f86c40ac740 2024-09-12T03:19:54.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.671+0000 7f86e1497640 1 -- 172.21.15.6:0/2852494233 >> [v2:172.21.15.6:6802/726729286,v1:172.21.15.6:6803/726729286] conn(0x7f86c40ac740 msgr2=0x7f86c40aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.671+0000 7f86e1497640 1 --2- 172.21.15.6:0/2852494233 >> [v2:172.21.15.6:6802/726729286,v1:172.21.15.6:6803/726729286] conn(0x7f86c40ac740 0x7f86c40aebe0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.671+0000 7f86e1497640 1 -- 172.21.15.6:0/2852494233 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f86c40a2c10 msgr2=0x7f86c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.671+0000 7f86e1497640 1 --2- 172.21.15.6:0/2852494233 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f86c40a2c10 0x7f86c40a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f86d0005940 tx=0x7f86d00058d0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.671+0000 7f86e1497640 1 -- 172.21.15.6:0/2852494233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86dc102d30 msgr2=0x7f86dc198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.671+0000 7f86e1497640 1 --2- 172.21.15.6:0/2852494233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86dc102d30 0x7f86dc198520 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00c920 tx=0x7f86cc00cde0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.671+0000 7f86db7fe640 1 -- 172.21.15.6:0/2852494233 reap_dead start 2024-09-12T03:19:54.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.671+0000 7f86e1497640 1 -- 172.21.15.6:0/2852494233 shutdown_connections 2024-09-12T03:19:54.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.671+0000 7f86e1497640 1 -- 172.21.15.6:0/2852494233 >> 172.21.15.6:0/2852494233 conn(0x7f86dc0fc7d0 msgr2=0x7f86dc105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:54.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.672+0000 7f86e1497640 1 -- 172.21.15.6:0/2852494233 shutdown_connections 2024-09-12T03:19:54.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.672+0000 7f86e1497640 1 -- 172.21.15.6:0/2852494233 wait complete. 2024-09-12T03:19:54.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:54 smithi118 ceph-mon[35048]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T03:19:54.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.822+0000 7ff24ea0f640 1 -- 172.21.15.6:0/1720990587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff248102ef0 msgr2=0x7ff24810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.822+0000 7ff24ea0f640 1 --2- 172.21.15.6:0/1720990587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff248102ef0 0x7ff24810da40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff240009ca0 tx=0x7ff24002a950 comp rx=0 tx=0).stop 2024-09-12T03:19:54.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.822+0000 7ff24ea0f640 1 -- 172.21.15.6:0/1720990587 shutdown_connections 2024-09-12T03:19:54.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.822+0000 7ff24ea0f640 1 --2- 172.21.15.6:0/1720990587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff248102ef0 0x7ff24810da40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.822+0000 7ff24ea0f640 1 --2- 172.21.15.6:0/1720990587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff248102570 0x7ff2481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.822+0000 7ff24ea0f640 1 --2- 172.21.15.6:0/1720990587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff248100e20 0x7ff248101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.822+0000 7ff24ea0f640 1 -- 172.21.15.6:0/1720990587 >> 172.21.15.6:0/1720990587 conn(0x7ff2480fc7d0 msgr2=0x7ff2480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:54.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.822+0000 7ff24ea0f640 1 -- 172.21.15.6:0/1720990587 shutdown_connections 2024-09-12T03:19:54.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.822+0000 7ff24ea0f640 1 -- 172.21.15.6:0/1720990587 wait complete. 2024-09-12T03:19:54.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.822+0000 7ff24ea0f640 1 Processor -- start 2024-09-12T03:19:54.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.823+0000 7ff24ea0f640 1 -- start start 2024-09-12T03:19:54.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.823+0000 7ff24ea0f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff248100e20 0x7ff2480fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.823+0000 7ff24ea0f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff248102570 0x7ff2480fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.823+0000 7ff24ea0f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff248102ef0 0x7ff2480fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.823+0000 7ff24ea0f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff248110f80 con 0x7ff248102ef0 2024-09-12T03:19:54.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.823+0000 7ff24ea0f640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7ff248110e00 con 0x7ff248100e20 2024-09-12T03:19:54.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.823+0000 7ff24ea0f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff248111100 con 0x7ff248102570 2024-09-12T03:19:54.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.823+0000 7ff24cfac640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff248102ef0 0x7ff2480fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.823+0000 7ff24cfac640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff248102ef0 0x7ff2480fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52892/0 (socket says 172.21.15.6:52892) 2024-09-12T03:19:54.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.823+0000 7ff24cfac640 1 -- 172.21.15.6:0/3494199808 learned_addr learned my addr 172.21.15.6:0/3494199808 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:54.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.823+0000 7ff23ffff640 1 --2- 172.21.15.6:0/3494199808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff248102570 0x7ff2480fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.823+0000 7ff247fff640 1 --2- 172.21.15.6:0/3494199808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff248100e20 0x7ff2480fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.823+0000 7ff24cfac640 1 -- 172.21.15.6:0/3494199808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff248102570 msgr2=0x7ff2480fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.823+0000 7ff24cfac640 1 --2- 172.21.15.6:0/3494199808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff248102570 0x7ff2480fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.823+0000 7ff24cfac640 1 -- 172.21.15.6:0/3494199808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff248100e20 msgr2=0x7ff2480fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.823+0000 7ff24cfac640 1 --2- 172.21.15.6:0/3494199808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff248100e20 0x7ff2480fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.823+0000 7ff24cfac640 1 -- 172.21.15.6:0/3494199808 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2481a6950 con 0x7ff248102ef0 2024-09-12T03:19:54.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.823+0000 7ff247fff640 1 --2- 172.21.15.6:0/3494199808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff248100e20 0x7ff2480fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:54.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.824+0000 7ff24cfac640 1 --2- 172.21.15.6:0/3494199808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff248102ef0 0x7ff2480fd700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff240004af0 tx=0x7ff24002f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:54.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.824+0000 7ff245ffb640 1 -- 172.21.15.6:0/3494199808 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff240034040 con 0x7ff248102ef0 2024-09-12T03:19:54.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.824+0000 7ff245ffb640 1 -- 172.21.15.6:0/3494199808 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff240030920 con 0x7ff248102ef0 2024-09-12T03:19:54.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.824+0000 7ff245ffb640 1 -- 172.21.15.6:0/3494199808 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff240004630 con 0x7ff248102ef0 2024-09-12T03:19:54.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.824+0000 7ff24ea0f640 1 -- 172.21.15.6:0/3494199808 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2481a6be0 con 0x7ff248102ef0 2024-09-12T03:19:54.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.824+0000 7ff24ea0f640 1 -- 172.21.15.6:0/3494199808 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2481a7110 con 0x7ff248102ef0 2024-09-12T03:19:54.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.825+0000 7ff24ea0f640 1 -- 172.21.15.6:0/3494199808 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff248101200 con 0x7ff248102ef0 2024-09-12T03:19:54.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.826+0000 7ff245ffb640 1 -- 172.21.15.6:0/3494199808 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff240037020 con 0x7ff248102ef0 2024-09-12T03:19:54.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.827+0000 7ff245ffb640 1 --2- 172.21.15.6:0/3494199808 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff2340a2c90 0x7ff2340a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.827+0000 7ff247fff640 1 --2- 172.21.15.6:0/3494199808 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff2340a2c90 0x7ff2340a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.827+0000 7ff245ffb640 1 -- 172.21.15.6:0/3494199808 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff2400efee0 con 0x7ff248102ef0 2024-09-12T03:19:54.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.827+0000 7ff247fff640 1 --2- 172.21.15.6:0/3494199808 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff2340a2c90 0x7ff2340a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff238009b50 tx=0x7ff238005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:54.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:54.832+0000 7ff245ffb640 1 -- 172.21.15.6:0/3494199808 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2400bce20 con 0x7ff248102ef0 2024-09-12T03:19:55.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:55.011+0000 7ff24ea0f640 1 -- 172.21.15.6:0/3494199808 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7ff24810a0a0 con 0x7ff248102ef0 2024-09-12T03:19:55.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:55.011+0000 7ff245ffb640 1 -- 172.21.15.6:0/3494199808 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff2400bc720 con 0x7ff248102ef0 2024-09-12T03:19:55.011 INFO:teuthology.orchestra.run.smithi006.stdout:317827579926 2024-09-12T03:19:55.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:55.013+0000 7ff24ea0f640 1 -- 172.21.15.6:0/3494199808 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff2340a2c90 msgr2=0x7ff2340a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:55.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:55.013+0000 7ff24ea0f640 1 --2- 172.21.15.6:0/3494199808 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff2340a2c90 0x7ff2340a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff238009b50 tx=0x7ff238005f90 comp rx=0 tx=0).stop 2024-09-12T03:19:55.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:55.013+0000 7ff24ea0f640 1 -- 172.21.15.6:0/3494199808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff248102ef0 msgr2=0x7ff2480fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:55.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:55.013+0000 7ff24ea0f640 1 --2- 172.21.15.6:0/3494199808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff248102ef0 0x7ff2480fd700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff240004af0 tx=0x7ff24002f3a0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:55.013+0000 7ff24ea0f640 1 -- 172.21.15.6:0/3494199808 shutdown_connections 2024-09-12T03:19:55.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:55.014+0000 7ff24ea0f640 1 --2- 172.21.15.6:0/3494199808 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff2340a2c90 0x7ff2340a5150 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:55.014+0000 7ff24ea0f640 1 --2- 172.21.15.6:0/3494199808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff248102ef0 0x7ff2480fd700 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:55.014+0000 7ff24ea0f640 1 --2- 172.21.15.6:0/3494199808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff248102570 0x7ff2480fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:55.014+0000 7ff24ea0f640 1 --2- 172.21.15.6:0/3494199808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff248100e20 0x7ff2480fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:55.014+0000 7ff24ea0f640 1 -- 172.21.15.6:0/3494199808 >> 172.21.15.6:0/3494199808 conn(0x7ff2480fc7d0 msgr2=0x7ff24806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:55.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:55.014+0000 7ff24ea0f640 1 -- 172.21.15.6:0/3494199808 shutdown_connections 2024-09-12T03:19:55.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:55.014+0000 7ff24ea0f640 1 -- 172.21.15.6:0/3494199808 wait complete. 2024-09-12T03:19:55.185 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:55.185 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:55.186 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:55.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:55 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3494199808' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T03:19:55.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:55 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3494199808' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T03:19:55.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:55 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3494199808' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T03:19:55.741 INFO:teuthology.orchestra.run.smithi006.stdout:34359738445 2024-09-12T03:19:55.741 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-12T03:19:55.949 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:56.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:56 smithi006 ceph-mon[30991]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:19:56.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:56 smithi161 ceph-mon[35485]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:19:56.661 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:56.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:56 smithi118 ceph-mon[35048]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:19:57.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.638+0000 7f077a371640 1 -- 172.21.15.6:0/268450240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0774106f40 msgr2=0x7f077410eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:57.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.638+0000 7f077a371640 1 --2- 172.21.15.6:0/268450240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0774106f40 0x7f077410eac0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f076c009ca0 tx=0x7f076c02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.638+0000 7f077a371640 1 -- 172.21.15.6:0/268450240 shutdown_connections 2024-09-12T03:19:57.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.638+0000 7f077a371640 1 --2- 172.21.15.6:0/268450240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0774106f40 0x7f077410eac0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.638+0000 7f077a371640 1 --2- 172.21.15.6:0/268450240 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f07741027c0 0x7f0774106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.638+0000 7f077a371640 1 --2- 172.21.15.6:0/268450240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0774101e10 0x7f07741021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.638+0000 7f077a371640 1 -- 172.21.15.6:0/268450240 >> 172.21.15.6:0/268450240 conn(0x7f07740fd8d0 msgr2=0x7f07740ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:57.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.638+0000 7f077a371640 1 -- 172.21.15.6:0/268450240 shutdown_connections 2024-09-12T03:19:57.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.638+0000 7f077a371640 1 -- 172.21.15.6:0/268450240 wait complete. 2024-09-12T03:19:57.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.639+0000 7f077a371640 1 Processor -- start 2024-09-12T03:19:57.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.639+0000 7f077a371640 1 -- start start 2024-09-12T03:19:57.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.639+0000 7f077a371640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0774101e10 0x7f07740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:57.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.639+0000 7f077a371640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07741027c0 0x7f0774074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:57.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.639+0000 7f077a371640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0774106f40 0x7f0774074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:57.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.639+0000 7f077a371640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0774111fe0 con 0x7f0774101e10 2024-09-12T03:19:57.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.639+0000 7f077a371640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f0774111e60 con 0x7f0774106f40 2024-09-12T03:19:57.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.640+0000 7f077a371640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0774112160 con 0x7f07741027c0 2024-09-12T03:19:57.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.640+0000 7f0773fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0774101e10 0x7f07740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:57.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.640+0000 7f0773fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0774101e10 0x7f07740744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52934/0 (socket says 172.21.15.6:52934) 2024-09-12T03:19:57.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.640+0000 7f0773fff640 1 -- 172.21.15.6:0/1668327332 learned_addr learned my addr 172.21.15.6:0/1668327332 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:57.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.640+0000 7f077890e640 1 --2- 172.21.15.6:0/1668327332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0774106f40 0x7f0774074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:57.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.640+0000 7f07737fe640 1 --2- 172.21.15.6:0/1668327332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07741027c0 0x7f0774074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:57.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.640+0000 7f0773fff640 1 -- 172.21.15.6:0/1668327332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07741027c0 msgr2=0x7f0774074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:57.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.640+0000 7f0773fff640 1 --2- 172.21.15.6:0/1668327332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07741027c0 0x7f0774074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.640+0000 7f0773fff640 1 -- 172.21.15.6:0/1668327332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0774106f40 msgr2=0x7f0774074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:57.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.640+0000 7f0773fff640 1 --2- 172.21.15.6:0/1668327332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0774106f40 0x7f0774074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.640+0000 7f0773fff640 1 -- 172.21.15.6:0/1668327332 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07741a6900 con 0x7f0774101e10 2024-09-12T03:19:57.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.640+0000 7f07737fe640 1 --2- 172.21.15.6:0/1668327332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07741027c0 0x7f0774074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:57.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.640+0000 7f0773fff640 1 --2- 172.21.15.6:0/1668327332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0774101e10 0x7f07740744f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f076400b450 tx=0x7f076400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:57.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.640+0000 7f07717fa640 1 -- 172.21.15.6:0/1668327332 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0764015020 con 0x7f0774101e10 2024-09-12T03:19:57.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.640+0000 7f077a371640 1 -- 172.21.15.6:0/1668327332 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07741a6bf0 con 0x7f0774101e10 2024-09-12T03:19:57.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.640+0000 7f07717fa640 1 -- 172.21.15.6:0/1668327332 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0764011d70 con 0x7f0774101e10 2024-09-12T03:19:57.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.640+0000 7f07717fa640 1 -- 172.21.15.6:0/1668327332 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076400f340 con 0x7f0774101e10 2024-09-12T03:19:57.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.640+0000 7f077a371640 1 -- 172.21.15.6:0/1668327332 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07741a70e0 con 0x7f0774101e10 2024-09-12T03:19:57.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.641+0000 7f077a371640 1 -- 172.21.15.6:0/1668327332 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07741021f0 con 0x7f0774101e10 2024-09-12T03:19:57.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.643+0000 7f07717fa640 1 -- 172.21.15.6:0/1668327332 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0764018020 con 0x7f0774101e10 2024-09-12T03:19:57.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.643+0000 7f07717fa640 1 --2- 172.21.15.6:0/1668327332 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f075c0a2c10 0x7f075c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:57.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.644+0000 7f07737fe640 1 --2- 172.21.15.6:0/1668327332 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f075c0a2c10 0x7f075c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:57.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.644+0000 7f07717fa640 1 -- 172.21.15.6:0/1668327332 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f07640ceec0 con 0x7f0774101e10 2024-09-12T03:19:57.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.644+0000 7f07737fe640 1 --2- 172.21.15.6:0/1668327332 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f075c0a2c10 0x7f075c0a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0774075f80 tx=0x7f0768005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:57.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.647+0000 7f07717fa640 1 -- 172.21.15.6:0/1668327332 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f076401f030 con 0x7f0774101e10 2024-09-12T03:19:57.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.343+0000 7f4e09793640 1 -- 172.21.15.6:0/760127768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e04101430 msgr2=0x7f4e0410d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:57.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.343+0000 7f4e09793640 1 --2- 172.21.15.6:0/760127768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e04101430 0x7f4e0410d960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009cd0 tx=0x7f4dfc02a8f0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.343+0000 7f4e09793640 1 -- 172.21.15.6:0/760127768 shutdown_connections 2024-09-12T03:19:57.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.343+0000 7f4e09793640 1 --2- 172.21.15.6:0/760127768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e04101430 0x7f4e0410d960 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.343+0000 7f4e09793640 1 --2- 172.21.15.6:0/760127768 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e04100a90 0x7f4e04100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.343+0000 7f4e09793640 1 --2- 172.21.15.6:0/760127768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e04106a90 0x7f4e04106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.343+0000 7f4e09793640 1 -- 172.21.15.6:0/760127768 >> 172.21.15.6:0/760127768 conn(0x7f4e040fc7d0 msgr2=0x7f4e040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:57.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.343+0000 7f4e09793640 1 -- 172.21.15.6:0/760127768 shutdown_connections 2024-09-12T03:19:57.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.343+0000 7f4e09793640 1 -- 172.21.15.6:0/760127768 wait complete. 2024-09-12T03:19:57.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.344+0000 7f4e09793640 1 Processor -- start 2024-09-12T03:19:57.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.344+0000 7f4e09793640 1 -- start start 2024-09-12T03:19:57.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.345+0000 7f4e09793640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e04100a90 0x7f4e04073990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:57.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.345+0000 7f4e09793640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e04101430 0x7f4e04073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:57.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.345+0000 7f4e09793640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e04106a90 0x7f4e0406cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:57.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.345+0000 7f4e037fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e04106a90 0x7f4e0406cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:57.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.345+0000 7f4e02ffd640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e04100a90 0x7f4e04073990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:57.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.345+0000 7f4e027fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e04101430 0x7f4e04073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:57.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.345+0000 7f4e02ffd640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e04100a90 0x7f4e04073990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:51084/0 (socket says 172.21.15.6:51084) 2024-09-12T03:19:57.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.345+0000 7f4e027fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e04101430 0x7f4e04073ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:35396/0 (socket says 172.21.15.6:35396) 2024-09-12T03:19:57.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.345+0000 7f4e02ffd640 1 -- 172.21.15.6:0/2387280696 learned_addr learned my addr 172.21.15.6:0/2387280696 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:57.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.345+0000 7f4e09793640 1 -- 172.21.15.6:0/2387280696 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0410fde0 con 0x7f4e04106a90 2024-09-12T03:19:57.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.345+0000 7f4e09793640 1 -- 172.21.15.6:0/2387280696 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0410fc60 con 0x7f4e04100a90 2024-09-12T03:19:57.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.345+0000 7f4e09793640 1 -- 172.21.15.6:0/2387280696 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0410ff60 con 0x7f4e04101430 2024-09-12T03:19:57.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.345+0000 7f4e02ffd640 1 -- 172.21.15.6:0/2387280696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e04101430 msgr2=0x7f4e04073ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:57.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.345+0000 7f4e02ffd640 1 --2- 172.21.15.6:0/2387280696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e04101430 0x7f4e04073ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.798 INFO:teuthology.orchestra.run.smithi006.stdout:184683593775 2024-09-12T03:19:57.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.345+0000 7f4e02ffd640 1 -- 172.21.15.6:0/2387280696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e04106a90 msgr2=0x7f4e0406cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:57.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.345+0000 7f4e02ffd640 1 --2- 172.21.15.6:0/2387280696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e04106a90 0x7f4e0406cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.345+0000 7f4e02ffd640 1 -- 172.21.15.6:0/2387280696 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0406d4c0 con 0x7f4e04100a90 2024-09-12T03:19:57.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.345+0000 7f4e027fc640 1 --2- 172.21.15.6:0/2387280696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e04101430 0x7f4e04073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:19:57.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.346+0000 7f4e037fe640 1 --2- 172.21.15.6:0/2387280696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e04106a90 0x7f4e0406cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:19:57.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.346+0000 7f4e02ffd640 1 --2- 172.21.15.6:0/2387280696 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e04100a90 0x7f4e04073990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4df400e9f0 tx=0x7f4df400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:57.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.346+0000 7f4debfff640 1 -- 172.21.15.6:0/2387280696 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df400cd80 con 0x7f4e04100a90 2024-09-12T03:19:57.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.346+0000 7f4e09793640 1 -- 172.21.15.6:0/2387280696 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e0406d7b0 con 0x7f4e04100a90 2024-09-12T03:19:57.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.346+0000 7f4debfff640 1 -- 172.21.15.6:0/2387280696 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df4005500 con 0x7f4e04100a90 2024-09-12T03:19:57.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.346+0000 7f4debfff640 1 -- 172.21.15.6:0/2387280696 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df4010690 con 0x7f4e04100a90 2024-09-12T03:19:57.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.346+0000 7f4e09793640 1 -- 172.21.15.6:0/2387280696 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e04071750 con 0x7f4e04100a90 2024-09-12T03:19:57.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.347+0000 7f4e09793640 1 -- 172.21.15.6:0/2387280696 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e041021d0 con 0x7f4e04100a90 2024-09-12T03:19:57.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.349+0000 7f4debfff640 1 -- 172.21.15.6:0/2387280696 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4df40046c0 con 0x7f4e04100a90 2024-09-12T03:19:57.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.350+0000 7f4debfff640 1 --2- 172.21.15.6:0/2387280696 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4dec0a2c10 0x7f4dec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:57.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.350+0000 7f4e027fc640 1 --2- 172.21.15.6:0/2387280696 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4dec0a2c10 0x7f4dec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:57.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.350+0000 7f4debfff640 1 -- 172.21.15.6:0/2387280696 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f4df40cf540 con 0x7f4e04100a90 2024-09-12T03:19:57.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.350+0000 7f4e027fc640 1 --2- 172.21.15.6:0/2387280696 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4dec0a2c10 0x7f4dec0a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4df800b800 tx=0x7f4df800d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:57.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.354+0000 7f4debfff640 1 -- 172.21.15.6:0/2387280696 <== mon.2 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4df409c370 con 0x7f4e04100a90 2024-09-12T03:19:57.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.543+0000 7f4e09793640 1 -- 172.21.15.6:0/2387280696 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f4e04071a10 con 0x7f4e04100a90 2024-09-12T03:19:57.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.544+0000 7f4debfff640 1 -- 172.21.15.6:0/2387280696 <== mon.2 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4df409bc70 con 0x7f4e04100a90 2024-09-12T03:19:57.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.546+0000 7f4e09793640 1 -- 172.21.15.6:0/2387280696 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4dec0a2c10 msgr2=0x7f4dec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:57.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.546+0000 7f4e09793640 1 --2- 172.21.15.6:0/2387280696 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4dec0a2c10 0x7f4dec0a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4df800b800 tx=0x7f4df800d040 comp rx=0 tx=0).stop 2024-09-12T03:19:57.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.546+0000 7f4e09793640 1 -- 172.21.15.6:0/2387280696 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e04100a90 msgr2=0x7f4e04073990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:57.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.546+0000 7f4e09793640 1 --2- 172.21.15.6:0/2387280696 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e04100a90 0x7f4e04073990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4df400e9f0 tx=0x7f4df400edf0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.546+0000 7f4e09793640 1 -- 172.21.15.6:0/2387280696 shutdown_connections 2024-09-12T03:19:57.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.546+0000 7f4e09793640 1 --2- 172.21.15.6:0/2387280696 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4dec0a2c10 0x7f4dec0a50d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.546+0000 7f4e09793640 1 --2- 172.21.15.6:0/2387280696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e04106a90 0x7f4e0406cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.546+0000 7f4e09793640 1 --2- 172.21.15.6:0/2387280696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e04101430 0x7f4e04073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.546+0000 7f4e09793640 1 --2- 172.21.15.6:0/2387280696 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e04100a90 0x7f4e04073990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.546+0000 7f4e09793640 1 -- 172.21.15.6:0/2387280696 >> 172.21.15.6:0/2387280696 conn(0x7f4e040fc7d0 msgr2=0x7f4e040fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:57.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.546+0000 7f4e09793640 1 -- 172.21.15.6:0/2387280696 shutdown_connections 2024-09-12T03:19:57.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.546+0000 7f4e09793640 1 -- 172.21.15.6:0/2387280696 wait complete. 2024-09-12T03:19:57.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.825+0000 7f077a371640 1 -- 172.21.15.6:0/1668327332 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f0774075a20 con 0x7f0774101e10 2024-09-12T03:19:57.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.825+0000 7f07717fa640 1 -- 172.21.15.6:0/1668327332 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0774075a20 con 0x7f0774101e10 2024-09-12T03:19:57.825 INFO:teuthology.orchestra.run.smithi006.stdout:77309411396 2024-09-12T03:19:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.827+0000 7f077a371640 1 -- 172.21.15.6:0/1668327332 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f075c0a2c10 msgr2=0x7f075c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.827+0000 7f077a371640 1 --2- 172.21.15.6:0/1668327332 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f075c0a2c10 0x7f075c0a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0774075f80 tx=0x7f0768005f90 comp rx=0 tx=0).stop 2024-09-12T03:19:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.827+0000 7f077a371640 1 -- 172.21.15.6:0/1668327332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0774101e10 msgr2=0x7f07740744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.827+0000 7f077a371640 1 --2- 172.21.15.6:0/1668327332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0774101e10 0x7f07740744f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f076400b450 tx=0x7f076400b910 comp rx=0 tx=0).stop 2024-09-12T03:19:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.827+0000 7f077a371640 1 -- 172.21.15.6:0/1668327332 shutdown_connections 2024-09-12T03:19:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.827+0000 7f077a371640 1 --2- 172.21.15.6:0/1668327332 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f075c0a2c10 0x7f075c0a50d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.827+0000 7f077a371640 1 --2- 172.21.15.6:0/1668327332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0774106f40 0x7f0774074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.827+0000 7f077a371640 1 --2- 172.21.15.6:0/1668327332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07741027c0 0x7f0774074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.827+0000 7f077a371640 1 --2- 172.21.15.6:0/1668327332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0774101e10 0x7f07740744f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.827+0000 7f077a371640 1 -- 172.21.15.6:0/1668327332 >> 172.21.15.6:0/1668327332 conn(0x7f07740fd8d0 msgr2=0x7f07740febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:57.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.827+0000 7f077a371640 1 -- 172.21.15.6:0/1668327332 shutdown_connections 2024-09-12T03:19:57.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.827+0000 7f077a371640 1 -- 172.21.15.6:0/1668327332 wait complete. 2024-09-12T03:19:58.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.297+0000 7fa23d624640 1 -- 172.21.15.6:0/238992764 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa238103660 msgr2=0x7fa23810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.297+0000 7fa23d624640 1 --2- 172.21.15.6:0/238992764 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa238103660 0x7fa23810bb50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009900 tx=0x7fa22c02a480 comp rx=0 tx=0).stop 2024-09-12T03:19:58.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.297+0000 7fa23d624640 1 -- 172.21.15.6:0/238992764 shutdown_connections 2024-09-12T03:19:58.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.297+0000 7fa23d624640 1 --2- 172.21.15.6:0/238992764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa23810c170 0x7fa23810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.297+0000 7fa23d624640 1 --2- 172.21.15.6:0/238992764 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa238103660 0x7fa23810bb50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.297+0000 7fa23d624640 1 --2- 172.21.15.6:0/238992764 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa238102d40 0x7fa238103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.297+0000 7fa23d624640 1 -- 172.21.15.6:0/238992764 >> 172.21.15.6:0/238992764 conn(0x7fa2380fc7a0 msgr2=0x7fa2380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:58.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.297+0000 7fa23d624640 1 -- 172.21.15.6:0/238992764 shutdown_connections 2024-09-12T03:19:58.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.297+0000 7fa23d624640 1 -- 172.21.15.6:0/238992764 wait complete. 2024-09-12T03:19:58.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.298+0000 7fa23d624640 1 Processor -- start 2024-09-12T03:19:58.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.298+0000 7fa23d624640 1 -- start start 2024-09-12T03:19:58.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.298+0000 7fa23d624640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa238102d40 0x7fa2381009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:58.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.298+0000 7fa23d624640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa238103660 0x7fa238100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:58.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.298+0000 7fa23d624640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa23810c170 0x7fa238101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:58.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.298+0000 7fa23d624640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa238111f70 con 0x7fa23810c170 2024-09-12T03:19:58.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.298+0000 7fa23d624640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa238111df0 con 0x7fa238103660 2024-09-12T03:19:58.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.298+0000 7fa23d624640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa2381120f0 con 0x7fa238102d40 2024-09-12T03:19:58.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.299+0000 7fa2377fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa23810c170 0x7fa238101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:58.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.299+0000 7fa2377fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa23810c170 0x7fa238101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52978/0 (socket says 172.21.15.6:52978) 2024-09-12T03:19:58.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.299+0000 7fa2377fe640 1 -- 172.21.15.6:0/4007262538 learned_addr learned my addr 172.21.15.6:0/4007262538 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:58.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.299+0000 7fa2367fc640 1 --2- 172.21.15.6:0/4007262538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa238103660 0x7fa238100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:58.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.299+0000 7fa236ffd640 1 --2- 172.21.15.6:0/4007262538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa238102d40 0x7fa2381009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:58.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.299+0000 7fa2377fe640 1 -- 172.21.15.6:0/4007262538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa238102d40 msgr2=0x7fa2381009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.299+0000 7fa2377fe640 1 --2- 172.21.15.6:0/4007262538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa238102d40 0x7fa2381009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.299+0000 7fa2377fe640 1 -- 172.21.15.6:0/4007262538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa238103660 msgr2=0x7fa238100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.299+0000 7fa2377fe640 1 --2- 172.21.15.6:0/4007262538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa238103660 0x7fa238100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.299+0000 7fa2377fe640 1 -- 172.21.15.6:0/4007262538 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2381a6930 con 0x7fa23810c170 2024-09-12T03:19:58.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.299+0000 7fa2367fc640 1 --2- 172.21.15.6:0/4007262538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa238103660 0x7fa238100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:58.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.299+0000 7fa2377fe640 1 --2- 172.21.15.6:0/4007262538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa23810c170 0x7fa238101420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa23000eaa0 tx=0x7fa23000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:58.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.299+0000 7fa21ffff640 1 -- 172.21.15.6:0/4007262538 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23000cdb0 con 0x7fa23810c170 2024-09-12T03:19:58.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.299+0000 7fa23d624640 1 -- 172.21.15.6:0/4007262538 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2381a6c20 con 0x7fa23810c170 2024-09-12T03:19:58.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.299+0000 7fa21ffff640 1 -- 172.21.15.6:0/4007262538 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa230005020 con 0x7fa23810c170 2024-09-12T03:19:58.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.299+0000 7fa21ffff640 1 -- 172.21.15.6:0/4007262538 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2300116b0 con 0x7fa23810c170 2024-09-12T03:19:58.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.299+0000 7fa23d624640 1 -- 172.21.15.6:0/4007262538 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2381a7160 con 0x7fa23810c170 2024-09-12T03:19:58.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.300+0000 7fa23d624640 1 -- 172.21.15.6:0/4007262538 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa238103120 con 0x7fa23810c170 2024-09-12T03:19:58.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.301+0000 7fa21ffff640 1 -- 172.21.15.6:0/4007262538 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa230005560 con 0x7fa23810c170 2024-09-12T03:19:58.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.302+0000 7fa21ffff640 1 --2- 172.21.15.6:0/4007262538 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa2200a2c10 0x7fa2200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:58.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.302+0000 7fa236ffd640 1 --2- 172.21.15.6:0/4007262538 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa2200a2c10 0x7fa2200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:58.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.302+0000 7fa21ffff640 1 -- 172.21.15.6:0/4007262538 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa2300cf4c0 con 0x7fa23810c170 2024-09-12T03:19:58.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.303+0000 7fa236ffd640 1 --2- 172.21.15.6:0/4007262538 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa2200a2c10 0x7fa2200a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa228005fd0 tx=0x7fa228009470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:58.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.306+0000 7fa21ffff640 1 -- 172.21.15.6:0/4007262538 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa230016030 con 0x7fa23810c170 2024-09-12T03:19:58.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.488+0000 7fa23d624640 1 -- 172.21.15.6:0/4007262538 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa23802d7c0 con 0x7fa23810c170 2024-09-12T03:19:58.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.488+0000 7fa21ffff640 1 -- 172.21.15.6:0/4007262538 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa23009c290 con 0x7fa23810c170 2024-09-12T03:19:58.489 INFO:teuthology.orchestra.run.smithi006.stdout:107374182461 2024-09-12T03:19:58.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.490+0000 7fa23d624640 1 -- 172.21.15.6:0/4007262538 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa2200a2c10 msgr2=0x7fa2200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.490+0000 7fa23d624640 1 --2- 172.21.15.6:0/4007262538 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa2200a2c10 0x7fa2200a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa228005fd0 tx=0x7fa228009470 comp rx=0 tx=0).stop 2024-09-12T03:19:58.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.490+0000 7fa23d624640 1 -- 172.21.15.6:0/4007262538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa23810c170 msgr2=0x7fa238101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.490+0000 7fa23d624640 1 --2- 172.21.15.6:0/4007262538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa23810c170 0x7fa238101420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa23000eaa0 tx=0x7fa23000ef60 comp rx=0 tx=0).stop 2024-09-12T03:19:58.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.490+0000 7fa23d624640 1 -- 172.21.15.6:0/4007262538 shutdown_connections 2024-09-12T03:19:58.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.490+0000 7fa23d624640 1 --2- 172.21.15.6:0/4007262538 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa2200a2c10 0x7fa2200a50d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.490+0000 7fa23d624640 1 --2- 172.21.15.6:0/4007262538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa23810c170 0x7fa238101420 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.490+0000 7fa23d624640 1 --2- 172.21.15.6:0/4007262538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa238103660 0x7fa238100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.490+0000 7fa23d624640 1 --2- 172.21.15.6:0/4007262538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa238102d40 0x7fa2381009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.490+0000 7fa23d624640 1 -- 172.21.15.6:0/4007262538 >> 172.21.15.6:0/4007262538 conn(0x7fa2380fc7a0 msgr2=0x7fa2380fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:58.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.491+0000 7fa23d624640 1 -- 172.21.15.6:0/4007262538 shutdown_connections 2024-09-12T03:19:58.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.491+0000 7fa23d624640 1 -- 172.21.15.6:0/4007262538 wait complete. 2024-09-12T03:19:58.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:58 smithi006 ceph-mon[30991]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:19:58.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:58 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2387280696' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T03:19:58.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:58 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/143861336' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T03:19:58.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:58 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1668327332' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T03:19:58.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:58 smithi161 ceph-mon[35485]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:19:58.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:58 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2387280696' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T03:19:58.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:58 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/143861336' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T03:19:58.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:58 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1668327332' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T03:19:58.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:58 smithi118 ceph-mon[35048]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:19:58.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:58 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2387280696' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T03:19:58.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:58 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/143861336' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T03:19:58.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:58 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1668327332' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T03:19:58.898 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593773 got 184683593775 for osd.6 2024-09-12T03:19:58.898 DEBUG:teuthology.parallel:result is None 2024-09-12T03:19:58.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.156+0000 7f830160d640 1 -- 172.21.15.6:0/3534718341 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82fc106f10 msgr2=0x7f82fc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.156+0000 7f830160d640 1 --2- 172.21.15.6:0/3534718341 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82fc106f10 0x7f82fc10ea90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009ca0 tx=0x7f82f402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.156+0000 7f830160d640 1 -- 172.21.15.6:0/3534718341 shutdown_connections 2024-09-12T03:19:58.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.156+0000 7f830160d640 1 --2- 172.21.15.6:0/3534718341 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82fc106f10 0x7f82fc10ea90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.156+0000 7f830160d640 1 --2- 172.21.15.6:0/3534718341 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82fc102790 0x7f82fc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.156+0000 7f830160d640 1 --2- 172.21.15.6:0/3534718341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82fc101de0 0x7f82fc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.156+0000 7f830160d640 1 -- 172.21.15.6:0/3534718341 >> 172.21.15.6:0/3534718341 conn(0x7f82fc0fd8c0 msgr2=0x7f82fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:58.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.156+0000 7f830160d640 1 -- 172.21.15.6:0/3534718341 shutdown_connections 2024-09-12T03:19:58.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.156+0000 7f830160d640 1 -- 172.21.15.6:0/3534718341 wait complete. 2024-09-12T03:19:58.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.156+0000 7f830160d640 1 Processor -- start 2024-09-12T03:19:58.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.157+0000 7f830160d640 1 -- start start 2024-09-12T03:19:58.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.157+0000 7f830160d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82fc101de0 0x7f82fc198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:58.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.157+0000 7f830160d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82fc102790 0x7f82fc1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:58.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.157+0000 7f830160d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82fc106f10 0x7f82fc19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:58.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.157+0000 7f830160d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc111fc0 con 0x7f82fc101de0 2024-09-12T03:19:58.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.157+0000 7f830160d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc111e40 con 0x7f82fc102790 2024-09-12T03:19:58.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.157+0000 7f830160d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc112140 con 0x7f82fc106f10 2024-09-12T03:19:58.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.157+0000 7f82faffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82fc101de0 0x7f82fc198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:58.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.157+0000 7f82faffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82fc101de0 0x7f82fc198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52952/0 (socket says 172.21.15.6:52952) 2024-09-12T03:19:58.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.157+0000 7f82faffd640 1 -- 172.21.15.6:0/877864878 learned_addr learned my addr 172.21.15.6:0/877864878 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:58.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.157+0000 7f82f27fc640 1 --2- 172.21.15.6:0/877864878 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82fc102790 0x7f82fc1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:58.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.157+0000 7f82fb7fe640 1 --2- 172.21.15.6:0/877864878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82fc106f10 0x7f82fc19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:58.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.157+0000 7f82faffd640 1 -- 172.21.15.6:0/877864878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82fc106f10 msgr2=0x7f82fc19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.157+0000 7f82faffd640 1 --2- 172.21.15.6:0/877864878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82fc106f10 0x7f82fc19cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.157+0000 7f82faffd640 1 -- 172.21.15.6:0/877864878 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82fc102790 msgr2=0x7f82fc1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.157+0000 7f82faffd640 1 --2- 172.21.15.6:0/877864878 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82fc102790 0x7f82fc1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.157+0000 7f82faffd640 1 -- 172.21.15.6:0/877864878 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82fc19d330 con 0x7f82fc101de0 2024-09-12T03:19:58.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.157+0000 7f82f27fc640 1 --2- 172.21.15.6:0/877864878 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82fc102790 0x7f82fc1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:58.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.158+0000 7f82faffd640 1 --2- 172.21.15.6:0/877864878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82fc101de0 0x7f82fc198380 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00e9f0 tx=0x7f82ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:58.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.158+0000 7f82f8ff9640 1 -- 172.21.15.6:0/877864878 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec00cd80 con 0x7f82fc101de0 2024-09-12T03:19:58.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.158+0000 7f82f8ff9640 1 -- 172.21.15.6:0/877864878 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82ec005500 con 0x7f82fc101de0 2024-09-12T03:19:58.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.158+0000 7f830160d640 1 -- 172.21.15.6:0/877864878 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82fc19d620 con 0x7f82fc101de0 2024-09-12T03:19:58.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.158+0000 7f82f8ff9640 1 -- 172.21.15.6:0/877864878 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec010690 con 0x7f82fc101de0 2024-09-12T03:19:58.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.158+0000 7f830160d640 1 -- 172.21.15.6:0/877864878 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82fc1a4f00 con 0x7f82fc101de0 2024-09-12T03:19:58.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.159+0000 7f830160d640 1 -- 172.21.15.6:0/877864878 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f82fc0fedf0 con 0x7f82fc101de0 2024-09-12T03:19:58.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.160+0000 7f82f8ff9640 1 -- 172.21.15.6:0/877864878 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f82ec0046c0 con 0x7f82fc101de0 2024-09-12T03:19:58.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.160+0000 7f82f8ff9640 1 --2- 172.21.15.6:0/877864878 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f82e80a2c90 0x7f82e80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:58.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.161+0000 7f82f8ff9640 1 -- 172.21.15.6:0/877864878 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f82ec0cfac0 con 0x7f82fc101de0 2024-09-12T03:19:58.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.161+0000 7f82f27fc640 1 --2- 172.21.15.6:0/877864878 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f82e80a2c90 0x7f82e80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:58.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.161+0000 7f82f8ff9640 1 --2- 172.21.15.6:0/877864878 >> [v2:172.21.15.118:6816/269980353,v1:172.21.15.118:6817/269980353] conn(0x7f82e80ac7c0 0x7f82e80aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:58.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.161+0000 7f82f8ff9640 1 -- 172.21.15.6:0/877864878 --> [v2:172.21.15.118:6816/269980353,v1:172.21.15.118:6817/269980353] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f82ec005350 con 0x7f82e80ac7c0 2024-09-12T03:19:58.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.161+0000 7f82f8ff9640 1 -- 172.21.15.6:0/877864878 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f82ec0cff10 con 0x7f82fc101de0 2024-09-12T03:19:58.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.161+0000 7f82f27fc640 1 --2- 172.21.15.6:0/877864878 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f82e80a2c90 0x7f82e80a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0fe130 tx=0x7f82e4005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:58.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.161+0000 7f82fb7fe640 1 --2- 172.21.15.6:0/877864878 >> [v2:172.21.15.118:6816/269980353,v1:172.21.15.118:6817/269980353] conn(0x7f82e80ac7c0 0x7f82e80aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:58.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.162+0000 7f82fb7fe640 1 --2- 172.21.15.6:0/877864878 >> [v2:172.21.15.118:6816/269980353,v1:172.21.15.118:6817/269980353] conn(0x7f82e80ac7c0 0x7f82e80aec60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:58.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.162+0000 7f82f8ff9640 1 -- 172.21.15.6:0/877864878 <== osd.7 v2:172.21.15.118:6816/269980353 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f82ec005350 con 0x7f82e80ac7c0 2024-09-12T03:19:58.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.176+0000 7f830160d640 1 -- 172.21.15.6:0/877864878 --> [v2:172.21.15.118:6816/269980353,v1:172.21.15.118:6817/269980353] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f82fc1032c0 con 0x7f82e80ac7c0 2024-09-12T03:19:58.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.176+0000 7f82f8ff9640 1 -- 172.21.15.6:0/877864878 <== osd.7 v2:172.21.15.118:6816/269980353 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f82fc1032c0 con 0x7f82e80ac7c0 2024-09-12T03:19:58.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.177+0000 7f830160d640 1 -- 172.21.15.6:0/877864878 >> [v2:172.21.15.118:6816/269980353,v1:172.21.15.118:6817/269980353] conn(0x7f82e80ac7c0 msgr2=0x7f82e80aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.177+0000 7f830160d640 1 --2- 172.21.15.6:0/877864878 >> [v2:172.21.15.118:6816/269980353,v1:172.21.15.118:6817/269980353] conn(0x7f82e80ac7c0 0x7f82e80aec60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.177+0000 7f830160d640 1 -- 172.21.15.6:0/877864878 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f82e80a2c90 msgr2=0x7f82e80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.177+0000 7f830160d640 1 --2- 172.21.15.6:0/877864878 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f82e80a2c90 0x7f82e80a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0fe130 tx=0x7f82e4005d20 comp rx=0 tx=0).stop 2024-09-12T03:19:58.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.177+0000 7f830160d640 1 -- 172.21.15.6:0/877864878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82fc101de0 msgr2=0x7f82fc198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.177+0000 7f830160d640 1 --2- 172.21.15.6:0/877864878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82fc101de0 0x7f82fc198380 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00e9f0 tx=0x7f82ec00edf0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.177+0000 7f82fb7fe640 1 -- 172.21.15.6:0/877864878 reap_dead start 2024-09-12T03:19:58.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.177+0000 7f830160d640 1 -- 172.21.15.6:0/877864878 shutdown_connections 2024-09-12T03:19:58.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.177+0000 7f830160d640 1 -- 172.21.15.6:0/877864878 >> 172.21.15.6:0/877864878 conn(0x7f82fc0fd8c0 msgr2=0x7f82fc0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:58.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.177+0000 7f830160d640 1 -- 172.21.15.6:0/877864878 shutdown_connections 2024-09-12T03:19:58.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:58.177+0000 7f830160d640 1 -- 172.21.15.6:0/877864878 wait complete. 2024-09-12T03:19:59.541 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:19:59 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/4007262538' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T03:19:59.573 INFO:teuthology.orchestra.run.smithi006.stdout:206158430251 2024-09-12T03:19:59.574 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-12T03:19:59.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:19:59 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/4007262538' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T03:19:59.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:19:59 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/4007262538' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T03:19:59.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.718+0000 7f94a78f1640 1 -- 172.21.15.6:0/3504161635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94a0102570 msgr2=0x7f94a01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.718+0000 7f94a78f1640 1 --2- 172.21.15.6:0/3504161635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94a0102570 0x7f94a01029b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9494009ca0 tx=0x7f949402a950 comp rx=0 tx=0).stop 2024-09-12T03:19:59.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.718+0000 7f94a78f1640 1 -- 172.21.15.6:0/3504161635 shutdown_connections 2024-09-12T03:19:59.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.718+0000 7f94a78f1640 1 --2- 172.21.15.6:0/3504161635 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94a0102ef0 0x7f94a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.718+0000 7f94a78f1640 1 --2- 172.21.15.6:0/3504161635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94a0102570 0x7f94a01029b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.718+0000 7f94a78f1640 1 --2- 172.21.15.6:0/3504161635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94a0100e20 0x7f94a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.718+0000 7f94a78f1640 1 -- 172.21.15.6:0/3504161635 >> 172.21.15.6:0/3504161635 conn(0x7f94a00fc7d0 msgr2=0x7f94a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:59.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.719+0000 7f94a78f1640 1 -- 172.21.15.6:0/3504161635 shutdown_connections 2024-09-12T03:19:59.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.719+0000 7f94a78f1640 1 -- 172.21.15.6:0/3504161635 wait complete. 2024-09-12T03:19:59.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.719+0000 7f94a78f1640 1 Processor -- start 2024-09-12T03:19:59.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.719+0000 7f94a78f1640 1 -- start start 2024-09-12T03:19:59.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.719+0000 7f94a78f1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94a0100e20 0x7f94a0199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:59.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.719+0000 7f94a78f1640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94a0102570 0x7f94a01998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:59.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.720+0000 7f94a78f1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94a0102ef0 0x7f94a01a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:59.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.720+0000 7f94a78f1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f94a0110f80 con 0x7f94a0102ef0 2024-09-12T03:19:59.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.720+0000 7f94a78f1640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f94a0110e00 con 0x7f94a0102570 2024-09-12T03:19:59.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.720+0000 7f94a78f1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f94a0111100 con 0x7f94a0100e20 2024-09-12T03:19:59.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.720+0000 7f94a5e8e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94a0102ef0 0x7f94a01a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:59.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.720+0000 7f94a5e8e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94a0102ef0 0x7f94a01a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42192/0 (socket says 172.21.15.6:42192) 2024-09-12T03:19:59.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.720+0000 7f94a5e8e640 1 -- 172.21.15.6:0/3707499059 learned_addr learned my addr 172.21.15.6:0/3707499059 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:19:59.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.720+0000 7f94a4e8c640 1 --2- 172.21.15.6:0/3707499059 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94a0102570 0x7f94a01998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:59.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.720+0000 7f94a568d640 1 --2- 172.21.15.6:0/3707499059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94a0100e20 0x7f94a0199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:59.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.720+0000 7f94a5e8e640 1 -- 172.21.15.6:0/3707499059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94a0100e20 msgr2=0x7f94a0199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.720+0000 7f94a5e8e640 1 --2- 172.21.15.6:0/3707499059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94a0100e20 0x7f94a0199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.720+0000 7f94a5e8e640 1 -- 172.21.15.6:0/3707499059 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94a0102570 msgr2=0x7f94a01998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.720+0000 7f94a5e8e640 1 --2- 172.21.15.6:0/3707499059 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94a0102570 0x7f94a01998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.720+0000 7f94a5e8e640 1 -- 172.21.15.6:0/3707499059 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a01a4930 con 0x7f94a0102ef0 2024-09-12T03:19:59.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.720+0000 7f94a4e8c640 1 --2- 172.21.15.6:0/3707499059 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94a0102570 0x7f94a01998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:59.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.720+0000 7f94a5e8e640 1 --2- 172.21.15.6:0/3707499059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94a0102ef0 0x7f94a01a43f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f949800e970 tx=0x7f949800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:59.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.720+0000 7f94927fc640 1 -- 172.21.15.6:0/3707499059 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949800ccc0 con 0x7f94a0102ef0 2024-09-12T03:19:59.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.720+0000 7f94a78f1640 1 -- 172.21.15.6:0/3707499059 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a01a4b00 con 0x7f94a0102ef0 2024-09-12T03:19:59.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.720+0000 7f94927fc640 1 -- 172.21.15.6:0/3707499059 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f949800ce60 con 0x7f94a0102ef0 2024-09-12T03:19:59.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.720+0000 7f94927fc640 1 -- 172.21.15.6:0/3707499059 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498010490 con 0x7f94a0102ef0 2024-09-12T03:19:59.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.721+0000 7f94a78f1640 1 -- 172.21.15.6:0/3707499059 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a01a5010 con 0x7f94a0102ef0 2024-09-12T03:19:59.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.722+0000 7f94a78f1640 1 -- 172.21.15.6:0/3707499059 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f94a0062940 con 0x7f94a0102ef0 2024-09-12T03:19:59.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.723+0000 7f94927fc640 1 -- 172.21.15.6:0/3707499059 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9498005000 con 0x7f94a0102ef0 2024-09-12T03:19:59.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.723+0000 7f94927fc640 1 --2- 172.21.15.6:0/3707499059 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f948c0a2c90 0x7f948c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:59.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.724+0000 7f94927fc640 1 -- 172.21.15.6:0/3707499059 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f94980cfb50 con 0x7f94a0102ef0 2024-09-12T03:19:59.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.724+0000 7f94a568d640 1 --2- 172.21.15.6:0/3707499059 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f948c0a2c90 0x7f948c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:59.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.724+0000 7f94927fc640 1 --2- 172.21.15.6:0/3707499059 >> [v2:172.21.15.161:6808/3758167854,v1:172.21.15.161:6809/3758167854] conn(0x7f948c0ac7c0 0x7f948c0aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:59.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.724+0000 7f94927fc640 1 -- 172.21.15.6:0/3707499059 --> [v2:172.21.15.161:6808/3758167854,v1:172.21.15.161:6809/3758167854] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f949801aec0 con 0x7f948c0ac7c0 2024-09-12T03:19:59.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.724+0000 7f94927fc640 1 -- 172.21.15.6:0/3707499059 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f94980cff00 con 0x7f94a0102ef0 2024-09-12T03:19:59.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.724+0000 7f94a4e8c640 1 --2- 172.21.15.6:0/3707499059 >> [v2:172.21.15.161:6808/3758167854,v1:172.21.15.161:6809/3758167854] conn(0x7f948c0ac7c0 0x7f948c0aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:59.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.724+0000 7f94a568d640 1 --2- 172.21.15.6:0/3707499059 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f948c0a2c90 0x7f948c0a5150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f949c00cac0 tx=0x7f949c00c4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:59.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.725+0000 7f94a4e8c640 1 --2- 172.21.15.6:0/3707499059 >> [v2:172.21.15.161:6808/3758167854,v1:172.21.15.161:6809/3758167854] conn(0x7f948c0ac7c0 0x7f948c0aec60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:59.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.725+0000 7f94927fc640 1 -- 172.21.15.6:0/3707499059 <== osd.5 v2:172.21.15.161:6808/3758167854 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f949801aec0 con 0x7f948c0ac7c0 2024-09-12T03:19:59.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.741+0000 7f94a78f1640 1 -- 172.21.15.6:0/3707499059 --> [v2:172.21.15.161:6808/3758167854,v1:172.21.15.161:6809/3758167854] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f94a0062dc0 con 0x7f948c0ac7c0 2024-09-12T03:19:59.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.741+0000 7f94927fc640 1 -- 172.21.15.6:0/3707499059 <== osd.5 v2:172.21.15.161:6808/3758167854 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f94a0062dc0 con 0x7f948c0ac7c0 2024-09-12T03:19:59.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.742+0000 7f94a78f1640 1 -- 172.21.15.6:0/3707499059 >> [v2:172.21.15.161:6808/3758167854,v1:172.21.15.161:6809/3758167854] conn(0x7f948c0ac7c0 msgr2=0x7f948c0aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.742+0000 7f94a78f1640 1 --2- 172.21.15.6:0/3707499059 >> [v2:172.21.15.161:6808/3758167854,v1:172.21.15.161:6809/3758167854] conn(0x7f948c0ac7c0 0x7f948c0aec60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.742+0000 7f94a78f1640 1 -- 172.21.15.6:0/3707499059 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f948c0a2c90 msgr2=0x7f948c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.742+0000 7f94a78f1640 1 --2- 172.21.15.6:0/3707499059 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f948c0a2c90 0x7f948c0a5150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f949c00cac0 tx=0x7f949c00c4a0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.742+0000 7f94a78f1640 1 -- 172.21.15.6:0/3707499059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94a0102ef0 msgr2=0x7f94a01a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.742+0000 7f94a78f1640 1 --2- 172.21.15.6:0/3707499059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94a0102ef0 0x7f94a01a43f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f949800e970 tx=0x7f949800ed70 comp rx=0 tx=0).stop 2024-09-12T03:19:59.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.742+0000 7f94a5e8e640 1 -- 172.21.15.6:0/3707499059 reap_dead start 2024-09-12T03:19:59.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.742+0000 7f94a78f1640 1 -- 172.21.15.6:0/3707499059 shutdown_connections 2024-09-12T03:19:59.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.742+0000 7f94a78f1640 1 -- 172.21.15.6:0/3707499059 >> 172.21.15.6:0/3707499059 conn(0x7f94a00fc7d0 msgr2=0x7f94a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:59.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.742+0000 7f94a78f1640 1 -- 172.21.15.6:0/3707499059 shutdown_connections 2024-09-12T03:19:59.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:59.742+0000 7f94a78f1640 1 -- 172.21.15.6:0/3707499059 wait complete. 2024-09-12T03:20:00.497 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:00 smithi006 ceph-mon[30991]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-12T03:20:00.497 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:00 smithi006 ceph-mon[30991]: overall HEALTH_OK 2024-09-12T03:20:00.524 INFO:teuthology.orchestra.run.smithi006.stdout:154618822708 2024-09-12T03:20:00.524 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-12T03:20:00.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:00 smithi161 ceph-mon[35485]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-12T03:20:00.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:00 smithi161 ceph-mon[35485]: overall HEALTH_OK 2024-09-12T03:20:00.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:00 smithi118 ceph-mon[35048]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-12T03:20:00.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:00 smithi118 ceph-mon[35048]: overall HEALTH_OK 2024-09-12T03:20:00.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.940+0000 7f46316da640 1 -- 172.21.15.6:0/1531900927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f462c106f10 msgr2=0x7f462c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.940+0000 7f46316da640 1 --2- 172.21.15.6:0/1531900927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f462c106f10 0x7f462c10ea90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4624009ca0 tx=0x7f462402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.940+0000 7f46316da640 1 -- 172.21.15.6:0/1531900927 shutdown_connections 2024-09-12T03:20:00.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.940+0000 7f46316da640 1 --2- 172.21.15.6:0/1531900927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f462c106f10 0x7f462c10ea90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.940+0000 7f46316da640 1 --2- 172.21.15.6:0/1531900927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462c102790 0x7f462c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.940+0000 7f46316da640 1 --2- 172.21.15.6:0/1531900927 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f462c101de0 0x7f462c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.940+0000 7f46316da640 1 -- 172.21.15.6:0/1531900927 >> 172.21.15.6:0/1531900927 conn(0x7f462c0fd8c0 msgr2=0x7f462c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:00.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.940+0000 7f46316da640 1 -- 172.21.15.6:0/1531900927 shutdown_connections 2024-09-12T03:20:00.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.940+0000 7f46316da640 1 -- 172.21.15.6:0/1531900927 wait complete. 2024-09-12T03:20:00.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.941+0000 7f46316da640 1 Processor -- start 2024-09-12T03:20:00.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.941+0000 7f46316da640 1 -- start start 2024-09-12T03:20:00.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.941+0000 7f46316da640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f462c101de0 0x7f462c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.941+0000 7f46316da640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f462c102790 0x7f462c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.941+0000 7f46316da640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462c106f10 0x7f462c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.941+0000 7f46316da640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f462c111f70 con 0x7f462c101de0 2024-09-12T03:20:00.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.941+0000 7f46316da640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f462c111df0 con 0x7f462c102790 2024-09-12T03:20:00.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.941+0000 7f46316da640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1120f0 con 0x7f462c106f10 2024-09-12T03:20:00.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.941+0000 7f462affd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f462c101de0 0x7f462c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.941+0000 7f462affd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f462c101de0 0x7f462c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42216/0 (socket says 172.21.15.6:42216) 2024-09-12T03:20:00.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.941+0000 7f462affd640 1 -- 172.21.15.6:0/488483426 learned_addr learned my addr 172.21.15.6:0/488483426 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:00.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.941+0000 7f4623fff640 1 --2- 172.21.15.6:0/488483426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f462c102790 0x7f462c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.941+0000 7f462affd640 1 -- 172.21.15.6:0/488483426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462c106f10 msgr2=0x7f462c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.941+0000 7f462affd640 1 --2- 172.21.15.6:0/488483426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462c106f10 0x7f462c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.941+0000 7f462affd640 1 -- 172.21.15.6:0/488483426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f462c102790 msgr2=0x7f462c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.941+0000 7f462affd640 1 --2- 172.21.15.6:0/488483426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f462c102790 0x7f462c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.941+0000 7f462affd640 1 -- 172.21.15.6:0/488483426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462c1a6800 con 0x7f462c101de0 2024-09-12T03:20:00.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.942+0000 7f462affd640 1 --2- 172.21.15.6:0/488483426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f462c101de0 0x7f462c0744f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f461c00c920 tx=0x7f461c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:00.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.942+0000 7f4628ff9640 1 -- 172.21.15.6:0/488483426 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c007c70 con 0x7f462c101de0 2024-09-12T03:20:00.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.942+0000 7f4628ff9640 1 -- 172.21.15.6:0/488483426 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f461c007e10 con 0x7f462c101de0 2024-09-12T03:20:00.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.942+0000 7f46316da640 1 -- 172.21.15.6:0/488483426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462c1a6af0 con 0x7f462c101de0 2024-09-12T03:20:00.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.942+0000 7f4628ff9640 1 -- 172.21.15.6:0/488483426 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c018890 con 0x7f462c101de0 2024-09-12T03:20:00.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.942+0000 7f46316da640 1 -- 172.21.15.6:0/488483426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f462c1a7080 con 0x7f462c101de0 2024-09-12T03:20:00.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.943+0000 7f46316da640 1 -- 172.21.15.6:0/488483426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f462c1021c0 con 0x7f462c101de0 2024-09-12T03:20:00.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.944+0000 7f4628ff9640 1 -- 172.21.15.6:0/488483426 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f461c018a70 con 0x7f462c101de0 2024-09-12T03:20:00.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.945+0000 7f4628ff9640 1 --2- 172.21.15.6:0/488483426 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f46180a2c90 0x7f46180a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.945+0000 7f4623fff640 1 --2- 172.21.15.6:0/488483426 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f46180a2c90 0x7f46180a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.945+0000 7f4628ff9640 1 -- 172.21.15.6:0/488483426 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f461c0cf360 con 0x7f462c101de0 2024-09-12T03:20:00.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.946+0000 7f4623fff640 1 --2- 172.21.15.6:0/488483426 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f46180a2c90 0x7f46180a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f462c075f80 tx=0x7f4614005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:00.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:00.950+0000 7f4628ff9640 1 -- 172.21.15.6:0/488483426 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f461c014030 con 0x7f462c101de0 2024-09-12T03:20:01.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:01.151+0000 7f46316da640 1 -- 172.21.15.6:0/488483426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f462c075a20 con 0x7f462c101de0 2024-09-12T03:20:01.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:01.152+0000 7f4628ff9640 1 -- 172.21.15.6:0/488483426 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f461c09c130 con 0x7f462c101de0 2024-09-12T03:20:01.152 INFO:teuthology.orchestra.run.smithi006.stdout:236223201318 2024-09-12T03:20:01.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:01.154+0000 7f46316da640 1 -- 172.21.15.6:0/488483426 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f46180a2c90 msgr2=0x7f46180a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:01.154+0000 7f46316da640 1 --2- 172.21.15.6:0/488483426 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f46180a2c90 0x7f46180a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f462c075f80 tx=0x7f4614005c50 comp rx=0 tx=0).stop 2024-09-12T03:20:01.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:01.154+0000 7f46316da640 1 -- 172.21.15.6:0/488483426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f462c101de0 msgr2=0x7f462c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:01.154+0000 7f46316da640 1 --2- 172.21.15.6:0/488483426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f462c101de0 0x7f462c0744f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f461c00c920 tx=0x7f461c00cde0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:01.154+0000 7f46316da640 1 -- 172.21.15.6:0/488483426 shutdown_connections 2024-09-12T03:20:01.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:01.154+0000 7f46316da640 1 --2- 172.21.15.6:0/488483426 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f46180a2c90 0x7f46180a5150 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:01.154+0000 7f46316da640 1 --2- 172.21.15.6:0/488483426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462c106f10 0x7f462c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:01.154+0000 7f46316da640 1 --2- 172.21.15.6:0/488483426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f462c102790 0x7f462c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:01.154+0000 7f46316da640 1 --2- 172.21.15.6:0/488483426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f462c101de0 0x7f462c0744f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:01.154+0000 7f46316da640 1 -- 172.21.15.6:0/488483426 >> 172.21.15.6:0/488483426 conn(0x7f462c0fd8c0 msgr2=0x7f462c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:01.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:01.154+0000 7f46316da640 1 -- 172.21.15.6:0/488483426 shutdown_connections 2024-09-12T03:20:01.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:01.154+0000 7f46316da640 1 -- 172.21.15.6:0/488483426 wait complete. 2024-09-12T03:20:01.369 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:01 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/488483426' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T03:20:01.404 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182459 got 107374182461 for osd.3 2024-09-12T03:20:01.404 DEBUG:teuthology.parallel:result is None 2024-09-12T03:20:01.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:01 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/488483426' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T03:20:01.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:01 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/488483426' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T03:20:02.040 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201315 got 236223201318 for osd.8 2024-09-12T03:20:02.040 DEBUG:teuthology.parallel:result is None 2024-09-12T03:20:02.255 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:02.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:02 smithi006 ceph-mon[30991]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:02.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:02 smithi161 ceph-mon[35485]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:02.613 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:02 smithi118 ceph-mon[35048]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:02.682 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411394 got 77309411396 for osd.2 2024-09-12T03:20:02.682 DEBUG:teuthology.parallel:result is None 2024-09-12T03:20:02.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.511+0000 7f55619ce640 1 -- 172.21.15.6:0/4155319539 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f555c106f10 msgr2=0x7f555c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.511+0000 7f55619ce640 1 --2- 172.21.15.6:0/4155319539 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f555c106f10 0x7f555c10ea90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5554009ca0 tx=0x7f555402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.511+0000 7f55619ce640 1 -- 172.21.15.6:0/4155319539 shutdown_connections 2024-09-12T03:20:02.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.511+0000 7f55619ce640 1 --2- 172.21.15.6:0/4155319539 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f555c106f10 0x7f555c10ea90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.511+0000 7f55619ce640 1 --2- 172.21.15.6:0/4155319539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f555c102790 0x7f555c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.511+0000 7f55619ce640 1 --2- 172.21.15.6:0/4155319539 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f555c101de0 0x7f555c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.511+0000 7f55619ce640 1 -- 172.21.15.6:0/4155319539 >> 172.21.15.6:0/4155319539 conn(0x7f555c0fd8c0 msgr2=0x7f555c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:02.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.511+0000 7f55619ce640 1 -- 172.21.15.6:0/4155319539 shutdown_connections 2024-09-12T03:20:02.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.511+0000 7f55619ce640 1 -- 172.21.15.6:0/4155319539 wait complete. 2024-09-12T03:20:02.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.512+0000 7f55619ce640 1 Processor -- start 2024-09-12T03:20:02.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.512+0000 7f55619ce640 1 -- start start 2024-09-12T03:20:02.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.513+0000 7f55619ce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f555c101de0 0x7f555c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.513+0000 7f55619ce640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f555c102790 0x7f555c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.513+0000 7f55619ce640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f555c106f10 0x7f555c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.513+0000 7f55619ce640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f555c111f70 con 0x7f555c102790 2024-09-12T03:20:02.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.513+0000 7f55619ce640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f555c111df0 con 0x7f555c106f10 2024-09-12T03:20:02.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.513+0000 7f55619ce640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f555c1120f0 con 0x7f555c101de0 2024-09-12T03:20:02.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.513+0000 7f5553fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f555c102790 0x7f555c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.513+0000 7f5553fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f555c102790 0x7f555c1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52930/0 (socket says 172.21.15.6:52930) 2024-09-12T03:20:02.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.513+0000 7f5553fff640 1 -- 172.21.15.6:0/143861336 learned_addr learned my addr 172.21.15.6:0/143861336 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:02.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.513+0000 7f555b7fe640 1 --2- 172.21.15.6:0/143861336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f555c106f10 0x7f555c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.513+0000 7f555affd640 1 --2- 172.21.15.6:0/143861336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f555c101de0 0x7f555c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.513+0000 7f5553fff640 1 -- 172.21.15.6:0/143861336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f555c101de0 msgr2=0x7f555c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.513+0000 7f5553fff640 1 --2- 172.21.15.6:0/143861336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f555c101de0 0x7f555c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.513+0000 7f5553fff640 1 -- 172.21.15.6:0/143861336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f555c106f10 msgr2=0x7f555c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.513+0000 7f5553fff640 1 --2- 172.21.15.6:0/143861336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f555c106f10 0x7f555c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.513+0000 7f5553fff640 1 -- 172.21.15.6:0/143861336 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555c19d340 con 0x7f555c102790 2024-09-12T03:20:02.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.513+0000 7f5553fff640 1 --2- 172.21.15.6:0/143861336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f555c102790 0x7f555c1988d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f554400c890 tx=0x7f554400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:02.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.514+0000 7f55609cc640 1 -- 172.21.15.6:0/143861336 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5544004220 con 0x7f555c102790 2024-09-12T03:20:02.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.514+0000 7f55609cc640 1 -- 172.21.15.6:0/143861336 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55440043c0 con 0x7f555c102790 2024-09-12T03:20:02.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.514+0000 7f55619ce640 1 -- 172.21.15.6:0/143861336 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f555c19d630 con 0x7f555c102790 2024-09-12T03:20:02.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.514+0000 7f55609cc640 1 -- 172.21.15.6:0/143861336 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55440049e0 con 0x7f555c102790 2024-09-12T03:20:02.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.514+0000 7f55619ce640 1 -- 172.21.15.6:0/143861336 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f555c1a4f10 con 0x7f555c102790 2024-09-12T03:20:02.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.515+0000 7f55619ce640 1 -- 172.21.15.6:0/143861336 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f555c1021c0 con 0x7f555c102790 2024-09-12T03:20:02.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.516+0000 7f55609cc640 1 -- 172.21.15.6:0/143861336 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f55440079b0 con 0x7f555c102790 2024-09-12T03:20:02.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.517+0000 7f55609cc640 1 --2- 172.21.15.6:0/143861336 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f55480a2c90 0x7f55480a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.771 INFO:teuthology.orchestra.run.smithi006.stdout:55834574921 2024-09-12T03:20:02.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.517+0000 7f555affd640 1 --2- 172.21.15.6:0/143861336 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f55480a2c90 0x7f55480a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.517+0000 7f55609cc640 1 -- 172.21.15.6:0/143861336 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f55440cf140 con 0x7f555c102790 2024-09-12T03:20:02.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.518+0000 7f555affd640 1 --2- 172.21.15.6:0/143861336 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f55480a2c90 0x7f55480a5150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f554c005fd0 tx=0x7f554c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:02.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.520+0000 7f55609cc640 1 -- 172.21.15.6:0/143861336 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5544014030 con 0x7f555c102790 2024-09-12T03:20:02.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.702+0000 7f55619ce640 1 -- 172.21.15.6:0/143861336 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f555c1996f0 con 0x7f555c102790 2024-09-12T03:20:02.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.702+0000 7f55609cc640 1 -- 172.21.15.6:0/143861336 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f555c1996f0 con 0x7f555c102790 2024-09-12T03:20:02.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.704+0000 7f55619ce640 1 -- 172.21.15.6:0/143861336 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f55480a2c90 msgr2=0x7f55480a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.704+0000 7f55619ce640 1 --2- 172.21.15.6:0/143861336 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f55480a2c90 0x7f55480a5150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f554c005fd0 tx=0x7f554c005850 comp rx=0 tx=0).stop 2024-09-12T03:20:02.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.704+0000 7f55619ce640 1 -- 172.21.15.6:0/143861336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f555c102790 msgr2=0x7f555c1988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.704+0000 7f55619ce640 1 --2- 172.21.15.6:0/143861336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f555c102790 0x7f555c1988d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f554400c890 tx=0x7f554400cd50 comp rx=0 tx=0).stop 2024-09-12T03:20:02.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.704+0000 7f55619ce640 1 -- 172.21.15.6:0/143861336 shutdown_connections 2024-09-12T03:20:02.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.704+0000 7f55619ce640 1 --2- 172.21.15.6:0/143861336 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f55480a2c90 0x7f55480a5150 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.704+0000 7f55619ce640 1 --2- 172.21.15.6:0/143861336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f555c106f10 0x7f555c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.704+0000 7f55619ce640 1 --2- 172.21.15.6:0/143861336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f555c102790 0x7f555c1988d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.704+0000 7f55619ce640 1 --2- 172.21.15.6:0/143861336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f555c101de0 0x7f555c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.704+0000 7f55619ce640 1 -- 172.21.15.6:0/143861336 >> 172.21.15.6:0/143861336 conn(0x7f555c0fd8c0 msgr2=0x7f555c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:02.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.704+0000 7f55619ce640 1 -- 172.21.15.6:0/143861336 shutdown_connections 2024-09-12T03:20:02.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:19:57.704+0000 7f55619ce640 1 -- 172.21.15.6:0/143861336 wait complete. 2024-09-12T03:20:03.396 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574919 got 55834574921 for osd.1 2024-09-12T03:20:03.397 DEBUG:teuthology.parallel:result is None 2024-09-12T03:20:03.409 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:03.409 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:03.857 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579925 got 317827579926 for osd.11 2024-09-12T03:20:03.858 DEBUG:teuthology.parallel:result is None 2024-09-12T03:20:04.092 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:04.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.552+0000 7f4caf612640 1 -- 172.21.15.6:0/2169998610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ca8101280 msgr2=0x7f4ca810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:04.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.552+0000 7f4caf612640 1 --2- 172.21.15.6:0/2169998610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ca8101280 0x7f4ca810df40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c009900 tx=0x7f4c9c02a410 comp rx=0 tx=0).stop 2024-09-12T03:20:04.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.552+0000 7f4caf612640 1 -- 172.21.15.6:0/2169998610 shutdown_connections 2024-09-12T03:20:04.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.552+0000 7f4caf612640 1 --2- 172.21.15.6:0/2169998610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ca810e480 0x7f4ca8110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.552+0000 7f4caf612640 1 --2- 172.21.15.6:0/2169998610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ca8101280 0x7f4ca810df40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.552+0000 7f4caf612640 1 --2- 172.21.15.6:0/2169998610 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ca81008d0 0x7f4ca8100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.552+0000 7f4caf612640 1 -- 172.21.15.6:0/2169998610 >> 172.21.15.6:0/2169998610 conn(0x7f4ca80fc7a0 msgr2=0x7f4ca80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:04.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.552+0000 7f4caf612640 1 -- 172.21.15.6:0/2169998610 shutdown_connections 2024-09-12T03:20:04.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.552+0000 7f4caf612640 1 -- 172.21.15.6:0/2169998610 wait complete. 2024-09-12T03:20:04.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.552+0000 7f4caf612640 1 Processor -- start 2024-09-12T03:20:04.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.552+0000 7f4caf612640 1 -- start start 2024-09-12T03:20:04.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.553+0000 7f4caf612640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ca81008d0 0x7f4ca819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.553+0000 7f4caf612640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ca8101280 0x7f4ca819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.553+0000 7f4caf612640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ca810e480 0x7f4ca819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.553+0000 7f4caf612640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81141a0 con 0x7f4ca810e480 2024-09-12T03:20:04.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.553+0000 7f4caf612640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca8114020 con 0x7f4ca81008d0 2024-09-12T03:20:04.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.553+0000 7f4caf612640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca8114320 con 0x7f4ca8101280 2024-09-12T03:20:04.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.553+0000 7f4cadbaf640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ca810e480 0x7f4ca819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.553+0000 7f4cadbaf640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ca810e480 0x7f4ca819ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42230/0 (socket says 172.21.15.6:42230) 2024-09-12T03:20:04.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.553+0000 7f4cadbaf640 1 -- 172.21.15.6:0/1251096055 learned_addr learned my addr 172.21.15.6:0/1251096055 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:04.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.553+0000 7f4cad3ae640 1 --2- 172.21.15.6:0/1251096055 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ca81008d0 0x7f4ca819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.553+0000 7f4cacbad640 1 --2- 172.21.15.6:0/1251096055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ca8101280 0x7f4ca819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.553+0000 7f4cadbaf640 1 -- 172.21.15.6:0/1251096055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ca8101280 msgr2=0x7f4ca819ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:04.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.553+0000 7f4cadbaf640 1 --2- 172.21.15.6:0/1251096055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ca8101280 0x7f4ca819ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.553+0000 7f4cadbaf640 1 -- 172.21.15.6:0/1251096055 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ca81008d0 msgr2=0x7f4ca819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:04.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.553+0000 7f4cadbaf640 1 --2- 172.21.15.6:0/1251096055 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ca81008d0 0x7f4ca819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.553+0000 7f4cadbaf640 1 -- 172.21.15.6:0/1251096055 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca819f670 con 0x7f4ca810e480 2024-09-12T03:20:04.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.553+0000 7f4cadbaf640 1 --2- 172.21.15.6:0/1251096055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ca810e480 0x7f4ca819ef90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4ca000d6a0 tx=0x7f4ca000db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:04.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.553+0000 7f4ca67fc640 1 -- 172.21.15.6:0/1251096055 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0017070 con 0x7f4ca810e480 2024-09-12T03:20:04.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.553+0000 7f4ca67fc640 1 -- 172.21.15.6:0/1251096055 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ca0009820 con 0x7f4ca810e480 2024-09-12T03:20:04.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.553+0000 7f4caf612640 1 -- 172.21.15.6:0/1251096055 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca819f960 con 0x7f4ca810e480 2024-09-12T03:20:04.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.553+0000 7f4ca67fc640 1 -- 172.21.15.6:0/1251096055 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0004ce0 con 0x7f4ca810e480 2024-09-12T03:20:04.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.554+0000 7f4caf612640 1 -- 172.21.15.6:0/1251096055 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca81a7240 con 0x7f4ca810e480 2024-09-12T03:20:04.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.554+0000 7f4caf612640 1 -- 172.21.15.6:0/1251096055 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ca8100cb0 con 0x7f4ca810e480 2024-09-12T03:20:04.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.556+0000 7f4ca67fc640 1 -- 172.21.15.6:0/1251096055 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4ca00099d0 con 0x7f4ca810e480 2024-09-12T03:20:04.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.556+0000 7f4ca67fc640 1 --2- 172.21.15.6:0/1251096055 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4c900a2c90 0x7f4c900a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.557+0000 7f4cad3ae640 1 --2- 172.21.15.6:0/1251096055 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4c900a2c90 0x7f4c900a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.557+0000 7f4ca67fc640 1 -- 172.21.15.6:0/1251096055 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f4ca00ced40 con 0x7f4ca810e480 2024-09-12T03:20:04.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.557+0000 7f4cad3ae640 1 --2- 172.21.15.6:0/1251096055 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4c900a2c90 0x7f4c900a5150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4c98005c40 tx=0x7f4c9800a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:04.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.560+0000 7f4ca67fc640 1 -- 172.21.15.6:0/1251096055 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4ca009b5a0 con 0x7f4ca810e480 2024-09-12T03:20:04.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:04 smithi006 ceph-mon[30991]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:20:04.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:04 smithi161 ceph-mon[35485]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:20:04.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:04 smithi118 ceph-mon[35048]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:20:04.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.743+0000 7f4caf612640 1 -- 172.21.15.6:0/1251096055 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f4ca8109080 con 0x7f4ca810e480 2024-09-12T03:20:04.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.743+0000 7f4ca67fc640 1 -- 172.21.15.6:0/1251096055 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4ca009f8a0 con 0x7f4ca810e480 2024-09-12T03:20:04.743 INFO:teuthology.orchestra.run.smithi006.stdout:137438953531 2024-09-12T03:20:04.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.746+0000 7f4caf612640 1 -- 172.21.15.6:0/1251096055 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4c900a2c90 msgr2=0x7f4c900a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:04.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.746+0000 7f4caf612640 1 --2- 172.21.15.6:0/1251096055 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4c900a2c90 0x7f4c900a5150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4c98005c40 tx=0x7f4c9800a590 comp rx=0 tx=0).stop 2024-09-12T03:20:04.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.746+0000 7f4caf612640 1 -- 172.21.15.6:0/1251096055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ca810e480 msgr2=0x7f4ca819ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:04.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.746+0000 7f4caf612640 1 --2- 172.21.15.6:0/1251096055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ca810e480 0x7f4ca819ef90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4ca000d6a0 tx=0x7f4ca000db60 comp rx=0 tx=0).stop 2024-09-12T03:20:04.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.746+0000 7f4caf612640 1 -- 172.21.15.6:0/1251096055 shutdown_connections 2024-09-12T03:20:04.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.746+0000 7f4caf612640 1 --2- 172.21.15.6:0/1251096055 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4c900a2c90 0x7f4c900a5150 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.746+0000 7f4caf612640 1 --2- 172.21.15.6:0/1251096055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ca810e480 0x7f4ca819ef90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.746+0000 7f4caf612640 1 --2- 172.21.15.6:0/1251096055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ca8101280 0x7f4ca819ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.746+0000 7f4caf612640 1 --2- 172.21.15.6:0/1251096055 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ca81008d0 0x7f4ca819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.746+0000 7f4caf612640 1 -- 172.21.15.6:0/1251096055 >> 172.21.15.6:0/1251096055 conn(0x7f4ca80fc7a0 msgr2=0x7f4ca80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:04.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.746+0000 7f4caf612640 1 -- 172.21.15.6:0/1251096055 shutdown_connections 2024-09-12T03:20:04.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:04.746+0000 7f4caf612640 1 -- 172.21.15.6:0/1251096055 wait complete. 2024-09-12T03:20:05.552 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953528 got 137438953531 for osd.4 2024-09-12T03:20:05.552 DEBUG:teuthology.parallel:result is None 2024-09-12T03:20:05.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:05 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1251096055' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T03:20:05.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:05 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1251096055' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T03:20:05.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:05 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1251096055' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T03:20:05.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.862+0000 7f28dc104640 1 -- 172.21.15.6:0/4109000271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28d4101de0 msgr2=0x7f28d41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:05.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.862+0000 7f28dc104640 1 --2- 172.21.15.6:0/4109000271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28d4101de0 0x7f28d41021c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009900 tx=0x7f28d002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.862+0000 7f28dc104640 1 -- 172.21.15.6:0/4109000271 shutdown_connections 2024-09-12T03:20:05.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.862+0000 7f28dc104640 1 --2- 172.21.15.6:0/4109000271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d4106f10 0x7f28d410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.862+0000 7f28dc104640 1 --2- 172.21.15.6:0/4109000271 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d4102790 0x7f28d41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.862+0000 7f28dc104640 1 --2- 172.21.15.6:0/4109000271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28d4101de0 0x7f28d41021c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.862+0000 7f28dc104640 1 -- 172.21.15.6:0/4109000271 >> 172.21.15.6:0/4109000271 conn(0x7f28d40fd8c0 msgr2=0x7f28d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:05.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.862+0000 7f28dc104640 1 -- 172.21.15.6:0/4109000271 shutdown_connections 2024-09-12T03:20:05.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.862+0000 7f28dc104640 1 -- 172.21.15.6:0/4109000271 wait complete. 2024-09-12T03:20:05.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.863+0000 7f28dc104640 1 Processor -- start 2024-09-12T03:20:05.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.863+0000 7f28dc104640 1 -- start start 2024-09-12T03:20:05.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.863+0000 7f28dc104640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28d4101de0 0x7f28d419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.863+0000 7f28dc104640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d4102790 0x7f28d419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.863+0000 7f28dc104640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d4106f10 0x7f28d419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.863+0000 7f28dc104640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f28d4111f30 con 0x7f28d4101de0 2024-09-12T03:20:05.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.863+0000 7f28dc104640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f28d4111db0 con 0x7f28d4106f10 2024-09-12T03:20:05.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.863+0000 7f28dc104640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f28d41120b0 con 0x7f28d4102790 2024-09-12T03:20:05.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.863+0000 7f28d9ea0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28d4101de0 0x7f28d419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.863+0000 7f28d9ea0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28d4101de0 0x7f28d419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42250/0 (socket says 172.21.15.6:42250) 2024-09-12T03:20:05.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.863+0000 7f28d9ea0640 1 -- 172.21.15.6:0/337634263 learned_addr learned my addr 172.21.15.6:0/337634263 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:05.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.863+0000 7f28da6a1640 1 --2- 172.21.15.6:0/337634263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d4106f10 0x7f28d419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.863+0000 7f28d969f640 1 --2- 172.21.15.6:0/337634263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d4102790 0x7f28d419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.863+0000 7f28d9ea0640 1 -- 172.21.15.6:0/337634263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d4102790 msgr2=0x7f28d419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:05.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.863+0000 7f28d9ea0640 1 --2- 172.21.15.6:0/337634263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d4102790 0x7f28d419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.864+0000 7f28d9ea0640 1 -- 172.21.15.6:0/337634263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d4106f10 msgr2=0x7f28d419ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:05.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.864+0000 7f28d9ea0640 1 --2- 172.21.15.6:0/337634263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d4106f10 0x7f28d419ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.864+0000 7f28d9ea0640 1 -- 172.21.15.6:0/337634263 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d419f560 con 0x7f28d4101de0 2024-09-12T03:20:05.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.864+0000 7f28d969f640 1 --2- 172.21.15.6:0/337634263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d4102790 0x7f28d419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:05.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.864+0000 7f28d9ea0640 1 --2- 172.21.15.6:0/337634263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28d4101de0 0x7f28d419a5b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f28d0005ab0 tx=0x7f28d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:05.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.864+0000 7f28caffd640 1 -- 172.21.15.6:0/337634263 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d003f030 con 0x7f28d4101de0 2024-09-12T03:20:05.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.864+0000 7f28caffd640 1 -- 172.21.15.6:0/337634263 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28d0003ce0 con 0x7f28d4101de0 2024-09-12T03:20:05.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.864+0000 7f28dc104640 1 -- 172.21.15.6:0/337634263 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d419f7f0 con 0x7f28d4101de0 2024-09-12T03:20:05.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.864+0000 7f28caffd640 1 -- 172.21.15.6:0/337634263 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0035070 con 0x7f28d4101de0 2024-09-12T03:20:05.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.864+0000 7f28dc104640 1 -- 172.21.15.6:0/337634263 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d41a7130 con 0x7f28d4101de0 2024-09-12T03:20:05.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.865+0000 7f28dc104640 1 -- 172.21.15.6:0/337634263 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d41032c0 con 0x7f28d4101de0 2024-09-12T03:20:05.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.866+0000 7f28caffd640 1 -- 172.21.15.6:0/337634263 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f28d0042030 con 0x7f28d4101de0 2024-09-12T03:20:05.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.867+0000 7f28caffd640 1 --2- 172.21.15.6:0/337634263 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f28c00a2c10 0x7f28c00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.867+0000 7f28d969f640 1 --2- 172.21.15.6:0/337634263 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f28c00a2c10 0x7f28c00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.867+0000 7f28caffd640 1 -- 172.21.15.6:0/337634263 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f28d00efba0 con 0x7f28d4101de0 2024-09-12T03:20:05.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.867+0000 7f28d969f640 1 --2- 172.21.15.6:0/337634263 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f28c00a2c10 0x7f28c00a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f28d4076510 tx=0x7f28c4005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:05.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:05.871+0000 7f28caffd640 1 -- 172.21.15.6:0/337634263 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f28d00bc970 con 0x7f28d4101de0 2024-09-12T03:20:06.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.047+0000 7f28dc104640 1 -- 172.21.15.6:0/337634263 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f28d41a7300 con 0x7f28d4101de0 2024-09-12T03:20:06.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.048+0000 7f28caffd640 1 -- 172.21.15.6:0/337634263 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f28d00bc270 con 0x7f28d4101de0 2024-09-12T03:20:06.048 INFO:teuthology.orchestra.run.smithi006.stdout:266287972385 2024-09-12T03:20:06.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.050+0000 7f28dc104640 1 -- 172.21.15.6:0/337634263 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f28c00a2c10 msgr2=0x7f28c00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.050+0000 7f28dc104640 1 --2- 172.21.15.6:0/337634263 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f28c00a2c10 0x7f28c00a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f28d4076510 tx=0x7f28c4005d90 comp rx=0 tx=0).stop 2024-09-12T03:20:06.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.050+0000 7f28dc104640 1 -- 172.21.15.6:0/337634263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28d4101de0 msgr2=0x7f28d419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.050+0000 7f28dc104640 1 --2- 172.21.15.6:0/337634263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28d4101de0 0x7f28d419a5b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f28d0005ab0 tx=0x7f28d00042f0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.050+0000 7f28dc104640 1 -- 172.21.15.6:0/337634263 shutdown_connections 2024-09-12T03:20:06.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.050+0000 7f28dc104640 1 --2- 172.21.15.6:0/337634263 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f28c00a2c10 0x7f28c00a50d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.050+0000 7f28dc104640 1 --2- 172.21.15.6:0/337634263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d4106f10 0x7f28d419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.050+0000 7f28dc104640 1 --2- 172.21.15.6:0/337634263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d4102790 0x7f28d419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.050+0000 7f28dc104640 1 --2- 172.21.15.6:0/337634263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28d4101de0 0x7f28d419a5b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.050+0000 7f28dc104640 1 -- 172.21.15.6:0/337634263 >> 172.21.15.6:0/337634263 conn(0x7f28d40fd8c0 msgr2=0x7f28d40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:06.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.050+0000 7f28dc104640 1 -- 172.21.15.6:0/337634263 shutdown_connections 2024-09-12T03:20:06.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.050+0000 7f28dc104640 1 -- 172.21.15.6:0/337634263 wait complete. 2024-09-12T03:20:06.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.067+0000 7f969864d640 1 -- 172.21.15.6:0/1898074903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96901027a0 msgr2=0x7f96901067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.067+0000 7f969864d640 1 --2- 172.21.15.6:0/1898074903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96901027a0 0x7f96901067f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9680009900 tx=0x7f968002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.067+0000 7f969864d640 1 -- 172.21.15.6:0/1898074903 shutdown_connections 2024-09-12T03:20:06.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.067+0000 7f969864d640 1 --2- 172.21.15.6:0/1898074903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9690106f20 0x7f969010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.067+0000 7f969864d640 1 --2- 172.21.15.6:0/1898074903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96901027a0 0x7f96901067f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.067+0000 7f969864d640 1 --2- 172.21.15.6:0/1898074903 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9690101df0 0x7f96901021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.067+0000 7f969864d640 1 -- 172.21.15.6:0/1898074903 >> 172.21.15.6:0/1898074903 conn(0x7f96900fd8f0 msgr2=0x7f96900ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:06.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.068+0000 7f969864d640 1 -- 172.21.15.6:0/1898074903 shutdown_connections 2024-09-12T03:20:06.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.068+0000 7f969864d640 1 -- 172.21.15.6:0/1898074903 wait complete. 2024-09-12T03:20:06.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.068+0000 7f969864d640 1 Processor -- start 2024-09-12T03:20:06.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.069+0000 7f969864d640 1 -- start start 2024-09-12T03:20:06.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.069+0000 7f969864d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9690101df0 0x7f96900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:06.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.069+0000 7f969864d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96901027a0 0x7f9690074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:06.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.069+0000 7f969864d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9690106f20 0x7f9690074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:06.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.069+0000 7f969864d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9690111f40 con 0x7f9690101df0 2024-09-12T03:20:06.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.069+0000 7f969864d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f9690111dc0 con 0x7f96901027a0 2024-09-12T03:20:06.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.069+0000 7f969864d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f96901120c0 con 0x7f9690106f20 2024-09-12T03:20:06.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.069+0000 7f96963e9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9690101df0 0x7f96900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:06.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.069+0000 7f96963e9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9690101df0 0x7f96900744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42276/0 (socket says 172.21.15.6:42276) 2024-09-12T03:20:06.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.069+0000 7f96963e9640 1 -- 172.21.15.6:0/2310080990 learned_addr learned my addr 172.21.15.6:0/2310080990 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:06.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.069+0000 7f9695be8640 1 --2- 172.21.15.6:0/2310080990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96901027a0 0x7f9690074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:06.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.069+0000 7f9696bea640 1 --2- 172.21.15.6:0/2310080990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9690106f20 0x7f9690074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:06.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.069+0000 7f96963e9640 1 -- 172.21.15.6:0/2310080990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9690106f20 msgr2=0x7f9690074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.069+0000 7f96963e9640 1 --2- 172.21.15.6:0/2310080990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9690106f20 0x7f9690074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.069+0000 7f96963e9640 1 -- 172.21.15.6:0/2310080990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96901027a0 msgr2=0x7f9690074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.069+0000 7f96963e9640 1 --2- 172.21.15.6:0/2310080990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96901027a0 0x7f9690074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.069+0000 7f96963e9640 1 -- 172.21.15.6:0/2310080990 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96901a6940 con 0x7f9690101df0 2024-09-12T03:20:06.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.069+0000 7f9695be8640 1 --2- 172.21.15.6:0/2310080990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96901027a0 0x7f9690074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:06.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.069+0000 7f9696bea640 1 --2- 172.21.15.6:0/2310080990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9690106f20 0x7f9690074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:06.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.070+0000 7f96963e9640 1 --2- 172.21.15.6:0/2310080990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9690101df0 0x7f96900744f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f968c00d950 tx=0x7f968c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:06.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.070+0000 7f96877fe640 1 -- 172.21.15.6:0/2310080990 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c017030 con 0x7f9690101df0 2024-09-12T03:20:06.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.070+0000 7f96877fe640 1 -- 172.21.15.6:0/2310080990 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f968c004020 con 0x7f9690101df0 2024-09-12T03:20:06.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.070+0000 7f969864d640 1 -- 172.21.15.6:0/2310080990 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96901a6c30 con 0x7f9690101df0 2024-09-12T03:20:06.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.070+0000 7f96877fe640 1 -- 172.21.15.6:0/2310080990 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c0047a0 con 0x7f9690101df0 2024-09-12T03:20:06.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.070+0000 7f969864d640 1 -- 172.21.15.6:0/2310080990 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96901a7170 con 0x7f9690101df0 2024-09-12T03:20:06.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.071+0000 7f969864d640 1 -- 172.21.15.6:0/2310080990 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96901021d0 con 0x7f9690101df0 2024-09-12T03:20:06.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.072+0000 7f96877fe640 1 -- 172.21.15.6:0/2310080990 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f968c0041d0 con 0x7f9690101df0 2024-09-12T03:20:06.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.073+0000 7f96877fe640 1 --2- 172.21.15.6:0/2310080990 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f967c0a2c10 0x7f967c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:06.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.074+0000 7f9695be8640 1 --2- 172.21.15.6:0/2310080990 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f967c0a2c10 0x7f967c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:06.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.074+0000 7f96877fe640 1 -- 172.21.15.6:0/2310080990 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f968c0cec40 con 0x7f9690101df0 2024-09-12T03:20:06.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.074+0000 7f9695be8640 1 --2- 172.21.15.6:0/2310080990 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f967c0a2c10 0x7f967c0a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9690075f80 tx=0x7f9680005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:06.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.081+0000 7f96877fe640 1 -- 172.21.15.6:0/2310080990 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f968c0d5080 con 0x7f9690101df0 2024-09-12T03:20:06.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.258+0000 7f969864d640 1 -- 172.21.15.6:0/2310080990 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f969002d7c0 con 0x7f9690101df0 2024-09-12T03:20:06.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.258+0000 7f96877fe640 1 -- 172.21.15.6:0/2310080990 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f968c09ba70 con 0x7f9690101df0 2024-09-12T03:20:06.258 INFO:teuthology.orchestra.run.smithi006.stdout:287762808861 2024-09-12T03:20:06.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.261+0000 7f969864d640 1 -- 172.21.15.6:0/2310080990 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f967c0a2c10 msgr2=0x7f967c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.261+0000 7f969864d640 1 --2- 172.21.15.6:0/2310080990 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f967c0a2c10 0x7f967c0a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9690075f80 tx=0x7f9680005a20 comp rx=0 tx=0).stop 2024-09-12T03:20:06.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.261+0000 7f969864d640 1 -- 172.21.15.6:0/2310080990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9690101df0 msgr2=0x7f96900744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.261+0000 7f969864d640 1 --2- 172.21.15.6:0/2310080990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9690101df0 0x7f96900744f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f968c00d950 tx=0x7f968c00de10 comp rx=0 tx=0).stop 2024-09-12T03:20:06.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.261+0000 7f969864d640 1 -- 172.21.15.6:0/2310080990 shutdown_connections 2024-09-12T03:20:06.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.261+0000 7f969864d640 1 --2- 172.21.15.6:0/2310080990 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f967c0a2c10 0x7f967c0a50d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.261+0000 7f969864d640 1 --2- 172.21.15.6:0/2310080990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9690106f20 0x7f9690074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.261+0000 7f969864d640 1 --2- 172.21.15.6:0/2310080990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96901027a0 0x7f9690074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.261+0000 7f969864d640 1 --2- 172.21.15.6:0/2310080990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9690101df0 0x7f96900744f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.261+0000 7f969864d640 1 -- 172.21.15.6:0/2310080990 >> 172.21.15.6:0/2310080990 conn(0x7f96900fd8f0 msgr2=0x7f96900fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:06.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.261+0000 7f969864d640 1 -- 172.21.15.6:0/2310080990 shutdown_connections 2024-09-12T03:20:06.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:06.261+0000 7f969864d640 1 -- 172.21.15.6:0/2310080990 wait complete. 2024-09-12T03:20:06.604 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:06 smithi006 ceph-mon[30991]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:20:06.604 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:06 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/337634263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T03:20:06.605 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:06 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2310080990' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T03:20:06.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:06 smithi161 ceph-mon[35485]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:20:06.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:06 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/337634263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T03:20:06.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:06 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2310080990' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T03:20:06.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:06 smithi118 ceph-mon[35048]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:20:06.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:06 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/337634263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T03:20:06.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:06 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2310080990' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T03:20:06.925 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808857 got 287762808861 for osd.10 2024-09-12T03:20:06.925 DEBUG:teuthology.parallel:result is None 2024-09-12T03:20:07.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.068+0000 7f0f05e7a640 1 -- 172.21.15.6:0/3058152362 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f001082a0 msgr2=0x7f0f0010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.068+0000 7f0f05e7a640 1 --2- 172.21.15.6:0/3058152362 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f001082a0 0x7f0f0010ec40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009cd0 tx=0x7f0ef802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.068+0000 7f0f05e7a640 1 -- 172.21.15.6:0/3058152362 shutdown_connections 2024-09-12T03:20:07.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.068+0000 7f0f05e7a640 1 --2- 172.21.15.6:0/3058152362 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f001082a0 0x7f0f0010ec40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.068+0000 7f0f05e7a640 1 --2- 172.21.15.6:0/3058152362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f001036e0 0x7f0f00107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.068+0000 7f0f05e7a640 1 --2- 172.21.15.6:0/3058152362 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0f00102d30 0x7f0f00103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.068+0000 7f0f05e7a640 1 -- 172.21.15.6:0/3058152362 >> 172.21.15.6:0/3058152362 conn(0x7f0f000fc7d0 msgr2=0x7f0f000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:07.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.068+0000 7f0f05e7a640 1 -- 172.21.15.6:0/3058152362 shutdown_connections 2024-09-12T03:20:07.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.068+0000 7f0f05e7a640 1 -- 172.21.15.6:0/3058152362 wait complete. 2024-09-12T03:20:07.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.069+0000 7f0f05e7a640 1 Processor -- start 2024-09-12T03:20:07.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.069+0000 7f0f05e7a640 1 -- start start 2024-09-12T03:20:07.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.069+0000 7f0f05e7a640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0f00102d30 0x7f0f001985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.069+0000 7f0f05e7a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f001036e0 0x7f0f00198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.069+0000 7f0f05e7a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f001082a0 0x7f0f0019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.069+0000 7f0f05e7a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0f00112170 con 0x7f0f001036e0 2024-09-12T03:20:07.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.069+0000 7f0f05e7a640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f0f00111ff0 con 0x7f0f00102d30 2024-09-12T03:20:07.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.069+0000 7f0f05e7a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0f001122f0 con 0x7f0f001082a0 2024-09-12T03:20:07.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.069+0000 7f0efeffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f001036e0 0x7f0f00198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.069+0000 7f0efeffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f001036e0 0x7f0f00198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42288/0 (socket says 172.21.15.6:42288) 2024-09-12T03:20:07.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.069+0000 7f0efeffd640 1 -- 172.21.15.6:0/1023130053 learned_addr learned my addr 172.21.15.6:0/1023130053 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:07.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.070+0000 7f0eff7fe640 1 --2- 172.21.15.6:0/1023130053 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0f00102d30 0x7f0f001985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.070+0000 7f0efffff640 1 --2- 172.21.15.6:0/1023130053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f001082a0 0x7f0f0019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.070+0000 7f0efeffd640 1 -- 172.21.15.6:0/1023130053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f001082a0 msgr2=0x7f0f0019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.070+0000 7f0efeffd640 1 --2- 172.21.15.6:0/1023130053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f001082a0 0x7f0f0019cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.070+0000 7f0efeffd640 1 -- 172.21.15.6:0/1023130053 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0f00102d30 msgr2=0x7f0f001985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.070+0000 7f0efeffd640 1 --2- 172.21.15.6:0/1023130053 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0f00102d30 0x7f0f001985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.070+0000 7f0efeffd640 1 -- 172.21.15.6:0/1023130053 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0019d5a0 con 0x7f0f001036e0 2024-09-12T03:20:07.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.070+0000 7f0efeffd640 1 --2- 172.21.15.6:0/1023130053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f001036e0 0x7f0f00198b30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400d750 tx=0x7f0ef400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:07.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.070+0000 7f0efcff9640 1 -- 172.21.15.6:0/1023130053 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4017070 con 0x7f0f001036e0 2024-09-12T03:20:07.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.070+0000 7f0efcff9640 1 -- 172.21.15.6:0/1023130053 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ef4005500 con 0x7f0f001036e0 2024-09-12T03:20:07.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.070+0000 7f0f05e7a640 1 -- 172.21.15.6:0/1023130053 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f0019d890 con 0x7f0f001036e0 2024-09-12T03:20:07.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.070+0000 7f0efcff9640 1 -- 172.21.15.6:0/1023130053 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4004e00 con 0x7f0f001036e0 2024-09-12T03:20:07.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.070+0000 7f0f05e7a640 1 -- 172.21.15.6:0/1023130053 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f001a5170 con 0x7f0f001036e0 2024-09-12T03:20:07.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.072+0000 7f0f05e7a640 1 -- 172.21.15.6:0/1023130053 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f00103110 con 0x7f0f001036e0 2024-09-12T03:20:07.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.073+0000 7f0efcff9640 1 -- 172.21.15.6:0/1023130053 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0ef4004fa0 con 0x7f0f001036e0 2024-09-12T03:20:07.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.073+0000 7f0efcff9640 1 --2- 172.21.15.6:0/1023130053 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0ee80a2c10 0x7f0ee80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.074+0000 7f0eff7fe640 1 --2- 172.21.15.6:0/1023130053 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0ee80a2c10 0x7f0ee80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.074+0000 7f0efcff9640 1 -- 172.21.15.6:0/1023130053 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f0ef40cec10 con 0x7f0f001036e0 2024-09-12T03:20:07.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.074+0000 7f0eff7fe640 1 --2- 172.21.15.6:0/1023130053 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0ee80a2c10 0x7f0ee80a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009890 tx=0x7f0ef0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:07.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.079+0000 7f0efcff9640 1 -- 172.21.15.6:0/1023130053 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0ef409ba40 con 0x7f0f001036e0 2024-09-12T03:20:07.089 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:07.090 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:07.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.268+0000 7f0f05e7a640 1 -- 172.21.15.6:0/1023130053 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f0f0010a2b0 con 0x7f0f001036e0 2024-09-12T03:20:07.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.268+0000 7f0efcff9640 1 -- 172.21.15.6:0/1023130053 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0ef409ba40 con 0x7f0f001036e0 2024-09-12T03:20:07.269 INFO:teuthology.orchestra.run.smithi006.stdout:206158430252 2024-09-12T03:20:07.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.271+0000 7f0f05e7a640 1 -- 172.21.15.6:0/1023130053 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0ee80a2c10 msgr2=0x7f0ee80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.271+0000 7f0f05e7a640 1 --2- 172.21.15.6:0/1023130053 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0ee80a2c10 0x7f0ee80a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009890 tx=0x7f0ef0008040 comp rx=0 tx=0).stop 2024-09-12T03:20:07.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.271+0000 7f0f05e7a640 1 -- 172.21.15.6:0/1023130053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f001036e0 msgr2=0x7f0f00198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.271+0000 7f0f05e7a640 1 --2- 172.21.15.6:0/1023130053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f001036e0 0x7f0f00198b30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400d750 tx=0x7f0ef400dc10 comp rx=0 tx=0).stop 2024-09-12T03:20:07.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.271+0000 7f0f05e7a640 1 -- 172.21.15.6:0/1023130053 shutdown_connections 2024-09-12T03:20:07.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.271+0000 7f0f05e7a640 1 --2- 172.21.15.6:0/1023130053 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0ee80a2c10 0x7f0ee80a50d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.271+0000 7f0f05e7a640 1 --2- 172.21.15.6:0/1023130053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f001082a0 0x7f0f0019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.271+0000 7f0f05e7a640 1 --2- 172.21.15.6:0/1023130053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f001036e0 0x7f0f00198b30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.271+0000 7f0f05e7a640 1 --2- 172.21.15.6:0/1023130053 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0f00102d30 0x7f0f001985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.271+0000 7f0f05e7a640 1 -- 172.21.15.6:0/1023130053 >> 172.21.15.6:0/1023130053 conn(0x7f0f000fc7d0 msgr2=0x7f0f000fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:07.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.271+0000 7f0f05e7a640 1 -- 172.21.15.6:0/1023130053 shutdown_connections 2024-09-12T03:20:07.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.271+0000 7f0f05e7a640 1 -- 172.21.15.6:0/1023130053 wait complete. 2024-09-12T03:20:07.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:07 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1023130053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T03:20:07.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:07 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1023130053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T03:20:07.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.791+0000 7f070dd47640 1 -- 172.21.15.6:0/67645467 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0708103660 msgr2=0x7f070810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.791+0000 7f070dd47640 1 --2- 172.21.15.6:0/67645467 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0708103660 0x7f070810bb50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f06f0009900 tx=0x7f06f002a480 comp rx=0 tx=0).stop 2024-09-12T03:20:07.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.791+0000 7f070dd47640 1 -- 172.21.15.6:0/67645467 shutdown_connections 2024-09-12T03:20:07.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.791+0000 7f070dd47640 1 --2- 172.21.15.6:0/67645467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f070810c170 0x7f070810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.791+0000 7f070dd47640 1 --2- 172.21.15.6:0/67645467 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0708103660 0x7f070810bb50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.791+0000 7f070dd47640 1 --2- 172.21.15.6:0/67645467 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0708102d40 0x7f0708103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.791+0000 7f070dd47640 1 -- 172.21.15.6:0/67645467 >> 172.21.15.6:0/67645467 conn(0x7f07080fc7a0 msgr2=0x7f07080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:07.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.791+0000 7f070dd47640 1 -- 172.21.15.6:0/67645467 shutdown_connections 2024-09-12T03:20:07.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.791+0000 7f070dd47640 1 -- 172.21.15.6:0/67645467 wait complete. 2024-09-12T03:20:07.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.792+0000 7f070dd47640 1 Processor -- start 2024-09-12T03:20:07.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.792+0000 7f070dd47640 1 -- start start 2024-09-12T03:20:07.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.792+0000 7f070dd47640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0708102d40 0x7f07081009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.792+0000 7f070dd47640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0708103660 0x7f0708100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.792+0000 7f070dd47640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f070810c170 0x7f0708101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.792+0000 7f070dd47640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0708112030 con 0x7f0708103660 2024-09-12T03:20:07.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.792+0000 7f070dd47640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f0708111eb0 con 0x7f070810c170 2024-09-12T03:20:07.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.792+0000 7f070dd47640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f07081121b0 con 0x7f0708102d40 2024-09-12T03:20:07.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.792+0000 7f0706ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0708103660 0x7f0708100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.792+0000 7f0706ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0708103660 0x7f0708100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42318/0 (socket says 172.21.15.6:42318) 2024-09-12T03:20:07.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.792+0000 7f0706ffd640 1 -- 172.21.15.6:0/3480539546 learned_addr learned my addr 172.21.15.6:0/3480539546 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:07.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.792+0000 7f07077fe640 1 --2- 172.21.15.6:0/3480539546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0708102d40 0x7f07081009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.793+0000 7f0707fff640 1 --2- 172.21.15.6:0/3480539546 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f070810c170 0x7f0708101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.793+0000 7f0706ffd640 1 -- 172.21.15.6:0/3480539546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0708102d40 msgr2=0x7f07081009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.793+0000 7f0706ffd640 1 --2- 172.21.15.6:0/3480539546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0708102d40 0x7f07081009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.793+0000 7f0706ffd640 1 -- 172.21.15.6:0/3480539546 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f070810c170 msgr2=0x7f0708101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.793+0000 7f0706ffd640 1 --2- 172.21.15.6:0/3480539546 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f070810c170 0x7f0708101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.793+0000 7f0706ffd640 1 -- 172.21.15.6:0/3480539546 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07081a6a50 con 0x7f0708103660 2024-09-12T03:20:07.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.793+0000 7f0706ffd640 1 --2- 172.21.15.6:0/3480539546 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0708103660 0x7f0708100ee0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f06f0005ab0 tx=0x7f06f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:07.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.793+0000 7f0704ff9640 1 -- 172.21.15.6:0/3480539546 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f003f030 con 0x7f0708103660 2024-09-12T03:20:07.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.793+0000 7f0704ff9640 1 -- 172.21.15.6:0/3480539546 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f0003ce0 con 0x7f0708103660 2024-09-12T03:20:07.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.793+0000 7f0704ff9640 1 -- 172.21.15.6:0/3480539546 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0035070 con 0x7f0708103660 2024-09-12T03:20:07.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.793+0000 7f070dd47640 1 -- 172.21.15.6:0/3480539546 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07081a6ce0 con 0x7f0708103660 2024-09-12T03:20:07.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.793+0000 7f070dd47640 1 -- 172.21.15.6:0/3480539546 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07081a71c0 con 0x7f0708103660 2024-09-12T03:20:07.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.794+0000 7f070dd47640 1 -- 172.21.15.6:0/3480539546 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07081031a0 con 0x7f0708103660 2024-09-12T03:20:07.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.795+0000 7f0704ff9640 1 -- 172.21.15.6:0/3480539546 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f06f0042030 con 0x7f0708103660 2024-09-12T03:20:07.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.797+0000 7f0704ff9640 1 --2- 172.21.15.6:0/3480539546 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f06f40a2c90 0x7f06f40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.797+0000 7f07077fe640 1 --2- 172.21.15.6:0/3480539546 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f06f40a2c90 0x7f06f40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.797+0000 7f0704ff9640 1 -- 172.21.15.6:0/3480539546 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f06f00efa20 con 0x7f0708103660 2024-09-12T03:20:07.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.797+0000 7f07077fe640 1 --2- 172.21.15.6:0/3480539546 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f06f40a2c90 0x7f06f40a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009890 tx=0x7f06f8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:07.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.801+0000 7f0704ff9640 1 -- 172.21.15.6:0/3480539546 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f06f00bc850 con 0x7f0708103660 2024-09-12T03:20:07.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:07 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1023130053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T03:20:07.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.980+0000 7f070dd47640 1 -- 172.21.15.6:0/3480539546 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f07081071e0 con 0x7f0708103660 2024-09-12T03:20:07.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.981+0000 7f0704ff9640 1 -- 172.21.15.6:0/3480539546 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f06f00bc150 con 0x7f0708103660 2024-09-12T03:20:07.981 INFO:teuthology.orchestra.run.smithi006.stdout:154618822710 2024-09-12T03:20:07.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.984+0000 7f070dd47640 1 -- 172.21.15.6:0/3480539546 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f06f40a2c90 msgr2=0x7f06f40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.984+0000 7f070dd47640 1 --2- 172.21.15.6:0/3480539546 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f06f40a2c90 0x7f06f40a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009890 tx=0x7f06f8008040 comp rx=0 tx=0).stop 2024-09-12T03:20:07.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.984+0000 7f070dd47640 1 -- 172.21.15.6:0/3480539546 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0708103660 msgr2=0x7f0708100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.984+0000 7f070dd47640 1 --2- 172.21.15.6:0/3480539546 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0708103660 0x7f0708100ee0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f06f0005ab0 tx=0x7f06f00042f0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.984+0000 7f070dd47640 1 -- 172.21.15.6:0/3480539546 shutdown_connections 2024-09-12T03:20:07.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.984+0000 7f070dd47640 1 --2- 172.21.15.6:0/3480539546 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f06f40a2c90 0x7f06f40a5150 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.984+0000 7f070dd47640 1 --2- 172.21.15.6:0/3480539546 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f070810c170 0x7f0708101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.984+0000 7f070dd47640 1 --2- 172.21.15.6:0/3480539546 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0708103660 0x7f0708100ee0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.984+0000 7f070dd47640 1 --2- 172.21.15.6:0/3480539546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0708102d40 0x7f07081009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.984+0000 7f070dd47640 1 -- 172.21.15.6:0/3480539546 >> 172.21.15.6:0/3480539546 conn(0x7f07080fc7a0 msgr2=0x7f07080fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:07.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.985+0000 7f070dd47640 1 -- 172.21.15.6:0/3480539546 shutdown_connections 2024-09-12T03:20:07.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:07.985+0000 7f070dd47640 1 -- 172.21.15.6:0/3480539546 wait complete. 2024-09-12T03:20:08.274 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430251 got 206158430252 for osd.7 2024-09-12T03:20:08.274 DEBUG:teuthology.parallel:result is None 2024-09-12T03:20:08.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:08 smithi118 ceph-mon[35048]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:08.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:08 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:08.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:08 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3480539546' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T03:20:08.514 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:08 smithi006 ceph-mon[30991]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:08.514 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:08 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:08.515 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:08 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3480539546' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T03:20:08.678 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972382 got 266287972385 for osd.9 2024-09-12T03:20:08.678 DEBUG:teuthology.parallel:result is None 2024-09-12T03:20:08.684 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:08 smithi161 ceph-mon[35485]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:08.684 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:08 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:08.685 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:08 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3480539546' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T03:20:09.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.076+0000 7fd3778e3640 1 -- 172.21.15.6:0/867057393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd370103780 msgr2=0x7fd370107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.076+0000 7fd3778e3640 1 --2- 172.21.15.6:0/867057393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd370103780 0x7fd370107bc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd360009930 tx=0x7fd36002a420 comp rx=0 tx=0).stop 2024-09-12T03:20:09.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.076+0000 7fd3778e3640 1 -- 172.21.15.6:0/867057393 shutdown_connections 2024-09-12T03:20:09.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.076+0000 7fd3778e3640 1 --2- 172.21.15.6:0/867057393 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd370108300 0x7fd37010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.076+0000 7fd3778e3640 1 --2- 172.21.15.6:0/867057393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd370103780 0x7fd370107bc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.076+0000 7fd3778e3640 1 --2- 172.21.15.6:0/867057393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd370102dd0 0x7fd3701031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.076+0000 7fd3778e3640 1 -- 172.21.15.6:0/867057393 >> 172.21.15.6:0/867057393 conn(0x7fd3700fc7d0 msgr2=0x7fd3700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:09.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.076+0000 7fd3778e3640 1 -- 172.21.15.6:0/867057393 shutdown_connections 2024-09-12T03:20:09.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.076+0000 7fd3778e3640 1 -- 172.21.15.6:0/867057393 wait complete. 2024-09-12T03:20:09.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.077+0000 7fd3778e3640 1 Processor -- start 2024-09-12T03:20:09.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.077+0000 7fd3778e3640 1 -- start start 2024-09-12T03:20:09.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.077+0000 7fd3778e3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd370102dd0 0x7fd3701985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.077+0000 7fd3778e3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd370103780 0x7fd370198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.077+0000 7fd3778e3640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd370108300 0x7fd37019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.077+0000 7fd3778e3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd370112070 con 0x7fd370102dd0 2024-09-12T03:20:09.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.077+0000 7fd3778e3640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fd370111ef0 con 0x7fd370108300 2024-09-12T03:20:09.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.077+0000 7fd3778e3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd3701121f0 con 0x7fd370103780 2024-09-12T03:20:09.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.077+0000 7fd37567f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd370102dd0 0x7fd3701985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.077+0000 7fd37567f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd370102dd0 0x7fd3701985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53866/0 (socket says 172.21.15.6:53866) 2024-09-12T03:20:09.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.077+0000 7fd37567f640 1 -- 172.21.15.6:0/2669960691 learned_addr learned my addr 172.21.15.6:0/2669960691 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:09.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.077+0000 7fd374e7e640 1 --2- 172.21.15.6:0/2669960691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd370103780 0x7fd370198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.077+0000 7fd375e80640 1 --2- 172.21.15.6:0/2669960691 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd370108300 0x7fd37019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.077+0000 7fd37567f640 1 -- 172.21.15.6:0/2669960691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd370103780 msgr2=0x7fd370198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.077+0000 7fd37567f640 1 --2- 172.21.15.6:0/2669960691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd370103780 0x7fd370198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.077+0000 7fd37567f640 1 -- 172.21.15.6:0/2669960691 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd370108300 msgr2=0x7fd37019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.077+0000 7fd37567f640 1 --2- 172.21.15.6:0/2669960691 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd370108300 0x7fd37019cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.077+0000 7fd37567f640 1 -- 172.21.15.6:0/2669960691 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd37019d5a0 con 0x7fd370102dd0 2024-09-12T03:20:09.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.078+0000 7fd37567f640 1 --2- 172.21.15.6:0/2669960691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd370102dd0 0x7fd3701985f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00e9f0 tx=0x7fd36c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:09.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.078+0000 7fd3667fc640 1 -- 172.21.15.6:0/2669960691 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36c00cd80 con 0x7fd370102dd0 2024-09-12T03:20:09.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.078+0000 7fd3778e3640 1 -- 172.21.15.6:0/2669960691 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd37019d890 con 0x7fd370102dd0 2024-09-12T03:20:09.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.078+0000 7fd3667fc640 1 -- 172.21.15.6:0/2669960691 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd36c005500 con 0x7fd370102dd0 2024-09-12T03:20:09.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.078+0000 7fd3667fc640 1 -- 172.21.15.6:0/2669960691 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36c010690 con 0x7fd370102dd0 2024-09-12T03:20:09.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.078+0000 7fd3778e3640 1 -- 172.21.15.6:0/2669960691 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3701a5170 con 0x7fd370102dd0 2024-09-12T03:20:09.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.079+0000 7fd3778e3640 1 -- 172.21.15.6:0/2669960691 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3701031b0 con 0x7fd370102dd0 2024-09-12T03:20:09.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.080+0000 7fd3667fc640 1 -- 172.21.15.6:0/2669960691 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fd36c0046c0 con 0x7fd370102dd0 2024-09-12T03:20:09.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.081+0000 7fd3667fc640 1 --2- 172.21.15.6:0/2669960691 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd35c0a2c10 0x7fd35c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.081+0000 7fd374e7e640 1 --2- 172.21.15.6:0/2669960691 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd35c0a2c10 0x7fd35c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.081+0000 7fd3667fc640 1 -- 172.21.15.6:0/2669960691 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd36c0cf580 con 0x7fd370102dd0 2024-09-12T03:20:09.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.081+0000 7fd374e7e640 1 --2- 172.21.15.6:0/2669960691 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd35c0a2c10 0x7fd35c0a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd360005940 tx=0x7fd3600058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:09.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.085+0000 7fd3667fc640 1 -- 172.21.15.6:0/2669960691 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd36c09c3b0 con 0x7fd370102dd0 2024-09-12T03:20:09.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.263+0000 7fd3778e3640 1 -- 172.21.15.6:0/2669960691 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd37010a290 con 0x7fd370102dd0 2024-09-12T03:20:09.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.264+0000 7fd3667fc640 1 -- 172.21.15.6:0/2669960691 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd36c09bcb0 con 0x7fd370102dd0 2024-09-12T03:20:09.264 INFO:teuthology.orchestra.run.smithi006.stdout:34359738448 2024-09-12T03:20:09.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.266+0000 7fd3778e3640 1 -- 172.21.15.6:0/2669960691 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd35c0a2c10 msgr2=0x7fd35c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.266+0000 7fd3778e3640 1 --2- 172.21.15.6:0/2669960691 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd35c0a2c10 0x7fd35c0a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd360005940 tx=0x7fd3600058d0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.266+0000 7fd3778e3640 1 -- 172.21.15.6:0/2669960691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd370102dd0 msgr2=0x7fd3701985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.266+0000 7fd3778e3640 1 --2- 172.21.15.6:0/2669960691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd370102dd0 0x7fd3701985f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00e9f0 tx=0x7fd36c00edf0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.266+0000 7fd3778e3640 1 -- 172.21.15.6:0/2669960691 shutdown_connections 2024-09-12T03:20:09.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.266+0000 7fd3778e3640 1 --2- 172.21.15.6:0/2669960691 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd35c0a2c10 0x7fd35c0a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.266+0000 7fd3778e3640 1 --2- 172.21.15.6:0/2669960691 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd370108300 0x7fd37019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.266+0000 7fd3778e3640 1 --2- 172.21.15.6:0/2669960691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd370103780 0x7fd370198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.267 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.266+0000 7fd3778e3640 1 --2- 172.21.15.6:0/2669960691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd370102dd0 0x7fd3701985f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.267 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.266+0000 7fd3778e3640 1 -- 172.21.15.6:0/2669960691 >> 172.21.15.6:0/2669960691 conn(0x7fd3700fc7d0 msgr2=0x7fd370105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:09.267 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.266+0000 7fd3778e3640 1 -- 172.21.15.6:0/2669960691 shutdown_connections 2024-09-12T03:20:09.267 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:09.266+0000 7fd3778e3640 1 -- 172.21.15.6:0/2669960691 wait complete. 2024-09-12T03:20:09.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:09 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2669960691' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T03:20:09.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:09 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:20:09.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:09 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:20:09.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:09 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:20:09.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:09 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:20:09.764 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:09 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2669960691' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T03:20:09.764 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:09 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:20:09.764 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:09 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:20:09.764 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:09 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:20:09.764 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:09 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:20:09.790 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822708 got 154618822710 for osd.5 2024-09-12T03:20:09.790 DEBUG:teuthology.parallel:result is None 2024-09-12T03:20:09.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:09 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2669960691' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T03:20:09.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:09 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:20:09.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:09 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:20:09.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:09 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:20:09.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:09 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:20:10.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:10 smithi006 ceph-mon[30991]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:20:10.458 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738448 for osd.0 2024-09-12T03:20:10.459 DEBUG:teuthology.parallel:result is None 2024-09-12T03:20:10.459 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T03:20:10.459 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:20:10.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:10 smithi118 ceph-mon[35048]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:20:10.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:10 smithi161 ceph-mon[35485]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:20:11.372 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:11.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.831+0000 7f7e9dec0640 1 -- 172.21.15.6:0/3941368056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e981036e0 msgr2=0x7f7e98107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:11.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.831+0000 7f7e9dec0640 1 --2- 172.21.15.6:0/3941368056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e981036e0 0x7f7e98107b90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009930 tx=0x7f7e8c02a420 comp rx=0 tx=0).stop 2024-09-12T03:20:11.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.831+0000 7f7e9dec0640 1 -- 172.21.15.6:0/3941368056 shutdown_connections 2024-09-12T03:20:11.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.831+0000 7f7e9dec0640 1 --2- 172.21.15.6:0/3941368056 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e981082a0 0x7f7e9810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.831+0000 7f7e9dec0640 1 --2- 172.21.15.6:0/3941368056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e981036e0 0x7f7e98107b90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.831+0000 7f7e9dec0640 1 --2- 172.21.15.6:0/3941368056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e98102d30 0x7f7e98103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.831+0000 7f7e9dec0640 1 -- 172.21.15.6:0/3941368056 >> 172.21.15.6:0/3941368056 conn(0x7f7e980fc7d0 msgr2=0x7f7e980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:11.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.831+0000 7f7e9dec0640 1 -- 172.21.15.6:0/3941368056 shutdown_connections 2024-09-12T03:20:11.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.831+0000 7f7e9dec0640 1 -- 172.21.15.6:0/3941368056 wait complete. 2024-09-12T03:20:11.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.831+0000 7f7e9dec0640 1 Processor -- start 2024-09-12T03:20:11.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.832+0000 7f7e9dec0640 1 -- start start 2024-09-12T03:20:11.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.832+0000 7f7e9dec0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e98102d30 0x7f7e981a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:11.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.832+0000 7f7e9dec0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e981036e0 0x7f7e981a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:11.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.832+0000 7f7e9dec0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e981082a0 0x7f7e9819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:11.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.832+0000 7f7e9dec0640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7e981120a0 con 0x7f7e98102d30 2024-09-12T03:20:11.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.832+0000 7f7e9dec0640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f7e98111f20 con 0x7f7e981082a0 2024-09-12T03:20:11.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.832+0000 7f7e9dec0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7e98112220 con 0x7f7e981036e0 2024-09-12T03:20:11.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.832+0000 7f7e977fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e98102d30 0x7f7e981a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:11.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.832+0000 7f7e977fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e98102d30 0x7f7e981a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53892/0 (socket says 172.21.15.6:53892) 2024-09-12T03:20:11.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.832+0000 7f7e977fe640 1 -- 172.21.15.6:0/1714227080 learned_addr learned my addr 172.21.15.6:0/1714227080 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:11.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.832+0000 7f7e96ffd640 1 --2- 172.21.15.6:0/1714227080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e981036e0 0x7f7e981a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:11.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.832+0000 7f7e97fff640 1 --2- 172.21.15.6:0/1714227080 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e981082a0 0x7f7e9819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:11.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.832+0000 7f7e977fe640 1 -- 172.21.15.6:0/1714227080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e981036e0 msgr2=0x7f7e981a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:11.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.832+0000 7f7e977fe640 1 --2- 172.21.15.6:0/1714227080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e981036e0 0x7f7e981a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.832+0000 7f7e977fe640 1 -- 172.21.15.6:0/1714227080 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e981082a0 msgr2=0x7f7e9819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:11.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.832+0000 7f7e977fe640 1 --2- 172.21.15.6:0/1714227080 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e981082a0 0x7f7e9819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.832+0000 7f7e977fe640 1 -- 172.21.15.6:0/1714227080 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9819acc0 con 0x7f7e98102d30 2024-09-12T03:20:11.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.832+0000 7f7e96ffd640 1 --2- 172.21.15.6:0/1714227080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e981036e0 0x7f7e981a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:11.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.832+0000 7f7e977fe640 1 --2- 172.21.15.6:0/1714227080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e98102d30 0x7f7e981a03d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800cce0 tx=0x7f7e8800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:11.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.833+0000 7f7e94ff9640 1 -- 172.21.15.6:0/1714227080 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88004b20 con 0x7f7e98102d30 2024-09-12T03:20:11.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.833+0000 7f7e9dec0640 1 -- 172.21.15.6:0/1714227080 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9819afb0 con 0x7f7e98102d30 2024-09-12T03:20:11.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.833+0000 7f7e94ff9640 1 -- 172.21.15.6:0/1714227080 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e88004cc0 con 0x7f7e98102d30 2024-09-12T03:20:11.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.833+0000 7f7e94ff9640 1 -- 172.21.15.6:0/1714227080 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8800f8c0 con 0x7f7e98102d30 2024-09-12T03:20:11.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.833+0000 7f7e9dec0640 1 -- 172.21.15.6:0/1714227080 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e981a72a0 con 0x7f7e98102d30 2024-09-12T03:20:11.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.834+0000 7f7e9dec0640 1 -- 172.21.15.6:0/1714227080 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e98103110 con 0x7f7e98102d30 2024-09-12T03:20:11.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.835+0000 7f7e94ff9640 1 -- 172.21.15.6:0/1714227080 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f7e8800fa60 con 0x7f7e98102d30 2024-09-12T03:20:11.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.835+0000 7f7e94ff9640 1 --2- 172.21.15.6:0/1714227080 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7e800a2c10 0x7f7e800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:11.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.836+0000 7f7e96ffd640 1 --2- 172.21.15.6:0/1714227080 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7e800a2c10 0x7f7e800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:11.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.836+0000 7f7e94ff9640 1 -- 172.21.15.6:0/1714227080 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7e880cf330 con 0x7f7e98102d30 2024-09-12T03:20:11.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.836+0000 7f7e96ffd640 1 --2- 172.21.15.6:0/1714227080 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7e800a2c10 0x7f7e800a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009900 tx=0x7f7e8c031690 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:11.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:11.840+0000 7f7e94ff9640 1 -- 172.21.15.6:0/1714227080 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7e88014030 con 0x7f7e98102d30 2024-09-12T03:20:12.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:12.037+0000 7f7e9dec0640 1 -- 172.21.15.6:0/1714227080 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e9810e6d0 con 0x7f7e800a2c10 2024-09-12T03:20:12.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:12.045+0000 7f7e94ff9640 1 -- 172.21.15.6:0/1714227080 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258717 (secure 0 0 0) 0x7f7e9810e6d0 con 0x7f7e800a2c10 2024-09-12T03:20:12.048 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:20:12.049 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-09-12T03:20:12.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:12.048+0000 7f7e9dec0640 1 -- 172.21.15.6:0/1714227080 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7e800a2c10 msgr2=0x7f7e800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:12.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:12.049+0000 7f7e9dec0640 1 --2- 172.21.15.6:0/1714227080 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7e800a2c10 0x7f7e800a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009900 tx=0x7f7e8c031690 comp rx=0 tx=0).stop 2024-09-12T03:20:12.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:12.049+0000 7f7e9dec0640 1 -- 172.21.15.6:0/1714227080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e98102d30 msgr2=0x7f7e981a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:12.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:12.049+0000 7f7e9dec0640 1 --2- 172.21.15.6:0/1714227080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e98102d30 0x7f7e981a03d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800cce0 tx=0x7f7e8800a620 comp rx=0 tx=0).stop 2024-09-12T03:20:12.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:12.049+0000 7f7e9dec0640 1 -- 172.21.15.6:0/1714227080 shutdown_connections 2024-09-12T03:20:12.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:12.049+0000 7f7e9dec0640 1 --2- 172.21.15.6:0/1714227080 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7e800a2c10 0x7f7e800a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:12.049+0000 7f7e9dec0640 1 --2- 172.21.15.6:0/1714227080 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e981082a0 0x7f7e9819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:12.049+0000 7f7e9dec0640 1 --2- 172.21.15.6:0/1714227080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e981036e0 0x7f7e981a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:12.049+0000 7f7e9dec0640 1 --2- 172.21.15.6:0/1714227080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e98102d30 0x7f7e981a03d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:12.049+0000 7f7e9dec0640 1 -- 172.21.15.6:0/1714227080 >> 172.21.15.6:0/1714227080 conn(0x7f7e980fc7d0 msgr2=0x7f7e9810cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:12.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:12.049+0000 7f7e9dec0640 1 -- 172.21.15.6:0/1714227080 shutdown_connections 2024-09-12T03:20:12.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:12.049+0000 7f7e9dec0640 1 -- 172.21.15.6:0/1714227080 wait complete. 2024-09-12T03:20:12.662 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:12 smithi006 ceph-mon[30991]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:12.662 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:12 smithi006 ceph-mon[30991]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:20:12.662 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:12 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:20:12.662 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:12 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:20:12.689 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-12T03:20:11.251649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654369,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":446,"num_read_kb":335,"num_write":347,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382608,"kb_used_data":12540,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559216,"statfs":{"total":1151940427776,"available":1151548637184,"internally_reserved":0,"allocated":12840960,"data_stored":7156359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002955"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.371249+0000","last_change":"2024-09-12T03:19:09.343829+0000","last_active":"2024-09-12T03:19:20.371249+0000","last_peered":"2024-09-12T03:19:20.371249+0000","last_clean":"2024-09-12T03:19:20.371249+0000","last_became_active":"2024-09-12T03:19:09.343228+0000","last_became_peered":"2024-09-12T03:19:09.343228+0000","last_unstale":"2024-09-12T03:19:20.371249+0000","last_undegraded":"2024-09-12T03:19:20.371249+0000","last_fullsized":"2024-09-12T03:19:20.371249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:08.720934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408528+0000","last_change":"2024-09-12T03:19:09.331829+0000","last_active":"2024-09-12T03:19:20.408528+0000","last_peered":"2024-09-12T03:19:20.408528+0000","last_clean":"2024-09-12T03:19:20.408528+0000","last_became_active":"2024-09-12T03:19:09.331276+0000","last_became_peered":"2024-09-12T03:19:09.331276+0000","last_unstale":"2024-09-12T03:19:20.408528+0000","last_undegraded":"2024-09-12T03:19:20.408528+0000","last_fullsized":"2024-09-12T03:19:20.408528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:48.672085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408509+0000","last_change":"2024-09-12T03:19:09.338120+0000","last_active":"2024-09-12T03:19:20.408509+0000","last_peered":"2024-09-12T03:19:20.408509+0000","last_clean":"2024-09-12T03:19:20.408509+0000","last_became_active":"2024-09-12T03:19:09.337773+0000","last_became_peered":"2024-09-12T03:19:09.337773+0000","last_unstale":"2024-09-12T03:19:20.408509+0000","last_undegraded":"2024-09-12T03:19:20.408509+0000","last_fullsized":"2024-09-12T03:19:20.408509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:13.716047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382087+0000","last_change":"2024-09-12T03:19:09.339404+0000","last_active":"2024-09-12T03:19:20.382087+0000","last_peered":"2024-09-12T03:19:20.382087+0000","last_clean":"2024-09-12T03:19:20.382087+0000","last_became_active":"2024-09-12T03:19:09.339105+0000","last_became_peered":"2024-09-12T03:19:09.339105+0000","last_unstale":"2024-09-12T03:19:20.382087+0000","last_undegraded":"2024-09-12T03:19:20.382087+0000","last_fullsized":"2024-09-12T03:19:20.382087+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:37.930963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373401+0000","last_change":"2024-09-12T03:19:09.326285+0000","last_active":"2024-09-12T03:19:20.373401+0000","last_peered":"2024-09-12T03:19:20.373401+0000","last_clean":"2024-09-12T03:19:20.373401+0000","last_became_active":"2024-09-12T03:19:09.325852+0000","last_became_peered":"2024-09-12T03:19:09.325852+0000","last_unstale":"2024-09-12T03:19:20.373401+0000","last_undegraded":"2024-09-12T03:19:20.373401+0000","last_fullsized":"2024-09-12T03:19:20.373401+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:44:25.387014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.376197+0000","last_change":"2024-09-12T03:19:20.376197+0000","last_active":"2024-09-12T03:19:20.376197+0000","last_peered":"2024-09-12T03:19:20.376197+0000","last_clean":"2024-09-12T03:19:20.376197+0000","last_became_active":"2024-09-12T03:19:20.375744+0000","last_became_peered":"2024-09-12T03:19:20.375744+0000","last_unstale":"2024-09-12T03:19:20.376197+0000","last_undegraded":"2024-09-12T03:19:20.376197+0000","last_fullsized":"2024-09-12T03:19:20.376197+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:08.663121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372152+0000","last_change":"2024-09-12T03:19:09.335284+0000","last_active":"2024-09-12T03:19:20.372152+0000","last_peered":"2024-09-12T03:19:20.372152+0000","last_clean":"2024-09-12T03:19:20.372152+0000","last_became_active":"2024-09-12T03:19:09.334948+0000","last_became_peered":"2024-09-12T03:19:09.334948+0000","last_unstale":"2024-09-12T03:19:20.372152+0000","last_undegraded":"2024-09-12T03:19:20.372152+0000","last_fullsized":"2024-09-12T03:19:20.372152+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:32.778433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372122+0000","last_change":"2024-09-12T03:19:09.338398+0000","last_active":"2024-09-12T03:19:20.372122+0000","last_peered":"2024-09-12T03:19:20.372122+0000","last_clean":"2024-09-12T03:19:20.372122+0000","last_became_active":"2024-09-12T03:19:09.337750+0000","last_became_peered":"2024-09-12T03:19:09.337750+0000","last_unstale":"2024-09-12T03:19:20.372122+0000","last_undegraded":"2024-09-12T03:19:20.372122+0000","last_fullsized":"2024-09-12T03:19:20.372122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:07.542322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370699+0000","last_change":"2024-09-12T03:19:09.340703+0000","last_active":"2024-09-12T03:19:20.370699+0000","last_peered":"2024-09-12T03:19:20.370699+0000","last_clean":"2024-09-12T03:19:20.370699+0000","last_became_active":"2024-09-12T03:19:09.340386+0000","last_became_peered":"2024-09-12T03:19:09.340386+0000","last_unstale":"2024-09-12T03:19:20.370699+0000","last_undegraded":"2024-09-12T03:19:20.370699+0000","last_fullsized":"2024-09-12T03:19:20.370699+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:19.990894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374331+0000","last_change":"2024-09-12T03:19:09.341489+0000","last_active":"2024-09-12T03:19:20.374331+0000","last_peered":"2024-09-12T03:19:20.374331+0000","last_clean":"2024-09-12T03:19:20.374331+0000","last_became_active":"2024-09-12T03:19:09.341183+0000","last_became_peered":"2024-09-12T03:19:09.341183+0000","last_unstale":"2024-09-12T03:19:20.374331+0000","last_undegraded":"2024-09-12T03:19:20.374331+0000","last_fullsized":"2024-09-12T03:19:20.374331+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:05.688402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372589+0000","last_change":"2024-09-12T03:19:09.339289+0000","last_active":"2024-09-12T03:19:20.372589+0000","last_peered":"2024-09-12T03:19:20.372589+0000","last_clean":"2024-09-12T03:19:20.372589+0000","last_became_active":"2024-09-12T03:19:09.338466+0000","last_became_peered":"2024-09-12T03:19:09.338466+0000","last_unstale":"2024-09-12T03:19:20.372589+0000","last_undegraded":"2024-09-12T03:19:20.372589+0000","last_fullsized":"2024-09-12T03:19:20.372589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:47:13.070057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382092+0000","last_change":"2024-09-12T03:19:09.337754+0000","last_active":"2024-09-12T03:19:20.382092+0000","last_peered":"2024-09-12T03:19:20.382092+0000","last_clean":"2024-09-12T03:19:20.382092+0000","last_became_active":"2024-09-12T03:19:09.332789+0000","last_became_peered":"2024-09-12T03:19:09.332789+0000","last_unstale":"2024-09-12T03:19:20.382092+0000","last_undegraded":"2024-09-12T03:19:20.382092+0000","last_fullsized":"2024-09-12T03:19:20.382092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:34.622136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375297+0000","last_change":"2024-09-12T03:19:09.333577+0000","last_active":"2024-09-12T03:19:20.375297+0000","last_peered":"2024-09-12T03:19:20.375297+0000","last_clean":"2024-09-12T03:19:20.375297+0000","last_became_active":"2024-09-12T03:19:09.333234+0000","last_became_peered":"2024-09-12T03:19:09.333234+0000","last_unstale":"2024-09-12T03:19:20.375297+0000","last_undegraded":"2024-09-12T03:19:20.375297+0000","last_fullsized":"2024-09-12T03:19:20.375297+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:22.604640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408267+0000","last_change":"2024-09-12T03:19:09.331607+0000","last_active":"2024-09-12T03:19:20.408267+0000","last_peered":"2024-09-12T03:19:20.408267+0000","last_clean":"2024-09-12T03:19:20.408267+0000","last_became_active":"2024-09-12T03:19:09.331253+0000","last_became_peered":"2024-09-12T03:19:09.331253+0000","last_unstale":"2024-09-12T03:19:20.408267+0000","last_undegraded":"2024-09-12T03:19:20.408267+0000","last_fullsized":"2024-09-12T03:19:20.408267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:11.969265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408245+0000","last_change":"2024-09-12T03:19:09.340768+0000","last_active":"2024-09-12T03:19:20.408245+0000","last_peered":"2024-09-12T03:19:20.408245+0000","last_clean":"2024-09-12T03:19:20.408245+0000","last_became_active":"2024-09-12T03:19:09.340437+0000","last_became_peered":"2024-09-12T03:19:09.340437+0000","last_unstale":"2024-09-12T03:19:20.408245+0000","last_undegraded":"2024-09-12T03:19:20.408245+0000","last_fullsized":"2024-09-12T03:19:20.408245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:32.645691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370081+0000","last_change":"2024-09-12T03:19:09.333829+0000","last_active":"2024-09-12T03:19:20.370081+0000","last_peered":"2024-09-12T03:19:20.370081+0000","last_clean":"2024-09-12T03:19:20.370081+0000","last_became_active":"2024-09-12T03:19:09.333462+0000","last_became_peered":"2024-09-12T03:19:09.333462+0000","last_unstale":"2024-09-12T03:19:20.370081+0000","last_undegraded":"2024-09-12T03:19:20.370081+0000","last_fullsized":"2024-09-12T03:19:20.370081+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:18:57.586836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.371273+0000","last_change":"2024-09-12T03:19:09.344068+0000","last_active":"2024-09-12T03:19:20.371273+0000","last_peered":"2024-09-12T03:19:20.371273+0000","last_clean":"2024-09-12T03:19:20.371273+0000","last_became_active":"2024-09-12T03:19:09.343751+0000","last_became_peered":"2024-09-12T03:19:09.343751+0000","last_unstale":"2024-09-12T03:19:20.371273+0000","last_undegraded":"2024-09-12T03:19:20.371273+0000","last_fullsized":"2024-09-12T03:19:20.371273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:52.425547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372644+0000","last_change":"2024-09-12T03:19:09.330828+0000","last_active":"2024-09-12T03:19:20.372644+0000","last_peered":"2024-09-12T03:19:20.372644+0000","last_clean":"2024-09-12T03:19:20.372644+0000","last_became_active":"2024-09-12T03:19:09.330497+0000","last_became_peered":"2024-09-12T03:19:09.330497+0000","last_unstale":"2024-09-12T03:19:20.372644+0000","last_undegraded":"2024-09-12T03:19:20.372644+0000","last_fullsized":"2024-09-12T03:19:20.372644+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:21:20.064177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375155+0000","last_change":"2024-09-12T03:19:09.344403+0000","last_active":"2024-09-12T03:19:20.375155+0000","last_peered":"2024-09-12T03:19:20.375155+0000","last_clean":"2024-09-12T03:19:20.375155+0000","last_became_active":"2024-09-12T03:19:09.343946+0000","last_became_peered":"2024-09-12T03:19:09.343946+0000","last_unstale":"2024-09-12T03:19:20.375155+0000","last_undegraded":"2024-09-12T03:19:20.375155+0000","last_fullsized":"2024-09-12T03:19:20.375155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:48.386117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370884+0000","last_change":"2024-09-12T03:19:09.341481+0000","last_active":"2024-09-12T03:19:20.370884+0000","last_peered":"2024-09-12T03:19:20.370884+0000","last_clean":"2024-09-12T03:19:20.370884+0000","last_became_active":"2024-09-12T03:19:09.341142+0000","last_became_peered":"2024-09-12T03:19:09.341142+0000","last_unstale":"2024-09-12T03:19:20.370884+0000","last_undegraded":"2024-09-12T03:19:20.370884+0000","last_fullsized":"2024-09-12T03:19:20.370884+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:42.371221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375324+0000","last_change":"2024-09-12T03:19:09.330621+0000","last_active":"2024-09-12T03:19:20.375324+0000","last_peered":"2024-09-12T03:19:20.375324+0000","last_clean":"2024-09-12T03:19:20.375324+0000","last_became_active":"2024-09-12T03:19:09.330169+0000","last_became_peered":"2024-09-12T03:19:09.330169+0000","last_unstale":"2024-09-12T03:19:20.375324+0000","last_undegraded":"2024-09-12T03:19:20.375324+0000","last_fullsized":"2024-09-12T03:19:20.375324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:16.634141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370190+0000","last_change":"2024-09-12T03:19:09.332817+0000","last_active":"2024-09-12T03:19:20.370190+0000","last_peered":"2024-09-12T03:19:20.370190+0000","last_clean":"2024-09-12T03:19:20.370190+0000","last_became_active":"2024-09-12T03:19:09.332236+0000","last_became_peered":"2024-09-12T03:19:09.332236+0000","last_unstale":"2024-09-12T03:19:20.370190+0000","last_undegraded":"2024-09-12T03:19:20.370190+0000","last_fullsized":"2024-09-12T03:19:20.370190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:28.024294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372100+0000","last_change":"2024-09-12T03:19:09.337910+0000","last_active":"2024-09-12T03:19:20.372100+0000","last_peered":"2024-09-12T03:19:20.372100+0000","last_clean":"2024-09-12T03:19:20.372100+0000","last_became_active":"2024-09-12T03:19:09.337176+0000","last_became_peered":"2024-09-12T03:19:09.337176+0000","last_unstale":"2024-09-12T03:19:20.372100+0000","last_undegraded":"2024-09-12T03:19:20.372100+0000","last_fullsized":"2024-09-12T03:19:20.372100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:53.795898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.385889+0000","last_change":"2024-09-12T03:19:09.337902+0000","last_active":"2024-09-12T03:19:20.385889+0000","last_peered":"2024-09-12T03:19:20.385889+0000","last_clean":"2024-09-12T03:19:20.385889+0000","last_became_active":"2024-09-12T03:19:09.337567+0000","last_became_peered":"2024-09-12T03:19:09.337567+0000","last_unstale":"2024-09-12T03:19:20.385889+0000","last_undegraded":"2024-09-12T03:19:20.385889+0000","last_fullsized":"2024-09-12T03:19:20.385889+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:08.471313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372957+0000","last_change":"2024-09-12T03:19:09.339850+0000","last_active":"2024-09-12T03:19:20.372957+0000","last_peered":"2024-09-12T03:19:20.372957+0000","last_clean":"2024-09-12T03:19:20.372957+0000","last_became_active":"2024-09-12T03:19:09.339563+0000","last_became_peered":"2024-09-12T03:19:09.339563+0000","last_unstale":"2024-09-12T03:19:20.372957+0000","last_undegraded":"2024-09-12T03:19:20.372957+0000","last_fullsized":"2024-09-12T03:19:20.372957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:27.473351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373965+0000","last_change":"2024-09-12T03:19:09.330555+0000","last_active":"2024-09-12T03:19:20.373965+0000","last_peered":"2024-09-12T03:19:20.373965+0000","last_clean":"2024-09-12T03:19:20.373965+0000","last_became_active":"2024-09-12T03:19:09.330183+0000","last_became_peered":"2024-09-12T03:19:09.330183+0000","last_unstale":"2024-09-12T03:19:20.373965+0000","last_undegraded":"2024-09-12T03:19:20.373965+0000","last_fullsized":"2024-09-12T03:19:20.373965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:39.334116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.371187+0000","last_change":"2024-09-12T03:19:09.338601+0000","last_active":"2024-09-12T03:19:20.371187+0000","last_peered":"2024-09-12T03:19:20.371187+0000","last_clean":"2024-09-12T03:19:20.371187+0000","last_became_active":"2024-09-12T03:19:09.337903+0000","last_became_peered":"2024-09-12T03:19:09.337903+0000","last_unstale":"2024-09-12T03:19:20.371187+0000","last_undegraded":"2024-09-12T03:19:20.371187+0000","last_fullsized":"2024-09-12T03:19:20.371187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:17.810139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375192+0000","last_change":"2024-09-12T03:19:09.326629+0000","last_active":"2024-09-12T03:19:20.375192+0000","last_peered":"2024-09-12T03:19:20.375192+0000","last_clean":"2024-09-12T03:19:20.375192+0000","last_became_active":"2024-09-12T03:19:09.326333+0000","last_became_peered":"2024-09-12T03:19:09.326333+0000","last_unstale":"2024-09-12T03:19:20.375192+0000","last_undegraded":"2024-09-12T03:19:20.375192+0000","last_fullsized":"2024-09-12T03:19:20.375192+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:20.807593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408435+0000","last_change":"2024-09-12T03:19:09.339919+0000","last_active":"2024-09-12T03:19:20.408435+0000","last_peered":"2024-09-12T03:19:20.408435+0000","last_clean":"2024-09-12T03:19:20.408435+0000","last_became_active":"2024-09-12T03:19:09.339618+0000","last_became_peered":"2024-09-12T03:19:09.339618+0000","last_unstale":"2024-09-12T03:19:20.408435+0000","last_undegraded":"2024-09-12T03:19:20.408435+0000","last_fullsized":"2024-09-12T03:19:20.408435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:11:03.740079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374805+0000","last_change":"2024-09-12T03:19:09.336148+0000","last_active":"2024-09-12T03:19:20.374805+0000","last_peered":"2024-09-12T03:19:20.374805+0000","last_clean":"2024-09-12T03:19:20.374805+0000","last_became_active":"2024-09-12T03:19:09.335779+0000","last_became_peered":"2024-09-12T03:19:09.335779+0000","last_unstale":"2024-09-12T03:19:20.374805+0000","last_undegraded":"2024-09-12T03:19:20.374805+0000","last_fullsized":"2024-09-12T03:19:20.374805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:18.234320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382536+0000","last_change":"2024-09-12T03:19:20.382536+0000","last_active":"2024-09-12T03:19:20.382536+0000","last_peered":"2024-09-12T03:19:20.382536+0000","last_clean":"2024-09-12T03:19:20.382536+0000","last_became_active":"2024-09-12T03:19:20.382118+0000","last_became_peered":"2024-09-12T03:19:20.382118+0000","last_unstale":"2024-09-12T03:19:20.382536+0000","last_undegraded":"2024-09-12T03:19:20.382536+0000","last_fullsized":"2024-09-12T03:19:20.382536+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:55.836646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370826+0000","last_change":"2024-09-12T03:19:09.345064+0000","last_active":"2024-09-12T03:19:20.370826+0000","last_peered":"2024-09-12T03:19:20.370826+0000","last_clean":"2024-09-12T03:19:20.370826+0000","last_became_active":"2024-09-12T03:19:09.344628+0000","last_became_peered":"2024-09-12T03:19:09.344628+0000","last_unstale":"2024-09-12T03:19:20.370826+0000","last_undegraded":"2024-09-12T03:19:20.370826+0000","last_fullsized":"2024-09-12T03:19:20.370826+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:50.011751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373663+0000","last_change":"2024-09-12T03:19:09.338052+0000","last_active":"2024-09-12T03:19:20.373663+0000","last_peered":"2024-09-12T03:19:20.373663+0000","last_clean":"2024-09-12T03:19:20.373663+0000","last_became_active":"2024-09-12T03:19:09.337710+0000","last_became_peered":"2024-09-12T03:19:09.337710+0000","last_unstale":"2024-09-12T03:19:20.373663+0000","last_undegraded":"2024-09-12T03:19:20.373663+0000","last_fullsized":"2024-09-12T03:19:20.373663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:36.521269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":83,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:09.453392+0000","last_change":"2024-09-12T03:19:07.208341+0000","last_active":"2024-09-12T03:20:09.453392+0000","last_peered":"2024-09-12T03:20:09.453392+0000","last_clean":"2024-09-12T03:20:09.453392+0000","last_became_active":"2024-09-12T03:19:07.207385+0000","last_became_peered":"2024-09-12T03:19:07.207385+0000","last_unstale":"2024-09-12T03:20:09.453392+0000","last_undegraded":"2024-09-12T03:20:09.453392+0000","last_fullsized":"2024-09-12T03:20:09.453392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:18.476795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386347+0000","last_change":"2024-09-12T03:19:09.339997+0000","last_active":"2024-09-12T03:19:20.386347+0000","last_peered":"2024-09-12T03:19:20.386347+0000","last_clean":"2024-09-12T03:19:20.386347+0000","last_became_active":"2024-09-12T03:19:09.339261+0000","last_became_peered":"2024-09-12T03:19:09.339261+0000","last_unstale":"2024-09-12T03:19:20.386347+0000","last_undegraded":"2024-09-12T03:19:20.386347+0000","last_fullsized":"2024-09-12T03:19:20.386347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:07.670345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374030+0000","last_change":"2024-09-12T03:19:07.200222+0000","last_active":"2024-09-12T03:19:20.374030+0000","last_peered":"2024-09-12T03:19:20.374030+0000","last_clean":"2024-09-12T03:19:20.374030+0000","last_became_active":"2024-09-12T03:19:07.199819+0000","last_became_peered":"2024-09-12T03:19:07.199819+0000","last_unstale":"2024-09-12T03:19:20.374030+0000","last_undegraded":"2024-09-12T03:19:20.374030+0000","last_fullsized":"2024-09-12T03:19:20.374030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:39:28.933609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375686+0000","last_change":"2024-09-12T03:19:07.207005+0000","last_active":"2024-09-12T03:19:20.375686+0000","last_peered":"2024-09-12T03:19:20.375686+0000","last_clean":"2024-09-12T03:19:20.375686+0000","last_became_active":"2024-09-12T03:19:07.206685+0000","last_became_peered":"2024-09-12T03:19:07.206685+0000","last_unstale":"2024-09-12T03:19:20.375686+0000","last_undegraded":"2024-09-12T03:19:20.375686+0000","last_fullsized":"2024-09-12T03:19:20.375686+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:31.111706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372028+0000","last_change":"2024-09-12T03:19:09.335871+0000","last_active":"2024-09-12T03:19:20.372028+0000","last_peered":"2024-09-12T03:19:20.372028+0000","last_clean":"2024-09-12T03:19:20.372028+0000","last_became_active":"2024-09-12T03:19:09.335568+0000","last_became_peered":"2024-09-12T03:19:09.335568+0000","last_unstale":"2024-09-12T03:19:20.372028+0000","last_undegraded":"2024-09-12T03:19:20.372028+0000","last_fullsized":"2024-09-12T03:19:20.372028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:52:34.471512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386049+0000","last_change":"2024-09-12T03:19:07.208320+0000","last_active":"2024-09-12T03:19:20.386049+0000","last_peered":"2024-09-12T03:19:20.386049+0000","last_clean":"2024-09-12T03:19:20.386049+0000","last_became_active":"2024-09-12T03:19:07.207873+0000","last_became_peered":"2024-09-12T03:19:07.207873+0000","last_unstale":"2024-09-12T03:19:20.386049+0000","last_undegraded":"2024-09-12T03:19:20.386049+0000","last_fullsized":"2024-09-12T03:19:20.386049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:00.128595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372467+0000","last_change":"2024-09-12T03:19:09.331930+0000","last_active":"2024-09-12T03:19:20.372467+0000","last_peered":"2024-09-12T03:19:20.372467+0000","last_clean":"2024-09-12T03:19:20.372467+0000","last_became_active":"2024-09-12T03:19:09.331558+0000","last_became_peered":"2024-09-12T03:19:09.331558+0000","last_unstale":"2024-09-12T03:19:20.372467+0000","last_undegraded":"2024-09-12T03:19:20.372467+0000","last_fullsized":"2024-09-12T03:19:20.372467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:00:27.935717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":74,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:06.449139+0000","last_change":"2024-09-12T03:19:07.205473+0000","last_active":"2024-09-12T03:20:06.449139+0000","last_peered":"2024-09-12T03:20:06.449139+0000","last_clean":"2024-09-12T03:20:06.449139+0000","last_became_active":"2024-09-12T03:19:07.205134+0000","last_became_peered":"2024-09-12T03:19:07.205134+0000","last_unstale":"2024-09-12T03:20:06.449139+0000","last_undegraded":"2024-09-12T03:20:06.449139+0000","last_fullsized":"2024-09-12T03:20:06.449139+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:23.443957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370511+0000","last_change":"2024-09-12T03:19:09.333868+0000","last_active":"2024-09-12T03:19:20.370511+0000","last_peered":"2024-09-12T03:19:20.370511+0000","last_clean":"2024-09-12T03:19:20.370511+0000","last_became_active":"2024-09-12T03:19:09.333484+0000","last_became_peered":"2024-09-12T03:19:09.333484+0000","last_unstale":"2024-09-12T03:19:20.370511+0000","last_undegraded":"2024-09-12T03:19:20.370511+0000","last_fullsized":"2024-09-12T03:19:20.370511+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:21:10.226821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386303+0000","last_change":"2024-09-12T03:19:20.386303+0000","last_active":"2024-09-12T03:19:20.386303+0000","last_peered":"2024-09-12T03:19:20.386303+0000","last_clean":"2024-09-12T03:19:20.386303+0000","last_became_active":"2024-09-12T03:19:20.383198+0000","last_became_peered":"2024-09-12T03:19:20.383198+0000","last_unstale":"2024-09-12T03:19:20.386303+0000","last_undegraded":"2024-09-12T03:19:20.386303+0000","last_fullsized":"2024-09-12T03:19:20.386303+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:42.224849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408655+0000","last_change":"2024-09-12T03:19:09.340849+0000","last_active":"2024-09-12T03:19:20.408655+0000","last_peered":"2024-09-12T03:19:20.408655+0000","last_clean":"2024-09-12T03:19:20.408655+0000","last_became_active":"2024-09-12T03:19:09.340517+0000","last_became_peered":"2024-09-12T03:19:09.340517+0000","last_unstale":"2024-09-12T03:19:20.408655+0000","last_undegraded":"2024-09-12T03:19:20.408655+0000","last_fullsized":"2024-09-12T03:19:20.408655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:42.018269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386240+0000","last_change":"2024-09-12T03:19:07.389147+0000","last_active":"2024-09-12T03:19:20.386240+0000","last_peered":"2024-09-12T03:19:20.386240+0000","last_clean":"2024-09-12T03:19:20.386240+0000","last_became_active":"2024-09-12T03:19:07.388744+0000","last_became_peered":"2024-09-12T03:19:07.388744+0000","last_unstale":"2024-09-12T03:19:20.386240+0000","last_undegraded":"2024-09-12T03:19:20.386240+0000","last_fullsized":"2024-09-12T03:19:20.386240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:16.700644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370280+0000","last_change":"2024-09-12T03:19:09.329243+0000","last_active":"2024-09-12T03:19:20.370280+0000","last_peered":"2024-09-12T03:19:20.370280+0000","last_clean":"2024-09-12T03:19:20.370280+0000","last_became_active":"2024-09-12T03:19:09.328610+0000","last_became_peered":"2024-09-12T03:19:09.328610+0000","last_unstale":"2024-09-12T03:19:20.370280+0000","last_undegraded":"2024-09-12T03:19:20.370280+0000","last_fullsized":"2024-09-12T03:19:20.370280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:49.452942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'28","reported_seq":155,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:07.450544+0000","last_change":"2024-09-12T03:19:07.207804+0000","last_active":"2024-09-12T03:20:07.450544+0000","last_peered":"2024-09-12T03:20:07.450544+0000","last_clean":"2024-09-12T03:20:07.450544+0000","last_became_active":"2024-09-12T03:19:07.207409+0000","last_became_peered":"2024-09-12T03:19:07.207409+0000","last_unstale":"2024-09-12T03:20:07.450544+0000","last_undegraded":"2024-09-12T03:20:07.450544+0000","last_fullsized":"2024-09-12T03:20:07.450544+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:10.732423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":105,"num_read_kb":110,"num_write":29,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373666+0000","last_change":"2024-09-12T03:19:09.333102+0000","last_active":"2024-09-12T03:19:20.373666+0000","last_peered":"2024-09-12T03:19:20.373666+0000","last_clean":"2024-09-12T03:19:20.373666+0000","last_became_active":"2024-09-12T03:19:09.332712+0000","last_became_peered":"2024-09-12T03:19:09.332712+0000","last_unstale":"2024-09-12T03:19:20.373666+0000","last_undegraded":"2024-09-12T03:19:20.373666+0000","last_fullsized":"2024-09-12T03:19:20.373666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:22.912213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382902+0000","last_change":"2024-09-12T03:19:20.382348+0000","last_active":"2024-09-12T03:19:20.382902+0000","last_peered":"2024-09-12T03:19:20.382902+0000","last_clean":"2024-09-12T03:19:20.382902+0000","last_became_active":"2024-09-12T03:19:20.375142+0000","last_became_peered":"2024-09-12T03:19:20.375142+0000","last_unstale":"2024-09-12T03:19:20.382902+0000","last_undegraded":"2024-09-12T03:19:20.382902+0000","last_fullsized":"2024-09-12T03:19:20.382902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:04.573599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375046+0000","last_change":"2024-09-12T03:19:09.333194+0000","last_active":"2024-09-12T03:19:20.375046+0000","last_peered":"2024-09-12T03:19:20.375046+0000","last_clean":"2024-09-12T03:19:20.375046+0000","last_became_active":"2024-09-12T03:19:09.332865+0000","last_became_peered":"2024-09-12T03:19:09.332865+0000","last_unstale":"2024-09-12T03:19:20.375046+0000","last_undegraded":"2024-09-12T03:19:20.375046+0000","last_fullsized":"2024-09-12T03:19:20.375046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:57.096138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386176+0000","last_change":"2024-09-12T03:19:07.209289+0000","last_active":"2024-09-12T03:19:20.386176+0000","last_peered":"2024-09-12T03:19:20.386176+0000","last_clean":"2024-09-12T03:19:20.386176+0000","last_became_active":"2024-09-12T03:19:07.208863+0000","last_became_peered":"2024-09-12T03:19:07.208863+0000","last_unstale":"2024-09-12T03:19:20.386176+0000","last_undegraded":"2024-09-12T03:19:20.386176+0000","last_fullsized":"2024-09-12T03:19:20.386176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:26.168148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372400+0000","last_change":"2024-09-12T03:19:09.343620+0000","last_active":"2024-09-12T03:19:20.372400+0000","last_peered":"2024-09-12T03:19:20.372400+0000","last_clean":"2024-09-12T03:19:20.372400+0000","last_became_active":"2024-09-12T03:19:09.343299+0000","last_became_peered":"2024-09-12T03:19:09.343299+0000","last_unstale":"2024-09-12T03:19:20.372400+0000","last_undegraded":"2024-09-12T03:19:20.372400+0000","last_fullsized":"2024-09-12T03:19:20.372400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:34.268886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":74,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:05.461421+0000","last_change":"2024-09-12T03:19:07.206160+0000","last_active":"2024-09-12T03:20:05.461421+0000","last_peered":"2024-09-12T03:20:05.461421+0000","last_clean":"2024-09-12T03:20:05.461421+0000","last_became_active":"2024-09-12T03:19:07.205509+0000","last_became_peered":"2024-09-12T03:19:07.205509+0000","last_unstale":"2024-09-12T03:20:05.461421+0000","last_undegraded":"2024-09-12T03:20:05.461421+0000","last_fullsized":"2024-09-12T03:20:05.461421+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:35.362831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372068+0000","last_change":"2024-09-12T03:19:09.337595+0000","last_active":"2024-09-12T03:19:20.372068+0000","last_peered":"2024-09-12T03:19:20.372068+0000","last_clean":"2024-09-12T03:19:20.372068+0000","last_became_active":"2024-09-12T03:19:09.337281+0000","last_became_peered":"2024-09-12T03:19:09.337281+0000","last_unstale":"2024-09-12T03:19:20.372068+0000","last_undegraded":"2024-09-12T03:19:20.372068+0000","last_fullsized":"2024-09-12T03:19:20.372068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:35.761679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373995+0000","last_change":"2024-09-12T03:19:07.389336+0000","last_active":"2024-09-12T03:19:20.373995+0000","last_peered":"2024-09-12T03:19:20.373995+0000","last_clean":"2024-09-12T03:19:20.373995+0000","last_became_active":"2024-09-12T03:19:07.388649+0000","last_became_peered":"2024-09-12T03:19:07.388649+0000","last_unstale":"2024-09-12T03:19:20.373995+0000","last_undegraded":"2024-09-12T03:19:20.373995+0000","last_fullsized":"2024-09-12T03:19:20.373995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:33.312700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386030+0000","last_change":"2024-09-12T03:19:09.340674+0000","last_active":"2024-09-12T03:19:20.386030+0000","last_peered":"2024-09-12T03:19:20.386030+0000","last_clean":"2024-09-12T03:19:20.386030+0000","last_became_active":"2024-09-12T03:19:09.340144+0000","last_became_peered":"2024-09-12T03:19:09.340144+0000","last_unstale":"2024-09-12T03:19:20.386030+0000","last_undegraded":"2024-09-12T03:19:20.386030+0000","last_fullsized":"2024-09-12T03:19:20.386030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:21:15.605086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373837+0000","last_change":"2024-09-12T03:19:07.206513+0000","last_active":"2024-09-12T03:19:20.373837+0000","last_peered":"2024-09-12T03:19:20.373837+0000","last_clean":"2024-09-12T03:19:20.373837+0000","last_became_active":"2024-09-12T03:19:07.205974+0000","last_became_peered":"2024-09-12T03:19:07.205974+0000","last_unstale":"2024-09-12T03:19:20.373837+0000","last_undegraded":"2024-09-12T03:19:20.373837+0000","last_fullsized":"2024-09-12T03:19:20.373837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:43:10.002123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.371052+0000","last_change":"2024-09-12T03:19:09.342200+0000","last_active":"2024-09-12T03:19:20.371052+0000","last_peered":"2024-09-12T03:19:20.371052+0000","last_clean":"2024-09-12T03:19:20.371052+0000","last_became_active":"2024-09-12T03:19:09.341901+0000","last_became_peered":"2024-09-12T03:19:09.341901+0000","last_unstale":"2024-09-12T03:19:20.371052+0000","last_undegraded":"2024-09-12T03:19:20.371052+0000","last_fullsized":"2024-09-12T03:19:20.371052+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:27.486214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375373+0000","last_change":"2024-09-12T03:19:07.207514+0000","last_active":"2024-09-12T03:19:20.375373+0000","last_peered":"2024-09-12T03:19:20.375373+0000","last_clean":"2024-09-12T03:19:20.375373+0000","last_became_active":"2024-09-12T03:19:07.205925+0000","last_became_peered":"2024-09-12T03:19:07.205925+0000","last_unstale":"2024-09-12T03:19:20.375373+0000","last_undegraded":"2024-09-12T03:19:20.375373+0000","last_fullsized":"2024-09-12T03:19:20.375373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:20.244703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372696+0000","last_change":"2024-09-12T03:19:09.337837+0000","last_active":"2024-09-12T03:19:20.372696+0000","last_peered":"2024-09-12T03:19:20.372696+0000","last_clean":"2024-09-12T03:19:20.372696+0000","last_became_active":"2024-09-12T03:19:09.337502+0000","last_became_peered":"2024-09-12T03:19:09.337502+0000","last_unstale":"2024-09-12T03:19:20.372696+0000","last_undegraded":"2024-09-12T03:19:20.372696+0000","last_fullsized":"2024-09-12T03:19:20.372696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:30.167139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386356+0000","last_change":"2024-09-12T03:19:07.209130+0000","last_active":"2024-09-12T03:19:20.386356+0000","last_peered":"2024-09-12T03:19:20.386356+0000","last_clean":"2024-09-12T03:19:20.386356+0000","last_became_active":"2024-09-12T03:19:07.208753+0000","last_became_peered":"2024-09-12T03:19:07.208753+0000","last_unstale":"2024-09-12T03:19:20.386356+0000","last_undegraded":"2024-09-12T03:19:20.386356+0000","last_fullsized":"2024-09-12T03:19:20.386356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:34.719292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372500+0000","last_change":"2024-09-12T03:19:09.336382+0000","last_active":"2024-09-12T03:19:20.372500+0000","last_peered":"2024-09-12T03:19:20.372500+0000","last_clean":"2024-09-12T03:19:20.372500+0000","last_became_active":"2024-09-12T03:19:09.336015+0000","last_became_peered":"2024-09-12T03:19:09.336015+0000","last_unstale":"2024-09-12T03:19:20.372500+0000","last_undegraded":"2024-09-12T03:19:20.372500+0000","last_fullsized":"2024-09-12T03:19:20.372500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:19.430849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375044+0000","last_change":"2024-09-12T03:19:07.528567+0000","last_active":"2024-09-12T03:19:20.375044+0000","last_peered":"2024-09-12T03:19:20.375044+0000","last_clean":"2024-09-12T03:19:20.375044+0000","last_became_active":"2024-09-12T03:19:07.528123+0000","last_became_peered":"2024-09-12T03:19:07.528123+0000","last_unstale":"2024-09-12T03:19:20.375044+0000","last_undegraded":"2024-09-12T03:19:20.375044+0000","last_fullsized":"2024-09-12T03:19:20.375044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:34.416714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372773+0000","last_change":"2024-09-12T03:19:09.338836+0000","last_active":"2024-09-12T03:19:20.372773+0000","last_peered":"2024-09-12T03:19:20.372773+0000","last_clean":"2024-09-12T03:19:20.372773+0000","last_became_active":"2024-09-12T03:19:09.337571+0000","last_became_peered":"2024-09-12T03:19:09.337571+0000","last_unstale":"2024-09-12T03:19:20.372773+0000","last_undegraded":"2024-09-12T03:19:20.372773+0000","last_fullsized":"2024-09-12T03:19:20.372773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:39.039878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":329,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372694+0000","last_change":"2024-09-12T03:18:01.051311+0000","last_active":"2024-09-12T03:19:20.372694+0000","last_peered":"2024-09-12T03:19:20.372694+0000","last_clean":"2024-09-12T03:19:20.372694+0000","last_became_active":"2024-09-12T03:18:01.050849+0000","last_became_peered":"2024-09-12T03:18:01.050849+0000","last_unstale":"2024-09-12T03:19:20.372694+0000","last_undegraded":"2024-09-12T03:19:20.372694+0000","last_fullsized":"2024-09-12T03:19:20.372694+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:14:38.096222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:14:38.096222+0000","last_clean_scrub_stamp":"2024-09-12T03:14:38.096222+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:33.067603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382035+0000","last_change":"2024-09-12T03:19:07.206045+0000","last_active":"2024-09-12T03:19:20.382035+0000","last_peered":"2024-09-12T03:19:20.382035+0000","last_clean":"2024-09-12T03:19:20.382035+0000","last_became_active":"2024-09-12T03:19:07.205594+0000","last_became_peered":"2024-09-12T03:19:07.205594+0000","last_unstale":"2024-09-12T03:19:20.382035+0000","last_undegraded":"2024-09-12T03:19:20.382035+0000","last_fullsized":"2024-09-12T03:19:20.382035+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:12.811906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.387690+0000","last_change":"2024-09-12T03:19:20.387274+0000","last_active":"2024-09-12T03:19:20.387690+0000","last_peered":"2024-09-12T03:19:20.387690+0000","last_clean":"2024-09-12T03:19:20.387690+0000","last_became_active":"2024-09-12T03:19:20.386878+0000","last_became_peered":"2024-09-12T03:19:20.386878+0000","last_unstale":"2024-09-12T03:19:20.387690+0000","last_undegraded":"2024-09-12T03:19:20.387690+0000","last_fullsized":"2024-09-12T03:19:20.387690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:26.674032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372302+0000","last_change":"2024-09-12T03:19:07.206212+0000","last_active":"2024-09-12T03:19:20.372302+0000","last_peered":"2024-09-12T03:19:20.372302+0000","last_clean":"2024-09-12T03:19:20.372302+0000","last_became_active":"2024-09-12T03:19:07.205899+0000","last_became_peered":"2024-09-12T03:19:07.205899+0000","last_unstale":"2024-09-12T03:19:20.372302+0000","last_undegraded":"2024-09-12T03:19:20.372302+0000","last_fullsized":"2024-09-12T03:19:20.372302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:02.169436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.377494+0000","last_change":"2024-09-12T03:19:20.377494+0000","last_active":"2024-09-12T03:19:20.377494+0000","last_peered":"2024-09-12T03:19:20.377494+0000","last_clean":"2024-09-12T03:19:20.377494+0000","last_became_active":"2024-09-12T03:19:20.377091+0000","last_became_peered":"2024-09-12T03:19:20.377091+0000","last_unstale":"2024-09-12T03:19:20.377494+0000","last_undegraded":"2024-09-12T03:19:20.377494+0000","last_fullsized":"2024-09-12T03:19:20.377494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:30.926853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":74,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:06.462297+0000","last_change":"2024-09-12T03:19:07.388486+0000","last_active":"2024-09-12T03:20:06.462297+0000","last_peered":"2024-09-12T03:20:06.462297+0000","last_clean":"2024-09-12T03:20:06.462297+0000","last_became_active":"2024-09-12T03:19:07.388125+0000","last_became_peered":"2024-09-12T03:19:07.388125+0000","last_unstale":"2024-09-12T03:20:06.462297+0000","last_undegraded":"2024-09-12T03:20:06.462297+0000","last_fullsized":"2024-09-12T03:20:06.462297+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:46.218601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.385912+0000","last_change":"2024-09-12T03:19:09.344308+0000","last_active":"2024-09-12T03:19:20.385912+0000","last_peered":"2024-09-12T03:19:20.385912+0000","last_clean":"2024-09-12T03:19:20.385912+0000","last_became_active":"2024-09-12T03:19:09.343987+0000","last_became_peered":"2024-09-12T03:19:09.343987+0000","last_unstale":"2024-09-12T03:19:20.385912+0000","last_undegraded":"2024-09-12T03:19:20.385912+0000","last_fullsized":"2024-09-12T03:19:20.385912+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:16:11.616555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372560+0000","last_change":"2024-09-12T03:19:07.208619+0000","last_active":"2024-09-12T03:19:20.372560+0000","last_peered":"2024-09-12T03:19:20.372560+0000","last_clean":"2024-09-12T03:19:20.372560+0000","last_became_active":"2024-09-12T03:19:07.208254+0000","last_became_peered":"2024-09-12T03:19:07.208254+0000","last_unstale":"2024-09-12T03:19:20.372560+0000","last_undegraded":"2024-09-12T03:19:20.372560+0000","last_fullsized":"2024-09-12T03:19:20.372560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:56.177948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374096+0000","last_change":"2024-09-12T03:19:09.341837+0000","last_active":"2024-09-12T03:19:20.374096+0000","last_peered":"2024-09-12T03:19:20.374096+0000","last_clean":"2024-09-12T03:19:20.374096+0000","last_became_active":"2024-09-12T03:19:09.341069+0000","last_became_peered":"2024-09-12T03:19:09.341069+0000","last_unstale":"2024-09-12T03:19:20.374096+0000","last_undegraded":"2024-09-12T03:19:20.374096+0000","last_fullsized":"2024-09-12T03:19:20.374096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:04.238986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372460+0000","last_change":"2024-09-12T03:19:07.388870+0000","last_active":"2024-09-12T03:19:20.372460+0000","last_peered":"2024-09-12T03:19:20.372460+0000","last_clean":"2024-09-12T03:19:20.372460+0000","last_became_active":"2024-09-12T03:19:07.388555+0000","last_became_peered":"2024-09-12T03:19:07.388555+0000","last_unstale":"2024-09-12T03:19:20.372460+0000","last_undegraded":"2024-09-12T03:19:20.372460+0000","last_fullsized":"2024-09-12T03:19:20.372460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:29.789454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374271+0000","last_change":"2024-09-12T03:19:09.334714+0000","last_active":"2024-09-12T03:19:20.374271+0000","last_peered":"2024-09-12T03:19:20.374271+0000","last_clean":"2024-09-12T03:19:20.374271+0000","last_became_active":"2024-09-12T03:19:09.334413+0000","last_became_peered":"2024-09-12T03:19:09.334413+0000","last_unstale":"2024-09-12T03:19:20.374271+0000","last_undegraded":"2024-09-12T03:19:20.374271+0000","last_fullsized":"2024-09-12T03:19:20.374271+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:52.222340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373797+0000","last_change":"2024-09-12T03:19:07.389055+0000","last_active":"2024-09-12T03:19:20.373797+0000","last_peered":"2024-09-12T03:19:20.373797+0000","last_clean":"2024-09-12T03:19:20.373797+0000","last_became_active":"2024-09-12T03:19:07.388673+0000","last_became_peered":"2024-09-12T03:19:07.388673+0000","last_unstale":"2024-09-12T03:19:20.373797+0000","last_undegraded":"2024-09-12T03:19:20.373797+0000","last_fullsized":"2024-09-12T03:19:20.373797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:20.974533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375304+0000","last_change":"2024-09-12T03:19:09.340105+0000","last_active":"2024-09-12T03:19:20.375304+0000","last_peered":"2024-09-12T03:19:20.375304+0000","last_clean":"2024-09-12T03:19:20.375304+0000","last_became_active":"2024-09-12T03:19:09.339642+0000","last_became_peered":"2024-09-12T03:19:09.339642+0000","last_unstale":"2024-09-12T03:19:20.375304+0000","last_undegraded":"2024-09-12T03:19:20.375304+0000","last_fullsized":"2024-09-12T03:19:20.375304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:01.624236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'12","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:33.331973+0000","last_change":"2024-09-12T03:19:07.207232+0000","last_active":"2024-09-12T03:19:33.331973+0000","last_peered":"2024-09-12T03:19:33.331973+0000","last_clean":"2024-09-12T03:19:33.331973+0000","last_became_active":"2024-09-12T03:19:07.206921+0000","last_became_peered":"2024-09-12T03:19:07.206921+0000","last_unstale":"2024-09-12T03:19:33.331973+0000","last_undegraded":"2024-09-12T03:19:33.331973+0000","last_fullsized":"2024-09-12T03:19:33.331973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:25.457853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372958+0000","last_change":"2024-09-12T03:19:09.330442+0000","last_active":"2024-09-12T03:19:20.372958+0000","last_peered":"2024-09-12T03:19:20.372958+0000","last_clean":"2024-09-12T03:19:20.372958+0000","last_became_active":"2024-09-12T03:19:09.330081+0000","last_became_peered":"2024-09-12T03:19:09.330081+0000","last_unstale":"2024-09-12T03:19:20.372958+0000","last_undegraded":"2024-09-12T03:19:20.372958+0000","last_fullsized":"2024-09-12T03:19:20.372958+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:56.152562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382392+0000","last_change":"2024-09-12T03:19:07.528646+0000","last_active":"2024-09-12T03:19:20.382392+0000","last_peered":"2024-09-12T03:19:20.382392+0000","last_clean":"2024-09-12T03:19:20.382392+0000","last_became_active":"2024-09-12T03:19:07.528334+0000","last_became_peered":"2024-09-12T03:19:07.528334+0000","last_unstale":"2024-09-12T03:19:20.382392+0000","last_undegraded":"2024-09-12T03:19:20.382392+0000","last_fullsized":"2024-09-12T03:19:20.382392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:40.425579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408541+0000","last_change":"2024-09-12T03:19:09.331993+0000","last_active":"2024-09-12T03:19:20.408541+0000","last_peered":"2024-09-12T03:19:20.408541+0000","last_clean":"2024-09-12T03:19:20.408541+0000","last_became_active":"2024-09-12T03:19:09.331698+0000","last_became_peered":"2024-09-12T03:19:09.331698+0000","last_unstale":"2024-09-12T03:19:20.408541+0000","last_undegraded":"2024-09-12T03:19:20.408541+0000","last_fullsized":"2024-09-12T03:19:20.408541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:03:54.182833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370179+0000","last_change":"2024-09-12T03:19:07.388297+0000","last_active":"2024-09-12T03:19:20.370179+0000","last_peered":"2024-09-12T03:19:20.370179+0000","last_clean":"2024-09-12T03:19:20.370179+0000","last_became_active":"2024-09-12T03:19:07.388038+0000","last_became_peered":"2024-09-12T03:19:07.388038+0000","last_unstale":"2024-09-12T03:19:20.370179+0000","last_undegraded":"2024-09-12T03:19:20.370179+0000","last_fullsized":"2024-09-12T03:19:20.370179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:45.874195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373463+0000","last_change":"2024-09-12T03:19:09.331525+0000","last_active":"2024-09-12T03:19:20.373463+0000","last_peered":"2024-09-12T03:19:20.373463+0000","last_clean":"2024-09-12T03:19:20.373463+0000","last_became_active":"2024-09-12T03:19:09.330903+0000","last_became_peered":"2024-09-12T03:19:09.330903+0000","last_unstale":"2024-09-12T03:19:20.373463+0000","last_undegraded":"2024-09-12T03:19:20.373463+0000","last_fullsized":"2024-09-12T03:19:20.373463+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:02.455132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372235+0000","last_change":"2024-09-12T03:19:07.205181+0000","last_active":"2024-09-12T03:19:20.372235+0000","last_peered":"2024-09-12T03:19:20.372235+0000","last_clean":"2024-09-12T03:19:20.372235+0000","last_became_active":"2024-09-12T03:19:07.204726+0000","last_became_peered":"2024-09-12T03:19:07.204726+0000","last_unstale":"2024-09-12T03:19:20.372235+0000","last_undegraded":"2024-09-12T03:19:20.372235+0000","last_fullsized":"2024-09-12T03:19:20.372235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:26.112800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370908+0000","last_change":"2024-09-12T03:19:09.345338+0000","last_active":"2024-09-12T03:19:20.370908+0000","last_peered":"2024-09-12T03:19:20.370908+0000","last_clean":"2024-09-12T03:19:20.370908+0000","last_became_active":"2024-09-12T03:19:09.345023+0000","last_became_peered":"2024-09-12T03:19:09.345023+0000","last_unstale":"2024-09-12T03:19:20.370908+0000","last_undegraded":"2024-09-12T03:19:20.370908+0000","last_fullsized":"2024-09-12T03:19:20.370908+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:39.184065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374938+0000","last_change":"2024-09-12T03:19:07.211954+0000","last_active":"2024-09-12T03:19:20.374938+0000","last_peered":"2024-09-12T03:19:20.374938+0000","last_clean":"2024-09-12T03:19:20.374938+0000","last_became_active":"2024-09-12T03:19:07.211509+0000","last_became_peered":"2024-09-12T03:19:07.211509+0000","last_unstale":"2024-09-12T03:19:20.374938+0000","last_undegraded":"2024-09-12T03:19:20.374938+0000","last_fullsized":"2024-09-12T03:19:20.374938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:27.461924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373018+0000","last_change":"2024-09-12T03:19:09.331168+0000","last_active":"2024-09-12T03:19:20.373018+0000","last_peered":"2024-09-12T03:19:20.373018+0000","last_clean":"2024-09-12T03:19:20.373018+0000","last_became_active":"2024-09-12T03:19:09.329592+0000","last_became_peered":"2024-09-12T03:19:09.329592+0000","last_unstale":"2024-09-12T03:19:20.373018+0000","last_undegraded":"2024-09-12T03:19:20.373018+0000","last_fullsized":"2024-09-12T03:19:20.373018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:46.127861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373572+0000","last_change":"2024-09-12T03:19:07.207802+0000","last_active":"2024-09-12T03:19:20.373572+0000","last_peered":"2024-09-12T03:19:20.373572+0000","last_clean":"2024-09-12T03:19:20.373572+0000","last_became_active":"2024-09-12T03:19:07.207428+0000","last_became_peered":"2024-09-12T03:19:07.207428+0000","last_unstale":"2024-09-12T03:19:20.373572+0000","last_undegraded":"2024-09-12T03:19:20.373572+0000","last_fullsized":"2024-09-12T03:19:20.373572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:45.135019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372336+0000","last_change":"2024-09-12T03:19:09.336094+0000","last_active":"2024-09-12T03:19:20.372336+0000","last_peered":"2024-09-12T03:19:20.372336+0000","last_clean":"2024-09-12T03:19:20.372336+0000","last_became_active":"2024-09-12T03:19:09.335779+0000","last_became_peered":"2024-09-12T03:19:09.335779+0000","last_unstale":"2024-09-12T03:19:20.372336+0000","last_undegraded":"2024-09-12T03:19:20.372336+0000","last_fullsized":"2024-09-12T03:19:20.372336+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:37.019082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386131+0000","last_change":"2024-09-12T03:19:09.344537+0000","last_active":"2024-09-12T03:19:20.386131+0000","last_peered":"2024-09-12T03:19:20.386131+0000","last_clean":"2024-09-12T03:19:20.386131+0000","last_became_active":"2024-09-12T03:19:09.343879+0000","last_became_peered":"2024-09-12T03:19:09.343879+0000","last_unstale":"2024-09-12T03:19:20.386131+0000","last_undegraded":"2024-09-12T03:19:20.386131+0000","last_fullsized":"2024-09-12T03:19:20.386131+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:40.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373973+0000","last_change":"2024-09-12T03:19:07.208040+0000","last_active":"2024-09-12T03:19:20.373973+0000","last_peered":"2024-09-12T03:19:20.373973+0000","last_clean":"2024-09-12T03:19:20.373973+0000","last_became_active":"2024-09-12T03:19:07.207658+0000","last_became_peered":"2024-09-12T03:19:07.207658+0000","last_unstale":"2024-09-12T03:19:20.373973+0000","last_undegraded":"2024-09-12T03:19:20.373973+0000","last_fullsized":"2024-09-12T03:19:20.373973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:48.620707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370231+0000","last_change":"2024-09-12T03:19:09.333837+0000","last_active":"2024-09-12T03:19:20.370231+0000","last_peered":"2024-09-12T03:19:20.370231+0000","last_clean":"2024-09-12T03:19:20.370231+0000","last_became_active":"2024-09-12T03:19:09.333194+0000","last_became_peered":"2024-09-12T03:19:09.333194+0000","last_unstale":"2024-09-12T03:19:20.370231+0000","last_undegraded":"2024-09-12T03:19:20.370231+0000","last_fullsized":"2024-09-12T03:19:20.370231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:18.766181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.385455+0000","last_change":"2024-09-12T03:19:20.384978+0000","last_active":"2024-09-12T03:19:20.385455+0000","last_peered":"2024-09-12T03:19:20.385455+0000","last_clean":"2024-09-12T03:19:20.385455+0000","last_became_active":"2024-09-12T03:19:20.376883+0000","last_became_peered":"2024-09-12T03:19:20.376883+0000","last_unstale":"2024-09-12T03:19:20.385455+0000","last_undegraded":"2024-09-12T03:19:20.385455+0000","last_fullsized":"2024-09-12T03:19:20.385455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:32.542334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408633+0000","last_change":"2024-09-12T03:19:09.334684+0000","last_active":"2024-09-12T03:19:20.408633+0000","last_peered":"2024-09-12T03:19:20.408633+0000","last_clean":"2024-09-12T03:19:20.408633+0000","last_became_active":"2024-09-12T03:19:09.334281+0000","last_became_peered":"2024-09-12T03:19:09.334281+0000","last_unstale":"2024-09-12T03:19:20.408633+0000","last_undegraded":"2024-09-12T03:19:20.408633+0000","last_fullsized":"2024-09-12T03:19:20.408633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:44.837067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372829+0000","last_change":"2024-09-12T03:19:07.388555+0000","last_active":"2024-09-12T03:19:20.372829+0000","last_peered":"2024-09-12T03:19:20.372829+0000","last_clean":"2024-09-12T03:19:20.372829+0000","last_became_active":"2024-09-12T03:19:07.388206+0000","last_became_peered":"2024-09-12T03:19:07.388206+0000","last_unstale":"2024-09-12T03:19:20.372829+0000","last_undegraded":"2024-09-12T03:19:20.372829+0000","last_fullsized":"2024-09-12T03:19:20.372829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:33.628294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372626+0000","last_change":"2024-09-12T03:19:09.339108+0000","last_active":"2024-09-12T03:19:20.372626+0000","last_peered":"2024-09-12T03:19:20.372626+0000","last_clean":"2024-09-12T03:19:20.372626+0000","last_became_active":"2024-09-12T03:19:09.338819+0000","last_became_peered":"2024-09-12T03:19:09.338819+0000","last_unstale":"2024-09-12T03:19:20.372626+0000","last_undegraded":"2024-09-12T03:19:20.372626+0000","last_fullsized":"2024-09-12T03:19:20.372626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:32.864900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372798+0000","last_change":"2024-09-12T03:19:07.207197+0000","last_active":"2024-09-12T03:19:20.372798+0000","last_peered":"2024-09-12T03:19:20.372798+0000","last_clean":"2024-09-12T03:19:20.372798+0000","last_became_active":"2024-09-12T03:19:07.206879+0000","last_became_peered":"2024-09-12T03:19:07.206879+0000","last_unstale":"2024-09-12T03:19:20.372798+0000","last_undegraded":"2024-09-12T03:19:20.372798+0000","last_fullsized":"2024-09-12T03:19:20.372798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:00:46.269636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63767,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":320,"num_read_kb":226,"num_write":164,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":191301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579929,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":892928,"data_stored":433387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808862,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36892,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708260,"statfs":{"total":95995035648,"available":95957258240,"internally_reserved":0,"allocated":1585152,"data_stored":1084598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972386,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":433589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201320,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":433509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430253,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1499136,"data_stored":1025441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593778,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":432978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822711,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1511424,"data_stored":1025125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":897024,"data_stored":433415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":433289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":494244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40436,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704716,"statfs":{"total":95995035648,"available":95953629184,"internally_reserved":0,"allocated":954368,"data_stored":492535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":913408,"data_stored":434249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:20:12.694 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:20:12.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:12 smithi118 ceph-mon[35048]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:12.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:12 smithi118 ceph-mon[35048]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:20:12.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:12 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:20:12.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:12 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:20:12.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:12 smithi161 ceph-mon[35485]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:12.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:12 smithi161 ceph-mon[35485]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:20:12.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:12 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:20:12.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:12 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:20:12.968 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:13.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.441+0000 7f67136ae640 1 -- 172.21.15.6:0/3047569783 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f670c106f10 msgr2=0x7f670c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.441+0000 7f67136ae640 1 --2- 172.21.15.6:0/3047569783 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f670c106f10 0x7f670c10ea90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6704009ca0 tx=0x7f670402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.441+0000 7f67136ae640 1 -- 172.21.15.6:0/3047569783 shutdown_connections 2024-09-12T03:20:13.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.441+0000 7f67136ae640 1 --2- 172.21.15.6:0/3047569783 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f670c106f10 0x7f670c10ea90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.441+0000 7f67136ae640 1 --2- 172.21.15.6:0/3047569783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f670c102790 0x7f670c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.441+0000 7f67136ae640 1 --2- 172.21.15.6:0/3047569783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f670c101de0 0x7f670c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.441+0000 7f67136ae640 1 -- 172.21.15.6:0/3047569783 >> 172.21.15.6:0/3047569783 conn(0x7f670c0fd8c0 msgr2=0x7f670c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:13.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.441+0000 7f67136ae640 1 -- 172.21.15.6:0/3047569783 shutdown_connections 2024-09-12T03:20:13.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.441+0000 7f67136ae640 1 -- 172.21.15.6:0/3047569783 wait complete. 2024-09-12T03:20:13.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.442+0000 7f67136ae640 1 Processor -- start 2024-09-12T03:20:13.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.442+0000 7f67136ae640 1 -- start start 2024-09-12T03:20:13.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.442+0000 7f67136ae640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f670c101de0 0x7f670c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.442+0000 7f67136ae640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f670c102790 0x7f670c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.442+0000 7f67136ae640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f670c106f10 0x7f670c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.442+0000 7f67136ae640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f670c111f70 con 0x7f670c106f10 2024-09-12T03:20:13.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.442+0000 7f67136ae640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f670c111df0 con 0x7f670c102790 2024-09-12T03:20:13.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.442+0000 7f67136ae640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f670c1120f0 con 0x7f670c101de0 2024-09-12T03:20:13.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.442+0000 7f6711c4b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f670c106f10 0x7f670c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.442+0000 7f6711c4b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f670c106f10 0x7f670c19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53900/0 (socket says 172.21.15.6:53900) 2024-09-12T03:20:13.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.442+0000 7f6711c4b640 1 -- 172.21.15.6:0/991026474 learned_addr learned my addr 172.21.15.6:0/991026474 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:13.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.442+0000 7f6710c49640 1 --2- 172.21.15.6:0/991026474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f670c102790 0x7f670c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.442+0000 7f671144a640 1 --2- 172.21.15.6:0/991026474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f670c101de0 0x7f670c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.443+0000 7f6711c4b640 1 -- 172.21.15.6:0/991026474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f670c101de0 msgr2=0x7f670c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.443+0000 7f6711c4b640 1 --2- 172.21.15.6:0/991026474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f670c101de0 0x7f670c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.443+0000 7f6711c4b640 1 -- 172.21.15.6:0/991026474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f670c102790 msgr2=0x7f670c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.443+0000 7f6711c4b640 1 --2- 172.21.15.6:0/991026474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f670c102790 0x7f670c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.443+0000 7f6711c4b640 1 -- 172.21.15.6:0/991026474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670c19d3a0 con 0x7f670c106f10 2024-09-12T03:20:13.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.443+0000 7f6711c4b640 1 --2- 172.21.15.6:0/991026474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f670c106f10 0x7f670c19ccc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6704031a20 tx=0x7f6704031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:13.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.443+0000 7f67027fc640 1 -- 172.21.15.6:0/991026474 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704003c30 con 0x7f670c106f10 2024-09-12T03:20:13.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.443+0000 7f67027fc640 1 -- 172.21.15.6:0/991026474 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6704003dd0 con 0x7f670c106f10 2024-09-12T03:20:13.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.443+0000 7f67136ae640 1 -- 172.21.15.6:0/991026474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670c19d630 con 0x7f670c106f10 2024-09-12T03:20:13.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.443+0000 7f67027fc640 1 -- 172.21.15.6:0/991026474 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670402fc40 con 0x7f670c106f10 2024-09-12T03:20:13.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.443+0000 7f67136ae640 1 -- 172.21.15.6:0/991026474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f670c1a4f70 con 0x7f670c106f10 2024-09-12T03:20:13.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.444+0000 7f67136ae640 1 -- 172.21.15.6:0/991026474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f670c1021c0 con 0x7f670c106f10 2024-09-12T03:20:13.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.445+0000 7f67027fc640 1 -- 172.21.15.6:0/991026474 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6704004050 con 0x7f670c106f10 2024-09-12T03:20:13.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.446+0000 7f67027fc640 1 --2- 172.21.15.6:0/991026474 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f66f80a2c90 0x7f66f80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.446+0000 7f67027fc640 1 -- 172.21.15.6:0/991026474 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f67040f0170 con 0x7f670c106f10 2024-09-12T03:20:13.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.446+0000 7f671144a640 1 --2- 172.21.15.6:0/991026474 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f66f80a2c90 0x7f66f80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.446+0000 7f671144a640 1 --2- 172.21.15.6:0/991026474 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f66f80a2c90 0x7f66f80a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6708009bd0 tx=0x7f6708005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:13.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.450+0000 7f67027fc640 1 -- 172.21.15.6:0/991026474 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f67040bd260 con 0x7f670c106f10 2024-09-12T03:20:13.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:13 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:13.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:13 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:13.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:13 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:20:13.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.623+0000 7f67136ae640 1 -- 172.21.15.6:0/991026474 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f670c062630 con 0x7f66f80a2c90 2024-09-12T03:20:13.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.629+0000 7f67027fc640 1 -- 172.21.15.6:0/991026474 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258717 (secure 0 0 0) 0x7f670c062630 con 0x7f66f80a2c90 2024-09-12T03:20:13.635 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:20:13.636 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-09-12T03:20:13.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.633+0000 7f67136ae640 1 -- 172.21.15.6:0/991026474 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f66f80a2c90 msgr2=0x7f66f80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.633+0000 7f67136ae640 1 --2- 172.21.15.6:0/991026474 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f66f80a2c90 0x7f66f80a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6708009bd0 tx=0x7f6708005f90 comp rx=0 tx=0).stop 2024-09-12T03:20:13.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.633+0000 7f67136ae640 1 -- 172.21.15.6:0/991026474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f670c106f10 msgr2=0x7f670c19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.633+0000 7f67136ae640 1 --2- 172.21.15.6:0/991026474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f670c106f10 0x7f670c19ccc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6704031a20 tx=0x7f6704031a50 comp rx=0 tx=0).stop 2024-09-12T03:20:13.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.633+0000 7f67136ae640 1 -- 172.21.15.6:0/991026474 shutdown_connections 2024-09-12T03:20:13.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.633+0000 7f67136ae640 1 --2- 172.21.15.6:0/991026474 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f66f80a2c90 0x7f66f80a5150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.633+0000 7f67136ae640 1 --2- 172.21.15.6:0/991026474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f670c106f10 0x7f670c19ccc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.633+0000 7f67136ae640 1 --2- 172.21.15.6:0/991026474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f670c102790 0x7f670c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.633+0000 7f67136ae640 1 --2- 172.21.15.6:0/991026474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f670c101de0 0x7f670c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.633+0000 7f67136ae640 1 -- 172.21.15.6:0/991026474 >> 172.21.15.6:0/991026474 conn(0x7f670c0fd8c0 msgr2=0x7f670c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:13.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.633+0000 7f67136ae640 1 -- 172.21.15.6:0/991026474 shutdown_connections 2024-09-12T03:20:13.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:13.633+0000 7f67136ae640 1 -- 172.21.15.6:0/991026474 wait complete. 2024-09-12T03:20:13.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:13 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:13.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:13 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:13.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:13 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:20:13.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:13 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:13.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:13 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:13.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:13 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:20:14.284 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2024-09-12T03:20:13.252091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654369,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":471,"num_read_kb":350,"num_write":347,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382608,"kb_used_data":12540,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559216,"statfs":{"total":1151940427776,"available":1151548637184,"internally_reserved":0,"allocated":12840960,"data_stored":7156359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002915"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.371249+0000","last_change":"2024-09-12T03:19:09.343829+0000","last_active":"2024-09-12T03:19:20.371249+0000","last_peered":"2024-09-12T03:19:20.371249+0000","last_clean":"2024-09-12T03:19:20.371249+0000","last_became_active":"2024-09-12T03:19:09.343228+0000","last_became_peered":"2024-09-12T03:19:09.343228+0000","last_unstale":"2024-09-12T03:19:20.371249+0000","last_undegraded":"2024-09-12T03:19:20.371249+0000","last_fullsized":"2024-09-12T03:19:20.371249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:08.720934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408528+0000","last_change":"2024-09-12T03:19:09.331829+0000","last_active":"2024-09-12T03:19:20.408528+0000","last_peered":"2024-09-12T03:19:20.408528+0000","last_clean":"2024-09-12T03:19:20.408528+0000","last_became_active":"2024-09-12T03:19:09.331276+0000","last_became_peered":"2024-09-12T03:19:09.331276+0000","last_unstale":"2024-09-12T03:19:20.408528+0000","last_undegraded":"2024-09-12T03:19:20.408528+0000","last_fullsized":"2024-09-12T03:19:20.408528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:48.672085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408509+0000","last_change":"2024-09-12T03:19:09.338120+0000","last_active":"2024-09-12T03:19:20.408509+0000","last_peered":"2024-09-12T03:19:20.408509+0000","last_clean":"2024-09-12T03:19:20.408509+0000","last_became_active":"2024-09-12T03:19:09.337773+0000","last_became_peered":"2024-09-12T03:19:09.337773+0000","last_unstale":"2024-09-12T03:19:20.408509+0000","last_undegraded":"2024-09-12T03:19:20.408509+0000","last_fullsized":"2024-09-12T03:19:20.408509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:13.716047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382087+0000","last_change":"2024-09-12T03:19:09.339404+0000","last_active":"2024-09-12T03:19:20.382087+0000","last_peered":"2024-09-12T03:19:20.382087+0000","last_clean":"2024-09-12T03:19:20.382087+0000","last_became_active":"2024-09-12T03:19:09.339105+0000","last_became_peered":"2024-09-12T03:19:09.339105+0000","last_unstale":"2024-09-12T03:19:20.382087+0000","last_undegraded":"2024-09-12T03:19:20.382087+0000","last_fullsized":"2024-09-12T03:19:20.382087+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:37.930963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373401+0000","last_change":"2024-09-12T03:19:09.326285+0000","last_active":"2024-09-12T03:19:20.373401+0000","last_peered":"2024-09-12T03:19:20.373401+0000","last_clean":"2024-09-12T03:19:20.373401+0000","last_became_active":"2024-09-12T03:19:09.325852+0000","last_became_peered":"2024-09-12T03:19:09.325852+0000","last_unstale":"2024-09-12T03:19:20.373401+0000","last_undegraded":"2024-09-12T03:19:20.373401+0000","last_fullsized":"2024-09-12T03:19:20.373401+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:44:25.387014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.376197+0000","last_change":"2024-09-12T03:19:20.376197+0000","last_active":"2024-09-12T03:19:20.376197+0000","last_peered":"2024-09-12T03:19:20.376197+0000","last_clean":"2024-09-12T03:19:20.376197+0000","last_became_active":"2024-09-12T03:19:20.375744+0000","last_became_peered":"2024-09-12T03:19:20.375744+0000","last_unstale":"2024-09-12T03:19:20.376197+0000","last_undegraded":"2024-09-12T03:19:20.376197+0000","last_fullsized":"2024-09-12T03:19:20.376197+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:08.663121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372152+0000","last_change":"2024-09-12T03:19:09.335284+0000","last_active":"2024-09-12T03:19:20.372152+0000","last_peered":"2024-09-12T03:19:20.372152+0000","last_clean":"2024-09-12T03:19:20.372152+0000","last_became_active":"2024-09-12T03:19:09.334948+0000","last_became_peered":"2024-09-12T03:19:09.334948+0000","last_unstale":"2024-09-12T03:19:20.372152+0000","last_undegraded":"2024-09-12T03:19:20.372152+0000","last_fullsized":"2024-09-12T03:19:20.372152+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:32.778433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372122+0000","last_change":"2024-09-12T03:19:09.338398+0000","last_active":"2024-09-12T03:19:20.372122+0000","last_peered":"2024-09-12T03:19:20.372122+0000","last_clean":"2024-09-12T03:19:20.372122+0000","last_became_active":"2024-09-12T03:19:09.337750+0000","last_became_peered":"2024-09-12T03:19:09.337750+0000","last_unstale":"2024-09-12T03:19:20.372122+0000","last_undegraded":"2024-09-12T03:19:20.372122+0000","last_fullsized":"2024-09-12T03:19:20.372122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:07.542322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370699+0000","last_change":"2024-09-12T03:19:09.340703+0000","last_active":"2024-09-12T03:19:20.370699+0000","last_peered":"2024-09-12T03:19:20.370699+0000","last_clean":"2024-09-12T03:19:20.370699+0000","last_became_active":"2024-09-12T03:19:09.340386+0000","last_became_peered":"2024-09-12T03:19:09.340386+0000","last_unstale":"2024-09-12T03:19:20.370699+0000","last_undegraded":"2024-09-12T03:19:20.370699+0000","last_fullsized":"2024-09-12T03:19:20.370699+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:19.990894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374331+0000","last_change":"2024-09-12T03:19:09.341489+0000","last_active":"2024-09-12T03:19:20.374331+0000","last_peered":"2024-09-12T03:19:20.374331+0000","last_clean":"2024-09-12T03:19:20.374331+0000","last_became_active":"2024-09-12T03:19:09.341183+0000","last_became_peered":"2024-09-12T03:19:09.341183+0000","last_unstale":"2024-09-12T03:19:20.374331+0000","last_undegraded":"2024-09-12T03:19:20.374331+0000","last_fullsized":"2024-09-12T03:19:20.374331+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:05.688402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372589+0000","last_change":"2024-09-12T03:19:09.339289+0000","last_active":"2024-09-12T03:19:20.372589+0000","last_peered":"2024-09-12T03:19:20.372589+0000","last_clean":"2024-09-12T03:19:20.372589+0000","last_became_active":"2024-09-12T03:19:09.338466+0000","last_became_peered":"2024-09-12T03:19:09.338466+0000","last_unstale":"2024-09-12T03:19:20.372589+0000","last_undegraded":"2024-09-12T03:19:20.372589+0000","last_fullsized":"2024-09-12T03:19:20.372589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:47:13.070057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382092+0000","last_change":"2024-09-12T03:19:09.337754+0000","last_active":"2024-09-12T03:19:20.382092+0000","last_peered":"2024-09-12T03:19:20.382092+0000","last_clean":"2024-09-12T03:19:20.382092+0000","last_became_active":"2024-09-12T03:19:09.332789+0000","last_became_peered":"2024-09-12T03:19:09.332789+0000","last_unstale":"2024-09-12T03:19:20.382092+0000","last_undegraded":"2024-09-12T03:19:20.382092+0000","last_fullsized":"2024-09-12T03:19:20.382092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:34.622136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375297+0000","last_change":"2024-09-12T03:19:09.333577+0000","last_active":"2024-09-12T03:19:20.375297+0000","last_peered":"2024-09-12T03:19:20.375297+0000","last_clean":"2024-09-12T03:19:20.375297+0000","last_became_active":"2024-09-12T03:19:09.333234+0000","last_became_peered":"2024-09-12T03:19:09.333234+0000","last_unstale":"2024-09-12T03:19:20.375297+0000","last_undegraded":"2024-09-12T03:19:20.375297+0000","last_fullsized":"2024-09-12T03:19:20.375297+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:22.604640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408267+0000","last_change":"2024-09-12T03:19:09.331607+0000","last_active":"2024-09-12T03:19:20.408267+0000","last_peered":"2024-09-12T03:19:20.408267+0000","last_clean":"2024-09-12T03:19:20.408267+0000","last_became_active":"2024-09-12T03:19:09.331253+0000","last_became_peered":"2024-09-12T03:19:09.331253+0000","last_unstale":"2024-09-12T03:19:20.408267+0000","last_undegraded":"2024-09-12T03:19:20.408267+0000","last_fullsized":"2024-09-12T03:19:20.408267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:11.969265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408245+0000","last_change":"2024-09-12T03:19:09.340768+0000","last_active":"2024-09-12T03:19:20.408245+0000","last_peered":"2024-09-12T03:19:20.408245+0000","last_clean":"2024-09-12T03:19:20.408245+0000","last_became_active":"2024-09-12T03:19:09.340437+0000","last_became_peered":"2024-09-12T03:19:09.340437+0000","last_unstale":"2024-09-12T03:19:20.408245+0000","last_undegraded":"2024-09-12T03:19:20.408245+0000","last_fullsized":"2024-09-12T03:19:20.408245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:32.645691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370081+0000","last_change":"2024-09-12T03:19:09.333829+0000","last_active":"2024-09-12T03:19:20.370081+0000","last_peered":"2024-09-12T03:19:20.370081+0000","last_clean":"2024-09-12T03:19:20.370081+0000","last_became_active":"2024-09-12T03:19:09.333462+0000","last_became_peered":"2024-09-12T03:19:09.333462+0000","last_unstale":"2024-09-12T03:19:20.370081+0000","last_undegraded":"2024-09-12T03:19:20.370081+0000","last_fullsized":"2024-09-12T03:19:20.370081+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:18:57.586836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.371273+0000","last_change":"2024-09-12T03:19:09.344068+0000","last_active":"2024-09-12T03:19:20.371273+0000","last_peered":"2024-09-12T03:19:20.371273+0000","last_clean":"2024-09-12T03:19:20.371273+0000","last_became_active":"2024-09-12T03:19:09.343751+0000","last_became_peered":"2024-09-12T03:19:09.343751+0000","last_unstale":"2024-09-12T03:19:20.371273+0000","last_undegraded":"2024-09-12T03:19:20.371273+0000","last_fullsized":"2024-09-12T03:19:20.371273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:52.425547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372644+0000","last_change":"2024-09-12T03:19:09.330828+0000","last_active":"2024-09-12T03:19:20.372644+0000","last_peered":"2024-09-12T03:19:20.372644+0000","last_clean":"2024-09-12T03:19:20.372644+0000","last_became_active":"2024-09-12T03:19:09.330497+0000","last_became_peered":"2024-09-12T03:19:09.330497+0000","last_unstale":"2024-09-12T03:19:20.372644+0000","last_undegraded":"2024-09-12T03:19:20.372644+0000","last_fullsized":"2024-09-12T03:19:20.372644+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:21:20.064177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375155+0000","last_change":"2024-09-12T03:19:09.344403+0000","last_active":"2024-09-12T03:19:20.375155+0000","last_peered":"2024-09-12T03:19:20.375155+0000","last_clean":"2024-09-12T03:19:20.375155+0000","last_became_active":"2024-09-12T03:19:09.343946+0000","last_became_peered":"2024-09-12T03:19:09.343946+0000","last_unstale":"2024-09-12T03:19:20.375155+0000","last_undegraded":"2024-09-12T03:19:20.375155+0000","last_fullsized":"2024-09-12T03:19:20.375155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:48.386117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370884+0000","last_change":"2024-09-12T03:19:09.341481+0000","last_active":"2024-09-12T03:19:20.370884+0000","last_peered":"2024-09-12T03:19:20.370884+0000","last_clean":"2024-09-12T03:19:20.370884+0000","last_became_active":"2024-09-12T03:19:09.341142+0000","last_became_peered":"2024-09-12T03:19:09.341142+0000","last_unstale":"2024-09-12T03:19:20.370884+0000","last_undegraded":"2024-09-12T03:19:20.370884+0000","last_fullsized":"2024-09-12T03:19:20.370884+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:42.371221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375324+0000","last_change":"2024-09-12T03:19:09.330621+0000","last_active":"2024-09-12T03:19:20.375324+0000","last_peered":"2024-09-12T03:19:20.375324+0000","last_clean":"2024-09-12T03:19:20.375324+0000","last_became_active":"2024-09-12T03:19:09.330169+0000","last_became_peered":"2024-09-12T03:19:09.330169+0000","last_unstale":"2024-09-12T03:19:20.375324+0000","last_undegraded":"2024-09-12T03:19:20.375324+0000","last_fullsized":"2024-09-12T03:19:20.375324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:16.634141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370190+0000","last_change":"2024-09-12T03:19:09.332817+0000","last_active":"2024-09-12T03:19:20.370190+0000","last_peered":"2024-09-12T03:19:20.370190+0000","last_clean":"2024-09-12T03:19:20.370190+0000","last_became_active":"2024-09-12T03:19:09.332236+0000","last_became_peered":"2024-09-12T03:19:09.332236+0000","last_unstale":"2024-09-12T03:19:20.370190+0000","last_undegraded":"2024-09-12T03:19:20.370190+0000","last_fullsized":"2024-09-12T03:19:20.370190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:28.024294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372100+0000","last_change":"2024-09-12T03:19:09.337910+0000","last_active":"2024-09-12T03:19:20.372100+0000","last_peered":"2024-09-12T03:19:20.372100+0000","last_clean":"2024-09-12T03:19:20.372100+0000","last_became_active":"2024-09-12T03:19:09.337176+0000","last_became_peered":"2024-09-12T03:19:09.337176+0000","last_unstale":"2024-09-12T03:19:20.372100+0000","last_undegraded":"2024-09-12T03:19:20.372100+0000","last_fullsized":"2024-09-12T03:19:20.372100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:53.795898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.385889+0000","last_change":"2024-09-12T03:19:09.337902+0000","last_active":"2024-09-12T03:19:20.385889+0000","last_peered":"2024-09-12T03:19:20.385889+0000","last_clean":"2024-09-12T03:19:20.385889+0000","last_became_active":"2024-09-12T03:19:09.337567+0000","last_became_peered":"2024-09-12T03:19:09.337567+0000","last_unstale":"2024-09-12T03:19:20.385889+0000","last_undegraded":"2024-09-12T03:19:20.385889+0000","last_fullsized":"2024-09-12T03:19:20.385889+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:08.471313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372957+0000","last_change":"2024-09-12T03:19:09.339850+0000","last_active":"2024-09-12T03:19:20.372957+0000","last_peered":"2024-09-12T03:19:20.372957+0000","last_clean":"2024-09-12T03:19:20.372957+0000","last_became_active":"2024-09-12T03:19:09.339563+0000","last_became_peered":"2024-09-12T03:19:09.339563+0000","last_unstale":"2024-09-12T03:19:20.372957+0000","last_undegraded":"2024-09-12T03:19:20.372957+0000","last_fullsized":"2024-09-12T03:19:20.372957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:27.473351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373965+0000","last_change":"2024-09-12T03:19:09.330555+0000","last_active":"2024-09-12T03:19:20.373965+0000","last_peered":"2024-09-12T03:19:20.373965+0000","last_clean":"2024-09-12T03:19:20.373965+0000","last_became_active":"2024-09-12T03:19:09.330183+0000","last_became_peered":"2024-09-12T03:19:09.330183+0000","last_unstale":"2024-09-12T03:19:20.373965+0000","last_undegraded":"2024-09-12T03:19:20.373965+0000","last_fullsized":"2024-09-12T03:19:20.373965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:39.334116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.371187+0000","last_change":"2024-09-12T03:19:09.338601+0000","last_active":"2024-09-12T03:19:20.371187+0000","last_peered":"2024-09-12T03:19:20.371187+0000","last_clean":"2024-09-12T03:19:20.371187+0000","last_became_active":"2024-09-12T03:19:09.337903+0000","last_became_peered":"2024-09-12T03:19:09.337903+0000","last_unstale":"2024-09-12T03:19:20.371187+0000","last_undegraded":"2024-09-12T03:19:20.371187+0000","last_fullsized":"2024-09-12T03:19:20.371187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:17.810139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375192+0000","last_change":"2024-09-12T03:19:09.326629+0000","last_active":"2024-09-12T03:19:20.375192+0000","last_peered":"2024-09-12T03:19:20.375192+0000","last_clean":"2024-09-12T03:19:20.375192+0000","last_became_active":"2024-09-12T03:19:09.326333+0000","last_became_peered":"2024-09-12T03:19:09.326333+0000","last_unstale":"2024-09-12T03:19:20.375192+0000","last_undegraded":"2024-09-12T03:19:20.375192+0000","last_fullsized":"2024-09-12T03:19:20.375192+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:20.807593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408435+0000","last_change":"2024-09-12T03:19:09.339919+0000","last_active":"2024-09-12T03:19:20.408435+0000","last_peered":"2024-09-12T03:19:20.408435+0000","last_clean":"2024-09-12T03:19:20.408435+0000","last_became_active":"2024-09-12T03:19:09.339618+0000","last_became_peered":"2024-09-12T03:19:09.339618+0000","last_unstale":"2024-09-12T03:19:20.408435+0000","last_undegraded":"2024-09-12T03:19:20.408435+0000","last_fullsized":"2024-09-12T03:19:20.408435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:11:03.740079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374805+0000","last_change":"2024-09-12T03:19:09.336148+0000","last_active":"2024-09-12T03:19:20.374805+0000","last_peered":"2024-09-12T03:19:20.374805+0000","last_clean":"2024-09-12T03:19:20.374805+0000","last_became_active":"2024-09-12T03:19:09.335779+0000","last_became_peered":"2024-09-12T03:19:09.335779+0000","last_unstale":"2024-09-12T03:19:20.374805+0000","last_undegraded":"2024-09-12T03:19:20.374805+0000","last_fullsized":"2024-09-12T03:19:20.374805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:18.234320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382536+0000","last_change":"2024-09-12T03:19:20.382536+0000","last_active":"2024-09-12T03:19:20.382536+0000","last_peered":"2024-09-12T03:19:20.382536+0000","last_clean":"2024-09-12T03:19:20.382536+0000","last_became_active":"2024-09-12T03:19:20.382118+0000","last_became_peered":"2024-09-12T03:19:20.382118+0000","last_unstale":"2024-09-12T03:19:20.382536+0000","last_undegraded":"2024-09-12T03:19:20.382536+0000","last_fullsized":"2024-09-12T03:19:20.382536+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:55.836646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370826+0000","last_change":"2024-09-12T03:19:09.345064+0000","last_active":"2024-09-12T03:19:20.370826+0000","last_peered":"2024-09-12T03:19:20.370826+0000","last_clean":"2024-09-12T03:19:20.370826+0000","last_became_active":"2024-09-12T03:19:09.344628+0000","last_became_peered":"2024-09-12T03:19:09.344628+0000","last_unstale":"2024-09-12T03:19:20.370826+0000","last_undegraded":"2024-09-12T03:19:20.370826+0000","last_fullsized":"2024-09-12T03:19:20.370826+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:50.011751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373663+0000","last_change":"2024-09-12T03:19:09.338052+0000","last_active":"2024-09-12T03:19:20.373663+0000","last_peered":"2024-09-12T03:19:20.373663+0000","last_clean":"2024-09-12T03:19:20.373663+0000","last_became_active":"2024-09-12T03:19:09.337710+0000","last_became_peered":"2024-09-12T03:19:09.337710+0000","last_unstale":"2024-09-12T03:19:20.373663+0000","last_undegraded":"2024-09-12T03:19:20.373663+0000","last_fullsized":"2024-09-12T03:19:20.373663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:36.521269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":83,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:09.453392+0000","last_change":"2024-09-12T03:19:07.208341+0000","last_active":"2024-09-12T03:20:09.453392+0000","last_peered":"2024-09-12T03:20:09.453392+0000","last_clean":"2024-09-12T03:20:09.453392+0000","last_became_active":"2024-09-12T03:19:07.207385+0000","last_became_peered":"2024-09-12T03:19:07.207385+0000","last_unstale":"2024-09-12T03:20:09.453392+0000","last_undegraded":"2024-09-12T03:20:09.453392+0000","last_fullsized":"2024-09-12T03:20:09.453392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:18.476795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386347+0000","last_change":"2024-09-12T03:19:09.339997+0000","last_active":"2024-09-12T03:19:20.386347+0000","last_peered":"2024-09-12T03:19:20.386347+0000","last_clean":"2024-09-12T03:19:20.386347+0000","last_became_active":"2024-09-12T03:19:09.339261+0000","last_became_peered":"2024-09-12T03:19:09.339261+0000","last_unstale":"2024-09-12T03:19:20.386347+0000","last_undegraded":"2024-09-12T03:19:20.386347+0000","last_fullsized":"2024-09-12T03:19:20.386347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:07.670345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374030+0000","last_change":"2024-09-12T03:19:07.200222+0000","last_active":"2024-09-12T03:19:20.374030+0000","last_peered":"2024-09-12T03:19:20.374030+0000","last_clean":"2024-09-12T03:19:20.374030+0000","last_became_active":"2024-09-12T03:19:07.199819+0000","last_became_peered":"2024-09-12T03:19:07.199819+0000","last_unstale":"2024-09-12T03:19:20.374030+0000","last_undegraded":"2024-09-12T03:19:20.374030+0000","last_fullsized":"2024-09-12T03:19:20.374030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:39:28.933609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375686+0000","last_change":"2024-09-12T03:19:07.207005+0000","last_active":"2024-09-12T03:19:20.375686+0000","last_peered":"2024-09-12T03:19:20.375686+0000","last_clean":"2024-09-12T03:19:20.375686+0000","last_became_active":"2024-09-12T03:19:07.206685+0000","last_became_peered":"2024-09-12T03:19:07.206685+0000","last_unstale":"2024-09-12T03:19:20.375686+0000","last_undegraded":"2024-09-12T03:19:20.375686+0000","last_fullsized":"2024-09-12T03:19:20.375686+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:31.111706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372028+0000","last_change":"2024-09-12T03:19:09.335871+0000","last_active":"2024-09-12T03:19:20.372028+0000","last_peered":"2024-09-12T03:19:20.372028+0000","last_clean":"2024-09-12T03:19:20.372028+0000","last_became_active":"2024-09-12T03:19:09.335568+0000","last_became_peered":"2024-09-12T03:19:09.335568+0000","last_unstale":"2024-09-12T03:19:20.372028+0000","last_undegraded":"2024-09-12T03:19:20.372028+0000","last_fullsized":"2024-09-12T03:19:20.372028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:52:34.471512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386049+0000","last_change":"2024-09-12T03:19:07.208320+0000","last_active":"2024-09-12T03:19:20.386049+0000","last_peered":"2024-09-12T03:19:20.386049+0000","last_clean":"2024-09-12T03:19:20.386049+0000","last_became_active":"2024-09-12T03:19:07.207873+0000","last_became_peered":"2024-09-12T03:19:07.207873+0000","last_unstale":"2024-09-12T03:19:20.386049+0000","last_undegraded":"2024-09-12T03:19:20.386049+0000","last_fullsized":"2024-09-12T03:19:20.386049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:00.128595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372467+0000","last_change":"2024-09-12T03:19:09.331930+0000","last_active":"2024-09-12T03:19:20.372467+0000","last_peered":"2024-09-12T03:19:20.372467+0000","last_clean":"2024-09-12T03:19:20.372467+0000","last_became_active":"2024-09-12T03:19:09.331558+0000","last_became_peered":"2024-09-12T03:19:09.331558+0000","last_unstale":"2024-09-12T03:19:20.372467+0000","last_undegraded":"2024-09-12T03:19:20.372467+0000","last_fullsized":"2024-09-12T03:19:20.372467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:00:27.935717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":79,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:11.455463+0000","last_change":"2024-09-12T03:19:07.205473+0000","last_active":"2024-09-12T03:20:11.455463+0000","last_peered":"2024-09-12T03:20:11.455463+0000","last_clean":"2024-09-12T03:20:11.455463+0000","last_became_active":"2024-09-12T03:19:07.205134+0000","last_became_peered":"2024-09-12T03:19:07.205134+0000","last_unstale":"2024-09-12T03:20:11.455463+0000","last_undegraded":"2024-09-12T03:20:11.455463+0000","last_fullsized":"2024-09-12T03:20:11.455463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:23.443957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370511+0000","last_change":"2024-09-12T03:19:09.333868+0000","last_active":"2024-09-12T03:19:20.370511+0000","last_peered":"2024-09-12T03:19:20.370511+0000","last_clean":"2024-09-12T03:19:20.370511+0000","last_became_active":"2024-09-12T03:19:09.333484+0000","last_became_peered":"2024-09-12T03:19:09.333484+0000","last_unstale":"2024-09-12T03:19:20.370511+0000","last_undegraded":"2024-09-12T03:19:20.370511+0000","last_fullsized":"2024-09-12T03:19:20.370511+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:21:10.226821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386303+0000","last_change":"2024-09-12T03:19:20.386303+0000","last_active":"2024-09-12T03:19:20.386303+0000","last_peered":"2024-09-12T03:19:20.386303+0000","last_clean":"2024-09-12T03:19:20.386303+0000","last_became_active":"2024-09-12T03:19:20.383198+0000","last_became_peered":"2024-09-12T03:19:20.383198+0000","last_unstale":"2024-09-12T03:19:20.386303+0000","last_undegraded":"2024-09-12T03:19:20.386303+0000","last_fullsized":"2024-09-12T03:19:20.386303+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:42.224849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408655+0000","last_change":"2024-09-12T03:19:09.340849+0000","last_active":"2024-09-12T03:19:20.408655+0000","last_peered":"2024-09-12T03:19:20.408655+0000","last_clean":"2024-09-12T03:19:20.408655+0000","last_became_active":"2024-09-12T03:19:09.340517+0000","last_became_peered":"2024-09-12T03:19:09.340517+0000","last_unstale":"2024-09-12T03:19:20.408655+0000","last_undegraded":"2024-09-12T03:19:20.408655+0000","last_fullsized":"2024-09-12T03:19:20.408655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:42.018269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386240+0000","last_change":"2024-09-12T03:19:07.389147+0000","last_active":"2024-09-12T03:19:20.386240+0000","last_peered":"2024-09-12T03:19:20.386240+0000","last_clean":"2024-09-12T03:19:20.386240+0000","last_became_active":"2024-09-12T03:19:07.388744+0000","last_became_peered":"2024-09-12T03:19:07.388744+0000","last_unstale":"2024-09-12T03:19:20.386240+0000","last_undegraded":"2024-09-12T03:19:20.386240+0000","last_fullsized":"2024-09-12T03:19:20.386240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:16.700644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370280+0000","last_change":"2024-09-12T03:19:09.329243+0000","last_active":"2024-09-12T03:19:20.370280+0000","last_peered":"2024-09-12T03:19:20.370280+0000","last_clean":"2024-09-12T03:19:20.370280+0000","last_became_active":"2024-09-12T03:19:09.328610+0000","last_became_peered":"2024-09-12T03:19:09.328610+0000","last_unstale":"2024-09-12T03:19:20.370280+0000","last_undegraded":"2024-09-12T03:19:20.370280+0000","last_fullsized":"2024-09-12T03:19:20.370280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:49.452942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'28","reported_seq":165,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:12.456910+0000","last_change":"2024-09-12T03:19:07.207804+0000","last_active":"2024-09-12T03:20:12.456910+0000","last_peered":"2024-09-12T03:20:12.456910+0000","last_clean":"2024-09-12T03:20:12.456910+0000","last_became_active":"2024-09-12T03:19:07.207409+0000","last_became_peered":"2024-09-12T03:19:07.207409+0000","last_unstale":"2024-09-12T03:20:12.456910+0000","last_undegraded":"2024-09-12T03:20:12.456910+0000","last_fullsized":"2024-09-12T03:20:12.456910+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:10.732423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":29,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373666+0000","last_change":"2024-09-12T03:19:09.333102+0000","last_active":"2024-09-12T03:19:20.373666+0000","last_peered":"2024-09-12T03:19:20.373666+0000","last_clean":"2024-09-12T03:19:20.373666+0000","last_became_active":"2024-09-12T03:19:09.332712+0000","last_became_peered":"2024-09-12T03:19:09.332712+0000","last_unstale":"2024-09-12T03:19:20.373666+0000","last_undegraded":"2024-09-12T03:19:20.373666+0000","last_fullsized":"2024-09-12T03:19:20.373666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:22.912213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382902+0000","last_change":"2024-09-12T03:19:20.382348+0000","last_active":"2024-09-12T03:19:20.382902+0000","last_peered":"2024-09-12T03:19:20.382902+0000","last_clean":"2024-09-12T03:19:20.382902+0000","last_became_active":"2024-09-12T03:19:20.375142+0000","last_became_peered":"2024-09-12T03:19:20.375142+0000","last_unstale":"2024-09-12T03:19:20.382902+0000","last_undegraded":"2024-09-12T03:19:20.382902+0000","last_fullsized":"2024-09-12T03:19:20.382902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:04.573599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375046+0000","last_change":"2024-09-12T03:19:09.333194+0000","last_active":"2024-09-12T03:19:20.375046+0000","last_peered":"2024-09-12T03:19:20.375046+0000","last_clean":"2024-09-12T03:19:20.375046+0000","last_became_active":"2024-09-12T03:19:09.332865+0000","last_became_peered":"2024-09-12T03:19:09.332865+0000","last_unstale":"2024-09-12T03:19:20.375046+0000","last_undegraded":"2024-09-12T03:19:20.375046+0000","last_fullsized":"2024-09-12T03:19:20.375046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:57.096138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386176+0000","last_change":"2024-09-12T03:19:07.209289+0000","last_active":"2024-09-12T03:19:20.386176+0000","last_peered":"2024-09-12T03:19:20.386176+0000","last_clean":"2024-09-12T03:19:20.386176+0000","last_became_active":"2024-09-12T03:19:07.208863+0000","last_became_peered":"2024-09-12T03:19:07.208863+0000","last_unstale":"2024-09-12T03:19:20.386176+0000","last_undegraded":"2024-09-12T03:19:20.386176+0000","last_fullsized":"2024-09-12T03:19:20.386176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:26.168148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372400+0000","last_change":"2024-09-12T03:19:09.343620+0000","last_active":"2024-09-12T03:19:20.372400+0000","last_peered":"2024-09-12T03:19:20.372400+0000","last_clean":"2024-09-12T03:19:20.372400+0000","last_became_active":"2024-09-12T03:19:09.343299+0000","last_became_peered":"2024-09-12T03:19:09.343299+0000","last_unstale":"2024-09-12T03:19:20.372400+0000","last_undegraded":"2024-09-12T03:19:20.372400+0000","last_fullsized":"2024-09-12T03:19:20.372400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:34.268886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":79,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:10.469323+0000","last_change":"2024-09-12T03:19:07.206160+0000","last_active":"2024-09-12T03:20:10.469323+0000","last_peered":"2024-09-12T03:20:10.469323+0000","last_clean":"2024-09-12T03:20:10.469323+0000","last_became_active":"2024-09-12T03:19:07.205509+0000","last_became_peered":"2024-09-12T03:19:07.205509+0000","last_unstale":"2024-09-12T03:20:10.469323+0000","last_undegraded":"2024-09-12T03:20:10.469323+0000","last_fullsized":"2024-09-12T03:20:10.469323+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:35.362831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372068+0000","last_change":"2024-09-12T03:19:09.337595+0000","last_active":"2024-09-12T03:19:20.372068+0000","last_peered":"2024-09-12T03:19:20.372068+0000","last_clean":"2024-09-12T03:19:20.372068+0000","last_became_active":"2024-09-12T03:19:09.337281+0000","last_became_peered":"2024-09-12T03:19:09.337281+0000","last_unstale":"2024-09-12T03:19:20.372068+0000","last_undegraded":"2024-09-12T03:19:20.372068+0000","last_fullsized":"2024-09-12T03:19:20.372068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:35.761679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373995+0000","last_change":"2024-09-12T03:19:07.389336+0000","last_active":"2024-09-12T03:19:20.373995+0000","last_peered":"2024-09-12T03:19:20.373995+0000","last_clean":"2024-09-12T03:19:20.373995+0000","last_became_active":"2024-09-12T03:19:07.388649+0000","last_became_peered":"2024-09-12T03:19:07.388649+0000","last_unstale":"2024-09-12T03:19:20.373995+0000","last_undegraded":"2024-09-12T03:19:20.373995+0000","last_fullsized":"2024-09-12T03:19:20.373995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:33.312700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386030+0000","last_change":"2024-09-12T03:19:09.340674+0000","last_active":"2024-09-12T03:19:20.386030+0000","last_peered":"2024-09-12T03:19:20.386030+0000","last_clean":"2024-09-12T03:19:20.386030+0000","last_became_active":"2024-09-12T03:19:09.340144+0000","last_became_peered":"2024-09-12T03:19:09.340144+0000","last_unstale":"2024-09-12T03:19:20.386030+0000","last_undegraded":"2024-09-12T03:19:20.386030+0000","last_fullsized":"2024-09-12T03:19:20.386030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:21:15.605086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373837+0000","last_change":"2024-09-12T03:19:07.206513+0000","last_active":"2024-09-12T03:19:20.373837+0000","last_peered":"2024-09-12T03:19:20.373837+0000","last_clean":"2024-09-12T03:19:20.373837+0000","last_became_active":"2024-09-12T03:19:07.205974+0000","last_became_peered":"2024-09-12T03:19:07.205974+0000","last_unstale":"2024-09-12T03:19:20.373837+0000","last_undegraded":"2024-09-12T03:19:20.373837+0000","last_fullsized":"2024-09-12T03:19:20.373837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:43:10.002123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.371052+0000","last_change":"2024-09-12T03:19:09.342200+0000","last_active":"2024-09-12T03:19:20.371052+0000","last_peered":"2024-09-12T03:19:20.371052+0000","last_clean":"2024-09-12T03:19:20.371052+0000","last_became_active":"2024-09-12T03:19:09.341901+0000","last_became_peered":"2024-09-12T03:19:09.341901+0000","last_unstale":"2024-09-12T03:19:20.371052+0000","last_undegraded":"2024-09-12T03:19:20.371052+0000","last_fullsized":"2024-09-12T03:19:20.371052+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:27.486214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375373+0000","last_change":"2024-09-12T03:19:07.207514+0000","last_active":"2024-09-12T03:19:20.375373+0000","last_peered":"2024-09-12T03:19:20.375373+0000","last_clean":"2024-09-12T03:19:20.375373+0000","last_became_active":"2024-09-12T03:19:07.205925+0000","last_became_peered":"2024-09-12T03:19:07.205925+0000","last_unstale":"2024-09-12T03:19:20.375373+0000","last_undegraded":"2024-09-12T03:19:20.375373+0000","last_fullsized":"2024-09-12T03:19:20.375373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:20.244703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372696+0000","last_change":"2024-09-12T03:19:09.337837+0000","last_active":"2024-09-12T03:19:20.372696+0000","last_peered":"2024-09-12T03:19:20.372696+0000","last_clean":"2024-09-12T03:19:20.372696+0000","last_became_active":"2024-09-12T03:19:09.337502+0000","last_became_peered":"2024-09-12T03:19:09.337502+0000","last_unstale":"2024-09-12T03:19:20.372696+0000","last_undegraded":"2024-09-12T03:19:20.372696+0000","last_fullsized":"2024-09-12T03:19:20.372696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:30.167139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386356+0000","last_change":"2024-09-12T03:19:07.209130+0000","last_active":"2024-09-12T03:19:20.386356+0000","last_peered":"2024-09-12T03:19:20.386356+0000","last_clean":"2024-09-12T03:19:20.386356+0000","last_became_active":"2024-09-12T03:19:07.208753+0000","last_became_peered":"2024-09-12T03:19:07.208753+0000","last_unstale":"2024-09-12T03:19:20.386356+0000","last_undegraded":"2024-09-12T03:19:20.386356+0000","last_fullsized":"2024-09-12T03:19:20.386356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:34.719292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372500+0000","last_change":"2024-09-12T03:19:09.336382+0000","last_active":"2024-09-12T03:19:20.372500+0000","last_peered":"2024-09-12T03:19:20.372500+0000","last_clean":"2024-09-12T03:19:20.372500+0000","last_became_active":"2024-09-12T03:19:09.336015+0000","last_became_peered":"2024-09-12T03:19:09.336015+0000","last_unstale":"2024-09-12T03:19:20.372500+0000","last_undegraded":"2024-09-12T03:19:20.372500+0000","last_fullsized":"2024-09-12T03:19:20.372500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:19.430849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375044+0000","last_change":"2024-09-12T03:19:07.528567+0000","last_active":"2024-09-12T03:19:20.375044+0000","last_peered":"2024-09-12T03:19:20.375044+0000","last_clean":"2024-09-12T03:19:20.375044+0000","last_became_active":"2024-09-12T03:19:07.528123+0000","last_became_peered":"2024-09-12T03:19:07.528123+0000","last_unstale":"2024-09-12T03:19:20.375044+0000","last_undegraded":"2024-09-12T03:19:20.375044+0000","last_fullsized":"2024-09-12T03:19:20.375044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:34.416714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372773+0000","last_change":"2024-09-12T03:19:09.338836+0000","last_active":"2024-09-12T03:19:20.372773+0000","last_peered":"2024-09-12T03:19:20.372773+0000","last_clean":"2024-09-12T03:19:20.372773+0000","last_became_active":"2024-09-12T03:19:09.337571+0000","last_became_peered":"2024-09-12T03:19:09.337571+0000","last_unstale":"2024-09-12T03:19:20.372773+0000","last_undegraded":"2024-09-12T03:19:20.372773+0000","last_fullsized":"2024-09-12T03:19:20.372773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:39.039878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":329,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372694+0000","last_change":"2024-09-12T03:18:01.051311+0000","last_active":"2024-09-12T03:19:20.372694+0000","last_peered":"2024-09-12T03:19:20.372694+0000","last_clean":"2024-09-12T03:19:20.372694+0000","last_became_active":"2024-09-12T03:18:01.050849+0000","last_became_peered":"2024-09-12T03:18:01.050849+0000","last_unstale":"2024-09-12T03:19:20.372694+0000","last_undegraded":"2024-09-12T03:19:20.372694+0000","last_fullsized":"2024-09-12T03:19:20.372694+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:14:38.096222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:14:38.096222+0000","last_clean_scrub_stamp":"2024-09-12T03:14:38.096222+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:33.067603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382035+0000","last_change":"2024-09-12T03:19:07.206045+0000","last_active":"2024-09-12T03:19:20.382035+0000","last_peered":"2024-09-12T03:19:20.382035+0000","last_clean":"2024-09-12T03:19:20.382035+0000","last_became_active":"2024-09-12T03:19:07.205594+0000","last_became_peered":"2024-09-12T03:19:07.205594+0000","last_unstale":"2024-09-12T03:19:20.382035+0000","last_undegraded":"2024-09-12T03:19:20.382035+0000","last_fullsized":"2024-09-12T03:19:20.382035+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:12.811906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.387690+0000","last_change":"2024-09-12T03:19:20.387274+0000","last_active":"2024-09-12T03:19:20.387690+0000","last_peered":"2024-09-12T03:19:20.387690+0000","last_clean":"2024-09-12T03:19:20.387690+0000","last_became_active":"2024-09-12T03:19:20.386878+0000","last_became_peered":"2024-09-12T03:19:20.386878+0000","last_unstale":"2024-09-12T03:19:20.387690+0000","last_undegraded":"2024-09-12T03:19:20.387690+0000","last_fullsized":"2024-09-12T03:19:20.387690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:26.674032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372302+0000","last_change":"2024-09-12T03:19:07.206212+0000","last_active":"2024-09-12T03:19:20.372302+0000","last_peered":"2024-09-12T03:19:20.372302+0000","last_clean":"2024-09-12T03:19:20.372302+0000","last_became_active":"2024-09-12T03:19:07.205899+0000","last_became_peered":"2024-09-12T03:19:07.205899+0000","last_unstale":"2024-09-12T03:19:20.372302+0000","last_undegraded":"2024-09-12T03:19:20.372302+0000","last_fullsized":"2024-09-12T03:19:20.372302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:02.169436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.377494+0000","last_change":"2024-09-12T03:19:20.377494+0000","last_active":"2024-09-12T03:19:20.377494+0000","last_peered":"2024-09-12T03:19:20.377494+0000","last_clean":"2024-09-12T03:19:20.377494+0000","last_became_active":"2024-09-12T03:19:20.377091+0000","last_became_peered":"2024-09-12T03:19:20.377091+0000","last_unstale":"2024-09-12T03:19:20.377494+0000","last_undegraded":"2024-09-12T03:19:20.377494+0000","last_fullsized":"2024-09-12T03:19:20.377494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:30.926853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":79,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:11.470230+0000","last_change":"2024-09-12T03:19:07.388486+0000","last_active":"2024-09-12T03:20:11.470230+0000","last_peered":"2024-09-12T03:20:11.470230+0000","last_clean":"2024-09-12T03:20:11.470230+0000","last_became_active":"2024-09-12T03:19:07.388125+0000","last_became_peered":"2024-09-12T03:19:07.388125+0000","last_unstale":"2024-09-12T03:20:11.470230+0000","last_undegraded":"2024-09-12T03:20:11.470230+0000","last_fullsized":"2024-09-12T03:20:11.470230+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:46.218601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.385912+0000","last_change":"2024-09-12T03:19:09.344308+0000","last_active":"2024-09-12T03:19:20.385912+0000","last_peered":"2024-09-12T03:19:20.385912+0000","last_clean":"2024-09-12T03:19:20.385912+0000","last_became_active":"2024-09-12T03:19:09.343987+0000","last_became_peered":"2024-09-12T03:19:09.343987+0000","last_unstale":"2024-09-12T03:19:20.385912+0000","last_undegraded":"2024-09-12T03:19:20.385912+0000","last_fullsized":"2024-09-12T03:19:20.385912+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:16:11.616555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372560+0000","last_change":"2024-09-12T03:19:07.208619+0000","last_active":"2024-09-12T03:19:20.372560+0000","last_peered":"2024-09-12T03:19:20.372560+0000","last_clean":"2024-09-12T03:19:20.372560+0000","last_became_active":"2024-09-12T03:19:07.208254+0000","last_became_peered":"2024-09-12T03:19:07.208254+0000","last_unstale":"2024-09-12T03:19:20.372560+0000","last_undegraded":"2024-09-12T03:19:20.372560+0000","last_fullsized":"2024-09-12T03:19:20.372560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:56.177948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374096+0000","last_change":"2024-09-12T03:19:09.341837+0000","last_active":"2024-09-12T03:19:20.374096+0000","last_peered":"2024-09-12T03:19:20.374096+0000","last_clean":"2024-09-12T03:19:20.374096+0000","last_became_active":"2024-09-12T03:19:09.341069+0000","last_became_peered":"2024-09-12T03:19:09.341069+0000","last_unstale":"2024-09-12T03:19:20.374096+0000","last_undegraded":"2024-09-12T03:19:20.374096+0000","last_fullsized":"2024-09-12T03:19:20.374096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:04.238986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372460+0000","last_change":"2024-09-12T03:19:07.388870+0000","last_active":"2024-09-12T03:19:20.372460+0000","last_peered":"2024-09-12T03:19:20.372460+0000","last_clean":"2024-09-12T03:19:20.372460+0000","last_became_active":"2024-09-12T03:19:07.388555+0000","last_became_peered":"2024-09-12T03:19:07.388555+0000","last_unstale":"2024-09-12T03:19:20.372460+0000","last_undegraded":"2024-09-12T03:19:20.372460+0000","last_fullsized":"2024-09-12T03:19:20.372460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:29.789454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374271+0000","last_change":"2024-09-12T03:19:09.334714+0000","last_active":"2024-09-12T03:19:20.374271+0000","last_peered":"2024-09-12T03:19:20.374271+0000","last_clean":"2024-09-12T03:19:20.374271+0000","last_became_active":"2024-09-12T03:19:09.334413+0000","last_became_peered":"2024-09-12T03:19:09.334413+0000","last_unstale":"2024-09-12T03:19:20.374271+0000","last_undegraded":"2024-09-12T03:19:20.374271+0000","last_fullsized":"2024-09-12T03:19:20.374271+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:52.222340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373797+0000","last_change":"2024-09-12T03:19:07.389055+0000","last_active":"2024-09-12T03:19:20.373797+0000","last_peered":"2024-09-12T03:19:20.373797+0000","last_clean":"2024-09-12T03:19:20.373797+0000","last_became_active":"2024-09-12T03:19:07.388673+0000","last_became_peered":"2024-09-12T03:19:07.388673+0000","last_unstale":"2024-09-12T03:19:20.373797+0000","last_undegraded":"2024-09-12T03:19:20.373797+0000","last_fullsized":"2024-09-12T03:19:20.373797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:20.974533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375304+0000","last_change":"2024-09-12T03:19:09.340105+0000","last_active":"2024-09-12T03:19:20.375304+0000","last_peered":"2024-09-12T03:19:20.375304+0000","last_clean":"2024-09-12T03:19:20.375304+0000","last_became_active":"2024-09-12T03:19:09.339642+0000","last_became_peered":"2024-09-12T03:19:09.339642+0000","last_unstale":"2024-09-12T03:19:20.375304+0000","last_undegraded":"2024-09-12T03:19:20.375304+0000","last_fullsized":"2024-09-12T03:19:20.375304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:01.624236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'12","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:33.331973+0000","last_change":"2024-09-12T03:19:07.207232+0000","last_active":"2024-09-12T03:19:33.331973+0000","last_peered":"2024-09-12T03:19:33.331973+0000","last_clean":"2024-09-12T03:19:33.331973+0000","last_became_active":"2024-09-12T03:19:07.206921+0000","last_became_peered":"2024-09-12T03:19:07.206921+0000","last_unstale":"2024-09-12T03:19:33.331973+0000","last_undegraded":"2024-09-12T03:19:33.331973+0000","last_fullsized":"2024-09-12T03:19:33.331973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:25.457853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372958+0000","last_change":"2024-09-12T03:19:09.330442+0000","last_active":"2024-09-12T03:19:20.372958+0000","last_peered":"2024-09-12T03:19:20.372958+0000","last_clean":"2024-09-12T03:19:20.372958+0000","last_became_active":"2024-09-12T03:19:09.330081+0000","last_became_peered":"2024-09-12T03:19:09.330081+0000","last_unstale":"2024-09-12T03:19:20.372958+0000","last_undegraded":"2024-09-12T03:19:20.372958+0000","last_fullsized":"2024-09-12T03:19:20.372958+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:56.152562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382392+0000","last_change":"2024-09-12T03:19:07.528646+0000","last_active":"2024-09-12T03:19:20.382392+0000","last_peered":"2024-09-12T03:19:20.382392+0000","last_clean":"2024-09-12T03:19:20.382392+0000","last_became_active":"2024-09-12T03:19:07.528334+0000","last_became_peered":"2024-09-12T03:19:07.528334+0000","last_unstale":"2024-09-12T03:19:20.382392+0000","last_undegraded":"2024-09-12T03:19:20.382392+0000","last_fullsized":"2024-09-12T03:19:20.382392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:40.425579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408541+0000","last_change":"2024-09-12T03:19:09.331993+0000","last_active":"2024-09-12T03:19:20.408541+0000","last_peered":"2024-09-12T03:19:20.408541+0000","last_clean":"2024-09-12T03:19:20.408541+0000","last_became_active":"2024-09-12T03:19:09.331698+0000","last_became_peered":"2024-09-12T03:19:09.331698+0000","last_unstale":"2024-09-12T03:19:20.408541+0000","last_undegraded":"2024-09-12T03:19:20.408541+0000","last_fullsized":"2024-09-12T03:19:20.408541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:03:54.182833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370179+0000","last_change":"2024-09-12T03:19:07.388297+0000","last_active":"2024-09-12T03:19:20.370179+0000","last_peered":"2024-09-12T03:19:20.370179+0000","last_clean":"2024-09-12T03:19:20.370179+0000","last_became_active":"2024-09-12T03:19:07.388038+0000","last_became_peered":"2024-09-12T03:19:07.388038+0000","last_unstale":"2024-09-12T03:19:20.370179+0000","last_undegraded":"2024-09-12T03:19:20.370179+0000","last_fullsized":"2024-09-12T03:19:20.370179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:45.874195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373463+0000","last_change":"2024-09-12T03:19:09.331525+0000","last_active":"2024-09-12T03:19:20.373463+0000","last_peered":"2024-09-12T03:19:20.373463+0000","last_clean":"2024-09-12T03:19:20.373463+0000","last_became_active":"2024-09-12T03:19:09.330903+0000","last_became_peered":"2024-09-12T03:19:09.330903+0000","last_unstale":"2024-09-12T03:19:20.373463+0000","last_undegraded":"2024-09-12T03:19:20.373463+0000","last_fullsized":"2024-09-12T03:19:20.373463+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:02.455132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372235+0000","last_change":"2024-09-12T03:19:07.205181+0000","last_active":"2024-09-12T03:19:20.372235+0000","last_peered":"2024-09-12T03:19:20.372235+0000","last_clean":"2024-09-12T03:19:20.372235+0000","last_became_active":"2024-09-12T03:19:07.204726+0000","last_became_peered":"2024-09-12T03:19:07.204726+0000","last_unstale":"2024-09-12T03:19:20.372235+0000","last_undegraded":"2024-09-12T03:19:20.372235+0000","last_fullsized":"2024-09-12T03:19:20.372235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:26.112800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370908+0000","last_change":"2024-09-12T03:19:09.345338+0000","last_active":"2024-09-12T03:19:20.370908+0000","last_peered":"2024-09-12T03:19:20.370908+0000","last_clean":"2024-09-12T03:19:20.370908+0000","last_became_active":"2024-09-12T03:19:09.345023+0000","last_became_peered":"2024-09-12T03:19:09.345023+0000","last_unstale":"2024-09-12T03:19:20.370908+0000","last_undegraded":"2024-09-12T03:19:20.370908+0000","last_fullsized":"2024-09-12T03:19:20.370908+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:39.184065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374938+0000","last_change":"2024-09-12T03:19:07.211954+0000","last_active":"2024-09-12T03:19:20.374938+0000","last_peered":"2024-09-12T03:19:20.374938+0000","last_clean":"2024-09-12T03:19:20.374938+0000","last_became_active":"2024-09-12T03:19:07.211509+0000","last_became_peered":"2024-09-12T03:19:07.211509+0000","last_unstale":"2024-09-12T03:19:20.374938+0000","last_undegraded":"2024-09-12T03:19:20.374938+0000","last_fullsized":"2024-09-12T03:19:20.374938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:27.461924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373018+0000","last_change":"2024-09-12T03:19:09.331168+0000","last_active":"2024-09-12T03:19:20.373018+0000","last_peered":"2024-09-12T03:19:20.373018+0000","last_clean":"2024-09-12T03:19:20.373018+0000","last_became_active":"2024-09-12T03:19:09.329592+0000","last_became_peered":"2024-09-12T03:19:09.329592+0000","last_unstale":"2024-09-12T03:19:20.373018+0000","last_undegraded":"2024-09-12T03:19:20.373018+0000","last_fullsized":"2024-09-12T03:19:20.373018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:46.127861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373572+0000","last_change":"2024-09-12T03:19:07.207802+0000","last_active":"2024-09-12T03:19:20.373572+0000","last_peered":"2024-09-12T03:19:20.373572+0000","last_clean":"2024-09-12T03:19:20.373572+0000","last_became_active":"2024-09-12T03:19:07.207428+0000","last_became_peered":"2024-09-12T03:19:07.207428+0000","last_unstale":"2024-09-12T03:19:20.373572+0000","last_undegraded":"2024-09-12T03:19:20.373572+0000","last_fullsized":"2024-09-12T03:19:20.373572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:45.135019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372336+0000","last_change":"2024-09-12T03:19:09.336094+0000","last_active":"2024-09-12T03:19:20.372336+0000","last_peered":"2024-09-12T03:19:20.372336+0000","last_clean":"2024-09-12T03:19:20.372336+0000","last_became_active":"2024-09-12T03:19:09.335779+0000","last_became_peered":"2024-09-12T03:19:09.335779+0000","last_unstale":"2024-09-12T03:19:20.372336+0000","last_undegraded":"2024-09-12T03:19:20.372336+0000","last_fullsized":"2024-09-12T03:19:20.372336+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:37.019082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386131+0000","last_change":"2024-09-12T03:19:09.344537+0000","last_active":"2024-09-12T03:19:20.386131+0000","last_peered":"2024-09-12T03:19:20.386131+0000","last_clean":"2024-09-12T03:19:20.386131+0000","last_became_active":"2024-09-12T03:19:09.343879+0000","last_became_peered":"2024-09-12T03:19:09.343879+0000","last_unstale":"2024-09-12T03:19:20.386131+0000","last_undegraded":"2024-09-12T03:19:20.386131+0000","last_fullsized":"2024-09-12T03:19:20.386131+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:40.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373973+0000","last_change":"2024-09-12T03:19:07.208040+0000","last_active":"2024-09-12T03:19:20.373973+0000","last_peered":"2024-09-12T03:19:20.373973+0000","last_clean":"2024-09-12T03:19:20.373973+0000","last_became_active":"2024-09-12T03:19:07.207658+0000","last_became_peered":"2024-09-12T03:19:07.207658+0000","last_unstale":"2024-09-12T03:19:20.373973+0000","last_undegraded":"2024-09-12T03:19:20.373973+0000","last_fullsized":"2024-09-12T03:19:20.373973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:48.620707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370231+0000","last_change":"2024-09-12T03:19:09.333837+0000","last_active":"2024-09-12T03:19:20.370231+0000","last_peered":"2024-09-12T03:19:20.370231+0000","last_clean":"2024-09-12T03:19:20.370231+0000","last_became_active":"2024-09-12T03:19:09.333194+0000","last_became_peered":"2024-09-12T03:19:09.333194+0000","last_unstale":"2024-09-12T03:19:20.370231+0000","last_undegraded":"2024-09-12T03:19:20.370231+0000","last_fullsized":"2024-09-12T03:19:20.370231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:18.766181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.385455+0000","last_change":"2024-09-12T03:19:20.384978+0000","last_active":"2024-09-12T03:19:20.385455+0000","last_peered":"2024-09-12T03:19:20.385455+0000","last_clean":"2024-09-12T03:19:20.385455+0000","last_became_active":"2024-09-12T03:19:20.376883+0000","last_became_peered":"2024-09-12T03:19:20.376883+0000","last_unstale":"2024-09-12T03:19:20.385455+0000","last_undegraded":"2024-09-12T03:19:20.385455+0000","last_fullsized":"2024-09-12T03:19:20.385455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:32.542334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.408633+0000","last_change":"2024-09-12T03:19:09.334684+0000","last_active":"2024-09-12T03:19:20.408633+0000","last_peered":"2024-09-12T03:19:20.408633+0000","last_clean":"2024-09-12T03:19:20.408633+0000","last_became_active":"2024-09-12T03:19:09.334281+0000","last_became_peered":"2024-09-12T03:19:09.334281+0000","last_unstale":"2024-09-12T03:19:20.408633+0000","last_undegraded":"2024-09-12T03:19:20.408633+0000","last_fullsized":"2024-09-12T03:19:20.408633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:44.837067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372829+0000","last_change":"2024-09-12T03:19:07.388555+0000","last_active":"2024-09-12T03:19:20.372829+0000","last_peered":"2024-09-12T03:19:20.372829+0000","last_clean":"2024-09-12T03:19:20.372829+0000","last_became_active":"2024-09-12T03:19:07.388206+0000","last_became_peered":"2024-09-12T03:19:07.388206+0000","last_unstale":"2024-09-12T03:19:20.372829+0000","last_undegraded":"2024-09-12T03:19:20.372829+0000","last_fullsized":"2024-09-12T03:19:20.372829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:33.628294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372626+0000","last_change":"2024-09-12T03:19:09.339108+0000","last_active":"2024-09-12T03:19:20.372626+0000","last_peered":"2024-09-12T03:19:20.372626+0000","last_clean":"2024-09-12T03:19:20.372626+0000","last_became_active":"2024-09-12T03:19:09.338819+0000","last_became_peered":"2024-09-12T03:19:09.338819+0000","last_unstale":"2024-09-12T03:19:20.372626+0000","last_undegraded":"2024-09-12T03:19:20.372626+0000","last_fullsized":"2024-09-12T03:19:20.372626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:32.864900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372798+0000","last_change":"2024-09-12T03:19:07.207197+0000","last_active":"2024-09-12T03:19:20.372798+0000","last_peered":"2024-09-12T03:19:20.372798+0000","last_clean":"2024-09-12T03:19:20.372798+0000","last_became_active":"2024-09-12T03:19:07.206879+0000","last_became_peered":"2024-09-12T03:19:07.206879+0000","last_unstale":"2024-09-12T03:19:20.372798+0000","last_undegraded":"2024-09-12T03:19:20.372798+0000","last_fullsized":"2024-09-12T03:19:20.372798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:00:46.269636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63767,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":345,"num_read_kb":241,"num_write":164,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":191301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579930,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":892928,"data_stored":433387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808863,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36892,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708260,"statfs":{"total":95995035648,"available":95957258240,"internally_reserved":0,"allocated":1585152,"data_stored":1084598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972386,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":433589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201321,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":433509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430254,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1499136,"data_stored":1025441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593779,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":432978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822711,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1511424,"data_stored":1025125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953533,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":897024,"data_stored":433415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":433289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":494244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40436,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704716,"statfs":{"total":95995035648,"available":95953629184,"internally_reserved":0,"allocated":954368,"data_stored":492535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":913408,"data_stored":434249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:20:14.289 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T03:20:14.289 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-12T03:20:14.289 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-12T03:20:14.289 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-12T03:20:14.548 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:14.956 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:14 smithi006 ceph-mon[30991]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:20:14.956 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:14 smithi006 ceph-mon[30991]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:20:15.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.037+0000 7f645592b640 1 -- 172.21.15.6:0/2236662381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6450101440 msgr2=0x7f645010df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.037+0000 7f645592b640 1 --2- 172.21.15.6:0/2236662381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6450101440 0x7f645010df30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6444009ca0 tx=0x7f644402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.037+0000 7f645592b640 1 -- 172.21.15.6:0/2236662381 shutdown_connections 2024-09-12T03:20:15.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.037+0000 7f645592b640 1 --2- 172.21.15.6:0/2236662381 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f645010e500 0x7f6450110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.037+0000 7f645592b640 1 --2- 172.21.15.6:0/2236662381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6450101440 0x7f645010df30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.037+0000 7f645592b640 1 --2- 172.21.15.6:0/2236662381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6450100b20 0x7f6450100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.037+0000 7f645592b640 1 -- 172.21.15.6:0/2236662381 >> 172.21.15.6:0/2236662381 conn(0x7f64500fc7d0 msgr2=0x7f64500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:15.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.037+0000 7f645592b640 1 -- 172.21.15.6:0/2236662381 shutdown_connections 2024-09-12T03:20:15.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.037+0000 7f645592b640 1 -- 172.21.15.6:0/2236662381 wait complete. 2024-09-12T03:20:15.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.037+0000 7f645592b640 1 Processor -- start 2024-09-12T03:20:15.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.037+0000 7f645592b640 1 -- start start 2024-09-12T03:20:15.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.038+0000 7f645592b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6450100b20 0x7f645019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.038+0000 7f645592b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6450101440 0x7f645019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.038+0000 7f645592b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645010e500 0x7f645019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.038+0000 7f645592b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6450114250 con 0x7f645010e500 2024-09-12T03:20:15.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.038+0000 7f645592b640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f64501140d0 con 0x7f6450100b20 2024-09-12T03:20:15.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.038+0000 7f645592b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f64501143d0 con 0x7f6450101440 2024-09-12T03:20:15.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.038+0000 7f644f7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645010e500 0x7f645019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.038+0000 7f644f7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645010e500 0x7f645019f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53922/0 (socket says 172.21.15.6:53922) 2024-09-12T03:20:15.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.038+0000 7f644f7fe640 1 -- 172.21.15.6:0/3276640265 learned_addr learned my addr 172.21.15.6:0/3276640265 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:15.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.038+0000 7f644effd640 1 --2- 172.21.15.6:0/3276640265 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6450100b20 0x7f645019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.038+0000 7f644e7fc640 1 --2- 172.21.15.6:0/3276640265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6450101440 0x7f645019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.038+0000 7f644f7fe640 1 -- 172.21.15.6:0/3276640265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6450101440 msgr2=0x7f645019ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.038+0000 7f644f7fe640 1 --2- 172.21.15.6:0/3276640265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6450101440 0x7f645019ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.038+0000 7f644f7fe640 1 -- 172.21.15.6:0/3276640265 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6450100b20 msgr2=0x7f645019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.038+0000 7f644f7fe640 1 --2- 172.21.15.6:0/3276640265 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6450100b20 0x7f645019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.038+0000 7f644f7fe640 1 -- 172.21.15.6:0/3276640265 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645019f700 con 0x7f645010e500 2024-09-12T03:20:15.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.038+0000 7f644effd640 1 --2- 172.21.15.6:0/3276640265 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6450100b20 0x7f645019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:15.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.038+0000 7f644f7fe640 1 --2- 172.21.15.6:0/3276640265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645010e500 0x7f645019f020 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f644800d720 tx=0x7f644800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:15.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.038+0000 7f6454929640 1 -- 172.21.15.6:0/3276640265 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448017070 con 0x7f645010e500 2024-09-12T03:20:15.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.039+0000 7f645592b640 1 -- 172.21.15.6:0/3276640265 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645019f9f0 con 0x7f645010e500 2024-09-12T03:20:15.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.039+0000 7f6454929640 1 -- 172.21.15.6:0/3276640265 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f644800de30 con 0x7f645010e500 2024-09-12T03:20:15.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.039+0000 7f6454929640 1 -- 172.21.15.6:0/3276640265 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448004d90 con 0x7f645010e500 2024-09-12T03:20:15.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.039+0000 7f645592b640 1 -- 172.21.15.6:0/3276640265 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64501a72d0 con 0x7f645010e500 2024-09-12T03:20:15.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.040+0000 7f645592b640 1 -- 172.21.15.6:0/3276640265 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6450100f00 con 0x7f645010e500 2024-09-12T03:20:15.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.041+0000 7f6454929640 1 -- 172.21.15.6:0/3276640265 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6448005020 con 0x7f645010e500 2024-09-12T03:20:15.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.041+0000 7f6454929640 1 --2- 172.21.15.6:0/3276640265 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f64380a2c10 0x7f64380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.041+0000 7f644effd640 1 --2- 172.21.15.6:0/3276640265 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f64380a2c10 0x7f64380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.042+0000 7f6454929640 1 -- 172.21.15.6:0/3276640265 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f64480ceb60 con 0x7f645010e500 2024-09-12T03:20:15.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.042+0000 7f644effd640 1 --2- 172.21.15.6:0/3276640265 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f64380a2c10 0x7f64380a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6440005fd0 tx=0x7f644000b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:15.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.045+0000 7f6454929640 1 -- 172.21.15.6:0/3276640265 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f644809b990 con 0x7f645010e500 2024-09-12T03:20:15.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:14 smithi118 ceph-mon[35048]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:20:15.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:14 smithi118 ceph-mon[35048]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:20:15.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.256+0000 7f645592b640 1 -- 172.21.15.6:0/3276640265 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6450109600 con 0x7f645010e500 2024-09-12T03:20:15.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.256+0000 7f6454929640 1 -- 172.21.15.6:0/3276640265 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f644809b290 con 0x7f645010e500 2024-09-12T03:20:15.257 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:20:15.257 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-12T03:20:15.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.258+0000 7f645592b640 1 -- 172.21.15.6:0/3276640265 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f64380a2c10 msgr2=0x7f64380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.258+0000 7f645592b640 1 --2- 172.21.15.6:0/3276640265 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f64380a2c10 0x7f64380a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6440005fd0 tx=0x7f644000b5f0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.258+0000 7f645592b640 1 -- 172.21.15.6:0/3276640265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645010e500 msgr2=0x7f645019f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.258+0000 7f645592b640 1 --2- 172.21.15.6:0/3276640265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645010e500 0x7f645019f020 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f644800d720 tx=0x7f644800dbe0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.259+0000 7f645592b640 1 -- 172.21.15.6:0/3276640265 shutdown_connections 2024-09-12T03:20:15.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.259+0000 7f645592b640 1 --2- 172.21.15.6:0/3276640265 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f64380a2c10 0x7f64380a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.259+0000 7f645592b640 1 --2- 172.21.15.6:0/3276640265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645010e500 0x7f645019f020 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.259+0000 7f645592b640 1 --2- 172.21.15.6:0/3276640265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6450101440 0x7f645019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.259+0000 7f645592b640 1 --2- 172.21.15.6:0/3276640265 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6450100b20 0x7f645019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.259+0000 7f645592b640 1 -- 172.21.15.6:0/3276640265 >> 172.21.15.6:0/3276640265 conn(0x7f64500fc7d0 msgr2=0x7f645010f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:15.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.259+0000 7f645592b640 1 -- 172.21.15.6:0/3276640265 shutdown_connections 2024-09-12T03:20:15.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:15.259+0000 7f645592b640 1 -- 172.21.15.6:0/3276640265 wait complete. 2024-09-12T03:20:15.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:14 smithi161 ceph-mon[35485]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:20:15.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:14 smithi161 ceph-mon[35485]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:20:15.969 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-12T03:20:15.970 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:20:16.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:15 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3276640265' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T03:20:16.225 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:16.347 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:15 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3276640265' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T03:20:16.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:15 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3276640265' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T03:20:16.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.702+0000 7f9d8d1a7640 1 -- 172.21.15.6:0/2074417852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d881037a0 msgr2=0x7f9d8810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.702+0000 7f9d8d1a7640 1 --2- 172.21.15.6:0/2074417852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d881037a0 0x7f9d8810bc00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009900 tx=0x7f9d7c02a410 comp rx=0 tx=0).stop 2024-09-12T03:20:16.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.702+0000 7f9d8d1a7640 1 -- 172.21.15.6:0/2074417852 shutdown_connections 2024-09-12T03:20:16.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.702+0000 7f9d8d1a7640 1 --2- 172.21.15.6:0/2074417852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d8810c2f0 0x7f9d8810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.702+0000 7f9d8d1a7640 1 --2- 172.21.15.6:0/2074417852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d881037a0 0x7f9d8810bc00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.702+0000 7f9d8d1a7640 1 --2- 172.21.15.6:0/2074417852 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d88102df0 0x7f9d881031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.702+0000 7f9d8d1a7640 1 -- 172.21.15.6:0/2074417852 >> 172.21.15.6:0/2074417852 conn(0x7f9d880fc7d0 msgr2=0x7f9d880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:16.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.702+0000 7f9d8d1a7640 1 -- 172.21.15.6:0/2074417852 shutdown_connections 2024-09-12T03:20:16.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.702+0000 7f9d8d1a7640 1 -- 172.21.15.6:0/2074417852 wait complete. 2024-09-12T03:20:16.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.703+0000 7f9d8d1a7640 1 Processor -- start 2024-09-12T03:20:16.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.703+0000 7f9d8d1a7640 1 -- start start 2024-09-12T03:20:16.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.703+0000 7f9d8d1a7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d88102df0 0x7f9d881a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.703+0000 7f9d8d1a7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d881037a0 0x7f9d881a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.703+0000 7f9d8d1a7640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d8810c2f0 0x7f9d8819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.703+0000 7f9d8d1a7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9d88112130 con 0x7f9d881037a0 2024-09-12T03:20:16.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.703+0000 7f9d8d1a7640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f9d88111fb0 con 0x7f9d8810c2f0 2024-09-12T03:20:16.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.703+0000 7f9d8d1a7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9d881122b0 con 0x7f9d88102df0 2024-09-12T03:20:16.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.704+0000 7f9d8659c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d881037a0 0x7f9d881a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.704+0000 7f9d8659c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d881037a0 0x7f9d881a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53930/0 (socket says 172.21.15.6:53930) 2024-09-12T03:20:16.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.704+0000 7f9d8659c640 1 -- 172.21.15.6:0/2492740583 learned_addr learned my addr 172.21.15.6:0/2492740583 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:16.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.704+0000 7f9d86d9d640 1 --2- 172.21.15.6:0/2492740583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d88102df0 0x7f9d881a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.704+0000 7f9d8759e640 1 --2- 172.21.15.6:0/2492740583 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d8810c2f0 0x7f9d8819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.704+0000 7f9d8659c640 1 -- 172.21.15.6:0/2492740583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d88102df0 msgr2=0x7f9d881a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.704+0000 7f9d8659c640 1 --2- 172.21.15.6:0/2492740583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d88102df0 0x7f9d881a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.704+0000 7f9d8659c640 1 -- 172.21.15.6:0/2492740583 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d8810c2f0 msgr2=0x7f9d8819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.704+0000 7f9d8659c640 1 --2- 172.21.15.6:0/2492740583 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d8810c2f0 0x7f9d8819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.704+0000 7f9d8659c640 1 -- 172.21.15.6:0/2492740583 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8819af10 con 0x7f9d881037a0 2024-09-12T03:20:16.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.704+0000 7f9d8759e640 1 --2- 172.21.15.6:0/2492740583 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d8810c2f0 0x7f9d8819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:16.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.704+0000 7f9d8659c640 1 --2- 172.21.15.6:0/2492740583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d881037a0 0x7f9d881a0b10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c005940 tx=0x7f9d7c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:16.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.704+0000 7f9d6ffff640 1 -- 172.21.15.6:0/2492740583 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c03f030 con 0x7f9d881037a0 2024-09-12T03:20:16.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.704+0000 7f9d6ffff640 1 -- 172.21.15.6:0/2492740583 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d7c003ce0 con 0x7f9d881037a0 2024-09-12T03:20:16.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.704+0000 7f9d8d1a7640 1 -- 172.21.15.6:0/2492740583 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8819b1a0 con 0x7f9d881037a0 2024-09-12T03:20:16.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.704+0000 7f9d6ffff640 1 -- 172.21.15.6:0/2492740583 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c035070 con 0x7f9d881037a0 2024-09-12T03:20:16.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.704+0000 7f9d8d1a7640 1 -- 172.21.15.6:0/2492740583 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d881a7380 con 0x7f9d881037a0 2024-09-12T03:20:16.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.706+0000 7f9d8d1a7640 1 -- 172.21.15.6:0/2492740583 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d8806a790 con 0x7f9d881037a0 2024-09-12T03:20:16.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.706+0000 7f9d6ffff640 1 -- 172.21.15.6:0/2492740583 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9d7c042030 con 0x7f9d881037a0 2024-09-12T03:20:16.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.707+0000 7f9d6ffff640 1 --2- 172.21.15.6:0/2492740583 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9d700a2c10 0x7f9d700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.707+0000 7f9d86d9d640 1 --2- 172.21.15.6:0/2492740583 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9d700a2c10 0x7f9d700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.707+0000 7f9d6ffff640 1 -- 172.21.15.6:0/2492740583 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f9d7c0ef760 con 0x7f9d881037a0 2024-09-12T03:20:16.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.708+0000 7f9d86d9d640 1 --2- 172.21.15.6:0/2492740583 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9d700a2c10 0x7f9d700a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009990 tx=0x7f9d78009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:16.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.711+0000 7f9d6ffff640 1 -- 172.21.15.6:0/2492740583 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9d7c0bc530 con 0x7f9d881037a0 2024-09-12T03:20:16.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.926+0000 7f9d8d1a7640 1 -- 172.21.15.6:0/2492740583 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d8819bbd0 con 0x7f9d881037a0 2024-09-12T03:20:16.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.927+0000 7f9d6ffff640 1 -- 172.21.15.6:0/2492740583 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11908 (secure 0 0 0) 0x7f9d7c0bbe30 con 0x7f9d881037a0 2024-09-12T03:20:16.927 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:20:16.927 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":24,"btime":"2024-09-12T03:19:21:375379+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:19:20.919190+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24463,24464,24494]},"id":1}]} 2024-09-12T03:20:16.928 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 24 2024-09-12T03:20:16.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.929+0000 7f9d8d1a7640 1 -- 172.21.15.6:0/2492740583 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9d700a2c10 msgr2=0x7f9d700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.929+0000 7f9d8d1a7640 1 --2- 172.21.15.6:0/2492740583 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9d700a2c10 0x7f9d700a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009990 tx=0x7f9d78009110 comp rx=0 tx=0).stop 2024-09-12T03:20:16.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.929+0000 7f9d8d1a7640 1 -- 172.21.15.6:0/2492740583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d881037a0 msgr2=0x7f9d881a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.929+0000 7f9d8d1a7640 1 --2- 172.21.15.6:0/2492740583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d881037a0 0x7f9d881a0b10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c005940 tx=0x7f9d7c0042f0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.929+0000 7f9d8d1a7640 1 -- 172.21.15.6:0/2492740583 shutdown_connections 2024-09-12T03:20:16.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.929+0000 7f9d8d1a7640 1 --2- 172.21.15.6:0/2492740583 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9d700a2c10 0x7f9d700a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.929+0000 7f9d8d1a7640 1 --2- 172.21.15.6:0/2492740583 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d8810c2f0 0x7f9d8819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.929+0000 7f9d8d1a7640 1 --2- 172.21.15.6:0/2492740583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d881037a0 0x7f9d881a0b10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.929+0000 7f9d8d1a7640 1 --2- 172.21.15.6:0/2492740583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d88102df0 0x7f9d881a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.929+0000 7f9d8d1a7640 1 -- 172.21.15.6:0/2492740583 >> 172.21.15.6:0/2492740583 conn(0x7f9d880fc7d0 msgr2=0x7f9d880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:16.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.930+0000 7f9d8d1a7640 1 -- 172.21.15.6:0/2492740583 shutdown_connections 2024-09-12T03:20:16.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:16.930+0000 7f9d8d1a7640 1 -- 172.21.15.6:0/2492740583 wait complete. 2024-09-12T03:20:17.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:16 smithi118 ceph-mon[35048]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:20:17.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:16 smithi006 ceph-mon[30991]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:20:17.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:16 smithi161 ceph-mon[35485]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:20:17.600 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:20:17.860 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:18.142 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:17 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2492740583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:18.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:17 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2492740583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:18.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.361+0000 7f075a717640 1 -- 172.21.15.6:0/3733363705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0754101bc0 msgr2=0x7f075410d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:18.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.361+0000 7f075a717640 1 --2- 172.21.15.6:0/3733363705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0754101bc0 0x7f075410d910 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f074c009ca0 tx=0x7f074c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.361+0000 7f075a717640 1 -- 172.21.15.6:0/3733363705 shutdown_connections 2024-09-12T03:20:18.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.361+0000 7f075a717640 1 --2- 172.21.15.6:0/3733363705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0754101bc0 0x7f075410d910 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.361+0000 7f075a717640 1 --2- 172.21.15.6:0/3733363705 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0754101240 0x7f0754101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.361+0000 7f075a717640 1 --2- 172.21.15.6:0/3733363705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0754105a90 0x7f0754105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.361+0000 7f075a717640 1 -- 172.21.15.6:0/3733363705 >> 172.21.15.6:0/3733363705 conn(0x7f0754077210 msgr2=0x7f07540fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:18.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.361+0000 7f075a717640 1 -- 172.21.15.6:0/3733363705 shutdown_connections 2024-09-12T03:20:18.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.361+0000 7f075a717640 1 -- 172.21.15.6:0/3733363705 wait complete. 2024-09-12T03:20:18.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.361+0000 7f075a717640 1 Processor -- start 2024-09-12T03:20:18.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.362+0000 7f075a717640 1 -- start start 2024-09-12T03:20:18.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.362+0000 7f075a717640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0754101240 0x7f0754198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:18.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.362+0000 7f075a717640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0754101bc0 0x7f0754198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:18.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.362+0000 7f075a717640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0754105a90 0x7f075419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:18.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.362+0000 7f075a717640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0754110fb0 con 0x7f0754105a90 2024-09-12T03:20:18.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.362+0000 7f075a717640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f0754110e30 con 0x7f0754101240 2024-09-12T03:20:18.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.362+0000 7f075a717640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0754111130 con 0x7f0754101bc0 2024-09-12T03:20:18.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.362+0000 7f0753fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0754101240 0x7f0754198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:18.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.362+0000 7f0753fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0754101240 0x7f0754198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:50190/0 (socket says 172.21.15.6:50190) 2024-09-12T03:20:18.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.362+0000 7f0753fff640 1 -- 172.21.15.6:0/355079321 learned_addr learned my addr 172.21.15.6:0/355079321 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:18.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.362+0000 7f0758cb4640 1 --2- 172.21.15.6:0/355079321 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0754105a90 0x7f075419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:18.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.362+0000 7f07537fe640 1 --2- 172.21.15.6:0/355079321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0754101bc0 0x7f0754198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:18.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.363+0000 7f0758cb4640 1 -- 172.21.15.6:0/355079321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0754101bc0 msgr2=0x7f0754198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:18.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.363+0000 7f0758cb4640 1 --2- 172.21.15.6:0/355079321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0754101bc0 0x7f0754198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.363+0000 7f0758cb4640 1 -- 172.21.15.6:0/355079321 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0754101240 msgr2=0x7f0754198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:18.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.363+0000 7f0758cb4640 1 --2- 172.21.15.6:0/355079321 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0754101240 0x7f0754198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.363+0000 7f0758cb4640 1 -- 172.21.15.6:0/355079321 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075419d3f0 con 0x7f0754105a90 2024-09-12T03:20:18.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.363+0000 7f0753fff640 1 --2- 172.21.15.6:0/355079321 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0754101240 0x7f0754198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:18.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.363+0000 7f0758cb4640 1 --2- 172.21.15.6:0/355079321 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0754105a90 0x7f075419cd10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f074c009ca0 tx=0x7f074c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:18.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.363+0000 7f07517fa640 1 -- 172.21.15.6:0/355079321 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c031750 con 0x7f0754105a90 2024-09-12T03:20:18.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.363+0000 7f07517fa640 1 -- 172.21.15.6:0/355079321 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f074c0318f0 con 0x7f0754105a90 2024-09-12T03:20:18.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.363+0000 7f075a717640 1 -- 172.21.15.6:0/355079321 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075419d680 con 0x7f0754105a90 2024-09-12T03:20:18.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.363+0000 7f07517fa640 1 -- 172.21.15.6:0/355079321 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c030c00 con 0x7f0754105a90 2024-09-12T03:20:18.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.363+0000 7f075a717640 1 -- 172.21.15.6:0/355079321 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07541a4fc0 con 0x7f0754105a90 2024-09-12T03:20:18.367 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:17 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2492740583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:18.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.364+0000 7f075a717640 1 -- 172.21.15.6:0/355079321 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0754101680 con 0x7f0754105a90 2024-09-12T03:20:18.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.365+0000 7f07517fa640 1 -- 172.21.15.6:0/355079321 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f074c004040 con 0x7f0754105a90 2024-09-12T03:20:18.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.366+0000 7f07517fa640 1 --2- 172.21.15.6:0/355079321 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f073c0a2c10 0x7f073c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:18.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.366+0000 7f0753fff640 1 --2- 172.21.15.6:0/355079321 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f073c0a2c10 0x7f073c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:18.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.366+0000 7f07517fa640 1 -- 172.21.15.6:0/355079321 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f074c0f06b0 con 0x7f0754105a90 2024-09-12T03:20:18.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.366+0000 7f0753fff640 1 --2- 172.21.15.6:0/355079321 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f073c0a2c10 0x7f073c0a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0744009e10 tx=0x7f07440091c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:18.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.370+0000 7f07517fa640 1 -- 172.21.15.6:0/355079321 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f074c0bd710 con 0x7f0754105a90 2024-09-12T03:20:18.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.581+0000 7f075a717640 1 -- 172.21.15.6:0/355079321 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07541a5190 con 0x7f0754105a90 2024-09-12T03:20:18.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.582+0000 7f07517fa640 1 -- 172.21.15.6:0/355079321 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11908 (secure 0 0 0) 0x7f074c0bd010 con 0x7f0754105a90 2024-09-12T03:20:18.583 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:20:18.583 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":24,"btime":"2024-09-12T03:19:21:375379+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:19:20.919190+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24463,24464,24494]},"id":1}]} 2024-09-12T03:20:18.583 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 24 2024-09-12T03:20:18.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.584+0000 7f075a717640 1 -- 172.21.15.6:0/355079321 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f073c0a2c10 msgr2=0x7f073c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:18.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.584+0000 7f075a717640 1 --2- 172.21.15.6:0/355079321 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f073c0a2c10 0x7f073c0a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0744009e10 tx=0x7f07440091c0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.584+0000 7f075a717640 1 -- 172.21.15.6:0/355079321 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0754105a90 msgr2=0x7f075419cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:18.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.584+0000 7f075a717640 1 --2- 172.21.15.6:0/355079321 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0754105a90 0x7f075419cd10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f074c009ca0 tx=0x7f074c003f10 comp rx=0 tx=0).stop 2024-09-12T03:20:18.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.585+0000 7f075a717640 1 -- 172.21.15.6:0/355079321 shutdown_connections 2024-09-12T03:20:18.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.585+0000 7f075a717640 1 --2- 172.21.15.6:0/355079321 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f073c0a2c10 0x7f073c0a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.585+0000 7f075a717640 1 --2- 172.21.15.6:0/355079321 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0754105a90 0x7f075419cd10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.585+0000 7f075a717640 1 --2- 172.21.15.6:0/355079321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0754101bc0 0x7f0754198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.585+0000 7f075a717640 1 --2- 172.21.15.6:0/355079321 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0754101240 0x7f0754198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.585+0000 7f075a717640 1 -- 172.21.15.6:0/355079321 >> 172.21.15.6:0/355079321 conn(0x7f0754077210 msgr2=0x7f07540fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:18.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.585+0000 7f075a717640 1 -- 172.21.15.6:0/355079321 shutdown_connections 2024-09-12T03:20:18.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:18.585+0000 7f075a717640 1 -- 172.21.15.6:0/355079321 wait complete. 2024-09-12T03:20:19.165 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:18 smithi006 ceph-mon[30991]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:19.166 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:18 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/355079321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:19.195 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:20:19.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:18 smithi118 ceph-mon[35048]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:19.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:18 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/355079321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:19.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:18 smithi161 ceph-mon[35485]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:19.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:18 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/355079321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:19.451 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:19.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.918+0000 7f74f8930640 1 -- 172.21.15.6:0/2417525717 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f4102790 msgr2=0x7f74f41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:19.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.918+0000 7f74f8930640 1 --2- 172.21.15.6:0/2417525717 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f4102790 0x7f74f41067e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f74dc009900 tx=0x7f74dc02a410 comp rx=0 tx=0).stop 2024-09-12T03:20:19.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.918+0000 7f74f8930640 1 -- 172.21.15.6:0/2417525717 shutdown_connections 2024-09-12T03:20:19.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.918+0000 7f74f8930640 1 --2- 172.21.15.6:0/2417525717 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74f4106f10 0x7f74f410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:19.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.918+0000 7f74f8930640 1 --2- 172.21.15.6:0/2417525717 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f4102790 0x7f74f41067e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:19.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.918+0000 7f74f8930640 1 --2- 172.21.15.6:0/2417525717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74f4101de0 0x7f74f41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:19.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.918+0000 7f74f8930640 1 -- 172.21.15.6:0/2417525717 >> 172.21.15.6:0/2417525717 conn(0x7f74f40fd8c0 msgr2=0x7f74f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:19.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.919+0000 7f74f8930640 1 -- 172.21.15.6:0/2417525717 shutdown_connections 2024-09-12T03:20:19.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.919+0000 7f74f8930640 1 -- 172.21.15.6:0/2417525717 wait complete. 2024-09-12T03:20:19.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.919+0000 7f74f8930640 1 Processor -- start 2024-09-12T03:20:19.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.919+0000 7f74f8930640 1 -- start start 2024-09-12T03:20:19.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.919+0000 7f74f8930640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f4101de0 0x7f74f40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:19.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.919+0000 7f74f8930640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74f4102790 0x7f74f4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:19.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.920+0000 7f74f8930640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74f4106f10 0x7f74f4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:19.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.920+0000 7f74f8930640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f74f4111f90 con 0x7f74f4101de0 2024-09-12T03:20:19.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.920+0000 7f74f8930640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f74f4111e10 con 0x7f74f4106f10 2024-09-12T03:20:19.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.920+0000 7f74f8930640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f74f4112110 con 0x7f74f4102790 2024-09-12T03:20:19.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.920+0000 7f74f259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f4101de0 0x7f74f40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:19.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.920+0000 7f74f259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f4101de0 0x7f74f40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35492/0 (socket says 172.21.15.6:35492) 2024-09-12T03:20:19.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.920+0000 7f74f259c640 1 -- 172.21.15.6:0/89652624 learned_addr learned my addr 172.21.15.6:0/89652624 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:19.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.920+0000 7f74ebfff640 1 --2- 172.21.15.6:0/89652624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74f4102790 0x7f74f4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:19.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.920+0000 7f74f2d9d640 1 --2- 172.21.15.6:0/89652624 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74f4106f10 0x7f74f4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:19.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.920+0000 7f74f259c640 1 -- 172.21.15.6:0/89652624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74f4102790 msgr2=0x7f74f4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:19.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.920+0000 7f74f259c640 1 --2- 172.21.15.6:0/89652624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74f4102790 0x7f74f4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:19.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.920+0000 7f74f259c640 1 -- 172.21.15.6:0/89652624 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74f4106f10 msgr2=0x7f74f4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:19.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.920+0000 7f74f259c640 1 --2- 172.21.15.6:0/89652624 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74f4106f10 0x7f74f4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:19.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.920+0000 7f74f259c640 1 -- 172.21.15.6:0/89652624 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f41a68d0 con 0x7f74f4101de0 2024-09-12T03:20:19.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.920+0000 7f74f259c640 1 --2- 172.21.15.6:0/89652624 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f4101de0 0x7f74f40744f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f74e400b4a0 tx=0x7f74e400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:19.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.920+0000 7f74eb7fe640 1 -- 172.21.15.6:0/89652624 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4014070 con 0x7f74f4101de0 2024-09-12T03:20:19.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.920+0000 7f74eb7fe640 1 -- 172.21.15.6:0/89652624 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74e4010af0 con 0x7f74f4101de0 2024-09-12T03:20:19.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.920+0000 7f74eb7fe640 1 -- 172.21.15.6:0/89652624 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4017d10 con 0x7f74f4101de0 2024-09-12T03:20:19.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.920+0000 7f74f8930640 1 -- 172.21.15.6:0/89652624 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f41a6bc0 con 0x7f74f4101de0 2024-09-12T03:20:19.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.921+0000 7f74f8930640 1 -- 172.21.15.6:0/89652624 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f41a70d0 con 0x7f74f4101de0 2024-09-12T03:20:19.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.922+0000 7f74f8930640 1 -- 172.21.15.6:0/89652624 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74f41021c0 con 0x7f74f4101de0 2024-09-12T03:20:19.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.923+0000 7f74eb7fe640 1 -- 172.21.15.6:0/89652624 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f74e4017070 con 0x7f74f4101de0 2024-09-12T03:20:19.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.923+0000 7f74eb7fe640 1 --2- 172.21.15.6:0/89652624 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f74e00a2c90 0x7f74e00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:19.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.924+0000 7f74ebfff640 1 --2- 172.21.15.6:0/89652624 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f74e00a2c90 0x7f74e00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:19.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.924+0000 7f74eb7fe640 1 -- 172.21.15.6:0/89652624 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f74e40ce7a0 con 0x7f74f4101de0 2024-09-12T03:20:19.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.924+0000 7f74ebfff640 1 --2- 172.21.15.6:0/89652624 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f74e00a2c90 0x7f74e00a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f74dc000c00 tx=0x7f74dc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:19.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:19.927+0000 7f74eb7fe640 1 -- 172.21.15.6:0/89652624 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f74e409b6e0 con 0x7f74f4101de0 2024-09-12T03:20:20.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:20.103+0000 7f74f8930640 1 -- 172.21.15.6:0/89652624 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74f402d7c0 con 0x7f74f4101de0 2024-09-12T03:20:20.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:20.104+0000 7f74eb7fe640 1 -- 172.21.15.6:0/89652624 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22607 (secure 0 0 0) 0x7f74e409afe0 con 0x7f74f4101de0 2024-09-12T03:20:20.105 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:20:20.105 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":82,"fsid":"89d99938-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:11:29.530545+0000","modified":"2024-09-12T03:19:20.366710+0000","last_up_change":"2024-09-12T03:18:21.292572+0000","last_in_change":"2024-09-12T03:17:58.013800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:14:37.662010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:06.143398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:08.305279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f55d68b-95a0-4ee2-a46f-bc422b38471e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6803","nonce":726729286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6805","nonce":726729286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6809","nonce":726729286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6807","nonce":726729286}]},"public_addr":"172.21.15.6:6803/726729286","cluster_addr":"172.21.15.6:6805/726729286","heartbeat_back_addr":"172.21.15.6:6809/726729286","heartbeat_front_addr":"172.21.15.6:6807/726729286","state":["exists","up"]},{"osd":1,"uuid":"83afdb01-c731-4de1-96c6-3f798549b1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6801","nonce":3377555415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6803","nonce":3377555415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6807","nonce":3377555415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6805","nonce":3377555415}]},"public_addr":"172.21.15.118:6801/3377555415","cluster_addr":"172.21.15.118:6803/3377555415","heartbeat_back_addr":"172.21.15.118:6807/3377555415","heartbeat_front_addr":"172.21.15.118:6805/3377555415","state":["exists","up"]},{"osd":2,"uuid":"aed718b4-b849-403a-b81c-3a140b5c380c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6801","nonce":392541009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6803","nonce":392541009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6807","nonce":392541009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6805","nonce":392541009}]},"public_addr":"172.21.15.161:6801/392541009","cluster_addr":"172.21.15.161:6803/392541009","heartbeat_back_addr":"172.21.15.161:6807/392541009","heartbeat_front_addr":"172.21.15.161:6805/392541009","state":["exists","up"]},{"osd":3,"uuid":"25ced4c1-bf31-402d-8397-d42fd57f615f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6811","nonce":1892039151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6813","nonce":1892039151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6817","nonce":1892039151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6815","nonce":1892039151}]},"public_addr":"172.21.15.6:6811/1892039151","cluster_addr":"172.21.15.6:6813/1892039151","heartbeat_back_addr":"172.21.15.6:6817/1892039151","heartbeat_front_addr":"172.21.15.6:6815/1892039151","state":["exists","up"]},{"osd":4,"uuid":"1d884604-920b-4c35-b41e-d629ad9f9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6809","nonce":150177024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6811","nonce":150177024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6815","nonce":150177024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6813","nonce":150177024}]},"public_addr":"172.21.15.118:6809/150177024","cluster_addr":"172.21.15.118:6811/150177024","heartbeat_back_addr":"172.21.15.118:6815/150177024","heartbeat_front_addr":"172.21.15.118:6813/150177024","state":["exists","up"]},{"osd":5,"uuid":"1d40c0ef-99f6-426f-90c7-27579edc66af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6809","nonce":3758167854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6811","nonce":3758167854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6815","nonce":3758167854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6813","nonce":3758167854}]},"public_addr":"172.21.15.161:6809/3758167854","cluster_addr":"172.21.15.161:6811/3758167854","heartbeat_back_addr":"172.21.15.161:6815/3758167854","heartbeat_front_addr":"172.21.15.161:6813/3758167854","state":["exists","up"]},{"osd":6,"uuid":"9d9f9510-b107-40c2-9a98-9012715944b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6819","nonce":2129224874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6821","nonce":2129224874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6825","nonce":2129224874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6823","nonce":2129224874}]},"public_addr":"172.21.15.6:6819/2129224874","cluster_addr":"172.21.15.6:6821/2129224874","heartbeat_back_addr":"172.21.15.6:6825/2129224874","heartbeat_front_addr":"172.21.15.6:6823/2129224874","state":["exists","up"]},{"osd":7,"uuid":"84635ae1-3240-4954-af14-76bb3aa162e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6817","nonce":269980353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6819","nonce":269980353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6823","nonce":269980353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6821","nonce":269980353}]},"public_addr":"172.21.15.118:6817/269980353","cluster_addr":"172.21.15.118:6819/269980353","heartbeat_back_addr":"172.21.15.118:6823/269980353","heartbeat_front_addr":"172.21.15.118:6821/269980353","state":["exists","up"]},{"osd":8,"uuid":"6fd58e55-1492-4595-8647-823ea3e3c5d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6817","nonce":1783416134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6819","nonce":1783416134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6823","nonce":1783416134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6821","nonce":1783416134}]},"public_addr":"172.21.15.161:6817/1783416134","cluster_addr":"172.21.15.161:6819/1783416134","heartbeat_back_addr":"172.21.15.161:6823/1783416134","heartbeat_front_addr":"172.21.15.161:6821/1783416134","state":["exists","up"]},{"osd":9,"uuid":"90508a6d-a49c-43c6-a600-5bbccd8702ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6827","nonce":2987332365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6829","nonce":2987332365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6833","nonce":2987332365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6831","nonce":2987332365}]},"public_addr":"172.21.15.6:6827/2987332365","cluster_addr":"172.21.15.6:6829/2987332365","heartbeat_back_addr":"172.21.15.6:6833/2987332365","heartbeat_front_addr":"172.21.15.6:6831/2987332365","state":["exists","up"]},{"osd":10,"uuid":"911e86d1-669a-4935-82c0-7da51d141df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6825","nonce":2167146341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6827","nonce":2167146341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6831","nonce":2167146341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6829","nonce":2167146341}]},"public_addr":"172.21.15.118:6825/2167146341","cluster_addr":"172.21.15.118:6827/2167146341","heartbeat_back_addr":"172.21.15.118:6831/2167146341","heartbeat_front_addr":"172.21.15.118:6829/2167146341","state":["exists","up"]},{"osd":11,"uuid":"5a15c6a7-ff43-473c-bb48-0c129c316df5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6825","nonce":2889795796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6827","nonce":2889795796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6831","nonce":2889795796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6829","nonce":2889795796}]},"public_addr":"172.21.15.161:6825/2889795796","cluster_addr":"172.21.15.161:6827/2889795796","heartbeat_back_addr":"172.21.15.161:6831/2889795796","heartbeat_front_addr":"172.21.15.161:6829/2889795796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:42.912205+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:08.410669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:34.299134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:59.093181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:24.116635+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:49.566433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:15.719320+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:40.541017+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:05.178178+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:31.163877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:55.405110+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:19.417156+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6800/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/2620491916":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/4037656392":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/1377116573":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/3062800125":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:6800/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/4145338968":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/825701128":"2024-09-13T03:11:55.244257+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:20:20.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:20.106+0000 7f74f8930640 1 -- 172.21.15.6:0/89652624 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f74e00a2c90 msgr2=0x7f74e00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:20.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:20.106+0000 7f74f8930640 1 --2- 172.21.15.6:0/89652624 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f74e00a2c90 0x7f74e00a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f74dc000c00 tx=0x7f74dc0058d0 comp rx=0 tx=0).stop 2024-09-12T03:20:20.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:20.106+0000 7f74f8930640 1 -- 172.21.15.6:0/89652624 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f4101de0 msgr2=0x7f74f40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:20.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:20.106+0000 7f74f8930640 1 --2- 172.21.15.6:0/89652624 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f4101de0 0x7f74f40744f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f74e400b4a0 tx=0x7f74e400b960 comp rx=0 tx=0).stop 2024-09-12T03:20:20.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:20.107+0000 7f74f8930640 1 -- 172.21.15.6:0/89652624 shutdown_connections 2024-09-12T03:20:20.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:20.107+0000 7f74f8930640 1 --2- 172.21.15.6:0/89652624 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f74e00a2c90 0x7f74e00a5150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:20.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:20.107+0000 7f74f8930640 1 --2- 172.21.15.6:0/89652624 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74f4106f10 0x7f74f4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:20.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:20.107+0000 7f74f8930640 1 --2- 172.21.15.6:0/89652624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74f4102790 0x7f74f4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:20.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:20.107+0000 7f74f8930640 1 --2- 172.21.15.6:0/89652624 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f4101de0 0x7f74f40744f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:20.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:20.107+0000 7f74f8930640 1 -- 172.21.15.6:0/89652624 >> 172.21.15.6:0/89652624 conn(0x7f74f40fd8c0 msgr2=0x7f74f40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:20.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:20.107+0000 7f74f8930640 1 -- 172.21.15.6:0/89652624 shutdown_connections 2024-09-12T03:20:20.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:20.107+0000 7f74f8930640 1 -- 172.21.15.6:0/89652624 wait complete. 2024-09-12T03:20:20.694 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:20:20.950 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:21.072 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:20 smithi006 ceph-mon[30991]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:20:21.072 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:20 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/89652624' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:20:21.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:20 smithi118 ceph-mon[35048]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:20:21.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:20 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/89652624' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:20:21.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:20 smithi161 ceph-mon[35485]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:20:21.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:20 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/89652624' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:20:21.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.441+0000 7f2b007f5640 1 -- 172.21.15.6:0/3240614164 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2af8103660 msgr2=0x7f2af810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:21.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.441+0000 7f2b007f5640 1 --2- 172.21.15.6:0/3240614164 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2af8103660 0x7f2af810bb50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009900 tx=0x7f2ae802a480 comp rx=0 tx=0).stop 2024-09-12T03:20:21.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.441+0000 7f2b007f5640 1 -- 172.21.15.6:0/3240614164 shutdown_connections 2024-09-12T03:20:21.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.441+0000 7f2b007f5640 1 --2- 172.21.15.6:0/3240614164 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2af810c170 0x7f2af810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.441+0000 7f2b007f5640 1 --2- 172.21.15.6:0/3240614164 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2af8103660 0x7f2af810bb50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.441+0000 7f2b007f5640 1 --2- 172.21.15.6:0/3240614164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af8102d40 0x7f2af8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.441+0000 7f2b007f5640 1 -- 172.21.15.6:0/3240614164 >> 172.21.15.6:0/3240614164 conn(0x7f2af80fc7a0 msgr2=0x7f2af80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:21.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.441+0000 7f2b007f5640 1 -- 172.21.15.6:0/3240614164 shutdown_connections 2024-09-12T03:20:21.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.441+0000 7f2b007f5640 1 -- 172.21.15.6:0/3240614164 wait complete. 2024-09-12T03:20:21.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.442+0000 7f2b007f5640 1 Processor -- start 2024-09-12T03:20:21.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.442+0000 7f2b007f5640 1 -- start start 2024-09-12T03:20:21.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.442+0000 7f2b007f5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2af8102d40 0x7f2af8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:21.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.442+0000 7f2b007f5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af8103660 0x7f2af8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:21.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.442+0000 7f2b007f5640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2af810c170 0x7f2af819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:21.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.442+0000 7f2b007f5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2af8111fc0 con 0x7f2af8102d40 2024-09-12T03:20:21.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.442+0000 7f2b007f5640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f2af8111e40 con 0x7f2af810c170 2024-09-12T03:20:21.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.442+0000 7f2b007f5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2af8112140 con 0x7f2af8103660 2024-09-12T03:20:21.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.442+0000 7f2afe591640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2af8102d40 0x7f2af8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:21.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.442+0000 7f2afe591640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2af8102d40 0x7f2af8198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35516/0 (socket says 172.21.15.6:35516) 2024-09-12T03:20:21.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.442+0000 7f2afe591640 1 -- 172.21.15.6:0/2501664733 learned_addr learned my addr 172.21.15.6:0/2501664733 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:21.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.442+0000 7f2afe591640 1 -- 172.21.15.6:0/2501664733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af8103660 msgr2=0x7f2af8198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:21.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.442+0000 7f2afe591640 1 --2- 172.21.15.6:0/2501664733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af8103660 0x7f2af8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.442+0000 7f2afed92640 1 --2- 172.21.15.6:0/2501664733 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2af810c170 0x7f2af819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:21.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.442+0000 7f2afe591640 1 -- 172.21.15.6:0/2501664733 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2af810c170 msgr2=0x7f2af819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:21.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.442+0000 7f2afe591640 1 --2- 172.21.15.6:0/2501664733 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2af810c170 0x7f2af819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.442+0000 7f2afe591640 1 -- 172.21.15.6:0/2501664733 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af819d3b0 con 0x7f2af8102d40 2024-09-12T03:20:21.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.442+0000 7f2afed92640 1 --2- 172.21.15.6:0/2501664733 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2af810c170 0x7f2af819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:21.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.442+0000 7f2afe591640 1 --2- 172.21.15.6:0/2501664733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2af8102d40 0x7f2af8198400 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2af400d950 tx=0x7f2af400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:21.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.443+0000 7f2aef7fe640 1 -- 172.21.15.6:0/2501664733 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4017070 con 0x7f2af8102d40 2024-09-12T03:20:21.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.443+0000 7f2aef7fe640 1 -- 172.21.15.6:0/2501664733 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2af4009c30 con 0x7f2af8102d40 2024-09-12T03:20:21.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.443+0000 7f2b007f5640 1 -- 172.21.15.6:0/2501664733 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af819d6a0 con 0x7f2af8102d40 2024-09-12T03:20:21.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.443+0000 7f2aef7fe640 1 -- 172.21.15.6:0/2501664733 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af400f6b0 con 0x7f2af8102d40 2024-09-12T03:20:21.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.443+0000 7f2b007f5640 1 -- 172.21.15.6:0/2501664733 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af81a4f80 con 0x7f2af8102d40 2024-09-12T03:20:21.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.444+0000 7f2b007f5640 1 -- 172.21.15.6:0/2501664733 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2af8103120 con 0x7f2af8102d40 2024-09-12T03:20:21.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.445+0000 7f2aef7fe640 1 -- 172.21.15.6:0/2501664733 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2af40109d0 con 0x7f2af8102d40 2024-09-12T03:20:21.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.445+0000 7f2aef7fe640 1 --2- 172.21.15.6:0/2501664733 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2ae40a2c90 0x7f2ae40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:21.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.446+0000 7f2afdd90640 1 --2- 172.21.15.6:0/2501664733 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2ae40a2c90 0x7f2ae40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:21.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.446+0000 7f2aef7fe640 1 -- 172.21.15.6:0/2501664733 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f2af40cee10 con 0x7f2af8102d40 2024-09-12T03:20:21.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.446+0000 7f2afdd90640 1 --2- 172.21.15.6:0/2501664733 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2ae40a2c90 0x7f2ae40a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80096c0 tx=0x7f2ae8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:21.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.449+0000 7f2aef7fe640 1 -- 172.21.15.6:0/2501664733 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2af4009de0 con 0x7f2af8102d40 2024-09-12T03:20:21.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.661+0000 7f2b007f5640 1 -- 172.21.15.6:0/2501664733 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2af8107240 con 0x7f2af8102d40 2024-09-12T03:20:21.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.662+0000 7f2aef7fe640 1 -- 172.21.15.6:0/2501664733 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11908 (secure 0 0 0) 0x7f2af409bcc0 con 0x7f2af8102d40 2024-09-12T03:20:21.662 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:20:21.662 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":24,"btime":"2024-09-12T03:19:21:375379+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:19:20.919190+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24463,24464,24494]},"id":1}]} 2024-09-12T03:20:21.662 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 24 2024-09-12T03:20:21.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.664+0000 7f2b007f5640 1 -- 172.21.15.6:0/2501664733 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2ae40a2c90 msgr2=0x7f2ae40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:21.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.664+0000 7f2b007f5640 1 --2- 172.21.15.6:0/2501664733 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2ae40a2c90 0x7f2ae40a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80096c0 tx=0x7f2ae8000bc0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.664+0000 7f2b007f5640 1 -- 172.21.15.6:0/2501664733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2af8102d40 msgr2=0x7f2af8198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:21.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.664+0000 7f2b007f5640 1 --2- 172.21.15.6:0/2501664733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2af8102d40 0x7f2af8198400 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2af400d950 tx=0x7f2af400de10 comp rx=0 tx=0).stop 2024-09-12T03:20:21.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.664+0000 7f2b007f5640 1 -- 172.21.15.6:0/2501664733 shutdown_connections 2024-09-12T03:20:21.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.664+0000 7f2b007f5640 1 --2- 172.21.15.6:0/2501664733 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2ae40a2c90 0x7f2ae40a5150 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.664+0000 7f2b007f5640 1 --2- 172.21.15.6:0/2501664733 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2af810c170 0x7f2af819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.664+0000 7f2b007f5640 1 --2- 172.21.15.6:0/2501664733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af8103660 0x7f2af8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.664+0000 7f2b007f5640 1 --2- 172.21.15.6:0/2501664733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2af8102d40 0x7f2af8198400 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.664+0000 7f2b007f5640 1 -- 172.21.15.6:0/2501664733 >> 172.21.15.6:0/2501664733 conn(0x7f2af80fc7a0 msgr2=0x7f2af80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:21.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.664+0000 7f2b007f5640 1 -- 172.21.15.6:0/2501664733 shutdown_connections 2024-09-12T03:20:21.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:21.664+0000 7f2b007f5640 1 -- 172.21.15.6:0/2501664733 wait complete. 2024-09-12T03:20:22.102 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:21 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2501664733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:22.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:21 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2501664733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:22.285 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T03:19:09.851402+0000', 'modified': '2024-09-12T03:19:20.919190+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24494, 'mds_1': 24464, 'mds_2': 24463}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24439': {'gid': 24439, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/955127607', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 955127607}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 955127607}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.118:6835/1389654073', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6834', 'nonce': 1389654073}, {'type': 'v1', 'addr': '172.21.15.118:6835', 'nonce': 1389654073}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6837/283024944', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 283024944}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 283024944}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.161:6839/3645555254', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 3645555254}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 3645555254}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.118:6837/3879191727', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6836', 'nonce': 3879191727}, {'type': 'v1', 'addr': '172.21.15.118:6837', 'nonce': 3879191727}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.118:6839/3035878957', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6838', 'nonce': 3035878957}, {'type': 'v1', 'addr': '172.21.15.118:6839', 'nonce': 3035878957}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24463, 24464, 24494]} 2024-09-12T03:20:22.285 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T03:20:22.285 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T03:20:22.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:21 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2501664733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:22.554 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:23.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.029+0000 7f15fe181640 1 -- 172.21.15.6:0/3022354136 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15f8106f20 msgr2=0x7f15f810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.029+0000 7f15fe181640 1 --2- 172.21.15.6:0/3022354136 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15f8106f20 0x7f15f810eaa0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009ca0 tx=0x7f15f002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.029+0000 7f15fe181640 1 -- 172.21.15.6:0/3022354136 shutdown_connections 2024-09-12T03:20:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.029+0000 7f15fe181640 1 --2- 172.21.15.6:0/3022354136 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15f8106f20 0x7f15f810eaa0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.029+0000 7f15fe181640 1 --2- 172.21.15.6:0/3022354136 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15f81027a0 0x7f15f81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.029+0000 7f15fe181640 1 --2- 172.21.15.6:0/3022354136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15f8101df0 0x7f15f81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.029+0000 7f15fe181640 1 -- 172.21.15.6:0/3022354136 >> 172.21.15.6:0/3022354136 conn(0x7f15f80fd8f0 msgr2=0x7f15f80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.029+0000 7f15fe181640 1 -- 172.21.15.6:0/3022354136 shutdown_connections 2024-09-12T03:20:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.029+0000 7f15fe181640 1 -- 172.21.15.6:0/3022354136 wait complete. 2024-09-12T03:20:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.030+0000 7f15fe181640 1 Processor -- start 2024-09-12T03:20:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.030+0000 7f15fe181640 1 -- start start 2024-09-12T03:20:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.030+0000 7f15fe181640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15f8101df0 0x7f15f81983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.030+0000 7f15fe181640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15f81027a0 0x7f15f81988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.030+0000 7f15fe181640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15f8106f20 0x7f15f819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.030+0000 7f15fe181640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f15f8111f80 con 0x7f15f8106f20 2024-09-12T03:20:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.030+0000 7f15fe181640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f15f8111e00 con 0x7f15f81027a0 2024-09-12T03:20:23.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.030+0000 7f15fe181640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f15f8112100 con 0x7f15f8101df0 2024-09-12T03:20:23.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.030+0000 7f15f7fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15f8106f20 0x7f15f819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.030+0000 7f15f7fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15f8106f20 0x7f15f819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35530/0 (socket says 172.21.15.6:35530) 2024-09-12T03:20:23.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.030+0000 7f15f77fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15f8101df0 0x7f15f81983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.030+0000 7f15f7fff640 1 -- 172.21.15.6:0/828266061 learned_addr learned my addr 172.21.15.6:0/828266061 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:23.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.030+0000 7f15f6ffd640 1 --2- 172.21.15.6:0/828266061 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15f81027a0 0x7f15f81988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.031+0000 7f15f7fff640 1 -- 172.21.15.6:0/828266061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15f8101df0 msgr2=0x7f15f81983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.031+0000 7f15f7fff640 1 --2- 172.21.15.6:0/828266061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15f8101df0 0x7f15f81983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.031+0000 7f15f7fff640 1 -- 172.21.15.6:0/828266061 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15f81027a0 msgr2=0x7f15f81988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.031+0000 7f15f7fff640 1 --2- 172.21.15.6:0/828266061 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15f81027a0 0x7f15f81988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.031+0000 7f15f7fff640 1 -- 172.21.15.6:0/828266061 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f819d350 con 0x7f15f8106f20 2024-09-12T03:20:23.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.031+0000 7f15f6ffd640 1 --2- 172.21.15.6:0/828266061 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15f81027a0 0x7f15f81988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:23.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.031+0000 7f15f7fff640 1 --2- 172.21.15.6:0/828266061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15f8106f20 0x7f15f819cc70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f15f0031a60 tx=0x7f15f0031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:23.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.031+0000 7f15f4ff9640 1 -- 172.21.15.6:0/828266061 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0003c30 con 0x7f15f8106f20 2024-09-12T03:20:23.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.031+0000 7f15f4ff9640 1 -- 172.21.15.6:0/828266061 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15f0003dd0 con 0x7f15f8106f20 2024-09-12T03:20:23.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.031+0000 7f15fe181640 1 -- 172.21.15.6:0/828266061 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f819d5e0 con 0x7f15f8106f20 2024-09-12T03:20:23.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.031+0000 7f15f4ff9640 1 -- 172.21.15.6:0/828266061 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f00047e0 con 0x7f15f8106f20 2024-09-12T03:20:23.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.031+0000 7f15fe181640 1 -- 172.21.15.6:0/828266061 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f81a4f20 con 0x7f15f8106f20 2024-09-12T03:20:23.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.034+0000 7f160055a740 1 Processor -- start 2024-09-12T03:20:23.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.034+0000 7f15f4ff9640 1 -- 172.21.15.6:0/828266061 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f15f002f790 con 0x7f15f8106f20 2024-09-12T03:20:23.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.034+0000 7f160055a740 1 -- start start 2024-09-12T03:20:23.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.035+0000 7f160055a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec309d0 0x5569cec30db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.035+0000 7f160055a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec312f0 0x5569cec3b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.035+0000 7f160055a740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5569cec3bbb0 0x5569cec3dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.035+0000 7f160055a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5569ceb97090 con 0x5569cec309d0 2024-09-12T03:20:23.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.035+0000 7f160055a740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5569ceb3ea70 con 0x5569cec3bbb0 2024-09-12T03:20:23.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.035+0000 7f160055a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5569ceb93050 con 0x5569cec312f0 2024-09-12T03:20:23.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.035+0000 7f15f6ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec309d0 0x5569cec30db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.035+0000 7f15f77fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec312f0 0x5569cec3b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.035+0000 7f15f77fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec312f0 0x5569cec3b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:51416/0 (socket says 172.21.15.6:51416) 2024-09-12T03:20:23.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.035+0000 7f15f77fe640 1 -- 172.21.15.6:0/3169182510 learned_addr learned my addr 172.21.15.6:0/3169182510 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:23.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.035+0000 7f15f6ffd640 1 --2- 172.21.15.6:0/3169182510 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5569cec3bbb0 0x5569cec3dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.035+0000 7f15f4ff9640 1 --2- 172.21.15.6:0/828266061 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f15e00a2c90 0x7f15e00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.035+0000 7f15f77fe640 1 -- 172.21.15.6:0/3169182510 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5569cec3bbb0 msgr2=0x5569cec3dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:23.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.035+0000 7f15f77fe640 1 --2- 172.21.15.6:0/3169182510 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5569cec3bbb0 0x5569cec3dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.035+0000 7f15f77fe640 1 -- 172.21.15.6:0/3169182510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec309d0 msgr2=0x5569cec30db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:23.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.035+0000 7f15f77fe640 1 --2- 172.21.15.6:0/3169182510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec309d0 0x5569cec30db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.035+0000 7f15f77fe640 1 -- 172.21.15.6:0/3169182510 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569ceb4b130 con 0x5569cec312f0 2024-09-12T03:20:23.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.035+0000 7f15f7fff640 1 --2- 172.21.15.6:0/828266061 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f15e00a2c90 0x7f15e00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.036+0000 7f15f4ff9640 1 -- 172.21.15.6:0/828266061 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f15f00f0440 con 0x7f15f8106f20 2024-09-12T03:20:23.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.036+0000 7f15f77fe640 1 --2- 172.21.15.6:0/3169182510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec312f0 0x5569cec3b670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f15e800ac10 tx=0x7f15e800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=374a2c66f02a46c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:23.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.036+0000 7f15ddffb640 1 -- 172.21.15.6:0/3169182510 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e801d070 con 0x5569cec312f0 2024-09-12T03:20:23.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.036+0000 7f15ddffb640 1 -- 172.21.15.6:0/3169182510 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15e8018b20 con 0x5569cec312f0 2024-09-12T03:20:23.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.036+0000 7f15ddffb640 1 -- 172.21.15.6:0/3169182510 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8010ad0 con 0x5569cec312f0 2024-09-12T03:20:23.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.037+0000 7f160055a740 1 -- 172.21.15.6:0/3169182510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec312f0 msgr2=0x5569cec3b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.037+0000 7f15f7fff640 1 --2- 172.21.15.6:0/828266061 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f15e00a2c90 0x7f15e00a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f15f00bcd30 tx=0x7f15f00e7810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:23.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.037+0000 7f160055a740 1 --2- 172.21.15.6:0/3169182510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec312f0 0x5569cec3b670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f15e800ac10 tx=0x7f15e800f790 comp rx=0 tx=0).stop 2024-09-12T03:20:23.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.037+0000 7f160055a740 1 -- 172.21.15.6:0/3169182510 shutdown_connections 2024-09-12T03:20:23.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.037+0000 7f160055a740 1 --2- 172.21.15.6:0/3169182510 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5569cec3bbb0 0x5569cec3dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.037+0000 7f160055a740 1 --2- 172.21.15.6:0/3169182510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec312f0 0x5569cec3b670 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.037+0000 7f160055a740 1 --2- 172.21.15.6:0/3169182510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec309d0 0x5569cec30db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.037+0000 7f160055a740 1 -- 172.21.15.6:0/3169182510 >> 172.21.15.6:0/3169182510 conn(0x5569cec2cf60 msgr2=0x5569cec2d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:23.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.037+0000 7f160055a740 1 -- 172.21.15.6:0/3169182510 shutdown_connections 2024-09-12T03:20:23.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.037+0000 7f160055a740 1 -- 172.21.15.6:0/3169182510 wait complete. 2024-09-12T03:20:23.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.037+0000 7f160055a740 1 -- start start 2024-09-12T03:20:23.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.038+0000 7f160055a740 1 Processor -- start 2024-09-12T03:20:23.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.038+0000 7f160055a740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5569cec309d0 0x5569cec3b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.038+0000 7f160055a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec312f0 0x5569cec387e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.038+0000 7f160055a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec3bbb0 0x5569cec38d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.038+0000 7f160055a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5569cec42b90 con 0x5569cec3bbb0 2024-09-12T03:20:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.038+0000 7f160055a740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5569ceb51f20 con 0x5569cec309d0 2024-09-12T03:20:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.038+0000 7f160055a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5569cec42d10 con 0x5569cec312f0 2024-09-12T03:20:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.038+0000 7f160055a740 10 client.0 _ll_register_callbacks cb 0x5569ceb1b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.038+0000 7f15f6ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec3bbb0 0x5569cec38d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.038+0000 7f15f77fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec312f0 0x5569cec387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.038+0000 7f15f6ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec3bbb0 0x5569cec38d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35548/0 (socket says 172.21.15.6:35548) 2024-09-12T03:20:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.038+0000 7f15f6ffd640 1 -- 172.21.15.6:0/2639975273 learned_addr learned my addr 172.21.15.6:0/2639975273 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.038+0000 7f15f6ffd640 1 --2- 172.21.15.6:0/2639975273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5569cec309d0 0x5569cec3b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.038+0000 7f15f77fe640 1 -- 172.21.15.6:0/2639975273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5569cec309d0 msgr2=0x5569cec3b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.038+0000 7f15f77fe640 1 --2- 172.21.15.6:0/2639975273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5569cec309d0 0x5569cec3b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.038+0000 7f15f77fe640 1 -- 172.21.15.6:0/2639975273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec3bbb0 msgr2=0x5569cec38d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.038+0000 7f15f77fe640 1 --2- 172.21.15.6:0/2639975273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec3bbb0 0x5569cec38d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.038+0000 7f15f77fe640 1 -- 172.21.15.6:0/2639975273 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569ce9082b0 con 0x5569cec312f0 2024-09-12T03:20:23.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.039+0000 7f15f77fe640 1 --2- 172.21.15.6:0/2639975273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec312f0 0x5569cec387e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f15e801b040 tx=0x7f15e8010790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=524e70eeee784d1c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:23.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.039+0000 7f15d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:20:23.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.039+0000 7f15d6ffd640 1 -- 172.21.15.6:0/2639975273 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e801d050 con 0x5569cec312f0 2024-09-12T03:20:23.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.039+0000 7f160055a740 1 -- 172.21.15.6:0/2639975273 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15e8018b20 con 0x5569cec312f0 2024-09-12T03:20:23.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.039+0000 7f15d6ffd640 1 -- 172.21.15.6:0/2639975273 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f15e8018650 con 0x5569cec312f0 2024-09-12T03:20:23.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.039+0000 7f15d6ffd640 1 -- 172.21.15.6:0/2639975273 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f15e8018850 con 0x5569cec312f0 2024-09-12T03:20:23.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.039+0000 7f160055a740 10 client.24670 fetch_fsmap learned FSMap version 24 2024-09-12T03:20:23.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.039+0000 7f160055a740 1 -- 172.21.15.6:0/2639975273 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5569cec4e500 con 0x5569cec312f0 2024-09-12T03:20:23.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.039+0000 7f15d6ffd640 1 -- 172.21.15.6:0/2639975273 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15e8018b20 con 0x5569cec312f0 2024-09-12T03:20:23.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.040+0000 7f15d6ffd640 1 -- 172.21.15.6:0/2639975273 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f15e80046c0 con 0x5569cec312f0 2024-09-12T03:20:23.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.040+0000 7f160055a740 10 client.24670 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:20:23.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.040+0000 7f160055a740 10 client.24670 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:20:23.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.040+0000 7f160055a740 20 client.24670 populate_metadata read hostname 'smithi006' 2024-09-12T03:20:23.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.040+0000 7f160055a740 1 --2- 172.21.15.6:0/2639975273 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5569cec50790 0x5569cec51c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.040+0000 7f160055a740 4 client.24670 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:23.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.040+0000 7f160055a740 1 -- 172.21.15.6:0/2639975273 --> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569ceba4600 con 0x5569cec50790 2024-09-12T03:20:23.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.040+0000 7f15f6ffd640 1 --2- 172.21.15.6:0/2639975273 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5569cec50790 0x5569cec51c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.040+0000 7f15d6ffd640 10 client.24670 ms_handle_connect on v2:172.21.15.161:6838/3645555254 2024-09-12T03:20:23.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.040+0000 7f15f6ffd640 1 --2- 172.21.15.6:0/2639975273 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5569cec50790 0x5569cec51c00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=901ef11afe54e22e server_cookie=22327647351d2be6 in_seq=0 out_seq=0 2024-09-12T03:20:23.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.041+0000 7f15d6ffd640 1 -- 172.21.15.6:0/2639975273 <== mds.2 v2:172.21.15.161:6838/3645555254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5569ceba4600 con 0x5569cec50790 2024-09-12T03:20:23.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.041+0000 7f15d6ffd640 10 client.24670 handle_command_reply: tid=0 2024-09-12T03:20:23.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.041+0000 7f160055a740 1 client.24670 shutdown 2024-09-12T03:20:23.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.042+0000 7f160055a740 1 -- 172.21.15.6:0/2639975273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec312f0 msgr2=0x5569cec387e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.042+0000 7f160055a740 1 --2- 172.21.15.6:0/2639975273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec312f0 0x5569cec387e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f15e801b040 tx=0x7f15e8010790 comp rx=0 tx=0).stop 2024-09-12T03:20:23.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.042+0000 7f160055a740 1 -- 172.21.15.6:0/2639975273 shutdown_connections 2024-09-12T03:20:23.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.042+0000 7f160055a740 1 --2- 172.21.15.6:0/2639975273 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5569cec50790 0x5569cec51c00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.042+0000 7f160055a740 1 --2- 172.21.15.6:0/2639975273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec3bbb0 0x5569cec38d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.042+0000 7f160055a740 1 --2- 172.21.15.6:0/2639975273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec312f0 0x5569cec387e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.042+0000 7f160055a740 1 --2- 172.21.15.6:0/2639975273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5569cec309d0 0x5569cec3b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.042+0000 7f160055a740 1 -- 172.21.15.6:0/2639975273 >> 172.21.15.6:0/2639975273 conn(0x5569cec2cf60 msgr2=0x5569cec32c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:23.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.042+0000 7f15d6ffd640 0 client.24670 ms_handle_reset on v2:172.21.15.161:6838/3645555254 2024-09-12T03:20:23.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.042+0000 7f160055a740 1 -- 172.21.15.6:0/2639975273 shutdown_connections 2024-09-12T03:20:23.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.042+0000 7f160055a740 1 -- 172.21.15.6:0/2639975273 wait complete. 2024-09-12T03:20:23.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.042+0000 7f160055a740 20 client.24670 trim_cache size 0 max 16384 2024-09-12T03:20:23.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.060+0000 7f160055a740 1 Processor -- start 2024-09-12T03:20:23.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.060+0000 7f160055a740 1 -- start start 2024-09-12T03:20:23.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.060+0000 7f160055a740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5569cec309d0 0x5569cec500c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.060+0000 7f160055a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec312f0 0x5569cec51530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.060+0000 7f160055a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec3bbb0 0x5569cec4f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.060+0000 7f160055a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5569cec4ff40 con 0x5569cec312f0 2024-09-12T03:20:23.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.060+0000 7f160055a740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5569cec50610 con 0x5569cec309d0 2024-09-12T03:20:23.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.060+0000 7f160055a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5569cec50ce0 con 0x5569cec3bbb0 2024-09-12T03:20:23.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.060+0000 7f15f77fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec312f0 0x5569cec51530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.060+0000 7f15f77fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec312f0 0x5569cec51530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35564/0 (socket says 172.21.15.6:35564) 2024-09-12T03:20:23.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.060+0000 7f15f77fe640 1 -- 172.21.15.6:0/2517296740 learned_addr learned my addr 172.21.15.6:0/2517296740 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:23.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.061+0000 7f15f6ffd640 1 --2- 172.21.15.6:0/2517296740 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5569cec309d0 0x5569cec500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.061+0000 7f15f6ffd640 1 --2- 172.21.15.6:0/2517296740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec3bbb0 0x5569cec4f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.061+0000 7f15f77fe640 1 -- 172.21.15.6:0/2517296740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec3bbb0 msgr2=0x5569cec4f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:23.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.061+0000 7f15f77fe640 1 --2- 172.21.15.6:0/2517296740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec3bbb0 0x5569cec4f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.061+0000 7f15f77fe640 1 -- 172.21.15.6:0/2517296740 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5569cec309d0 msgr2=0x5569cec500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:23.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.061+0000 7f15f77fe640 1 --2- 172.21.15.6:0/2517296740 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5569cec309d0 0x5569cec500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.061+0000 7f15f77fe640 1 -- 172.21.15.6:0/2517296740 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569cec36d30 con 0x5569cec312f0 2024-09-12T03:20:23.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.061+0000 7f15f6ffd640 1 --2- 172.21.15.6:0/2517296740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec3bbb0 0x5569cec4f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:23.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.061+0000 7f15f77fe640 1 --2- 172.21.15.6:0/2517296740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec312f0 0x5569cec51530 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f15e8004c80 tx=0x7f15e8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed210750f24e55c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:23.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.061+0000 7f15dd7fa640 1 -- 172.21.15.6:0/2517296740 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e801d040 con 0x5569cec312f0 2024-09-12T03:20:23.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.061+0000 7f15dd7fa640 1 -- 172.21.15.6:0/2517296740 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15e801e040 con 0x5569cec312f0 2024-09-12T03:20:23.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.061+0000 7f15dd7fa640 1 -- 172.21.15.6:0/2517296740 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e80189f0 con 0x5569cec312f0 2024-09-12T03:20:23.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.062+0000 7f160055a740 1 -- 172.21.15.6:0/2517296740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec312f0 msgr2=0x5569cec51530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.062+0000 7f160055a740 1 --2- 172.21.15.6:0/2517296740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec312f0 0x5569cec51530 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f15e8004c80 tx=0x7f15e8000bc0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.062+0000 7f160055a740 1 -- 172.21.15.6:0/2517296740 shutdown_connections 2024-09-12T03:20:23.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.062+0000 7f160055a740 1 --2- 172.21.15.6:0/2517296740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec3bbb0 0x5569cec4f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.062+0000 7f160055a740 1 --2- 172.21.15.6:0/2517296740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec312f0 0x5569cec51530 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.062+0000 7f160055a740 1 --2- 172.21.15.6:0/2517296740 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5569cec309d0 0x5569cec500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.062+0000 7f160055a740 1 -- 172.21.15.6:0/2517296740 >> 172.21.15.6:0/2517296740 conn(0x5569cec2cf60 msgr2=0x5569cec32c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:23.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.062+0000 7f160055a740 1 -- 172.21.15.6:0/2517296740 shutdown_connections 2024-09-12T03:20:23.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.062+0000 7f160055a740 1 -- 172.21.15.6:0/2517296740 wait complete. 2024-09-12T03:20:23.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.062+0000 7f160055a740 1 -- start start 2024-09-12T03:20:23.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.063+0000 7f160055a740 1 Processor -- start 2024-09-12T03:20:23.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.063+0000 7f160055a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec309d0 0x5569cec52d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.063+0000 7f160055a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec312f0 0x5569cec53250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.063+0000 7f160055a740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5569cec3bbb0 0x5569cec53790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.063+0000 7f160055a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5569cec51a80 con 0x5569cec309d0 2024-09-12T03:20:23.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.063+0000 7f160055a740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5569cec513b0 con 0x5569cec3bbb0 2024-09-12T03:20:23.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.063+0000 7f160055a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5569cec52150 con 0x5569cec312f0 2024-09-12T03:20:23.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.063+0000 7f160055a740 10 client.0 _ll_register_callbacks cb 0x7f15e801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:23.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.063+0000 7f15f6ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec309d0 0x5569cec52d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.063+0000 7f15f6ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec309d0 0x5569cec52d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35572/0 (socket says 172.21.15.6:35572) 2024-09-12T03:20:23.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.063+0000 7f15f6ffd640 1 -- 172.21.15.6:0/1827347838 learned_addr learned my addr 172.21.15.6:0/1827347838 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:23.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.063+0000 7f15f77fe640 1 --2- 172.21.15.6:0/1827347838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec312f0 0x5569cec53250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.063+0000 7f15f6ffd640 1 -- 172.21.15.6:0/1827347838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec312f0 msgr2=0x5569cec53250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:23.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.063+0000 7f15f6ffd640 1 --2- 172.21.15.6:0/1827347838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec312f0 0x5569cec53250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.063+0000 7f15f6ffd640 1 -- 172.21.15.6:0/1827347838 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5569cec3bbb0 msgr2=0x5569cec53790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:23.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.063+0000 7f15f6ffd640 1 --2- 172.21.15.6:0/1827347838 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5569cec3bbb0 0x5569cec53790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.063+0000 7f15f6ffd640 1 -- 172.21.15.6:0/1827347838 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569cecb8ec0 con 0x5569cec309d0 2024-09-12T03:20:23.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.063+0000 7f15f77fe640 1 --2- 172.21.15.6:0/1827347838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec312f0 0x5569cec53250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:23.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.063+0000 7f15f6ffd640 1 --2- 172.21.15.6:0/1827347838 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec309d0 0x5569cec52d10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x5569cec39450 tx=0x7f15ec013ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d270ce6c915bd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:23.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.063+0000 7f15d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:20:23.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.064+0000 7f15d6ffd640 1 -- 172.21.15.6:0/1827347838 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec02e060 con 0x5569cec309d0 2024-09-12T03:20:23.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.064+0000 7f15d6ffd640 1 -- 172.21.15.6:0/1827347838 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f15ec010e10 con 0x5569cec309d0 2024-09-12T03:20:23.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.064+0000 7f160055a740 1 -- 172.21.15.6:0/1827347838 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15e801e040 con 0x5569cec309d0 2024-09-12T03:20:23.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.064+0000 7f15d6ffd640 1 -- 172.21.15.6:0/1827347838 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f15ec026070 con 0x5569cec309d0 2024-09-12T03:20:23.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.064+0000 7f160055a740 10 client.14955 fetch_fsmap learned FSMap version 24 2024-09-12T03:20:23.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.064+0000 7f160055a740 1 -- 172.21.15.6:0/1827347838 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5569cecba190 con 0x5569cec309d0 2024-09-12T03:20:23.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.064+0000 7f15d6ffd640 1 -- 172.21.15.6:0/1827347838 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15e801e040 con 0x5569cec309d0 2024-09-12T03:20:23.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.064+0000 7f15d6ffd640 1 -- 172.21.15.6:0/1827347838 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f15e8019ce0 con 0x5569cec309d0 2024-09-12T03:20:23.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.064+0000 7f160055a740 10 client.14955 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:20:23.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.064+0000 7f160055a740 10 client.14955 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:20:23.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.064+0000 7f160055a740 20 client.14955 populate_metadata read hostname 'smithi006' 2024-09-12T03:20:23.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.064+0000 7f160055a740 1 --2- 172.21.15.6:0/1827347838 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5569cecbed40 0x5569cecbda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.064+0000 7f160055a740 4 client.14955 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:20:23.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.064+0000 7f160055a740 1 -- 172.21.15.6:0/1827347838 --> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] -- command(tid 0: {"prefix": "status"}) -- 0x5569ceb9eb10 con 0x5569cecbed40 2024-09-12T03:20:23.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.065+0000 7f15f77fe640 1 --2- 172.21.15.6:0/1827347838 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5569cecbed40 0x5569cecbda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.065+0000 7f15f77fe640 1 --2- 172.21.15.6:0/1827347838 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5569cecbed40 0x5569cecbda50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=93efe13f039cbc80 server_cookie=b8d8e10c69dcfc5 in_seq=0 out_seq=0 2024-09-12T03:20:23.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.065+0000 7f15d6ffd640 10 client.14955 ms_handle_connect on v2:172.21.15.161:6838/3645555254 2024-09-12T03:20:23.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.065+0000 7f15d6ffd640 1 -- 172.21.15.6:0/1827347838 <== mds.2 v2:172.21.15.161:6838/3645555254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5569ceb9eb10 con 0x5569cecbed40 2024-09-12T03:20:23.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.065+0000 7f15d6ffd640 10 client.14955 handle_command_reply: tid=0 2024-09-12T03:20:23.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.065+0000 7f160055a740 1 client.14955 shutdown 2024-09-12T03:20:23.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.066+0000 7f160055a740 1 -- 172.21.15.6:0/1827347838 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec309d0 msgr2=0x5569cec52d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.066+0000 7f160055a740 1 --2- 172.21.15.6:0/1827347838 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec309d0 0x5569cec52d10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x5569cec39450 tx=0x7f15ec013ee0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.066+0000 7f160055a740 1 -- 172.21.15.6:0/1827347838 shutdown_connections 2024-09-12T03:20:23.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.066+0000 7f160055a740 1 --2- 172.21.15.6:0/1827347838 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5569cecbed40 0x5569cecbda50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.066+0000 7f15d6ffd640 0 client.14955 ms_handle_reset on v2:172.21.15.161:6838/3645555254 2024-09-12T03:20:23.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.066+0000 7f160055a740 1 --2- 172.21.15.6:0/1827347838 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5569cec3bbb0 0x5569cec53790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.066+0000 7f160055a740 1 --2- 172.21.15.6:0/1827347838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cec312f0 0x5569cec53250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.066+0000 7f160055a740 1 --2- 172.21.15.6:0/1827347838 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5569cec309d0 0x5569cec52d10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.066+0000 7f160055a740 1 -- 172.21.15.6:0/1827347838 >> 172.21.15.6:0/1827347838 conn(0x5569cec2cf60 msgr2=0x5569cec335f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:23.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.066+0000 7f160055a740 1 -- 172.21.15.6:0/1827347838 shutdown_connections 2024-09-12T03:20:23.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.066+0000 7f160055a740 1 -- 172.21.15.6:0/1827347838 wait complete. 2024-09-12T03:20:23.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.066+0000 7f160055a740 20 client.14955 trim_cache size 0 max 16384 2024-09-12T03:20:23.071 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:20:23.071 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "89d99938-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:20:23.071 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 2, 2024-09-12T03:20:23.071 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24463, 2024-09-12T03:20:23.071 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-09-12T03:20:23.071 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-09-12T03:20:23.071 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-09-12T03:20:23.072 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 64.703781473000006, 2024-09-12T03:20:23.072 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 24, 2024-09-12T03:20:23.072 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 82, 2024-09-12T03:20:23.072 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T03:20:23.072 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 90.330581620999993 2024-09-12T03:20:23.072 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:20:23.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.066+0000 7f15fe181640 1 -- 172.21.15.6:0/828266061 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f15e00a2c90 msgr2=0x7f15e00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.066+0000 7f15fe181640 1 --2- 172.21.15.6:0/828266061 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f15e00a2c90 0x7f15e00a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f15f00bcd30 tx=0x7f15f00e7810 comp rx=0 tx=0).stop 2024-09-12T03:20:23.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.066+0000 7f15fe181640 1 -- 172.21.15.6:0/828266061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15f8106f20 msgr2=0x7f15f819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.066+0000 7f15fe181640 1 --2- 172.21.15.6:0/828266061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15f8106f20 0x7f15f819cc70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f15f0031a60 tx=0x7f15f0031a90 comp rx=0 tx=0).stop 2024-09-12T03:20:23.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.067+0000 7f15fe181640 1 -- 172.21.15.6:0/828266061 shutdown_connections 2024-09-12T03:20:23.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.067+0000 7f15fe181640 1 --2- 172.21.15.6:0/828266061 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f15e00a2c90 0x7f15e00a5150 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.067+0000 7f15fe181640 1 --2- 172.21.15.6:0/828266061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15f8106f20 0x7f15f819cc70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.067+0000 7f15fe181640 1 --2- 172.21.15.6:0/828266061 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15f81027a0 0x7f15f81988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.067+0000 7f15fe181640 1 --2- 172.21.15.6:0/828266061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15f8101df0 0x7f15f81983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.067+0000 7f15fe181640 1 -- 172.21.15.6:0/828266061 >> 172.21.15.6:0/828266061 conn(0x7f15f80fd8f0 msgr2=0x7f15f80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:23.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.067+0000 7f15fe181640 1 -- 172.21.15.6:0/828266061 shutdown_connections 2024-09-12T03:20:23.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:23.067+0000 7f15fe181640 1 -- 172.21.15.6:0/828266061 wait complete. 2024-09-12T03:20:23.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:22 smithi118 ceph-mon[35048]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:23.354 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:22 smithi006 ceph-mon[30991]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:23.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:22 smithi161 ceph-mon[35485]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:23.752 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-12T03:20:24.008 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:24.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.510+0000 7fd32c3d3640 1 -- 172.21.15.6:0/2779689332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd324103660 msgr2=0x7fd32410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:24.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.510+0000 7fd32c3d3640 1 --2- 172.21.15.6:0/2779689332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd324103660 0x7fd32410bb50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd314009900 tx=0x7fd31402a480 comp rx=0 tx=0).stop 2024-09-12T03:20:24.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.511+0000 7fd32c3d3640 1 -- 172.21.15.6:0/2779689332 shutdown_connections 2024-09-12T03:20:24.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.511+0000 7fd32c3d3640 1 --2- 172.21.15.6:0/2779689332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd32410c170 0x7fd32410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.511+0000 7fd32c3d3640 1 --2- 172.21.15.6:0/2779689332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd324103660 0x7fd32410bb50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.511+0000 7fd32c3d3640 1 --2- 172.21.15.6:0/2779689332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd324102d40 0x7fd324103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.511+0000 7fd32c3d3640 1 -- 172.21.15.6:0/2779689332 >> 172.21.15.6:0/2779689332 conn(0x7fd3240fc7a0 msgr2=0x7fd3240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:24.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.511+0000 7fd32c3d3640 1 -- 172.21.15.6:0/2779689332 shutdown_connections 2024-09-12T03:20:24.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.511+0000 7fd32c3d3640 1 -- 172.21.15.6:0/2779689332 wait complete. 2024-09-12T03:20:24.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.511+0000 7fd32c3d3640 1 Processor -- start 2024-09-12T03:20:24.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.511+0000 7fd32c3d3640 1 -- start start 2024-09-12T03:20:24.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd32c3d3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd324102d40 0x7fd324198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd32c3d3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd324103660 0x7fd324198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd32c3d3640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd32410c170 0x7fd32419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd32c3d3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd324111fb0 con 0x7fd324102d40 2024-09-12T03:20:24.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd32c3d3640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fd324111e30 con 0x7fd32410c170 2024-09-12T03:20:24.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd32c3d3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd324112130 con 0x7fd324103660 2024-09-12T03:20:24.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd32a16f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd324102d40 0x7fd324198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd32a16f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd324102d40 0x7fd324198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35598/0 (socket says 172.21.15.6:35598) 2024-09-12T03:20:24.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd32a16f640 1 -- 172.21.15.6:0/2235832674 learned_addr learned my addr 172.21.15.6:0/2235832674 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:24.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd32996e640 1 --2- 172.21.15.6:0/2235832674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd324103660 0x7fd324198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd32a970640 1 --2- 172.21.15.6:0/2235832674 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd32410c170 0x7fd32419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd32a16f640 1 -- 172.21.15.6:0/2235832674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd324103660 msgr2=0x7fd324198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:24.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd32a16f640 1 --2- 172.21.15.6:0/2235832674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd324103660 0x7fd324198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd32a16f640 1 -- 172.21.15.6:0/2235832674 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd32410c170 msgr2=0x7fd32419cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:24.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd32a16f640 1 --2- 172.21.15.6:0/2235832674 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd32410c170 0x7fd32419cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd32a16f640 1 -- 172.21.15.6:0/2235832674 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32419d5c0 con 0x7fd324102d40 2024-09-12T03:20:24.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd32996e640 1 --2- 172.21.15.6:0/2235832674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd324103660 0x7fd324198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:24.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd32a970640 1 --2- 172.21.15.6:0/2235832674 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd32410c170 0x7fd32419cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:24.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd32a16f640 1 --2- 172.21.15.6:0/2235832674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd324102d40 0x7fd324198610 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd32000b4a0 tx=0x7fd32000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:24.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd31b7fe640 1 -- 172.21.15.6:0/2235832674 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd320014070 con 0x7fd324102d40 2024-09-12T03:20:24.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd31b7fe640 1 -- 172.21.15.6:0/2235832674 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd320010af0 con 0x7fd324102d40 2024-09-12T03:20:24.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd32c3d3640 1 -- 172.21.15.6:0/2235832674 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32419d8b0 con 0x7fd324102d40 2024-09-12T03:20:24.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd31b7fe640 1 -- 172.21.15.6:0/2235832674 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd320017d10 con 0x7fd324102d40 2024-09-12T03:20:24.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.512+0000 7fd32c3d3640 1 -- 172.21.15.6:0/2235832674 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3241a5190 con 0x7fd324102d40 2024-09-12T03:20:24.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.515+0000 7fd31b7fe640 1 -- 172.21.15.6:0/2235832674 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fd320017070 con 0x7fd324102d40 2024-09-12T03:20:24.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.515+0000 7fd32e7ac740 1 Processor -- start 2024-09-12T03:20:24.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.515+0000 7fd32e7ac740 1 -- start start 2024-09-12T03:20:24.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.515+0000 7fd32e7ac740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0dd9d0 0x56242a0dddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.515+0000 7fd32e7ac740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0de2f0 0x56242a0e8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.515+0000 7fd32e7ac740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56242a0e8bb0 0x56242a0eafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.515+0000 7fd32e7ac740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56242a044090 con 0x56242a0dd9d0 2024-09-12T03:20:24.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.515+0000 7fd32e7ac740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x562429feba70 con 0x56242a0e8bb0 2024-09-12T03:20:24.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.516+0000 7fd32e7ac740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56242a040050 con 0x56242a0de2f0 2024-09-12T03:20:24.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.516+0000 7fd31b7fe640 1 --2- 172.21.15.6:0/2235832674 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd3100a2c10 0x7fd3100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.516+0000 7fd32a970640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0dd9d0 0x56242a0dddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.516+0000 7fd32a970640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0dd9d0 0x56242a0dddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35602/0 (socket says 172.21.15.6:35602) 2024-09-12T03:20:24.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.516+0000 7fd32a970640 1 -- 172.21.15.6:0/3947351086 learned_addr learned my addr 172.21.15.6:0/3947351086 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:24.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.516+0000 7fd32996e640 1 --2- 172.21.15.6:0/3947351086 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56242a0e8bb0 0x56242a0eafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.516+0000 7fd32a16f640 1 --2- 172.21.15.6:0/3947351086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0de2f0 0x56242a0e8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.516+0000 7fd31b7fe640 1 -- 172.21.15.6:0/2235832674 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd3200ce580 con 0x7fd324102d40 2024-09-12T03:20:24.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.516+0000 7fd32a970640 1 --2- 172.21.15.6:0/2235832674 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd3100a2c10 0x7fd3100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.516+0000 7fd32a970640 1 -- 172.21.15.6:0/3947351086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0de2f0 msgr2=0x56242a0e8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:24.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.516+0000 7fd32a970640 1 --2- 172.21.15.6:0/3947351086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0de2f0 0x56242a0e8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.516+0000 7fd32a970640 1 -- 172.21.15.6:0/3947351086 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56242a0e8bb0 msgr2=0x56242a0eafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:24.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.516+0000 7fd32a970640 1 --2- 172.21.15.6:0/3947351086 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56242a0e8bb0 0x56242a0eafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.516+0000 7fd32a970640 1 -- 172.21.15.6:0/3947351086 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562429ff8130 con 0x56242a0dd9d0 2024-09-12T03:20:24.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.516+0000 7fd32a16f640 1 --2- 172.21.15.6:0/3947351086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0de2f0 0x56242a0e8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:24.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.516+0000 7fd32a970640 1 --2- 172.21.15.6:0/3947351086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0dd9d0 0x56242a0dddb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0149a0 tx=0x7fd31c014e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cf4d481a7a7f82 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:24.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.516+0000 7fd318dea640 1 -- 172.21.15.6:0/3947351086 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c021070 con 0x56242a0dd9d0 2024-09-12T03:20:24.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.517+0000 7fd318dea640 1 -- 172.21.15.6:0/3947351086 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd31c013460 con 0x56242a0dd9d0 2024-09-12T03:20:24.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.517+0000 7fd318dea640 1 -- 172.21.15.6:0/3947351086 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c00f5f0 con 0x56242a0dd9d0 2024-09-12T03:20:24.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.517+0000 7fd32e7ac740 1 -- 172.21.15.6:0/3947351086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0dd9d0 msgr2=0x56242a0dddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:24.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.517+0000 7fd32a970640 1 --2- 172.21.15.6:0/2235832674 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd3100a2c10 0x7fd3100a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd31c01e000 tx=0x7fd31c019000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:24.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.517+0000 7fd32e7ac740 1 --2- 172.21.15.6:0/3947351086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0dd9d0 0x56242a0dddb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0149a0 tx=0x7fd31c014e60 comp rx=0 tx=0).stop 2024-09-12T03:20:24.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.517+0000 7fd32e7ac740 1 -- 172.21.15.6:0/3947351086 shutdown_connections 2024-09-12T03:20:24.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.517+0000 7fd32e7ac740 1 --2- 172.21.15.6:0/3947351086 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56242a0e8bb0 0x56242a0eafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.517+0000 7fd32e7ac740 1 --2- 172.21.15.6:0/3947351086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0de2f0 0x56242a0e8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.517+0000 7fd32e7ac740 1 --2- 172.21.15.6:0/3947351086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0dd9d0 0x56242a0dddb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.517+0000 7fd32e7ac740 1 -- 172.21.15.6:0/3947351086 >> 172.21.15.6:0/3947351086 conn(0x56242a0d9f60 msgr2=0x56242a0da340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:24.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.517+0000 7fd32e7ac740 1 -- 172.21.15.6:0/3947351086 shutdown_connections 2024-09-12T03:20:24.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.517+0000 7fd32e7ac740 1 -- 172.21.15.6:0/3947351086 wait complete. 2024-09-12T03:20:24.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.518+0000 7fd32e7ac740 1 -- start start 2024-09-12T03:20:24.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.518+0000 7fd32e7ac740 1 Processor -- start 2024-09-12T03:20:24.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.518+0000 7fd32e7ac740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0dd9d0 0x56242a0e8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.518+0000 7fd32e7ac740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0de2f0 0x56242a0e5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.518+0000 7fd32e7ac740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56242a0e8bb0 0x56242a0e5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.518+0000 7fd32e7ac740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56242a0eb780 con 0x56242a0de2f0 2024-09-12T03:20:24.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.518+0000 7fd32e7ac740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x562429ffef20 con 0x56242a0e8bb0 2024-09-12T03:20:24.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.518+0000 7fd32e7ac740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56242a0eb900 con 0x56242a0dd9d0 2024-09-12T03:20:24.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.518+0000 7fd32e7ac740 10 client.0 _ll_register_callbacks cb 0x562429fc8030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:24.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.518+0000 7fd32996e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0de2f0 0x56242a0e5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.518+0000 7fd32996e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0de2f0 0x56242a0e5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35604/0 (socket says 172.21.15.6:35604) 2024-09-12T03:20:24.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.518+0000 7fd32996e640 1 -- 172.21.15.6:0/3908129213 learned_addr learned my addr 172.21.15.6:0/3908129213 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:24.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.518+0000 7fd32a16f640 1 --2- 172.21.15.6:0/3908129213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0dd9d0 0x56242a0e8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.519+0000 7fd32996e640 1 -- 172.21.15.6:0/3908129213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0dd9d0 msgr2=0x56242a0e8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:24.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.519+0000 7fd32996e640 1 --2- 172.21.15.6:0/3908129213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0dd9d0 0x56242a0e8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.519+0000 7fd32996e640 1 -- 172.21.15.6:0/3908129213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56242a0e8bb0 msgr2=0x56242a0e5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:24.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.519+0000 7fd32996e640 1 --2- 172.21.15.6:0/3908129213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56242a0e8bb0 0x56242a0e5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.519+0000 7fd32996e640 1 -- 172.21.15.6:0/3908129213 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562429db52b0 con 0x56242a0de2f0 2024-09-12T03:20:24.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.519+0000 7fd32996e640 1 --2- 172.21.15.6:0/3908129213 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0de2f0 0x56242a0e5770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd314031550 tx=0x7fd314032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b485481e6275137 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:24.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.519+0000 7fd30dffb640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:20:24.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.520+0000 7fd30dffb640 1 -- 172.21.15.6:0/3908129213 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314035040 con 0x56242a0de2f0 2024-09-12T03:20:24.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.520+0000 7fd30dffb640 1 -- 172.21.15.6:0/3908129213 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd314004040 con 0x56242a0de2f0 2024-09-12T03:20:24.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.520+0000 7fd32e7ac740 1 -- 172.21.15.6:0/3908129213 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd31c013460 con 0x56242a0de2f0 2024-09-12T03:20:24.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.520+0000 7fd30dffb640 1 -- 172.21.15.6:0/3908129213 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd314004240 con 0x56242a0de2f0 2024-09-12T03:20:24.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.520+0000 7fd32e7ac740 10 client.14967 fetch_fsmap learned FSMap version 24 2024-09-12T03:20:24.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.520+0000 7fd32e7ac740 1 -- 172.21.15.6:0/3908129213 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56242a0f6fe0 con 0x56242a0de2f0 2024-09-12T03:20:24.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.520+0000 7fd30dffb640 1 -- 172.21.15.6:0/3908129213 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd31c013460 con 0x56242a0de2f0 2024-09-12T03:20:24.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.520+0000 7fd30dffb640 1 -- 172.21.15.6:0/3908129213 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fd31403fbb0 con 0x56242a0de2f0 2024-09-12T03:20:24.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.520+0000 7fd32e7ac740 10 client.14967 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:20:24.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.520+0000 7fd32e7ac740 10 client.14967 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T03:20:24.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.520+0000 7fd32e7ac740 20 client.14967 populate_metadata read hostname 'smithi006' 2024-09-12T03:20:24.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.520+0000 7fd32e7ac740 1 --2- 172.21.15.6:0/3908129213 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x56242a0f9eb0 0x56242a0fab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.520+0000 7fd32e7ac740 4 client.14967 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:24.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.520+0000 7fd32e7ac740 1 -- 172.21.15.6:0/3908129213 --> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56242a051600 con 0x56242a0f9eb0 2024-09-12T03:20:24.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.521+0000 7fd32a16f640 1 --2- 172.21.15.6:0/3908129213 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x56242a0f9eb0 0x56242a0fab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.521+0000 7fd32a16f640 1 --2- 172.21.15.6:0/3908129213 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x56242a0f9eb0 0x56242a0fab10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=452617767b05b948 server_cookie=34284f1d3423afd4 in_seq=0 out_seq=0 2024-09-12T03:20:24.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.521+0000 7fd30dffb640 10 client.14967 ms_handle_connect on v2:172.21.15.118:6836/3879191727 2024-09-12T03:20:24.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.522+0000 7fd30dffb640 1 -- 172.21.15.6:0/3908129213 <== mds.1 v2:172.21.15.118:6836/3879191727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56242a051600 con 0x56242a0f9eb0 2024-09-12T03:20:24.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.522+0000 7fd30dffb640 10 client.14967 handle_command_reply: tid=0 2024-09-12T03:20:24.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.522+0000 7fd32e7ac740 1 client.14967 shutdown 2024-09-12T03:20:24.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.522+0000 7fd32e7ac740 1 -- 172.21.15.6:0/3908129213 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0de2f0 msgr2=0x56242a0e5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:24.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.522+0000 7fd32e7ac740 1 --2- 172.21.15.6:0/3908129213 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0de2f0 0x56242a0e5770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd314031550 tx=0x7fd314032ed0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.522+0000 7fd32e7ac740 1 -- 172.21.15.6:0/3908129213 shutdown_connections 2024-09-12T03:20:24.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.522+0000 7fd32e7ac740 1 --2- 172.21.15.6:0/3908129213 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x56242a0f9eb0 0x56242a0fab10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.522+0000 7fd30dffb640 0 client.14967 ms_handle_reset on v2:172.21.15.118:6836/3879191727 2024-09-12T03:20:24.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.522+0000 7fd32e7ac740 1 --2- 172.21.15.6:0/3908129213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56242a0e8bb0 0x56242a0e5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.522+0000 7fd32e7ac740 1 --2- 172.21.15.6:0/3908129213 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0de2f0 0x56242a0e5770 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.522+0000 7fd32e7ac740 1 --2- 172.21.15.6:0/3908129213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0dd9d0 0x56242a0e8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.522+0000 7fd32e7ac740 1 -- 172.21.15.6:0/3908129213 >> 172.21.15.6:0/3908129213 conn(0x56242a0d9f60 msgr2=0x56242a0dfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:24.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.522+0000 7fd32e7ac740 1 -- 172.21.15.6:0/3908129213 shutdown_connections 2024-09-12T03:20:24.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.522+0000 7fd32e7ac740 1 -- 172.21.15.6:0/3908129213 wait complete. 2024-09-12T03:20:24.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.522+0000 7fd32e7ac740 20 client.14967 trim_cache size 0 max 16384 2024-09-12T03:20:24.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.540+0000 7fd32e7ac740 1 Processor -- start 2024-09-12T03:20:24.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.540+0000 7fd32e7ac740 1 -- start start 2024-09-12T03:20:24.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.540+0000 7fd32e7ac740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56242a0dd9d0 0x56242a0fa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.540+0000 7fd32e7ac740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0de2f0 0x56242a102df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.540+0000 7fd32e7ac740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0e8bb0 0x56242a103330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.540+0000 7fd32e7ac740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56242a0f9010 con 0x56242a0de2f0 2024-09-12T03:20:24.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.540+0000 7fd32e7ac740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x56242a0f96e0 con 0x56242a0dd9d0 2024-09-12T03:20:24.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.540+0000 7fd32e7ac740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56242a0f9d30 con 0x56242a0e8bb0 2024-09-12T03:20:24.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.540+0000 7fd32996e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0de2f0 0x56242a102df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.541+0000 7fd32996e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0de2f0 0x56242a102df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35616/0 (socket says 172.21.15.6:35616) 2024-09-12T03:20:24.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.541+0000 7fd32996e640 1 -- 172.21.15.6:0/1411918953 learned_addr learned my addr 172.21.15.6:0/1411918953 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:24.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.541+0000 7fd32a16f640 1 --2- 172.21.15.6:0/1411918953 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56242a0dd9d0 0x56242a0fa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.541+0000 7fd32a970640 1 --2- 172.21.15.6:0/1411918953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0e8bb0 0x56242a103330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.541+0000 7fd32996e640 1 -- 172.21.15.6:0/1411918953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0e8bb0 msgr2=0x56242a103330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:24.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.541+0000 7fd32996e640 1 --2- 172.21.15.6:0/1411918953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0e8bb0 0x56242a103330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.541+0000 7fd32996e640 1 -- 172.21.15.6:0/1411918953 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56242a0dd9d0 msgr2=0x56242a0fa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:24.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.541+0000 7fd32996e640 1 --2- 172.21.15.6:0/1411918953 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56242a0dd9d0 0x56242a0fa440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.541+0000 7fd32996e640 1 -- 172.21.15.6:0/1411918953 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56242a0db0d0 con 0x56242a0de2f0 2024-09-12T03:20:24.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.541+0000 7fd32996e640 1 --2- 172.21.15.6:0/1411918953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0de2f0 0x56242a102df0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd314006010 tx=0x7fd314032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efe5a461dba9a83 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:24.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.541+0000 7fd30ffff640 1 -- 172.21.15.6:0/1411918953 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314035070 con 0x56242a0de2f0 2024-09-12T03:20:24.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.541+0000 7fd30ffff640 1 -- 172.21.15.6:0/1411918953 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd314057070 con 0x56242a0de2f0 2024-09-12T03:20:24.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.541+0000 7fd30ffff640 1 -- 172.21.15.6:0/1411918953 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3140486d0 con 0x56242a0de2f0 2024-09-12T03:20:24.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.542+0000 7fd32e7ac740 1 -- 172.21.15.6:0/1411918953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0de2f0 msgr2=0x56242a102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:24.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.542+0000 7fd32e7ac740 1 --2- 172.21.15.6:0/1411918953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0de2f0 0x56242a102df0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd314006010 tx=0x7fd314032f40 comp rx=0 tx=0).stop 2024-09-12T03:20:24.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.542+0000 7fd32e7ac740 1 -- 172.21.15.6:0/1411918953 shutdown_connections 2024-09-12T03:20:24.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.542+0000 7fd32e7ac740 1 --2- 172.21.15.6:0/1411918953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0e8bb0 0x56242a103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.542+0000 7fd32e7ac740 1 --2- 172.21.15.6:0/1411918953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0de2f0 0x56242a102df0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.542+0000 7fd32e7ac740 1 --2- 172.21.15.6:0/1411918953 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56242a0dd9d0 0x56242a0fa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.542+0000 7fd32e7ac740 1 -- 172.21.15.6:0/1411918953 >> 172.21.15.6:0/1411918953 conn(0x56242a0d9f60 msgr2=0x56242a0e4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:24.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.542+0000 7fd32e7ac740 1 -- 172.21.15.6:0/1411918953 shutdown_connections 2024-09-12T03:20:24.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.542+0000 7fd32e7ac740 1 -- 172.21.15.6:0/1411918953 wait complete. 2024-09-12T03:20:24.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.542+0000 7fd32e7ac740 1 -- start start 2024-09-12T03:20:24.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.542+0000 7fd32e7ac740 1 Processor -- start 2024-09-12T03:20:24.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd32e7ac740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0dd9d0 0x56242a1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd32e7ac740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0de2f0 0x56242a1035f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd32e7ac740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56242a0e8bb0 0x56242a0fe440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd32e7ac740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56242a0fa990 con 0x56242a0de2f0 2024-09-12T03:20:24.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd32e7ac740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x56242a0fa2c0 con 0x56242a0e8bb0 2024-09-12T03:20:24.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd32e7ac740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56242a0fb060 con 0x56242a0dd9d0 2024-09-12T03:20:24.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd32e7ac740 10 client.0 _ll_register_callbacks cb 0x7fd314009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:24.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd32996e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0de2f0 0x56242a1035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd32996e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0de2f0 0x56242a1035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35624/0 (socket says 172.21.15.6:35624) 2024-09-12T03:20:24.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd32996e640 1 -- 172.21.15.6:0/2374261622 learned_addr learned my addr 172.21.15.6:0/2374261622 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:24.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd32a16f640 1 --2- 172.21.15.6:0/2374261622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0dd9d0 0x56242a1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd32a970640 1 --2- 172.21.15.6:0/2374261622 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56242a0e8bb0 0x56242a0fe440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd32996e640 1 -- 172.21.15.6:0/2374261622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0dd9d0 msgr2=0x56242a1030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:24.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd32996e640 1 --2- 172.21.15.6:0/2374261622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0dd9d0 0x56242a1030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd32996e640 1 -- 172.21.15.6:0/2374261622 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56242a0e8bb0 msgr2=0x56242a0fe440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:24.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd32996e640 1 --2- 172.21.15.6:0/2374261622 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56242a0e8bb0 0x56242a0fe440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd32996e640 1 -- 172.21.15.6:0/2374261622 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56242a0e67b0 con 0x56242a0de2f0 2024-09-12T03:20:24.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd32a16f640 1 --2- 172.21.15.6:0/2374261622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0dd9d0 0x56242a1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:24.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd32a970640 1 --2- 172.21.15.6:0/2374261622 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56242a0e8bb0 0x56242a0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:24.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd32996e640 1 --2- 172.21.15.6:0/2374261622 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0de2f0 0x56242a1035f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd314000c00 tx=0x7fd31403fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dab938ba01a0d07 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:24.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd30dffb640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:20:24.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd30dffb640 1 -- 172.21.15.6:0/2374261622 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314035070 con 0x56242a0de2f0 2024-09-12T03:20:24.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd30dffb640 1 -- 172.21.15.6:0/2374261622 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd314048470 con 0x56242a0de2f0 2024-09-12T03:20:24.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.543+0000 7fd32e7ac740 1 -- 172.21.15.6:0/2374261622 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd314057070 con 0x56242a0de2f0 2024-09-12T03:20:24.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.544+0000 7fd30dffb640 1 -- 172.21.15.6:0/2374261622 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd31406d070 con 0x56242a0de2f0 2024-09-12T03:20:24.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.544+0000 7fd32e7ac740 10 client.14973 fetch_fsmap learned FSMap version 24 2024-09-12T03:20:24.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.544+0000 7fd32e7ac740 1 -- 172.21.15.6:0/2374261622 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56242a163770 con 0x56242a0de2f0 2024-09-12T03:20:24.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.544+0000 7fd30dffb640 1 -- 172.21.15.6:0/2374261622 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd314057070 con 0x56242a0de2f0 2024-09-12T03:20:24.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.544+0000 7fd30dffb640 1 -- 172.21.15.6:0/2374261622 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fd314064020 con 0x56242a0de2f0 2024-09-12T03:20:24.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.544+0000 7fd32e7ac740 10 client.14973 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:20:24.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.544+0000 7fd32e7ac740 10 client.14973 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T03:20:24.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.544+0000 7fd32e7ac740 20 client.14973 populate_metadata read hostname 'smithi006' 2024-09-12T03:20:24.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.544+0000 7fd32e7ac740 1 --2- 172.21.15.6:0/2374261622 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x56242a0f9eb0 0x56242a166e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.544+0000 7fd32e7ac740 4 client.14973 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:20:24.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.544+0000 7fd32e7ac740 1 -- 172.21.15.6:0/2374261622 --> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] -- command(tid 0: {"prefix": "status"}) -- 0x56242a04bb10 con 0x56242a0f9eb0 2024-09-12T03:20:24.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.544+0000 7fd32a16f640 1 --2- 172.21.15.6:0/2374261622 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x56242a0f9eb0 0x56242a166e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.545+0000 7fd32a16f640 1 --2- 172.21.15.6:0/2374261622 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x56242a0f9eb0 0x56242a166e60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b5a595f23da11d1 server_cookie=718a223e01755ca1 in_seq=0 out_seq=0 2024-09-12T03:20:24.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.545+0000 7fd30dffb640 10 client.14973 ms_handle_connect on v2:172.21.15.118:6836/3879191727 2024-09-12T03:20:24.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.545+0000 7fd30dffb640 1 -- 172.21.15.6:0/2374261622 <== mds.1 v2:172.21.15.118:6836/3879191727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56242a04bb10 con 0x56242a0f9eb0 2024-09-12T03:20:24.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.545+0000 7fd30dffb640 10 client.14973 handle_command_reply: tid=0 2024-09-12T03:20:24.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.545+0000 7fd32e7ac740 1 client.14973 shutdown 2024-09-12T03:20:24.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.545+0000 7fd32e7ac740 1 -- 172.21.15.6:0/2374261622 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0de2f0 msgr2=0x56242a1035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:24.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.545+0000 7fd32e7ac740 1 --2- 172.21.15.6:0/2374261622 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0de2f0 0x56242a1035f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd314000c00 tx=0x7fd31403fe40 comp rx=0 tx=0).stop 2024-09-12T03:20:24.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.545+0000 7fd32e7ac740 1 -- 172.21.15.6:0/2374261622 shutdown_connections 2024-09-12T03:20:24.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.545+0000 7fd32e7ac740 1 --2- 172.21.15.6:0/2374261622 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x56242a0f9eb0 0x56242a166e60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.545+0000 7fd32e7ac740 1 --2- 172.21.15.6:0/2374261622 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56242a0e8bb0 0x56242a0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.545+0000 7fd32e7ac740 1 --2- 172.21.15.6:0/2374261622 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56242a0de2f0 0x56242a1035f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.545+0000 7fd32e7ac740 1 --2- 172.21.15.6:0/2374261622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56242a0dd9d0 0x56242a1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.545+0000 7fd32e7ac740 1 -- 172.21.15.6:0/2374261622 >> 172.21.15.6:0/2374261622 conn(0x56242a0d9f60 msgr2=0x56242a0e4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:24.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.545+0000 7fd30dffb640 0 client.14973 ms_handle_reset on v2:172.21.15.118:6836/3879191727 2024-09-12T03:20:24.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.545+0000 7fd32e7ac740 1 -- 172.21.15.6:0/2374261622 shutdown_connections 2024-09-12T03:20:24.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.545+0000 7fd32e7ac740 1 -- 172.21.15.6:0/2374261622 wait complete. 2024-09-12T03:20:24.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.545+0000 7fd32e7ac740 20 client.14973 trim_cache size 0 max 16384 2024-09-12T03:20:24.552 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:20:24.552 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "89d99938-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:20:24.552 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 1, 2024-09-12T03:20:24.552 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24464, 2024-09-12T03:20:24.552 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-09-12T03:20:24.552 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-09-12T03:20:24.552 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-09-12T03:20:24.552 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 67.188869609999998, 2024-09-12T03:20:24.552 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 24, 2024-09-12T03:20:24.552 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 82, 2024-09-12T03:20:24.553 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T03:20:24.553 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 94.457113188999998 2024-09-12T03:20:24.553 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:20:24.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.546+0000 7fd32c3d3640 1 -- 172.21.15.6:0/2235832674 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd3100a2c10 msgr2=0x7fd3100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:24.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.546+0000 7fd32c3d3640 1 --2- 172.21.15.6:0/2235832674 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd3100a2c10 0x7fd3100a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd31c01e000 tx=0x7fd31c019000 comp rx=0 tx=0).stop 2024-09-12T03:20:24.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.546+0000 7fd32c3d3640 1 -- 172.21.15.6:0/2235832674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd324102d40 msgr2=0x7fd324198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:24.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.546+0000 7fd32c3d3640 1 --2- 172.21.15.6:0/2235832674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd324102d40 0x7fd324198610 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd32000b4a0 tx=0x7fd32000b960 comp rx=0 tx=0).stop 2024-09-12T03:20:24.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.546+0000 7fd32c3d3640 1 -- 172.21.15.6:0/2235832674 shutdown_connections 2024-09-12T03:20:24.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.546+0000 7fd32c3d3640 1 --2- 172.21.15.6:0/2235832674 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd3100a2c10 0x7fd3100a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.546+0000 7fd32c3d3640 1 --2- 172.21.15.6:0/2235832674 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd32410c170 0x7fd32419cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.546+0000 7fd32c3d3640 1 --2- 172.21.15.6:0/2235832674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd324103660 0x7fd324198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.546+0000 7fd32c3d3640 1 --2- 172.21.15.6:0/2235832674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd324102d40 0x7fd324198610 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.546+0000 7fd32c3d3640 1 -- 172.21.15.6:0/2235832674 >> 172.21.15.6:0/2235832674 conn(0x7fd3240fc7a0 msgr2=0x7fd3240fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:24.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.546+0000 7fd32c3d3640 1 -- 172.21.15.6:0/2235832674 shutdown_connections 2024-09-12T03:20:24.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:24.546+0000 7fd32c3d3640 1 -- 172.21.15.6:0/2235832674 wait complete. 2024-09-12T03:20:25.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:24 smithi118 ceph-mon[35048]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:20:25.265 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:24 smithi006 ceph-mon[30991]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:20:25.295 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T03:20:25.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:24 smithi161 ceph-mon[35485]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:20:25.553 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:26.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.160+0000 7f7bce33e640 1 -- 172.21.15.6:0/2573214778 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bc8100a80 msgr2=0x7f7bc8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:26.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.160+0000 7f7bce33e640 1 --2- 172.21.15.6:0/2573214778 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bc8100a80 0x7f7bc8100ee0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc009930 tx=0x7f7bbc02a420 comp rx=0 tx=0).stop 2024-09-12T03:20:26.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.160+0000 7f7bce33e640 1 -- 172.21.15.6:0/2573214778 shutdown_connections 2024-09-12T03:20:26.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.160+0000 7f7bce33e640 1 --2- 172.21.15.6:0/2573214778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bc8101420 0x7f7bc810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.160+0000 7f7bce33e640 1 --2- 172.21.15.6:0/2573214778 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bc8100a80 0x7f7bc8100ee0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.160+0000 7f7bce33e640 1 --2- 172.21.15.6:0/2573214778 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7bc8106a80 0x7f7bc8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.160+0000 7f7bce33e640 1 -- 172.21.15.6:0/2573214778 >> 172.21.15.6:0/2573214778 conn(0x7f7bc80fc7a0 msgr2=0x7f7bc80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:26.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.160+0000 7f7bce33e640 1 -- 172.21.15.6:0/2573214778 shutdown_connections 2024-09-12T03:20:26.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.160+0000 7f7bce33e640 1 -- 172.21.15.6:0/2573214778 wait complete. 2024-09-12T03:20:26.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.161+0000 7f7bce33e640 1 Processor -- start 2024-09-12T03:20:26.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.161+0000 7f7bce33e640 1 -- start start 2024-09-12T03:20:26.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.162+0000 7f7bce33e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bc8100a80 0x7f7bc8197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.162+0000 7f7bce33e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bc8101420 0x7f7bc81978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.162+0000 7f7bce33e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7bc8106a80 0x7f7bc819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.162+0000 7f7bce33e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc8110e80 con 0x7f7bc8100a80 2024-09-12T03:20:26.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.162+0000 7f7bce33e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc8110d00 con 0x7f7bc8106a80 2024-09-12T03:20:26.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.162+0000 7f7bce33e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc8111000 con 0x7f7bc8101420 2024-09-12T03:20:26.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.162+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bc8100a80 0x7f7bc8197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.162+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bc8100a80 0x7f7bc8197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35638/0 (socket says 172.21.15.6:35638) 2024-09-12T03:20:26.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.162+0000 7f7bc7fff640 1 -- 172.21.15.6:0/1828920943 learned_addr learned my addr 172.21.15.6:0/1828920943 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:26.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.162+0000 7f7bc77fe640 1 --2- 172.21.15.6:0/1828920943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bc8101420 0x7f7bc81978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.162+0000 7f7bcc8db640 1 --2- 172.21.15.6:0/1828920943 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7bc8106a80 0x7f7bc819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.162+0000 7f7bc7fff640 1 -- 172.21.15.6:0/1828920943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bc8101420 msgr2=0x7f7bc81978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:26.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.162+0000 7f7bc7fff640 1 --2- 172.21.15.6:0/1828920943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bc8101420 0x7f7bc81978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.162+0000 7f7bc7fff640 1 -- 172.21.15.6:0/1828920943 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7bc8106a80 msgr2=0x7f7bc819bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:26.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.162+0000 7f7bc7fff640 1 --2- 172.21.15.6:0/1828920943 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7bc8106a80 0x7f7bc819bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.162+0000 7f7bc7fff640 1 -- 172.21.15.6:0/1828920943 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc819c330 con 0x7f7bc8100a80 2024-09-12T03:20:26.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.162+0000 7f7bcc8db640 1 --2- 172.21.15.6:0/1828920943 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7bc8106a80 0x7f7bc819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:26.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.162+0000 7f7bc7fff640 1 --2- 172.21.15.6:0/1828920943 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bc8100a80 0x7f7bc8197380 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800e9f0 tx=0x7f7bb800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:26.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.162+0000 7f7bc57fa640 1 -- 172.21.15.6:0/1828920943 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb800cd80 con 0x7f7bc8100a80 2024-09-12T03:20:26.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.162+0000 7f7bc57fa640 1 -- 172.21.15.6:0/1828920943 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bb8010910 con 0x7f7bc8100a80 2024-09-12T03:20:26.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.163+0000 7f7bce33e640 1 -- 172.21.15.6:0/1828920943 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc819c620 con 0x7f7bc8100a80 2024-09-12T03:20:26.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.163+0000 7f7bc57fa640 1 -- 172.21.15.6:0/1828920943 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb80188f0 con 0x7f7bc8100a80 2024-09-12T03:20:26.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.163+0000 7f7bce33e640 1 -- 172.21.15.6:0/1828920943 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc81a3f00 con 0x7f7bc8100a80 2024-09-12T03:20:26.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.165+0000 7f7bc57fa640 1 -- 172.21.15.6:0/1828920943 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f7bb8010ac0 con 0x7f7bc8100a80 2024-09-12T03:20:26.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.165+0000 7f7bd0717740 1 Processor -- start 2024-09-12T03:20:26.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.165+0000 7f7bd0717740 1 -- start start 2024-09-12T03:20:26.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.165+0000 7f7bd0717740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c00d425810 0x55c00d425bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.165+0000 7f7bd0717740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d426130 0x55c00d432880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.165+0000 7f7bd0717740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c00d432dc0 0x55c00d4351b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.165+0000 7f7bd0717740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55c00d38c090 con 0x55c00d426130 2024-09-12T03:20:26.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.165+0000 7f7bd0717740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55c00d333a70 con 0x55c00d425810 2024-09-12T03:20:26.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.165+0000 7f7bd0717740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c00d388050 con 0x55c00d432dc0 2024-09-12T03:20:26.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.166+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d426130 0x55c00d432880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.166+0000 7f7bcc8db640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c00d425810 0x55c00d425bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.166+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d426130 0x55c00d432880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35648/0 (socket says 172.21.15.6:35648) 2024-09-12T03:20:26.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.166+0000 7f7bc7fff640 1 -- 172.21.15.6:0/3137477886 learned_addr learned my addr 172.21.15.6:0/3137477886 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:26.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.166+0000 7f7bc77fe640 1 --2- 172.21.15.6:0/3137477886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c00d432dc0 0x55c00d4351b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.166+0000 7f7bc7fff640 1 -- 172.21.15.6:0/3137477886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c00d432dc0 msgr2=0x55c00d4351b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:26.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.166+0000 7f7bc7fff640 1 --2- 172.21.15.6:0/3137477886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c00d432dc0 0x55c00d4351b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.166+0000 7f7bc7fff640 1 -- 172.21.15.6:0/3137477886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c00d425810 msgr2=0x55c00d425bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:26.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.166+0000 7f7bc7fff640 1 --2- 172.21.15.6:0/3137477886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c00d425810 0x55c00d425bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.166+0000 7f7bc7fff640 1 -- 172.21.15.6:0/3137477886 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c00d340130 con 0x55c00d426130 2024-09-12T03:20:26.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.166+0000 7f7bc7fff640 1 --2- 172.21.15.6:0/3137477886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d426130 0x55c00d432880 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80cced0 tx=0x7f7bb80d7690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0e00e5b084718a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:26.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.166+0000 7f7bc57fa640 1 --2- 172.21.15.6:0/1828920943 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7bb00a2c10 0x7f7bb00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.166+0000 7f7bcc8db640 1 --2- 172.21.15.6:0/1828920943 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7bb00a2c10 0x7f7bb00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.166+0000 7f7bae7fc640 1 -- 172.21.15.6:0/3137477886 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb80de070 con 0x55c00d426130 2024-09-12T03:20:26.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.167+0000 7f7bc57fa640 1 -- 172.21.15.6:0/1828920943 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7bb80cf5b0 con 0x7f7bc8100a80 2024-09-12T03:20:26.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.167+0000 7f7bae7fc640 1 -- 172.21.15.6:0/3137477886 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bb809c3e0 con 0x55c00d426130 2024-09-12T03:20:26.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.167+0000 7f7bae7fc640 1 -- 172.21.15.6:0/3137477886 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb809bab0 con 0x55c00d426130 2024-09-12T03:20:26.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.167+0000 7f7bd0717740 1 -- 172.21.15.6:0/3137477886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d426130 msgr2=0x55c00d432880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:26.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.167+0000 7f7bcc8db640 1 --2- 172.21.15.6:0/1828920943 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7bb00a2c10 0x7f7bb00a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0005fd0 tx=0x7f7bc0009700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:26.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.167+0000 7f7bd0717740 1 --2- 172.21.15.6:0/3137477886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d426130 0x55c00d432880 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80cced0 tx=0x7f7bb80d7690 comp rx=0 tx=0).stop 2024-09-12T03:20:26.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.167+0000 7f7bd0717740 1 -- 172.21.15.6:0/3137477886 shutdown_connections 2024-09-12T03:20:26.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.167+0000 7f7bd0717740 1 --2- 172.21.15.6:0/3137477886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c00d432dc0 0x55c00d4351b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.167+0000 7f7bd0717740 1 --2- 172.21.15.6:0/3137477886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d426130 0x55c00d432880 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.167+0000 7f7bd0717740 1 --2- 172.21.15.6:0/3137477886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c00d425810 0x55c00d425bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.167+0000 7f7bd0717740 1 -- 172.21.15.6:0/3137477886 >> 172.21.15.6:0/3137477886 conn(0x55c00d421f60 msgr2=0x55c00d422340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:26.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.167+0000 7f7bd0717740 1 -- 172.21.15.6:0/3137477886 shutdown_connections 2024-09-12T03:20:26.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.167+0000 7f7bd0717740 1 -- 172.21.15.6:0/3137477886 wait complete. 2024-09-12T03:20:26.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.168+0000 7f7bd0717740 1 -- start start 2024-09-12T03:20:26.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.168+0000 7f7bd0717740 1 Processor -- start 2024-09-12T03:20:26.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.168+0000 7f7bd0717740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c00d425810 0x55c00d42de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.168+0000 7f7bd0717740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d426130 0x55c00d42e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.168+0000 7f7bd0717740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c00d432dc0 0x55c00d42e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.168+0000 7f7bd0717740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55c00d428850 con 0x55c00d426130 2024-09-12T03:20:26.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.168+0000 7f7bd0717740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55c00d346f20 con 0x55c00d425810 2024-09-12T03:20:26.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.168+0000 7f7bd0717740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c00d4289d0 con 0x55c00d432dc0 2024-09-12T03:20:26.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.168+0000 7f7bd0717740 10 client.0 _ll_register_callbacks cb 0x55c00d310030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:26.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.168+0000 7f7bc77fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d426130 0x55c00d42e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.169+0000 7f7bc77fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d426130 0x55c00d42e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35660/0 (socket says 172.21.15.6:35660) 2024-09-12T03:20:26.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.169+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c00d425810 0x55c00d42de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.169+0000 7f7bc77fe640 1 -- 172.21.15.6:0/3573425179 learned_addr learned my addr 172.21.15.6:0/3573425179 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:26.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.169+0000 7f7bcc8db640 1 --2- 172.21.15.6:0/3573425179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c00d432dc0 0x55c00d42e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.169+0000 7f7bc77fe640 1 -- 172.21.15.6:0/3573425179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c00d432dc0 msgr2=0x55c00d42e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:26.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.169+0000 7f7bc77fe640 1 --2- 172.21.15.6:0/3573425179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c00d432dc0 0x55c00d42e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.169+0000 7f7bc77fe640 1 -- 172.21.15.6:0/3573425179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c00d425810 msgr2=0x55c00d42de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:26.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.169+0000 7f7bc77fe640 1 --2- 172.21.15.6:0/3573425179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c00d425810 0x55c00d42de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.169+0000 7f7bc77fe640 1 -- 172.21.15.6:0/3573425179 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c00d0fd2b0 con 0x55c00d426130 2024-09-12T03:20:26.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.169+0000 7f7bcc8db640 1 --2- 172.21.15.6:0/3573425179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c00d432dc0 0x55c00d42e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:26.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.169+0000 7f7bc7fff640 1 --2- 172.21.15.6:0/3573425179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c00d425810 0x55c00d42de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:26.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.169+0000 7f7bc77fe640 1 --2- 172.21.15.6:0/3573425179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d426130 0x55c00d42e370 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc02a940 tx=0x7f7bbc0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260670d8c99c1262 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:26.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.169+0000 7f7ba77fe640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:20:26.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.169+0000 7f7ba77fe640 1 -- 172.21.15.6:0/3573425179 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc035040 con 0x55c00d426130 2024-09-12T03:20:26.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.169+0000 7f7bd0717740 1 -- 172.21.15.6:0/3573425179 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bb809c3e0 con 0x55c00d426130 2024-09-12T03:20:26.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.169+0000 7f7ba77fe640 1 -- 172.21.15.6:0/3573425179 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7bbc004240 con 0x55c00d426130 2024-09-12T03:20:26.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.170+0000 7f7ba77fe640 1 -- 172.21.15.6:0/3573425179 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7bbc04c070 con 0x55c00d426130 2024-09-12T03:20:26.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.170+0000 7f7bd0717740 10 client.14985 fetch_fsmap learned FSMap version 24 2024-09-12T03:20:26.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.170+0000 7f7bd0717740 1 -- 172.21.15.6:0/3573425179 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c00d4434e0 con 0x55c00d426130 2024-09-12T03:20:26.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.170+0000 7f7ba77fe640 1 -- 172.21.15.6:0/3573425179 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bb809c3e0 con 0x55c00d426130 2024-09-12T03:20:26.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.170+0000 7f7ba77fe640 1 -- 172.21.15.6:0/3573425179 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7bbc045640 con 0x55c00d426130 2024-09-12T03:20:26.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.170+0000 7f7bd0717740 10 client.14985 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:20:26.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.170+0000 7f7bd0717740 10 client.14985 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:20:26.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.170+0000 7f7bd0717740 20 client.14985 populate_metadata read hostname 'smithi006' 2024-09-12T03:20:26.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.170+0000 7f7bd0717740 1 --2- 172.21.15.6:0/3573425179 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55c00d445ae0 0x55c00d446f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.170+0000 7f7bd0717740 4 client.14985 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:26.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.170+0000 7f7bd0717740 1 -- 172.21.15.6:0/3573425179 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c00d399600 con 0x55c00d445ae0 2024-09-12T03:20:26.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.170+0000 7f7bc7fff640 1 --2- 172.21.15.6:0/3573425179 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55c00d445ae0 0x55c00d446f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.171+0000 7f7bc7fff640 1 --2- 172.21.15.6:0/3573425179 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55c00d445ae0 0x55c00d446f50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d514e20ac7b8633d server_cookie=8358101407b1340d in_seq=0 out_seq=0 2024-09-12T03:20:26.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.171+0000 7f7ba77fe640 10 client.14985 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:20:26.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.172+0000 7f7ba77fe640 1 -- 172.21.15.6:0/3573425179 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c00d399600 con 0x55c00d445ae0 2024-09-12T03:20:26.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.172+0000 7f7ba77fe640 10 client.14985 handle_command_reply: tid=0 2024-09-12T03:20:26.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.172+0000 7f7bd0717740 1 client.14985 shutdown 2024-09-12T03:20:26.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.172+0000 7f7bd0717740 1 -- 172.21.15.6:0/3573425179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d426130 msgr2=0x55c00d42e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:26.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.172+0000 7f7bd0717740 1 --2- 172.21.15.6:0/3573425179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d426130 0x55c00d42e370 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc02a940 tx=0x7f7bbc0317e0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.172+0000 7f7bd0717740 1 -- 172.21.15.6:0/3573425179 shutdown_connections 2024-09-12T03:20:26.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.172+0000 7f7bd0717740 1 --2- 172.21.15.6:0/3573425179 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55c00d445ae0 0x55c00d446f50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.172+0000 7f7bd0717740 1 --2- 172.21.15.6:0/3573425179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c00d432dc0 0x55c00d42e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.172+0000 7f7bd0717740 1 --2- 172.21.15.6:0/3573425179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d426130 0x55c00d42e370 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.172+0000 7f7bd0717740 1 --2- 172.21.15.6:0/3573425179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c00d425810 0x55c00d42de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.172+0000 7f7bd0717740 1 -- 172.21.15.6:0/3573425179 >> 172.21.15.6:0/3573425179 conn(0x55c00d421f60 msgr2=0x55c00d427c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:26.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.172+0000 7f7ba77fe640 0 client.14985 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:20:26.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.172+0000 7f7bd0717740 1 -- 172.21.15.6:0/3573425179 shutdown_connections 2024-09-12T03:20:26.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.172+0000 7f7bd0717740 1 -- 172.21.15.6:0/3573425179 wait complete. 2024-09-12T03:20:26.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.172+0000 7f7bd0717740 20 client.14985 trim_cache size 0 max 16384 2024-09-12T03:20:26.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.190+0000 7f7bd0717740 1 Processor -- start 2024-09-12T03:20:26.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.190+0000 7f7bd0717740 1 -- start start 2024-09-12T03:20:26.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.190+0000 7f7bd0717740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d425810 0x55c00d446f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.190+0000 7f7bd0717740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c00d426130 0x55c00d422340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.190+0000 7f7bd0717740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c00d432dc0 0x55c00d42e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.190+0000 7f7bd0717740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55c00d445290 con 0x55c00d425810 2024-09-12T03:20:26.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.190+0000 7f7bd0717740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55c00d445960 con 0x55c00d426130 2024-09-12T03:20:26.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.190+0000 7f7bd0717740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c00d446030 con 0x55c00d432dc0 2024-09-12T03:20:26.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.190+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d425810 0x55c00d446f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.190+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d425810 0x55c00d446f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35668/0 (socket says 172.21.15.6:35668) 2024-09-12T03:20:26.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.190+0000 7f7bc7fff640 1 -- 172.21.15.6:0/2972245037 learned_addr learned my addr 172.21.15.6:0/2972245037 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:26.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.190+0000 7f7bc7fff640 1 -- 172.21.15.6:0/2972245037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c00d432dc0 msgr2=0x55c00d42e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:26.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.190+0000 7f7bc7fff640 1 --2- 172.21.15.6:0/2972245037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c00d432dc0 0x55c00d42e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.190+0000 7f7bc7fff640 1 -- 172.21.15.6:0/2972245037 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c00d426130 msgr2=0x55c00d422340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:26.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.190+0000 7f7bc7fff640 1 --2- 172.21.15.6:0/2972245037 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c00d426130 0x55c00d422340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.190+0000 7f7bc7fff640 1 -- 172.21.15.6:0/2972245037 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c00d42c380 con 0x55c00d425810 2024-09-12T03:20:26.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.191+0000 7f7bc7fff640 1 --2- 172.21.15.6:0/2972245037 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d425810 0x55c00d446f50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8098140 tx=0x7f7bb80d5600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bda371f774ad4e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:26.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.191+0000 7f7badffb640 1 -- 172.21.15.6:0/2972245037 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb80de040 con 0x55c00d425810 2024-09-12T03:20:26.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.191+0000 7f7badffb640 1 -- 172.21.15.6:0/2972245037 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bb809c3e0 con 0x55c00d425810 2024-09-12T03:20:26.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.191+0000 7f7badffb640 1 -- 172.21.15.6:0/2972245037 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb809b640 con 0x55c00d425810 2024-09-12T03:20:26.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.191+0000 7f7bd0717740 1 -- 172.21.15.6:0/2972245037 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d425810 msgr2=0x55c00d446f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:26.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.191+0000 7f7bd0717740 1 --2- 172.21.15.6:0/2972245037 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d425810 0x55c00d446f50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8098140 tx=0x7f7bb80d5600 comp rx=0 tx=0).stop 2024-09-12T03:20:26.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.191+0000 7f7bd0717740 1 -- 172.21.15.6:0/2972245037 shutdown_connections 2024-09-12T03:20:26.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.191+0000 7f7bd0717740 1 --2- 172.21.15.6:0/2972245037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c00d432dc0 0x55c00d42e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.191+0000 7f7bd0717740 1 --2- 172.21.15.6:0/2972245037 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c00d426130 0x55c00d422340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.191+0000 7f7bd0717740 1 --2- 172.21.15.6:0/2972245037 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d425810 0x55c00d446f50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.191+0000 7f7bd0717740 1 -- 172.21.15.6:0/2972245037 >> 172.21.15.6:0/2972245037 conn(0x55c00d421f60 msgr2=0x55c00d42cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:26.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.191+0000 7f7bd0717740 1 -- 172.21.15.6:0/2972245037 shutdown_connections 2024-09-12T03:20:26.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.191+0000 7f7bd0717740 1 -- 172.21.15.6:0/2972245037 wait complete. 2024-09-12T03:20:26.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.192+0000 7f7bd0717740 1 -- start start 2024-09-12T03:20:26.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.192+0000 7f7bd0717740 1 Processor -- start 2024-09-12T03:20:26.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.192+0000 7f7bd0717740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c00d425810 0x55c00d439b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.192+0000 7f7bd0717740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d426130 0x55c00d43a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.192+0000 7f7bd0717740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c00d432dc0 0x55c00d43ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.192+0000 7f7bd0717740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55c00d446dd0 con 0x55c00d426130 2024-09-12T03:20:26.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.192+0000 7f7bd0717740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55c00d446700 con 0x55c00d432dc0 2024-09-12T03:20:26.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.192+0000 7f7bd0717740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c00d4474a0 con 0x55c00d425810 2024-09-12T03:20:26.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.192+0000 7f7bd0717740 10 client.0 _ll_register_callbacks cb 0x7f7bbc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:26.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.192+0000 7f7bc77fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d426130 0x55c00d43a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.192+0000 7f7bc77fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d426130 0x55c00d43a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35684/0 (socket says 172.21.15.6:35684) 2024-09-12T03:20:26.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.192+0000 7f7bc77fe640 1 -- 172.21.15.6:0/2821284167 learned_addr learned my addr 172.21.15.6:0/2821284167 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:26.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.193+0000 7f7bc7fff640 1 --2- 172.21.15.6:0/2821284167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c00d425810 0x55c00d439b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.193+0000 7f7bcc8db640 1 --2- 172.21.15.6:0/2821284167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c00d432dc0 0x55c00d43ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.193+0000 7f7bc77fe640 1 -- 172.21.15.6:0/2821284167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c00d425810 msgr2=0x55c00d439b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:26.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.193+0000 7f7bc77fe640 1 --2- 172.21.15.6:0/2821284167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c00d425810 0x55c00d439b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.193+0000 7f7bc77fe640 1 -- 172.21.15.6:0/2821284167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c00d432dc0 msgr2=0x55c00d43ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:26.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.193+0000 7f7bc77fe640 1 --2- 172.21.15.6:0/2821284167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c00d432dc0 0x55c00d43ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.193+0000 7f7bc77fe640 1 -- 172.21.15.6:0/2821284167 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c00d43f0f0 con 0x55c00d426130 2024-09-12T03:20:26.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.193+0000 7f7bcc8db640 1 --2- 172.21.15.6:0/2821284167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c00d432dc0 0x55c00d43ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:26.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.193+0000 7f7bc77fe640 1 --2- 172.21.15.6:0/2821284167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d426130 0x55c00d43a0a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc045b10 tx=0x7f7bbc005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3140dd6c6e46e27 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:26.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.193+0000 7f7ba77fe640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:20:26.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.193+0000 7f7ba77fe640 1 -- 172.21.15.6:0/2821284167 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc057070 con 0x55c00d426130 2024-09-12T03:20:26.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.193+0000 7f7bd0717740 1 -- 172.21.15.6:0/2821284167 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bb809c3e0 con 0x55c00d426130 2024-09-12T03:20:26.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.193+0000 7f7ba77fe640 1 -- 172.21.15.6:0/2821284167 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7bbc045680 con 0x55c00d426130 2024-09-12T03:20:26.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.193+0000 7f7ba77fe640 1 -- 172.21.15.6:0/2821284167 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7bbc045880 con 0x55c00d426130 2024-09-12T03:20:26.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.193+0000 7f7bd0717740 10 client.14991 fetch_fsmap learned FSMap version 24 2024-09-12T03:20:26.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.193+0000 7f7bd0717740 1 -- 172.21.15.6:0/2821284167 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c00d4403b0 con 0x55c00d426130 2024-09-12T03:20:26.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.193+0000 7f7ba77fe640 1 -- 172.21.15.6:0/2821284167 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bb809c3e0 con 0x55c00d426130 2024-09-12T03:20:26.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.194+0000 7f7ba77fe640 1 -- 172.21.15.6:0/2821284167 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7bbc0053a0 con 0x55c00d426130 2024-09-12T03:20:26.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.194+0000 7f7bd0717740 10 client.14991 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:20:26.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.194+0000 7f7bd0717740 10 client.14991 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:20:26.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.194+0000 7f7bd0717740 20 client.14991 populate_metadata read hostname 'smithi006' 2024-09-12T03:20:26.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.194+0000 7f7bd0717740 1 --2- 172.21.15.6:0/2821284167 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55c00d4b44a0 0x55c00d4b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.194+0000 7f7bd0717740 4 client.14991 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:20:26.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.194+0000 7f7bd0717740 1 -- 172.21.15.6:0/2821284167 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "status"}) -- 0x55c00d393b10 con 0x55c00d4b44a0 2024-09-12T03:20:26.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.194+0000 7f7bc7fff640 1 --2- 172.21.15.6:0/2821284167 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55c00d4b44a0 0x55c00d4b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.194+0000 7f7bc7fff640 1 --2- 172.21.15.6:0/2821284167 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55c00d4b44a0 0x55c00d4b31b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fe6f66368d98c3b server_cookie=7480733e681c0b50 in_seq=0 out_seq=0 2024-09-12T03:20:26.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.194+0000 7f7ba77fe640 10 client.14991 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:20:26.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.195+0000 7f7ba77fe640 1 -- 172.21.15.6:0/2821284167 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c00d393b10 con 0x55c00d4b44a0 2024-09-12T03:20:26.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.195+0000 7f7ba77fe640 10 client.14991 handle_command_reply: tid=0 2024-09-12T03:20:26.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.195+0000 7f7bd0717740 1 client.14991 shutdown 2024-09-12T03:20:26.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.195+0000 7f7bd0717740 1 -- 172.21.15.6:0/2821284167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d426130 msgr2=0x55c00d43a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:26.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.195+0000 7f7bd0717740 1 --2- 172.21.15.6:0/2821284167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d426130 0x55c00d43a0a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc045b10 tx=0x7f7bbc005a20 comp rx=0 tx=0).stop 2024-09-12T03:20:26.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.195+0000 7f7bd0717740 1 -- 172.21.15.6:0/2821284167 shutdown_connections 2024-09-12T03:20:26.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.195+0000 7f7bd0717740 1 --2- 172.21.15.6:0/2821284167 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55c00d4b44a0 0x55c00d4b31b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.195+0000 7f7ba77fe640 0 client.14991 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:20:26.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.195+0000 7f7bd0717740 1 --2- 172.21.15.6:0/2821284167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c00d432dc0 0x55c00d43ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.195+0000 7f7bd0717740 1 --2- 172.21.15.6:0/2821284167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c00d426130 0x55c00d43a0a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.195+0000 7f7bd0717740 1 --2- 172.21.15.6:0/2821284167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c00d425810 0x55c00d439b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.195+0000 7f7bd0717740 1 -- 172.21.15.6:0/2821284167 >> 172.21.15.6:0/2821284167 conn(0x55c00d421f60 msgr2=0x55c00d428c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:26.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.195+0000 7f7bd0717740 1 -- 172.21.15.6:0/2821284167 shutdown_connections 2024-09-12T03:20:26.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.195+0000 7f7bd0717740 1 -- 172.21.15.6:0/2821284167 wait complete. 2024-09-12T03:20:26.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.195+0000 7f7bd0717740 20 client.14991 trim_cache size 0 max 16384 2024-09-12T03:20:26.201 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:20:26.201 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "89d99938-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:20:26.201 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 0, 2024-09-12T03:20:26.201 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24494, 2024-09-12T03:20:26.202 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-09-12T03:20:26.202 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-09-12T03:20:26.202 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-09-12T03:20:26.202 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 75.872498673999999, 2024-09-12T03:20:26.202 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 24, 2024-09-12T03:20:26.202 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 82, 2024-09-12T03:20:26.202 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T03:20:26.202 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 85.137768360999999 2024-09-12T03:20:26.202 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:20:26.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.196+0000 7f7bce33e640 1 -- 172.21.15.6:0/1828920943 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7bb00a2c10 msgr2=0x7f7bb00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:26.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.196+0000 7f7bce33e640 1 --2- 172.21.15.6:0/1828920943 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7bb00a2c10 0x7f7bb00a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0005fd0 tx=0x7f7bc0009700 comp rx=0 tx=0).stop 2024-09-12T03:20:26.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.196+0000 7f7bce33e640 1 -- 172.21.15.6:0/1828920943 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bc8100a80 msgr2=0x7f7bc8197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:26.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.196+0000 7f7bce33e640 1 --2- 172.21.15.6:0/1828920943 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bc8100a80 0x7f7bc8197380 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800e9f0 tx=0x7f7bb800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.196+0000 7f7bce33e640 1 -- 172.21.15.6:0/1828920943 shutdown_connections 2024-09-12T03:20:26.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.196+0000 7f7bce33e640 1 --2- 172.21.15.6:0/1828920943 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7bb00a2c10 0x7f7bb00a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.196+0000 7f7bce33e640 1 --2- 172.21.15.6:0/1828920943 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7bc8106a80 0x7f7bc819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.196+0000 7f7bce33e640 1 --2- 172.21.15.6:0/1828920943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bc8101420 0x7f7bc81978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.196+0000 7f7bce33e640 1 --2- 172.21.15.6:0/1828920943 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bc8100a80 0x7f7bc8197380 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.196+0000 7f7bce33e640 1 -- 172.21.15.6:0/1828920943 >> 172.21.15.6:0/1828920943 conn(0x7f7bc80fc7a0 msgr2=0x7f7bc80fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:26.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.196+0000 7f7bce33e640 1 -- 172.21.15.6:0/1828920943 shutdown_connections 2024-09-12T03:20:26.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:26.196+0000 7f7bce33e640 1 -- 172.21.15.6:0/1828920943 wait complete. 2024-09-12T03:20:26.874 INFO:tasks.cephadm:Setup complete, yielding 2024-09-12T03:20:26.874 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T03:20:26.884 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi006.front.sepia.ceph.com 2024-09-12T03:20:26.884 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-12T03:20:27.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:26 smithi006 ceph-mon[30991]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:20:27.144 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:27.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:26 smithi118 ceph-mon[35048]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:20:27.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:26 smithi161 ceph-mon[35485]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:20:27.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.643+0000 7fceff181640 1 -- 172.21.15.6:0/1142619540 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef810e1e0 msgr2=0x7fcef81105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:27.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.643+0000 7fceff181640 1 --2- 172.21.15.6:0/1142619540 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef810e1e0 0x7fcef81105d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009ca0 tx=0x7fcef002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:20:27.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.643+0000 7fceff181640 1 -- 172.21.15.6:0/1142619540 shutdown_connections 2024-09-12T03:20:27.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.643+0000 7fceff181640 1 --2- 172.21.15.6:0/1142619540 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef810e1e0 0x7fcef81105d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:27.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.643+0000 7fceff181640 1 --2- 172.21.15.6:0/1142619540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcef8074c50 0x7fcef80750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:27.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.643+0000 7fceff181640 1 --2- 172.21.15.6:0/1142619540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcef8076220 0x7fcef8074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:27.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.643+0000 7fceff181640 1 -- 172.21.15.6:0/1142619540 >> 172.21.15.6:0/1142619540 conn(0x7fcef80fd390 msgr2=0x7fcef80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:27.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.643+0000 7fceff181640 1 -- 172.21.15.6:0/1142619540 shutdown_connections 2024-09-12T03:20:27.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.643+0000 7fceff181640 1 -- 172.21.15.6:0/1142619540 wait complete. 2024-09-12T03:20:27.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.644+0000 7fceff181640 1 Processor -- start 2024-09-12T03:20:27.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.644+0000 7fceff181640 1 -- start start 2024-09-12T03:20:27.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.644+0000 7fceff181640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef8074c50 0x7fcef819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:27.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.644+0000 7fceff181640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcef8076220 0x7fcef819e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:27.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.644+0000 7fceff181640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcef810e1e0 0x7fcef81a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:27.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.644+0000 7fceff181640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcef8113ea0 con 0x7fcef8074c50 2024-09-12T03:20:27.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.644+0000 7fceff181640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fcef8113d20 con 0x7fcef8076220 2024-09-12T03:20:27.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.644+0000 7fceff181640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcef8114020 con 0x7fcef810e1e0 2024-09-12T03:20:27.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.644+0000 7fcefcf1d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef8074c50 0x7fcef819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:27.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.644+0000 7fcefcf1d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef8074c50 0x7fcef819e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35696/0 (socket says 172.21.15.6:35696) 2024-09-12T03:20:27.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.644+0000 7fcefcf1d640 1 -- 172.21.15.6:0/2387974837 learned_addr learned my addr 172.21.15.6:0/2387974837 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:27.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.645+0000 7fcef7fff640 1 --2- 172.21.15.6:0/2387974837 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcef8076220 0x7fcef819e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:27.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.645+0000 7fcefd71e640 1 --2- 172.21.15.6:0/2387974837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcef810e1e0 0x7fcef81a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:27.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.645+0000 7fcefcf1d640 1 -- 172.21.15.6:0/2387974837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcef810e1e0 msgr2=0x7fcef81a5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:27.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.645+0000 7fcefcf1d640 1 --2- 172.21.15.6:0/2387974837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcef810e1e0 0x7fcef81a5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:27.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.645+0000 7fcefcf1d640 1 -- 172.21.15.6:0/2387974837 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcef8076220 msgr2=0x7fcef819e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:27.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.645+0000 7fcefcf1d640 1 --2- 172.21.15.6:0/2387974837 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcef8076220 0x7fcef819e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:27.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.645+0000 7fcefcf1d640 1 -- 172.21.15.6:0/2387974837 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef81a60c0 con 0x7fcef8074c50 2024-09-12T03:20:27.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.645+0000 7fcefd71e640 1 --2- 172.21.15.6:0/2387974837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcef810e1e0 0x7fcef81a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:27.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.645+0000 7fcefcf1d640 1 --2- 172.21.15.6:0/2387974837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef8074c50 0x7fcef819e490 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fceec00d740 tx=0x7fceec00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:27.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.645+0000 7fcef5ffb640 1 -- 172.21.15.6:0/2387974837 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec017070 con 0x7fcef8074c50 2024-09-12T03:20:27.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.645+0000 7fceff181640 1 -- 172.21.15.6:0/2387974837 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef81a63b0 con 0x7fcef8074c50 2024-09-12T03:20:27.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.645+0000 7fcef5ffb640 1 -- 172.21.15.6:0/2387974837 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fceec00de50 con 0x7fcef8074c50 2024-09-12T03:20:27.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.645+0000 7fcef5ffb640 1 -- 172.21.15.6:0/2387974837 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec004e00 con 0x7fcef8074c50 2024-09-12T03:20:27.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.645+0000 7fceff181640 1 -- 172.21.15.6:0/2387974837 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef81a68a0 con 0x7fcef8074c50 2024-09-12T03:20:27.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.646+0000 7fceff181640 1 -- 172.21.15.6:0/2387974837 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcef8104290 con 0x7fcef8074c50 2024-09-12T03:20:27.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.647+0000 7fcef5ffb640 1 -- 172.21.15.6:0/2387974837 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fceec004fa0 con 0x7fcef8074c50 2024-09-12T03:20:27.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.648+0000 7fcef5ffb640 1 --2- 172.21.15.6:0/2387974837 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcee80a2c40 0x7fcee80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:27.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.648+0000 7fcef7fff640 1 --2- 172.21.15.6:0/2387974837 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcee80a2c40 0x7fcee80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:27.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.648+0000 7fcef5ffb640 1 -- 172.21.15.6:0/2387974837 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fceec0cebc0 con 0x7fcef8074c50 2024-09-12T03:20:27.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.649+0000 7fcef7fff640 1 --2- 172.21.15.6:0/2387974837 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcee80a2c40 0x7fcee80a5100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcef819f9b0 tx=0x7fcee400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:27.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.652+0000 7fcef5ffb640 1 -- 172.21.15.6:0/2387974837 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fceec09b990 con 0x7fcef8074c50 2024-09-12T03:20:27.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.844+0000 7fceff181640 1 -- 172.21.15.6:0/2387974837 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fcef8062630 con 0x7fcee80a2c40 2024-09-12T03:20:27.845 INFO:teuthology.orchestra.run.smithi006.stdout:Backend: cephadm 2024-09-12T03:20:27.846 INFO:teuthology.orchestra.run.smithi006.stdout:Available: Yes 2024-09-12T03:20:27.846 INFO:teuthology.orchestra.run.smithi006.stdout:Paused: No 2024-09-12T03:20:27.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.845+0000 7fcef5ffb640 1 -- 172.21.15.6:0/2387974837 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fcef8062630 con 0x7fcee80a2c40 2024-09-12T03:20:27.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.848+0000 7fceff181640 1 -- 172.21.15.6:0/2387974837 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcee80a2c40 msgr2=0x7fcee80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:27.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.848+0000 7fceff181640 1 --2- 172.21.15.6:0/2387974837 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcee80a2c40 0x7fcee80a5100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcef819f9b0 tx=0x7fcee400b390 comp rx=0 tx=0).stop 2024-09-12T03:20:27.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.848+0000 7fceff181640 1 -- 172.21.15.6:0/2387974837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef8074c50 msgr2=0x7fcef819e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:27.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.848+0000 7fceff181640 1 --2- 172.21.15.6:0/2387974837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef8074c50 0x7fcef819e490 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fceec00d740 tx=0x7fceec00dc00 comp rx=0 tx=0).stop 2024-09-12T03:20:27.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.848+0000 7fceff181640 1 -- 172.21.15.6:0/2387974837 shutdown_connections 2024-09-12T03:20:27.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.848+0000 7fceff181640 1 --2- 172.21.15.6:0/2387974837 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcee80a2c40 0x7fcee80a5100 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:27.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.848+0000 7fceff181640 1 --2- 172.21.15.6:0/2387974837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcef810e1e0 0x7fcef81a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:27.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.848+0000 7fceff181640 1 --2- 172.21.15.6:0/2387974837 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcef8076220 0x7fcef819e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:27.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.848+0000 7fceff181640 1 --2- 172.21.15.6:0/2387974837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef8074c50 0x7fcef819e490 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:27.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.848+0000 7fceff181640 1 -- 172.21.15.6:0/2387974837 >> 172.21.15.6:0/2387974837 conn(0x7fcef80fd390 msgr2=0x7fcef8101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:27.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.848+0000 7fceff181640 1 -- 172.21.15.6:0/2387974837 shutdown_connections 2024-09-12T03:20:27.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:27.848+0000 7fceff181640 1 -- 172.21.15.6:0/2387974837 wait complete. 2024-09-12T03:20:28.524 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-12T03:20:28.784 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:29.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:28 smithi118 ceph-mon[35048]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:29.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:28 smithi118 ceph-mon[35048]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:29.233 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:28 smithi006 ceph-mon[30991]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:29.233 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:28 smithi006 ceph-mon[30991]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:29.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.233+0000 7f302da46640 1 -- 172.21.15.6:0/482412766 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3028101d10 msgr2=0x7f3028102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.233+0000 7f302da46640 1 --2- 172.21.15.6:0/482412766 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3028101d10 0x7f3028102190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3010009900 tx=0x7f301002a410 comp rx=0 tx=0).stop 2024-09-12T03:20:29.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.233+0000 7f302da46640 1 -- 172.21.15.6:0/482412766 shutdown_connections 2024-09-12T03:20:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.233+0000 7f302da46640 1 --2- 172.21.15.6:0/482412766 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f30281026d0 0x7f3028108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.233+0000 7f302da46640 1 --2- 172.21.15.6:0/482412766 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3028101d10 0x7f3028102190 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.233+0000 7f302da46640 1 --2- 172.21.15.6:0/482412766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3028100b10 0x7f3028100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.233+0000 7f302da46640 1 -- 172.21.15.6:0/482412766 >> 172.21.15.6:0/482412766 conn(0x7f30280fc2a0 msgr2=0x7f30280fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.233+0000 7f302da46640 1 -- 172.21.15.6:0/482412766 shutdown_connections 2024-09-12T03:20:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.233+0000 7f302da46640 1 -- 172.21.15.6:0/482412766 wait complete. 2024-09-12T03:20:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.234+0000 7f302da46640 1 Processor -- start 2024-09-12T03:20:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.234+0000 7f302da46640 1 -- start start 2024-09-12T03:20:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.234+0000 7f302da46640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3028100b10 0x7f3028197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.234+0000 7f302da46640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3028101d10 0x7f3028198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.234+0000 7f302da46640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30281026d0 0x7f302819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.234+0000 7f302da46640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f302810c770 con 0x7f30281026d0 2024-09-12T03:20:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.234+0000 7f302da46640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f302810c5f0 con 0x7f3028100b10 2024-09-12T03:20:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.234+0000 7f302da46640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f302810c8f0 con 0x7f3028101d10 2024-09-12T03:20:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.234+0000 7f30277fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30281026d0 0x7f302819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.234+0000 7f30277fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30281026d0 0x7f302819f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59956/0 (socket says 172.21.15.6:59956) 2024-09-12T03:20:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.234+0000 7f30277fe640 1 -- 172.21.15.6:0/1802336004 learned_addr learned my addr 172.21.15.6:0/1802336004 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.234+0000 7f3026ffd640 1 --2- 172.21.15.6:0/1802336004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3028100b10 0x7f3028197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.234+0000 7f30267fc640 1 --2- 172.21.15.6:0/1802336004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3028101d10 0x7f3028198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.235+0000 7f30277fe640 1 -- 172.21.15.6:0/1802336004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3028101d10 msgr2=0x7f3028198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.235+0000 7f30277fe640 1 --2- 172.21.15.6:0/1802336004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3028101d10 0x7f3028198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.235+0000 7f30277fe640 1 -- 172.21.15.6:0/1802336004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3028100b10 msgr2=0x7f3028197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.235+0000 7f30277fe640 1 --2- 172.21.15.6:0/1802336004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3028100b10 0x7f3028197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.235+0000 7f30277fe640 1 -- 172.21.15.6:0/1802336004 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302819fb40 con 0x7f30281026d0 2024-09-12T03:20:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.235+0000 7f30277fe640 1 --2- 172.21.15.6:0/1802336004 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30281026d0 0x7f302819f4d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f302000eaa0 tx=0x7f302000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.235+0000 7f302ca44640 1 -- 172.21.15.6:0/1802336004 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302000cdb0 con 0x7f30281026d0 2024-09-12T03:20:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.235+0000 7f302da46640 1 -- 172.21.15.6:0/1802336004 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302819fe30 con 0x7f30281026d0 2024-09-12T03:20:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.235+0000 7f302ca44640 1 -- 172.21.15.6:0/1802336004 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3020005020 con 0x7f30281026d0 2024-09-12T03:20:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.235+0000 7f302da46640 1 -- 172.21.15.6:0/1802336004 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30281a0290 con 0x7f30281026d0 2024-09-12T03:20:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.235+0000 7f302ca44640 1 -- 172.21.15.6:0/1802336004 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302001f060 con 0x7f30281026d0 2024-09-12T03:20:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.236+0000 7f302da46640 1 -- 172.21.15.6:0/1802336004 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30281a0570 con 0x7f30281026d0 2024-09-12T03:20:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.237+0000 7f302ca44640 1 -- 172.21.15.6:0/1802336004 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f3020005560 con 0x7f30281026d0 2024-09-12T03:20:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.238+0000 7f302ca44640 1 --2- 172.21.15.6:0/1802336004 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f30140a2c40 0x7f30140a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.239+0000 7f3026ffd640 1 --2- 172.21.15.6:0/1802336004 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f30140a2c40 0x7f30140a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.239+0000 7f302ca44640 1 -- 172.21.15.6:0/1802336004 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f30200cf3e0 con 0x7f30281026d0 2024-09-12T03:20:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.239+0000 7f3026ffd640 1 --2- 172.21.15.6:0/1802336004 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f30140a2c40 0x7f30140a5100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3028101b70 tx=0x7f301800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.243+0000 7f302ca44640 1 -- 172.21.15.6:0/1802336004 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3020016030 con 0x7f30281026d0 2024-09-12T03:20:29.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:28 smithi161 ceph-mon[35485]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:29.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:28 smithi161 ceph-mon[35485]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:29.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.432+0000 7f302da46640 1 -- 172.21.15.6:0/1802336004 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f30281a0740 con 0x7f30140a2c40 2024-09-12T03:20:29.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.444+0000 7f302ca44640 1 -- 172.21.15.6:0/1802336004 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7f30281a0740 con 0x7f30140a2c40 2024-09-12T03:20:29.445 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-12T03:20:29.445 INFO:teuthology.orchestra.run.smithi006.stdout:mds.a smithi006 running (91s) 17s ago 90s 14.3M - 19.1.1-298-gb53b6d3e 54467abc55f8 c12ad20163c7 2024-09-12T03:20:29.445 INFO:teuthology.orchestra.run.smithi006.stdout:mds.b smithi118 running (88s) 20s ago 88s 25.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 e937849e5ddb 2024-09-12T03:20:29.445 INFO:teuthology.orchestra.run.smithi006.stdout:mds.c smithi161 running (102s) 20s ago 101s 18.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 9abf9587f1ed 2024-09-12T03:20:29.445 INFO:teuthology.orchestra.run.smithi006.stdout:mds.d smithi006 running (110s) 17s ago 109s 15.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 1076a0428422 2024-09-12T03:20:29.445 INFO:teuthology.orchestra.run.smithi006.stdout:mds.e smithi118 running (107s) 20s ago 107s 17.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 3075a89eb6ea 2024-09-12T03:20:29.445 INFO:teuthology.orchestra.run.smithi006.stdout:mds.f smithi161 running (105s) 20s ago 104s 18.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 7aa5e7821250 2024-09-12T03:20:29.445 INFO:teuthology.orchestra.run.smithi006.stdout:mds.g smithi006 running (94s) 17s ago 93s 14.3M - 19.1.1-298-gb53b6d3e 54467abc55f8 d6b829cebcbd 2024-09-12T03:20:29.445 INFO:teuthology.orchestra.run.smithi006.stdout:mds.h smithi118 running (99s) 20s ago 99s 20.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 8070a91c59c6 2024-09-12T03:20:29.445 INFO:teuthology.orchestra.run.smithi006.stdout:mds.i smithi161 running (97s) 20s ago 96s 17.3M - 19.1.1-298-gb53b6d3e 54467abc55f8 12bd8a72daec 2024-09-12T03:20:29.446 INFO:teuthology.orchestra.run.smithi006.stdout:mds.j smithi006 running (113s) 17s ago 113s 14.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 cda4ab06d3e1 2024-09-12T03:20:29.446 INFO:teuthology.orchestra.run.smithi006.stdout:mds.k smithi118 running (116s) 20s ago 116s 16.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 89deea317135 2024-09-12T03:20:29.446 INFO:teuthology.orchestra.run.smithi006.stdout:mds.l smithi161 running (119s) 20s ago 118s 15.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 456b8c4a58a3 2024-09-12T03:20:29.446 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.x smithi006 *:9283,8765 running (8m) 17s ago 8m 508M - 19.1.1-298-gb53b6d3e 54467abc55f8 7792be15ecec 2024-09-12T03:20:29.446 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.y smithi118 *:8443,8765 running (7m) 20s ago 7m 441M - 19.1.1-298-gb53b6d3e 54467abc55f8 19c6b21ba90f 2024-09-12T03:20:29.446 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.z smithi161 *:8443,8765 running (7m) 20s ago 7m 441M - 19.1.1-298-gb53b6d3e 54467abc55f8 27802b55bbb0 2024-09-12T03:20:29.446 INFO:teuthology.orchestra.run.smithi006.stdout:mon.a smithi006 running (8m) 17s ago 8m 49.5M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 29ec88f97057 2024-09-12T03:20:29.446 INFO:teuthology.orchestra.run.smithi006.stdout:mon.b smithi118 running (7m) 20s ago 7m 40.3M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 7cca8700c631 2024-09-12T03:20:29.447 INFO:teuthology.orchestra.run.smithi006.stdout:mon.c smithi161 running (7m) 20s ago 7m 40.0M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 892bf1db843a 2024-09-12T03:20:29.447 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (6m) 17s ago 6m 56.9M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 a72f5b2efba9 2024-09-12T03:20:29.447 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi118 running (6m) 20s ago 6m 53.9M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 cdbe673d9f8e 2024-09-12T03:20:29.447 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi161 running (6m) 20s ago 6m 57.8M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 8f9438d60b4d 2024-09-12T03:20:29.447 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi006 running (5m) 17s ago 5m 48.9M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 4e170146bd51 2024-09-12T03:20:29.447 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi118 running (5m) 20s ago 5m 51.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 25dddd345d37 2024-09-12T03:20:29.447 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi161 running (4m) 20s ago 4m 46.4M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 faec383de158 2024-09-12T03:20:29.447 INFO:teuthology.orchestra.run.smithi006.stdout:osd.6 smithi006 running (4m) 17s ago 4m 52.2M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 9a117b4b6b03 2024-09-12T03:20:29.448 INFO:teuthology.orchestra.run.smithi006.stdout:osd.7 smithi118 running (3m) 20s ago 3m 48.6M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 62fe00bb357d 2024-09-12T03:20:29.448 INFO:teuthology.orchestra.run.smithi006.stdout:osd.8 smithi161 running (3m) 20s ago 3m 56.4M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 4b3a0dd4c5cb 2024-09-12T03:20:29.448 INFO:teuthology.orchestra.run.smithi006.stdout:osd.9 smithi006 running (3m) 17s ago 3m 44.3M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 0b3e2ef4936e 2024-09-12T03:20:29.448 INFO:teuthology.orchestra.run.smithi006.stdout:osd.10 smithi118 running (2m) 20s ago 2m 53.8M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 6a4aeb014b96 2024-09-12T03:20:29.448 INFO:teuthology.orchestra.run.smithi006.stdout:osd.11 smithi161 running (2m) 20s ago 2m 43.0M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 db84530bb035 2024-09-12T03:20:29.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.447+0000 7f302da46640 1 -- 172.21.15.6:0/1802336004 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f30140a2c40 msgr2=0x7f30140a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.447+0000 7f302da46640 1 --2- 172.21.15.6:0/1802336004 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f30140a2c40 0x7f30140a5100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3028101b70 tx=0x7f301800a2d0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.447+0000 7f302da46640 1 -- 172.21.15.6:0/1802336004 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30281026d0 msgr2=0x7f302819f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.447+0000 7f302da46640 1 --2- 172.21.15.6:0/1802336004 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30281026d0 0x7f302819f4d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f302000eaa0 tx=0x7f302000ef60 comp rx=0 tx=0).stop 2024-09-12T03:20:29.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.447+0000 7f302da46640 1 -- 172.21.15.6:0/1802336004 shutdown_connections 2024-09-12T03:20:29.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.447+0000 7f302da46640 1 --2- 172.21.15.6:0/1802336004 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f30140a2c40 0x7f30140a5100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.447+0000 7f302da46640 1 --2- 172.21.15.6:0/1802336004 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30281026d0 0x7f302819f4d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.447+0000 7f302da46640 1 --2- 172.21.15.6:0/1802336004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3028101d10 0x7f3028198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.447+0000 7f302da46640 1 --2- 172.21.15.6:0/1802336004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3028100b10 0x7f3028197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.447+0000 7f302da46640 1 -- 172.21.15.6:0/1802336004 >> 172.21.15.6:0/1802336004 conn(0x7f30280fc2a0 msgr2=0x7f302806a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:29.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.447+0000 7f302da46640 1 -- 172.21.15.6:0/1802336004 shutdown_connections 2024-09-12T03:20:29.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:29.447+0000 7f302da46640 1 -- 172.21.15.6:0/1802336004 wait complete. 2024-09-12T03:20:30.086 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-12T03:20:30.346 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:30.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.828+0000 7f2ac3f68640 1 -- 172.21.15.6:0/1211145421 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2abc100ad0 msgr2=0x7f2abc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:30.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.828+0000 7f2ac3f68640 1 --2- 172.21.15.6:0/1211145421 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2abc100ad0 0x7f2abc100ed0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009900 tx=0x7f2ab802a410 comp rx=0 tx=0).stop 2024-09-12T03:20:30.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.828+0000 7f2ac3f68640 1 -- 172.21.15.6:0/1211145421 shutdown_connections 2024-09-12T03:20:30.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.828+0000 7f2ac3f68640 1 --2- 172.21.15.6:0/1211145421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2abc102690 0x7f2abc108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:30.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.828+0000 7f2ac3f68640 1 --2- 172.21.15.6:0/1211145421 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2abc101cd0 0x7f2abc102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:30.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.828+0000 7f2ac3f68640 1 --2- 172.21.15.6:0/1211145421 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2abc100ad0 0x7f2abc100ed0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:30.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.828+0000 7f2ac3f68640 1 -- 172.21.15.6:0/1211145421 >> 172.21.15.6:0/1211145421 conn(0x7f2abc0fc280 msgr2=0x7f2abc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:30.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.828+0000 7f2ac3f68640 1 -- 172.21.15.6:0/1211145421 shutdown_connections 2024-09-12T03:20:30.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.828+0000 7f2ac3f68640 1 -- 172.21.15.6:0/1211145421 wait complete. 2024-09-12T03:20:30.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.829+0000 7f2ac3f68640 1 Processor -- start 2024-09-12T03:20:30.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.829+0000 7f2ac3f68640 1 -- start start 2024-09-12T03:20:30.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.829+0000 7f2ac3f68640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2abc100ad0 0x7f2abc197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:30.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.829+0000 7f2ac3f68640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2abc101cd0 0x7f2abc1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:30.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.829+0000 7f2ac3f68640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2abc102690 0x7f2abc19f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:30.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.829+0000 7f2ac3f68640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc10c7d0 con 0x7f2abc100ad0 2024-09-12T03:20:30.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.829+0000 7f2ac3f68640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc10c650 con 0x7f2abc101cd0 2024-09-12T03:20:30.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.829+0000 7f2ac3f68640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc10c950 con 0x7f2abc102690 2024-09-12T03:20:30.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.829+0000 7f2ac1d04640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2abc100ad0 0x7f2abc197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:30.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.829+0000 7f2ac1d04640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2abc100ad0 0x7f2abc197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59972/0 (socket says 172.21.15.6:59972) 2024-09-12T03:20:30.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.829+0000 7f2ac1d04640 1 -- 172.21.15.6:0/1931226076 learned_addr learned my addr 172.21.15.6:0/1931226076 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:30.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.829+0000 7f2ac2505640 1 --2- 172.21.15.6:0/1931226076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2abc102690 0x7f2abc19f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:30.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.829+0000 7f2ac1503640 1 --2- 172.21.15.6:0/1931226076 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2abc101cd0 0x7f2abc1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:30.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.829+0000 7f2ac1d04640 1 -- 172.21.15.6:0/1931226076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2abc102690 msgr2=0x7f2abc19f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:30.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.829+0000 7f2ac1d04640 1 --2- 172.21.15.6:0/1931226076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2abc102690 0x7f2abc19f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:30.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.829+0000 7f2ac1d04640 1 -- 172.21.15.6:0/1931226076 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2abc101cd0 msgr2=0x7f2abc1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:30.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.829+0000 7f2ac1d04640 1 --2- 172.21.15.6:0/1931226076 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2abc101cd0 0x7f2abc1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:30.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.829+0000 7f2ac1d04640 1 -- 172.21.15.6:0/1931226076 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2abc19fbd0 con 0x7f2abc100ad0 2024-09-12T03:20:30.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.830+0000 7f2ac1d04640 1 --2- 172.21.15.6:0/1931226076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2abc100ad0 0x7f2abc197fa0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8031610 tx=0x7f2ab80043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:30.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.830+0000 7f2ab2ffd640 1 -- 172.21.15.6:0/1931226076 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab803f030 con 0x7f2abc100ad0 2024-09-12T03:20:30.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.830+0000 7f2ab2ffd640 1 -- 172.21.15.6:0/1931226076 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ab8003ce0 con 0x7f2abc100ad0 2024-09-12T03:20:30.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.830+0000 7f2ac3f68640 1 -- 172.21.15.6:0/1931226076 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2abc19fe60 con 0x7f2abc100ad0 2024-09-12T03:20:30.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.830+0000 7f2ab2ffd640 1 -- 172.21.15.6:0/1931226076 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8035070 con 0x7f2abc100ad0 2024-09-12T03:20:30.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.830+0000 7f2ac3f68640 1 -- 172.21.15.6:0/1931226076 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2abc1a02e0 con 0x7f2abc100ad0 2024-09-12T03:20:30.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.831+0000 7f2ac3f68640 1 -- 172.21.15.6:0/1931226076 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2abc1a05d0 con 0x7f2abc100ad0 2024-09-12T03:20:30.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.832+0000 7f2ab2ffd640 1 -- 172.21.15.6:0/1931226076 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2ab8042030 con 0x7f2abc100ad0 2024-09-12T03:20:30.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.833+0000 7f2ab2ffd640 1 --2- 172.21.15.6:0/1931226076 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2aa80a2c40 0x7f2aa80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:30.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.833+0000 7f2ac1503640 1 --2- 172.21.15.6:0/1931226076 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2aa80a2c40 0x7f2aa80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:30.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.833+0000 7f2ab2ffd640 1 -- 172.21.15.6:0/1931226076 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f2ab80ef9a0 con 0x7f2abc100ad0 2024-09-12T03:20:30.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.834+0000 7f2ac1503640 1 --2- 172.21.15.6:0/1931226076 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2aa80a2c40 0x7f2aa80a5100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2abc1994c0 tx=0x7f2aac008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:30.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:30.837+0000 7f2ab2ffd640 1 -- 172.21.15.6:0/1931226076 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ab80bc7d0 con 0x7f2abc100ad0 2024-09-12T03:20:31.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:31.022+0000 7f2ac3f68640 1 -- 172.21.15.6:0/1931226076 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f2abc06a730 con 0x7f2aa80a2c40 2024-09-12T03:20:31.023 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:30 smithi006 ceph-mon[30991]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:20:31.023 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:30 smithi006 ceph-mon[30991]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:31.028 INFO:teuthology.orchestra.run.smithi006.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-12T03:20:31.028 INFO:teuthology.orchestra.run.smithi006.stdout:mds.all 12/12 21s ago 2m smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi118=b;smithi118=e;smithi118=h;smithi118=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-09-12T03:20:31.028 INFO:teuthology.orchestra.run.smithi006.stdout:mgr 3/3 21s ago 7m smithi006=x;smithi118=y;smithi161=z;count:3 2024-09-12T03:20:31.028 INFO:teuthology.orchestra.run.smithi006.stdout:mon 3/3 21s ago 7m smithi006:172.21.15.6=a;smithi118:172.21.15.118=b;smithi161:172.21.15.161=c;count:3 2024-09-12T03:20:31.028 INFO:teuthology.orchestra.run.smithi006.stdout:osd 12 21s ago - 2024-09-12T03:20:31.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:31.028+0000 7f2ab2ffd640 1 -- 172.21.15.6:0/1931226076 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f2abc06a730 con 0x7f2aa80a2c40 2024-09-12T03:20:31.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:31.030+0000 7f2ac3f68640 1 -- 172.21.15.6:0/1931226076 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2aa80a2c40 msgr2=0x7f2aa80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:31.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:31.030+0000 7f2ac3f68640 1 --2- 172.21.15.6:0/1931226076 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2aa80a2c40 0x7f2aa80a5100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2abc1994c0 tx=0x7f2aac008040 comp rx=0 tx=0).stop 2024-09-12T03:20:31.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:31.030+0000 7f2ac3f68640 1 -- 172.21.15.6:0/1931226076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2abc100ad0 msgr2=0x7f2abc197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:31.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:31.030+0000 7f2ac3f68640 1 --2- 172.21.15.6:0/1931226076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2abc100ad0 0x7f2abc197fa0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8031610 tx=0x7f2ab80043b0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:31.030+0000 7f2ac3f68640 1 -- 172.21.15.6:0/1931226076 shutdown_connections 2024-09-12T03:20:31.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:31.030+0000 7f2ac3f68640 1 --2- 172.21.15.6:0/1931226076 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2aa80a2c40 0x7f2aa80a5100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:31.030+0000 7f2ac3f68640 1 --2- 172.21.15.6:0/1931226076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2abc102690 0x7f2abc19f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:31.030+0000 7f2ac3f68640 1 --2- 172.21.15.6:0/1931226076 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2abc101cd0 0x7f2abc1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:31.030+0000 7f2ac3f68640 1 --2- 172.21.15.6:0/1931226076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2abc100ad0 0x7f2abc197fa0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:31.030+0000 7f2ac3f68640 1 -- 172.21.15.6:0/1931226076 >> 172.21.15.6:0/1931226076 conn(0x7f2abc0fc280 msgr2=0x7f2abc06a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:31.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:31.031+0000 7f2ac3f68640 1 -- 172.21.15.6:0/1931226076 shutdown_connections 2024-09-12T03:20:31.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:31.031+0000 7f2ac3f68640 1 -- 172.21.15.6:0/1931226076 wait complete. 2024-09-12T03:20:31.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:30 smithi118 ceph-mon[35048]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:20:31.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:30 smithi118 ceph-mon[35048]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:31.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:30 smithi161 ceph-mon[35485]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:20:31.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:30 smithi161 ceph-mon[35485]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:31.708 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-12T03:20:31.966 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:32.087 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:31 smithi006 ceph-mon[30991]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:32.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:31 smithi118 ceph-mon[35048]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:32.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:31 smithi161 ceph-mon[35485]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:32.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.481+0000 7f4c87b5a640 1 -- 172.21.15.6:0/2741448515 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80109a80 msgr2=0x7f4c8010bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:32.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.481+0000 7f4c87b5a640 1 --2- 172.21.15.6:0/2741448515 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80109a80 0x7f4c8010bf40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009900 tx=0x7f4c7802a410 comp rx=0 tx=0).stop 2024-09-12T03:20:32.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.481+0000 7f4c87b5a640 1 -- 172.21.15.6:0/2741448515 shutdown_connections 2024-09-12T03:20:32.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.481+0000 7f4c87b5a640 1 --2- 172.21.15.6:0/2741448515 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80109a80 0x7f4c8010bf40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.481+0000 7f4c87b5a640 1 --2- 172.21.15.6:0/2741448515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c80074c50 0x7f4c800750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.481+0000 7f4c87b5a640 1 --2- 172.21.15.6:0/2741448515 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c80076220 0x7f4c80074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.481+0000 7f4c87b5a640 1 -- 172.21.15.6:0/2741448515 >> 172.21.15.6:0/2741448515 conn(0x7f4c800fd390 msgr2=0x7f4c800ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:32.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.481+0000 7f4c87b5a640 1 -- 172.21.15.6:0/2741448515 shutdown_connections 2024-09-12T03:20:32.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.481+0000 7f4c87b5a640 1 -- 172.21.15.6:0/2741448515 wait complete. 2024-09-12T03:20:32.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.481+0000 7f4c87b5a640 1 Processor -- start 2024-09-12T03:20:32.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.481+0000 7f4c87b5a640 1 -- start start 2024-09-12T03:20:32.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.482+0000 7f4c87b5a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c80074c50 0x7f4c8019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:32.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.482+0000 7f4c87b5a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80076220 0x7f4c8019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:32.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.482+0000 7f4c87b5a640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c80109a80 0x7f4c801a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:32.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.482+0000 7f4c87b5a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8010fb00 con 0x7f4c80076220 2024-09-12T03:20:32.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.482+0000 7f4c87b5a640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8010f980 con 0x7f4c80109a80 2024-09-12T03:20:32.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.482+0000 7f4c87b5a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8010fc80 con 0x7f4c80074c50 2024-09-12T03:20:32.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.482+0000 7f4c850f5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80076220 0x7f4c8019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:32.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.482+0000 7f4c850f5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80076220 0x7f4c8019a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59980/0 (socket says 172.21.15.6:59980) 2024-09-12T03:20:32.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.482+0000 7f4c850f5640 1 -- 172.21.15.6:0/361707394 learned_addr learned my addr 172.21.15.6:0/361707394 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:32.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.482+0000 7f4c858f6640 1 --2- 172.21.15.6:0/361707394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c80074c50 0x7f4c8019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:32.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.482+0000 7f4c860f7640 1 --2- 172.21.15.6:0/361707394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c80109a80 0x7f4c801a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:32.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.482+0000 7f4c850f5640 1 -- 172.21.15.6:0/361707394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c80074c50 msgr2=0x7f4c8019a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:32.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.482+0000 7f4c850f5640 1 --2- 172.21.15.6:0/361707394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c80074c50 0x7f4c8019a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.482+0000 7f4c850f5640 1 -- 172.21.15.6:0/361707394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c80109a80 msgr2=0x7f4c801a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:32.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.482+0000 7f4c850f5640 1 --2- 172.21.15.6:0/361707394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c80109a80 0x7f4c801a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.482+0000 7f4c850f5640 1 -- 172.21.15.6:0/361707394 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c801a1cd0 con 0x7f4c80076220 2024-09-12T03:20:32.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.482+0000 7f4c858f6640 1 --2- 172.21.15.6:0/361707394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c80074c50 0x7f4c8019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:32.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.482+0000 7f4c860f7640 1 --2- 172.21.15.6:0/361707394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c80109a80 0x7f4c801a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:32.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.482+0000 7f4c850f5640 1 --2- 172.21.15.6:0/361707394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80076220 0x7f4c8019a5e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4c7000b4a0 tx=0x7f4c7000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:32.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.483+0000 7f4c76ffd640 1 -- 172.21.15.6:0/361707394 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c70014070 con 0x7f4c80076220 2024-09-12T03:20:32.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.483+0000 7f4c76ffd640 1 -- 172.21.15.6:0/361707394 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c70010af0 con 0x7f4c80076220 2024-09-12T03:20:32.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.483+0000 7f4c87b5a640 1 -- 172.21.15.6:0/361707394 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c801a1fc0 con 0x7f4c80076220 2024-09-12T03:20:32.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.483+0000 7f4c76ffd640 1 -- 172.21.15.6:0/361707394 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c70017d10 con 0x7f4c80076220 2024-09-12T03:20:32.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.483+0000 7f4c87b5a640 1 -- 172.21.15.6:0/361707394 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c801a24f0 con 0x7f4c80076220 2024-09-12T03:20:32.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.484+0000 7f4c87b5a640 1 -- 172.21.15.6:0/361707394 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c80108800 con 0x7f4c80076220 2024-09-12T03:20:32.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.485+0000 7f4c76ffd640 1 -- 172.21.15.6:0/361707394 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4c70017070 con 0x7f4c80076220 2024-09-12T03:20:32.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.486+0000 7f4c76ffd640 1 --2- 172.21.15.6:0/361707394 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4c6c0a2c40 0x7f4c6c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:32.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.486+0000 7f4c858f6640 1 --2- 172.21.15.6:0/361707394 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4c6c0a2c40 0x7f4c6c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:32.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.486+0000 7f4c76ffd640 1 -- 172.21.15.6:0/361707394 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f4c700ce460 con 0x7f4c80076220 2024-09-12T03:20:32.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.486+0000 7f4c858f6640 1 --2- 172.21.15.6:0/361707394 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4c6c0a2c40 0x7f4c6c0a5100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4c80076600 tx=0x7f4c7c007850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:32.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.490+0000 7f4c76ffd640 1 -- 172.21.15.6:0/361707394 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4c7009b3a0 con 0x7f4c80076220 2024-09-12T03:20:32.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.674+0000 7f4c87b5a640 1 -- 172.21.15.6:0/361707394 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f4c80062630 con 0x7f4c6c0a2c40 2024-09-12T03:20:32.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.675+0000 7f4c76ffd640 1 -- 172.21.15.6:0/361707394 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f4c80062630 con 0x7f4c6c0a2c40 2024-09-12T03:20:32.675 INFO:teuthology.orchestra.run.smithi006.stdout:HOST ADDR LABELS STATUS 2024-09-12T03:20:32.675 INFO:teuthology.orchestra.run.smithi006.stdout:smithi006 172.21.15.6 2024-09-12T03:20:32.675 INFO:teuthology.orchestra.run.smithi006.stdout:smithi118 172.21.15.118 2024-09-12T03:20:32.675 INFO:teuthology.orchestra.run.smithi006.stdout:smithi161 172.21.15.161 2024-09-12T03:20:32.675 INFO:teuthology.orchestra.run.smithi006.stdout:3 hosts in cluster 2024-09-12T03:20:32.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.677+0000 7f4c87b5a640 1 -- 172.21.15.6:0/361707394 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4c6c0a2c40 msgr2=0x7f4c6c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:32.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.677+0000 7f4c87b5a640 1 --2- 172.21.15.6:0/361707394 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4c6c0a2c40 0x7f4c6c0a5100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4c80076600 tx=0x7f4c7c007850 comp rx=0 tx=0).stop 2024-09-12T03:20:32.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.677+0000 7f4c87b5a640 1 -- 172.21.15.6:0/361707394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80076220 msgr2=0x7f4c8019a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:32.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.677+0000 7f4c87b5a640 1 --2- 172.21.15.6:0/361707394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80076220 0x7f4c8019a5e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4c7000b4a0 tx=0x7f4c7000b960 comp rx=0 tx=0).stop 2024-09-12T03:20:32.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.677+0000 7f4c87b5a640 1 -- 172.21.15.6:0/361707394 shutdown_connections 2024-09-12T03:20:32.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.677+0000 7f4c87b5a640 1 --2- 172.21.15.6:0/361707394 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4c6c0a2c40 0x7f4c6c0a5100 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.677+0000 7f4c87b5a640 1 --2- 172.21.15.6:0/361707394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c80109a80 0x7f4c801a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.677+0000 7f4c87b5a640 1 --2- 172.21.15.6:0/361707394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80076220 0x7f4c8019a5e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.677+0000 7f4c87b5a640 1 --2- 172.21.15.6:0/361707394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c80074c50 0x7f4c8019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.677+0000 7f4c87b5a640 1 -- 172.21.15.6:0/361707394 >> 172.21.15.6:0/361707394 conn(0x7f4c800fd390 msgr2=0x7f4c801059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:32.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.677+0000 7f4c87b5a640 1 -- 172.21.15.6:0/361707394 shutdown_connections 2024-09-12T03:20:32.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:32.677+0000 7f4c87b5a640 1 -- 172.21.15.6:0/361707394 wait complete. 2024-09-12T03:20:33.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:32 smithi006 ceph-mon[30991]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:33.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:32 smithi118 ceph-mon[35048]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:33.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:32 smithi161 ceph-mon[35485]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:33.364 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-12T03:20:33.622 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:34.011 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:33 smithi006 ceph-mon[30991]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:34.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.088+0000 7f971126a640 1 -- 172.21.15.6:0/211853863 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f970c074c50 msgr2=0x7f970c0750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:34.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.088+0000 7f971126a640 1 --2- 172.21.15.6:0/211853863 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f970c074c50 0x7f970c0750b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9700009900 tx=0x7f970002a410 comp rx=0 tx=0).stop 2024-09-12T03:20:34.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.088+0000 7f971126a640 1 -- 172.21.15.6:0/211853863 shutdown_connections 2024-09-12T03:20:34.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.088+0000 7f971126a640 1 --2- 172.21.15.6:0/211853863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f970c10e1e0 0x7f970c1105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:34.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.088+0000 7f971126a640 1 --2- 172.21.15.6:0/211853863 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f970c074c50 0x7f970c0750b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:34.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.088+0000 7f971126a640 1 --2- 172.21.15.6:0/211853863 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f970c076220 0x7f970c074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:34.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.088+0000 7f971126a640 1 -- 172.21.15.6:0/211853863 >> 172.21.15.6:0/211853863 conn(0x7f970c0fd390 msgr2=0x7f970c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:34.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.088+0000 7f971126a640 1 -- 172.21.15.6:0/211853863 shutdown_connections 2024-09-12T03:20:34.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.088+0000 7f971126a640 1 -- 172.21.15.6:0/211853863 wait complete. 2024-09-12T03:20:34.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.089+0000 7f971126a640 1 Processor -- start 2024-09-12T03:20:34.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.089+0000 7f971126a640 1 -- start start 2024-09-12T03:20:34.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.089+0000 7f971126a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f970c074c50 0x7f970c19e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:34.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.089+0000 7f971126a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f970c076220 0x7f970c19ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:34.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.089+0000 7f971126a640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f970c10e1e0 0x7f970c1a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:34.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.089+0000 7f971126a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f970c113ea0 con 0x7f970c076220 2024-09-12T03:20:34.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.089+0000 7f971126a640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f970c113d20 con 0x7f970c10e1e0 2024-09-12T03:20:34.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.089+0000 7f971126a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f970c114020 con 0x7f970c074c50 2024-09-12T03:20:34.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.089+0000 7f970a7fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f970c076220 0x7f970c19ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:34.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.089+0000 7f970a7fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f970c076220 0x7f970c19ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60004/0 (socket says 172.21.15.6:60004) 2024-09-12T03:20:34.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.089+0000 7f970a7fc640 1 -- 172.21.15.6:0/1834647272 learned_addr learned my addr 172.21.15.6:0/1834647272 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:34.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.089+0000 7f970b7fe640 1 --2- 172.21.15.6:0/1834647272 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f970c10e1e0 0x7f970c1a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:34.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.089+0000 7f970affd640 1 --2- 172.21.15.6:0/1834647272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f970c074c50 0x7f970c19e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:34.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.090+0000 7f970a7fc640 1 -- 172.21.15.6:0/1834647272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f970c074c50 msgr2=0x7f970c19e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:34.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.090+0000 7f970a7fc640 1 --2- 172.21.15.6:0/1834647272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f970c074c50 0x7f970c19e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:34.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.090+0000 7f970a7fc640 1 -- 172.21.15.6:0/1834647272 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f970c10e1e0 msgr2=0x7f970c1a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:34.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.090+0000 7f970a7fc640 1 --2- 172.21.15.6:0/1834647272 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f970c10e1e0 0x7f970c1a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:34.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.090+0000 7f970a7fc640 1 -- 172.21.15.6:0/1834647272 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970c1a6160 con 0x7f970c076220 2024-09-12T03:20:34.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.090+0000 7f970a7fc640 1 --2- 172.21.15.6:0/1834647272 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f970c076220 0x7f970c19ea70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9700009230 tx=0x7f9700004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:34.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.090+0000 7f96f3fff640 1 -- 172.21.15.6:0/1834647272 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970003f030 con 0x7f970c076220 2024-09-12T03:20:34.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.090+0000 7f971126a640 1 -- 172.21.15.6:0/1834647272 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970c1a63f0 con 0x7f970c076220 2024-09-12T03:20:34.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.090+0000 7f96f3fff640 1 -- 172.21.15.6:0/1834647272 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9700003ce0 con 0x7f970c076220 2024-09-12T03:20:34.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.090+0000 7f96f3fff640 1 -- 172.21.15.6:0/1834647272 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700035070 con 0x7f970c076220 2024-09-12T03:20:34.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.090+0000 7f971126a640 1 -- 172.21.15.6:0/1834647272 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f970c1a6920 con 0x7f970c076220 2024-09-12T03:20:34.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.091+0000 7f971126a640 1 -- 172.21.15.6:0/1834647272 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f970c1043b0 con 0x7f970c076220 2024-09-12T03:20:34.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.092+0000 7f96f3fff640 1 -- 172.21.15.6:0/1834647272 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9700042030 con 0x7f970c076220 2024-09-12T03:20:34.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.093+0000 7f96f3fff640 1 --2- 172.21.15.6:0/1834647272 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f96f40a2c40 0x7f96f40a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:34.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.093+0000 7f96f3fff640 1 -- 172.21.15.6:0/1834647272 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f97000ef4a0 con 0x7f970c076220 2024-09-12T03:20:34.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.093+0000 7f970affd640 1 --2- 172.21.15.6:0/1834647272 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f96f40a2c40 0x7f96f40a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:34.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.093+0000 7f970affd640 1 --2- 172.21.15.6:0/1834647272 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f96f40a2c40 0x7f96f40a5100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f96fc005d00 tx=0x7f96fc005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:34.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.097+0000 7f96f3fff640 1 -- 172.21.15.6:0/1834647272 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f97000bc2d0 con 0x7f970c076220 2024-09-12T03:20:34.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:33 smithi118 ceph-mon[35048]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:34.265 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:33 smithi161 ceph-mon[35485]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:34.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.280+0000 7f971126a640 1 -- 172.21.15.6:0/1834647272 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f970c062630 con 0x7f96f40a2c40 2024-09-12T03:20:34.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.282+0000 7f96f3fff640 1 -- 172.21.15.6:0/1834647272 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f970c062630 con 0x7f96f40a2c40 2024-09-12T03:20:34.282 INFO:teuthology.orchestra.run.smithi006.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-12T03:20:34.282 INFO:teuthology.orchestra.run.smithi006.stdout:smithi006 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330000H400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-12T03:20:34.282 INFO:teuthology.orchestra.run.smithi006.stdout:smithi118 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747500JS480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-09-12T03:20:34.282 INFO:teuthology.orchestra.run.smithi006.stdout:smithi161 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001X400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-12T03:20:34.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.285+0000 7f971126a640 1 -- 172.21.15.6:0/1834647272 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f96f40a2c40 msgr2=0x7f96f40a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:34.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.285+0000 7f971126a640 1 --2- 172.21.15.6:0/1834647272 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f96f40a2c40 0x7f96f40a5100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f96fc005d00 tx=0x7f96fc005c50 comp rx=0 tx=0).stop 2024-09-12T03:20:34.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.285+0000 7f971126a640 1 -- 172.21.15.6:0/1834647272 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f970c076220 msgr2=0x7f970c19ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:34.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.285+0000 7f971126a640 1 --2- 172.21.15.6:0/1834647272 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f970c076220 0x7f970c19ea70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9700009230 tx=0x7f9700004360 comp rx=0 tx=0).stop 2024-09-12T03:20:34.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.285+0000 7f971126a640 1 -- 172.21.15.6:0/1834647272 shutdown_connections 2024-09-12T03:20:34.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.285+0000 7f971126a640 1 --2- 172.21.15.6:0/1834647272 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f96f40a2c40 0x7f96f40a5100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:34.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.285+0000 7f971126a640 1 --2- 172.21.15.6:0/1834647272 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f970c10e1e0 0x7f970c1a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:34.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.285+0000 7f971126a640 1 --2- 172.21.15.6:0/1834647272 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f970c076220 0x7f970c19ea70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:34.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.285+0000 7f971126a640 1 --2- 172.21.15.6:0/1834647272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f970c074c50 0x7f970c19e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:34.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.285+0000 7f971126a640 1 -- 172.21.15.6:0/1834647272 >> 172.21.15.6:0/1834647272 conn(0x7f970c0fd390 msgr2=0x7f970c101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:34.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.285+0000 7f971126a640 1 -- 172.21.15.6:0/1834647272 shutdown_connections 2024-09-12T03:20:34.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:34.285+0000 7f971126a640 1 -- 172.21.15.6:0/1834647272 wait complete. 2024-09-12T03:20:34.932 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T03:20:34.943 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi006.front.sepia.ceph.com 2024-09-12T03:20:34.943 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-12T03:20:35.202 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:35.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:34 smithi118 ceph-mon[35048]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:20:35.308 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:34 smithi006 ceph-mon[30991]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:20:35.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:34 smithi161 ceph-mon[35485]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:20:35.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.652+0000 7f059df54640 1 -- 172.21.15.6:0/3946567010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0598105070 msgr2=0x7f0598107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:35.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.652+0000 7f059df54640 1 --2- 172.21.15.6:0/3946567010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0598105070 0x7f0598107460 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0580009900 tx=0x7f058002a410 comp rx=0 tx=0).stop 2024-09-12T03:20:35.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.652+0000 7f059df54640 1 -- 172.21.15.6:0/3946567010 shutdown_connections 2024-09-12T03:20:35.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.652+0000 7f059df54640 1 --2- 172.21.15.6:0/3946567010 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f05981079a0 0x7f0598109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.652+0000 7f059df54640 1 --2- 172.21.15.6:0/3946567010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0598105070 0x7f0598107460 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.652+0000 7f059df54640 1 --2- 172.21.15.6:0/3946567010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0598102740 0x7f0598104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.652+0000 7f059df54640 1 -- 172.21.15.6:0/3946567010 >> 172.21.15.6:0/3946567010 conn(0x7f05980fc2a0 msgr2=0x7f05980fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:35.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.652+0000 7f059df54640 1 -- 172.21.15.6:0/3946567010 shutdown_connections 2024-09-12T03:20:35.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.652+0000 7f059df54640 1 -- 172.21.15.6:0/3946567010 wait complete. 2024-09-12T03:20:35.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.652+0000 7f059df54640 1 Processor -- start 2024-09-12T03:20:35.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.652+0000 7f059df54640 1 -- start start 2024-09-12T03:20:35.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.653+0000 7f059df54640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0598102740 0x7f05981020a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:35.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.653+0000 7f059df54640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0598105070 0x7f05981006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:35.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.653+0000 7f059df54640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05981079a0 0x7f0598100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:35.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.653+0000 7f059df54640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f059810d900 con 0x7f05981079a0 2024-09-12T03:20:35.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.653+0000 7f059df54640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f059810d780 con 0x7f0598102740 2024-09-12T03:20:35.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.653+0000 7f059df54640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f059810da80 con 0x7f0598105070 2024-09-12T03:20:35.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.653+0000 7f0597fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05981079a0 0x7f0598100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:35.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.653+0000 7f0597fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05981079a0 0x7f0598100c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60026/0 (socket says 172.21.15.6:60026) 2024-09-12T03:20:35.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.653+0000 7f0597fff640 1 -- 172.21.15.6:0/1859939169 learned_addr learned my addr 172.21.15.6:0/1859939169 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:35.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.653+0000 7f05977fe640 1 --2- 172.21.15.6:0/1859939169 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0598102740 0x7f05981020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:35.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.653+0000 7f058effd640 1 --2- 172.21.15.6:0/1859939169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0598105070 0x7f05981006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:35.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.653+0000 7f0597fff640 1 -- 172.21.15.6:0/1859939169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0598105070 msgr2=0x7f05981006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:35.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.653+0000 7f0597fff640 1 --2- 172.21.15.6:0/1859939169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0598105070 0x7f05981006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.653+0000 7f0597fff640 1 -- 172.21.15.6:0/1859939169 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0598102740 msgr2=0x7f05981020a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:35.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.653+0000 7f0597fff640 1 --2- 172.21.15.6:0/1859939169 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0598102740 0x7f05981020a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.653+0000 7f0597fff640 1 -- 172.21.15.6:0/1859939169 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05981014f0 con 0x7f05981079a0 2024-09-12T03:20:35.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.653+0000 7f05977fe640 1 --2- 172.21.15.6:0/1859939169 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0598102740 0x7f05981020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:35.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.653+0000 7f0597fff640 1 --2- 172.21.15.6:0/1859939169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05981079a0 0x7f0598100c30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f059000b970 tx=0x7f059000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:35.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.654+0000 7f05957fa640 1 -- 172.21.15.6:0/1859939169 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590019070 con 0x7f05981079a0 2024-09-12T03:20:35.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.654+0000 7f05957fa640 1 -- 172.21.15.6:0/1859939169 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f059000cb40 con 0x7f05981079a0 2024-09-12T03:20:35.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.654+0000 7f059df54640 1 -- 172.21.15.6:0/1859939169 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05981a21a0 con 0x7f05981079a0 2024-09-12T03:20:35.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.654+0000 7f05957fa640 1 -- 172.21.15.6:0/1859939169 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590007650 con 0x7f05981079a0 2024-09-12T03:20:35.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.654+0000 7f059df54640 1 -- 172.21.15.6:0/1859939169 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05981a2730 con 0x7f05981079a0 2024-09-12T03:20:35.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.655+0000 7f059df54640 1 -- 172.21.15.6:0/1859939169 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05980fdd90 con 0x7f05981079a0 2024-09-12T03:20:35.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.656+0000 7f05957fa640 1 -- 172.21.15.6:0/1859939169 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f059001c070 con 0x7f05981079a0 2024-09-12T03:20:35.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.657+0000 7f05957fa640 1 --2- 172.21.15.6:0/1859939169 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f05840a2c90 0x7f05840a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:35.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.657+0000 7f05977fe640 1 --2- 172.21.15.6:0/1859939169 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f05840a2c90 0x7f05840a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:35.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.657+0000 7f05957fa640 1 -- 172.21.15.6:0/1859939169 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f05900ce5e0 con 0x7f05981079a0 2024-09-12T03:20:35.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.657+0000 7f05977fe640 1 --2- 172.21.15.6:0/1859939169 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f05840a2c90 0x7f05840a5150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f05880058c0 tx=0x7f0588005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:35.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.661+0000 7f05957fa640 1 -- 172.21.15.6:0/1859939169 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f059009aed0 con 0x7f05981079a0 2024-09-12T03:20:35.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.874+0000 7f059df54640 1 -- 172.21.15.6:0/1859939169 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f059802d7c0 con 0x7f05981079a0 2024-09-12T03:20:35.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.874+0000 7f05957fa640 1 -- 172.21.15.6:0/1859939169 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3084 (secure 0 0 0) 0x7f059009f1d0 con 0x7f05981079a0 2024-09-12T03:20:35.874 INFO:teuthology.orchestra.run.smithi006.stdout:e24 2024-09-12T03:20:35.875 INFO:teuthology.orchestra.run.smithi006.stdout:btime 2024-09-12T03:19:21:375379+0000 2024-09-12T03:20:35.875 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T03:20:35.875 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T03:20:35.875 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-09-12T03:20:35.875 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:20:35.875 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-09-12T03:20:35.875 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-09-12T03:20:35.875 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 24 2024-09-12T03:20:35.875 INFO:teuthology.orchestra.run.smithi006.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-12T03:20:35.875 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-09-12T03:19:09.851402+0000 2024-09-12T03:20:35.875 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-09-12T03:19:20.919190+0000 2024-09-12T03:20:35.875 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-09-12T03:20:35.876 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-09-12T03:20:35.876 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 300 2024-09-12T03:20:35.876 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-09-12T03:20:35.876 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-09-12T03:20:35.876 INFO:teuthology.orchestra.run.smithi006.stdout:max_xattr_size 65536 2024-09-12T03:20:35.876 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-09-12T03:20:35.876 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-09-12T03:20:35.876 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-09-12T03:20:35.876 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T03:20:35.876 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 3 2024-09-12T03:20:35.876 INFO:teuthology.orchestra.run.smithi006.stdout:in 0,1,2 2024-09-12T03:20:35.876 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=24494,1=24464,2=24463} 2024-09-12T03:20:35.876 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-09-12T03:20:35.877 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-09-12T03:20:35.877 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-09-12T03:20:35.877 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-09-12T03:20:35.877 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-09-12T03:20:35.877 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-09-12T03:20:35.877 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-09-12T03:20:35.877 INFO:teuthology.orchestra.run.smithi006.stdout:bal_rank_mask -1 2024-09-12T03:20:35.877 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 0 2024-09-12T03:20:35.877 INFO:teuthology.orchestra.run.smithi006.stdout:qdb_cluster leader: 24494 members: 24463,24464,24494 2024-09-12T03:20:35.877 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.b{0:24494} state up:active seq 4 addr [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:20:35.877 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.c{0:24451} state up:standby-replay seq 1 addr [v2:172.21.15.161:6836/283024944,v1:172.21.15.161:6837/283024944] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:20:35.877 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.h{1:24464} state up:active seq 8 addr [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:20:35.877 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.e{1:24440} state up:standby-replay seq 1 addr [v2:172.21.15.118:6834/1389654073,v1:172.21.15.118:6835/1389654073] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:20:35.878 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.i{2:24463} state up:active seq 8 addr [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:20:35.878 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.f{2:24439} state up:standby-replay seq 1 addr [v2:172.21.15.161:6834/955127607,v1:172.21.15.161:6835/955127607] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:20:35.878 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:20:35.878 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:20:35.878 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-09-12T03:20:35.878 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:20:35.878 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.j{-1:14523} state up:standby seq 1 addr [v2:172.21.15.6:6834/4139960526,v1:172.21.15.6:6835/4139960526] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:20:35.878 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.d{-1:14535} state up:standby seq 1 addr [v2:172.21.15.6:6836/3369703756,v1:172.21.15.6:6837/3369703756] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:20:35.878 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.g{-1:14586} state up:standby seq 1 addr [v2:172.21.15.6:6838/3301072029,v1:172.21.15.6:6839/3301072029] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:20:35.878 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.a{-1:14598} state up:standby seq 1 addr [v2:172.21.15.6:6840/2670411625,v1:172.21.15.6:6841/2670411625] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:20:35.878 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.l{-1:24403} state up:standby seq 1 addr [v2:172.21.15.161:6832/1749185252,v1:172.21.15.161:6833/1749185252] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:20:35.878 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.k{-1:24413} state up:standby seq 1 addr [v2:172.21.15.118:6832/3592007490,v1:172.21.15.118:6833/3592007490] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:20:35.879 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 24 2024-09-12T03:20:35.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.877+0000 7f059df54640 1 -- 172.21.15.6:0/1859939169 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f05840a2c90 msgr2=0x7f05840a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:35.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.877+0000 7f059df54640 1 --2- 172.21.15.6:0/1859939169 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f05840a2c90 0x7f05840a5150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f05880058c0 tx=0x7f0588005850 comp rx=0 tx=0).stop 2024-09-12T03:20:35.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.877+0000 7f059df54640 1 -- 172.21.15.6:0/1859939169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05981079a0 msgr2=0x7f0598100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:35.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.877+0000 7f059df54640 1 --2- 172.21.15.6:0/1859939169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05981079a0 0x7f0598100c30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f059000b970 tx=0x7f059000be30 comp rx=0 tx=0).stop 2024-09-12T03:20:35.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.878+0000 7f059df54640 1 -- 172.21.15.6:0/1859939169 shutdown_connections 2024-09-12T03:20:35.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.878+0000 7f059df54640 1 --2- 172.21.15.6:0/1859939169 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f05840a2c90 0x7f05840a5150 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.878+0000 7f059df54640 1 --2- 172.21.15.6:0/1859939169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05981079a0 0x7f0598100c30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.878+0000 7f059df54640 1 --2- 172.21.15.6:0/1859939169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0598105070 0x7f05981006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.878+0000 7f059df54640 1 --2- 172.21.15.6:0/1859939169 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0598102740 0x7f05981020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.878+0000 7f059df54640 1 -- 172.21.15.6:0/1859939169 >> 172.21.15.6:0/1859939169 conn(0x7f05980fc2a0 msgr2=0x7f0598103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:35.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.878+0000 7f059df54640 1 -- 172.21.15.6:0/1859939169 shutdown_connections 2024-09-12T03:20:35.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:35.878+0000 7f059df54640 1 -- 172.21.15.6:0/1859939169 wait complete. 2024-09-12T03:20:36.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:35 smithi006 ceph-mon[30991]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:36.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:35 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1859939169' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T03:20:36.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:35 smithi118 ceph-mon[35048]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:36.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:35 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1859939169' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T03:20:36.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:35 smithi161 ceph-mon[35485]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:36.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:35 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1859939169' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T03:20:36.561 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-12T03:20:36.818 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:37.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:36 smithi118 ceph-mon[35048]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:20:37.227 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:36 smithi006 ceph-mon[30991]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:20:37.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.310+0000 7f730ec81640 1 -- 172.21.15.6:0/4250001702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73081026a0 msgr2=0x7f7308104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:37.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.310+0000 7f730ec81640 1 --2- 172.21.15.6:0/4250001702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73081026a0 0x7f7308104a90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f72f8009900 tx=0x7f72f802a410 comp rx=0 tx=0).stop 2024-09-12T03:20:37.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.310+0000 7f730ec81640 1 -- 172.21.15.6:0/4250001702 shutdown_connections 2024-09-12T03:20:37.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.310+0000 7f730ec81640 1 --2- 172.21.15.6:0/4250001702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7308107900 0x7f7308109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.310+0000 7f730ec81640 1 --2- 172.21.15.6:0/4250001702 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7308104fd0 0x7f73081073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.310+0000 7f730ec81640 1 --2- 172.21.15.6:0/4250001702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73081026a0 0x7f7308104a90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.310+0000 7f730ec81640 1 -- 172.21.15.6:0/4250001702 >> 172.21.15.6:0/4250001702 conn(0x7f73080fc2a0 msgr2=0x7f73080fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:37.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.310+0000 7f730ec81640 1 -- 172.21.15.6:0/4250001702 shutdown_connections 2024-09-12T03:20:37.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.310+0000 7f730ec81640 1 -- 172.21.15.6:0/4250001702 wait complete. 2024-09-12T03:20:37.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.310+0000 7f730ec81640 1 Processor -- start 2024-09-12T03:20:37.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.311+0000 7f730ec81640 1 -- start start 2024-09-12T03:20:37.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.311+0000 7f730ec81640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73081026a0 0x7f7308197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.311+0000 7f730ec81640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7308104fd0 0x7f7308198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.311+0000 7f730ec81640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7308107900 0x7f730819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.311+0000 7f730ec81640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f730810d8b0 con 0x7f73081026a0 2024-09-12T03:20:37.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.311+0000 7f730ec81640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f730810d730 con 0x7f7308104fd0 2024-09-12T03:20:37.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.311+0000 7f730ec81640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f730810da30 con 0x7f7308107900 2024-09-12T03:20:37.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.311+0000 7f730ca1d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73081026a0 0x7f7308197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.311+0000 7f730ca1d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73081026a0 0x7f7308197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60038/0 (socket says 172.21.15.6:60038) 2024-09-12T03:20:37.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.311+0000 7f730ca1d640 1 -- 172.21.15.6:0/75311735 learned_addr learned my addr 172.21.15.6:0/75311735 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:37.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.311+0000 7f7307fff640 1 --2- 172.21.15.6:0/75311735 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7308104fd0 0x7f7308198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.311+0000 7f730ca1d640 1 -- 172.21.15.6:0/75311735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7308107900 msgr2=0x7f730819f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:37.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.311+0000 7f730d21e640 1 --2- 172.21.15.6:0/75311735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7308107900 0x7f730819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.311+0000 7f730ca1d640 1 --2- 172.21.15.6:0/75311735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7308107900 0x7f730819f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.311+0000 7f730ca1d640 1 -- 172.21.15.6:0/75311735 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7308104fd0 msgr2=0x7f7308198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:37.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.311+0000 7f730ca1d640 1 --2- 172.21.15.6:0/75311735 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7308104fd0 0x7f7308198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.311+0000 7f730ca1d640 1 -- 172.21.15.6:0/75311735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730819fb10 con 0x7f73081026a0 2024-09-12T03:20:37.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.311+0000 7f7307fff640 1 --2- 172.21.15.6:0/75311735 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7308104fd0 0x7f7308198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:37.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.311+0000 7f730d21e640 1 --2- 172.21.15.6:0/75311735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7308107900 0x7f730819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:37.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.312+0000 7f730ca1d640 1 --2- 172.21.15.6:0/75311735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73081026a0 0x7f7308197ee0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f72f8009230 tx=0x7f72f8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:37.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.312+0000 7f7305ffb640 1 -- 172.21.15.6:0/75311735 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f803f030 con 0x7f73081026a0 2024-09-12T03:20:37.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.312+0000 7f730ec81640 1 -- 172.21.15.6:0/75311735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f730819fda0 con 0x7f73081026a0 2024-09-12T03:20:37.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.312+0000 7f7305ffb640 1 -- 172.21.15.6:0/75311735 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72f8003ce0 con 0x7f73081026a0 2024-09-12T03:20:37.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.312+0000 7f7305ffb640 1 -- 172.21.15.6:0/75311735 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f8035070 con 0x7f73081026a0 2024-09-12T03:20:37.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.312+0000 7f730ec81640 1 -- 172.21.15.6:0/75311735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73081a01c0 con 0x7f73081026a0 2024-09-12T03:20:37.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.313+0000 7f730ec81640 1 -- 172.21.15.6:0/75311735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73080fdd90 con 0x7f73081026a0 2024-09-12T03:20:37.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.314+0000 7f7305ffb640 1 -- 172.21.15.6:0/75311735 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f72f8042030 con 0x7f73081026a0 2024-09-12T03:20:37.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.315+0000 7f7305ffb640 1 --2- 172.21.15.6:0/75311735 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f72f40a2c40 0x7f72f40a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.315+0000 7f7307fff640 1 --2- 172.21.15.6:0/75311735 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f72f40a2c40 0x7f72f40a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.315+0000 7f7305ffb640 1 -- 172.21.15.6:0/75311735 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f72f80ef8c0 con 0x7f73081026a0 2024-09-12T03:20:37.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.315+0000 7f7307fff640 1 --2- 172.21.15.6:0/75311735 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f72f40a2c40 0x7f72f40a5100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f72f00058c0 tx=0x7f72f0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:37.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.319+0000 7f7305ffb640 1 -- 172.21.15.6:0/75311735 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f72f80bc690 con 0x7f73081026a0 2024-09-12T03:20:37.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:36 smithi161 ceph-mon[35485]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:20:37.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.497+0000 7f730ec81640 1 -- 172.21.15.6:0/75311735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f730802d7c0 con 0x7f73081026a0 2024-09-12T03:20:37.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.498+0000 7f7305ffb640 1 -- 172.21.15.6:0/75311735 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v82) ==== 56+0+5227 (secure 0 0 0) 0x7f72f80bbf90 con 0x7f73081026a0 2024-09-12T03:20:37.498 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 82 2024-09-12T03:20:37.498 INFO:teuthology.orchestra.run.smithi006.stdout:fsid 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:20:37.498 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-09-12T03:11:29.530545+0000 2024-09-12T03:20:37.498 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-09-12T03:19:20.366710+0000 2024-09-12T03:20:37.499 INFO:teuthology.orchestra.run.smithi006.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-12T03:20:37.499 INFO:teuthology.orchestra.run.smithi006.stdout:crush_version 26 2024-09-12T03:20:37.499 INFO:teuthology.orchestra.run.smithi006.stdout:full_ratio 0.95 2024-09-12T03:20:37.499 INFO:teuthology.orchestra.run.smithi006.stdout:backfillfull_ratio 0.9 2024-09-12T03:20:37.499 INFO:teuthology.orchestra.run.smithi006.stdout:nearfull_ratio 0.85 2024-09-12T03:20:37.499 INFO:teuthology.orchestra.run.smithi006.stdout:require_min_compat_client luminous 2024-09-12T03:20:37.499 INFO:teuthology.orchestra.run.smithi006.stdout:min_compat_client luminous 2024-09-12T03:20:37.499 INFO:teuthology.orchestra.run.smithi006.stdout:require_osd_release squid 2024-09-12T03:20:37.499 INFO:teuthology.orchestra.run.smithi006.stdout:stretch_mode_enabled false 2024-09-12T03:20:37.499 INFO:teuthology.orchestra.run.smithi006.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-12T03:20:37.500 INFO:teuthology.orchestra.run.smithi006.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-12T03:20:37.500 INFO:teuthology.orchestra.run.smithi006.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-12T03:20:37.500 INFO:teuthology.orchestra.run.smithi006.stdout:max_osd 12 2024-09-12T03:20:37.500 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 up in weight 1 up_from 8 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.6:6802/726729286,v1:172.21.15.6:6803/726729286] [v2:172.21.15.6:6804/726729286,v1:172.21.15.6:6805/726729286] exists,up 0f55d68b-95a0-4ee2-a46f-bc422b38471e 2024-09-12T03:20:37.500 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 up in weight 1 up_from 13 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6800/3377555415,v1:172.21.15.118:6801/3377555415] [v2:172.21.15.118:6802/3377555415,v1:172.21.15.118:6803/3377555415] exists,up 83afdb01-c731-4de1-96c6-3f798549b1eb 2024-09-12T03:20:37.500 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 up in weight 1 up_from 18 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6800/392541009,v1:172.21.15.161:6801/392541009] [v2:172.21.15.161:6802/392541009,v1:172.21.15.161:6803/392541009] exists,up aed718b4-b849-403a-b81c-3a140b5c380c 2024-09-12T03:20:37.500 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 up in weight 1 up_from 25 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.6:6810/1892039151,v1:172.21.15.6:6811/1892039151] [v2:172.21.15.6:6812/1892039151,v1:172.21.15.6:6813/1892039151] exists,up 25ced4c1-bf31-402d-8397-d42fd57f615f 2024-09-12T03:20:37.500 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 up in weight 1 up_from 32 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6808/150177024,v1:172.21.15.118:6809/150177024] [v2:172.21.15.118:6810/150177024,v1:172.21.15.118:6811/150177024] exists,up 1d884604-920b-4c35-b41e-d629ad9f9279 2024-09-12T03:20:37.500 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 up in weight 1 up_from 36 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6808/3758167854,v1:172.21.15.161:6809/3758167854] [v2:172.21.15.161:6810/3758167854,v1:172.21.15.161:6811/3758167854] exists,up 1d40c0ef-99f6-426f-90c7-27579edc66af 2024-09-12T03:20:37.500 INFO:teuthology.orchestra.run.smithi006.stdout:osd.6 up in weight 1 up_from 43 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.6:6818/2129224874,v1:172.21.15.6:6819/2129224874] [v2:172.21.15.6:6820/2129224874,v1:172.21.15.6:6821/2129224874] exists,up 9d9f9510-b107-40c2-9a98-9012715944b3 2024-09-12T03:20:37.500 INFO:teuthology.orchestra.run.smithi006.stdout:osd.7 up in weight 1 up_from 48 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6816/269980353,v1:172.21.15.118:6817/269980353] [v2:172.21.15.118:6818/269980353,v1:172.21.15.118:6819/269980353] exists,up 84635ae1-3240-4954-af14-76bb3aa162e8 2024-09-12T03:20:37.500 INFO:teuthology.orchestra.run.smithi006.stdout:osd.8 up in weight 1 up_from 55 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6816/1783416134,v1:172.21.15.161:6817/1783416134] [v2:172.21.15.161:6818/1783416134,v1:172.21.15.161:6819/1783416134] exists,up 6fd58e55-1492-4595-8647-823ea3e3c5d7 2024-09-12T03:20:37.501 INFO:teuthology.orchestra.run.smithi006.stdout:osd.9 up in weight 1 up_from 62 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.6:6826/2987332365,v1:172.21.15.6:6827/2987332365] [v2:172.21.15.6:6828/2987332365,v1:172.21.15.6:6829/2987332365] exists,up 90508a6d-a49c-43c6-a600-5bbccd8702ea 2024-09-12T03:20:37.501 INFO:teuthology.orchestra.run.smithi006.stdout:osd.10 up in weight 1 up_from 67 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6824/2167146341,v1:172.21.15.118:6825/2167146341] [v2:172.21.15.118:6826/2167146341,v1:172.21.15.118:6827/2167146341] exists,up 911e86d1-669a-4935-82c0-7da51d141df8 2024-09-12T03:20:37.501 INFO:teuthology.orchestra.run.smithi006.stdout:osd.11 up in weight 1 up_from 74 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6824/2889795796,v1:172.21.15.161:6825/2889795796] [v2:172.21.15.161:6826/2889795796,v1:172.21.15.161:6827/2889795796] exists,up 5a15c6a7-ff43-473c-bb48-0c129c316df5 2024-09-12T03:20:37.501 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 2.13 [2,11] 2024-09-12T03:20:37.501 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 2.16 [2,4] 2024-09-12T03:20:37.501 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 2.1d [8,11] 2024-09-12T03:20:37.501 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 3.1 [4,9] 2024-09-12T03:20:37.501 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 3.2 [3,9] 2024-09-12T03:20:37.501 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 3.21 [5,9] 2024-09-12T03:20:37.501 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 3.3a [4,8] 2024-09-12T03:20:37.501 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:6800/1684477653 expires 2024-09-13T03:12:19.155264+0000 2024-09-12T03:20:37.501 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/2620491916 expires 2024-09-13T03:12:19.155264+0000 2024-09-12T03:20:37.501 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/4037656392 expires 2024-09-13T03:12:19.155264+0000 2024-09-12T03:20:37.501 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/1377116573 expires 2024-09-13T03:12:19.155264+0000 2024-09-12T03:20:37.502 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/3062800125 expires 2024-09-13T03:11:55.244257+0000 2024-09-12T03:20:37.502 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:6801/1684477653 expires 2024-09-13T03:12:19.155264+0000 2024-09-12T03:20:37.502 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:6800/836933784 expires 2024-09-13T03:11:55.244257+0000 2024-09-12T03:20:37.502 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:6801/836933784 expires 2024-09-13T03:11:55.244257+0000 2024-09-12T03:20:37.502 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/4145338968 expires 2024-09-13T03:11:55.244257+0000 2024-09-12T03:20:37.502 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/825701128 expires 2024-09-13T03:11:55.244257+0000 2024-09-12T03:20:37.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.501+0000 7f730ec81640 1 -- 172.21.15.6:0/75311735 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f72f40a2c40 msgr2=0x7f72f40a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:37.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.501+0000 7f730ec81640 1 --2- 172.21.15.6:0/75311735 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f72f40a2c40 0x7f72f40a5100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f72f00058c0 tx=0x7f72f0005850 comp rx=0 tx=0).stop 2024-09-12T03:20:37.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.501+0000 7f730ec81640 1 -- 172.21.15.6:0/75311735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73081026a0 msgr2=0x7f7308197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:37.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.501+0000 7f730ec81640 1 --2- 172.21.15.6:0/75311735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73081026a0 0x7f7308197ee0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f72f8009230 tx=0x7f72f8004360 comp rx=0 tx=0).stop 2024-09-12T03:20:37.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.501+0000 7f730ec81640 1 -- 172.21.15.6:0/75311735 shutdown_connections 2024-09-12T03:20:37.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.501+0000 7f730ec81640 1 --2- 172.21.15.6:0/75311735 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f72f40a2c40 0x7f72f40a5100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.501+0000 7f730ec81640 1 --2- 172.21.15.6:0/75311735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7308107900 0x7f730819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.501+0000 7f730ec81640 1 --2- 172.21.15.6:0/75311735 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7308104fd0 0x7f7308198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.501+0000 7f730ec81640 1 --2- 172.21.15.6:0/75311735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73081026a0 0x7f7308197ee0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.501+0000 7f730ec81640 1 -- 172.21.15.6:0/75311735 >> 172.21.15.6:0/75311735 conn(0x7f73080fc2a0 msgr2=0x7f7308102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:37.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.501+0000 7f730ec81640 1 -- 172.21.15.6:0/75311735 shutdown_connections 2024-09-12T03:20:37.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:37.501+0000 7f730ec81640 1 -- 172.21.15.6:0/75311735 wait complete. 2024-09-12T03:20:38.188 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-12T03:20:39.175 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:20:39.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:37 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/75311735' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T03:20:39.177 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:37 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/75311735' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T03:20:39.177 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:37 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/75311735' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T03:20:39.205 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:38 smithi006 ceph-mon[30991]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:39.211 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:38 smithi118 ceph-mon[35048]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:39.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:38 smithi161 ceph-mon[35485]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:39.446 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:39.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.940+0000 7f981c5b1640 1 -- 172.21.15.6:0/3465074711 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9814101de0 msgr2=0x7f98141021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:39.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.940+0000 7f981c5b1640 1 --2- 172.21.15.6:0/3465074711 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9814101de0 0x7f98141021c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9810009900 tx=0x7f981002a410 comp rx=0 tx=0).stop 2024-09-12T03:20:39.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.940+0000 7f981c5b1640 1 -- 172.21.15.6:0/3465074711 shutdown_connections 2024-09-12T03:20:39.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.940+0000 7f981c5b1640 1 --2- 172.21.15.6:0/3465074711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9814106f10 0x7f981410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:39.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.940+0000 7f981c5b1640 1 --2- 172.21.15.6:0/3465074711 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9814102790 0x7f98141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:39.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.940+0000 7f981c5b1640 1 --2- 172.21.15.6:0/3465074711 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9814101de0 0x7f98141021c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.940+0000 7f981c5b1640 1 -- 172.21.15.6:0/3465074711 >> 172.21.15.6:0/3465074711 conn(0x7f98140fd8c0 msgr2=0x7f98140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.940+0000 7f981c5b1640 1 -- 172.21.15.6:0/3465074711 shutdown_connections 2024-09-12T03:20:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.940+0000 7f981c5b1640 1 -- 172.21.15.6:0/3465074711 wait complete. 2024-09-12T03:20:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.941+0000 7f981c5b1640 1 Processor -- start 2024-09-12T03:20:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.941+0000 7f981c5b1640 1 -- start start 2024-09-12T03:20:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.941+0000 7f981c5b1640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9814101de0 0x7f98140744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.941+0000 7f981c5b1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9814102790 0x7f9814074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.941+0000 7f981c5b1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9814106f10 0x7f9814074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.941+0000 7f981c5b1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9814111fe0 con 0x7f9814106f10 2024-09-12T03:20:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.941+0000 7f981c5b1640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f9814111e60 con 0x7f9814101de0 2024-09-12T03:20:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.941+0000 7f981c5b1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9814112160 con 0x7f9814102790 2024-09-12T03:20:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.941+0000 7f981ab4e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9814106f10 0x7f9814074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:39.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.941+0000 7f981ab4e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9814106f10 0x7f9814074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56432/0 (socket says 172.21.15.6:56432) 2024-09-12T03:20:39.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.941+0000 7f981ab4e640 1 -- 172.21.15.6:0/216760168 learned_addr learned my addr 172.21.15.6:0/216760168 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:39.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.941+0000 7f981a34d640 1 --2- 172.21.15.6:0/216760168 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9814101de0 0x7f98140744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:39.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.942+0000 7f9819b4c640 1 --2- 172.21.15.6:0/216760168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9814102790 0x7f9814074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:39.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.942+0000 7f981ab4e640 1 -- 172.21.15.6:0/216760168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9814102790 msgr2=0x7f9814074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:39.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.942+0000 7f981ab4e640 1 --2- 172.21.15.6:0/216760168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9814102790 0x7f9814074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:39.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.942+0000 7f981ab4e640 1 -- 172.21.15.6:0/216760168 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9814101de0 msgr2=0x7f98140744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:39.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.942+0000 7f981ab4e640 1 --2- 172.21.15.6:0/216760168 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9814101de0 0x7f98140744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:39.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.942+0000 7f981ab4e640 1 -- 172.21.15.6:0/216760168 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98141a6890 con 0x7f9814106f10 2024-09-12T03:20:39.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.942+0000 7f981ab4e640 1 --2- 172.21.15.6:0/216760168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9814106f10 0x7f9814074f70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f980c00b970 tx=0x7f980c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:39.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.942+0000 7f980b7fe640 1 -- 172.21.15.6:0/216760168 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c019070 con 0x7f9814106f10 2024-09-12T03:20:39.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.942+0000 7f980b7fe640 1 -- 172.21.15.6:0/216760168 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f980c00cb40 con 0x7f9814106f10 2024-09-12T03:20:39.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.942+0000 7f981c5b1640 1 -- 172.21.15.6:0/216760168 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98141a6b80 con 0x7f9814106f10 2024-09-12T03:20:39.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.942+0000 7f980b7fe640 1 -- 172.21.15.6:0/216760168 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c007650 con 0x7f9814106f10 2024-09-12T03:20:39.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.942+0000 7f981c5b1640 1 -- 172.21.15.6:0/216760168 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98141a7110 con 0x7f9814106f10 2024-09-12T03:20:39.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.943+0000 7f981c5b1640 1 -- 172.21.15.6:0/216760168 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98141021c0 con 0x7f9814106f10 2024-09-12T03:20:39.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.947+0000 7f980b7fe640 1 -- 172.21.15.6:0/216760168 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f980c01c070 con 0x7f9814106f10 2024-09-12T03:20:39.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.948+0000 7f980b7fe640 1 --2- 172.21.15.6:0/216760168 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f98000a2c90 0x7f98000a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:39.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.948+0000 7f980b7fe640 1 -- 172.21.15.6:0/216760168 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f980c0ceab0 con 0x7f9814106f10 2024-09-12T03:20:39.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.948+0000 7f981a34d640 1 --2- 172.21.15.6:0/216760168 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f98000a2c90 0x7f98000a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:39.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.949+0000 7f981a34d640 1 --2- 172.21.15.6:0/216760168 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f98000a2c90 0x7f98000a5150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9810000c00 tx=0x7f9810005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:39.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:39.951+0000 7f980b7fe640 1 -- 172.21.15.6:0/216760168 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f980c09b9f0 con 0x7f9814106f10 2024-09-12T03:20:40.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:40.163+0000 7f981c5b1640 1 -- 172.21.15.6:0/216760168 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f981402d7c0 con 0x7f9814106f10 2024-09-12T03:20:40.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:40.164+0000 7f980b7fe640 1 -- 172.21.15.6:0/216760168 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11908 (secure 0 0 0) 0x7f980c09b2f0 con 0x7f9814106f10 2024-09-12T03:20:40.164 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:20:40.164 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":24,"btime":"2024-09-12T03:19:21:375379+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:19:20.919190+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24463,24464,24494]},"id":1}]} 2024-09-12T03:20:40.165 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 24 2024-09-12T03:20:40.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:40.166+0000 7f981c5b1640 1 -- 172.21.15.6:0/216760168 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f98000a2c90 msgr2=0x7f98000a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:40.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:40.166+0000 7f981c5b1640 1 --2- 172.21.15.6:0/216760168 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f98000a2c90 0x7f98000a5150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9810000c00 tx=0x7f9810005a20 comp rx=0 tx=0).stop 2024-09-12T03:20:40.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:40.166+0000 7f981c5b1640 1 -- 172.21.15.6:0/216760168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9814106f10 msgr2=0x7f9814074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:40.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:40.166+0000 7f981c5b1640 1 --2- 172.21.15.6:0/216760168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9814106f10 0x7f9814074f70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f980c00b970 tx=0x7f980c00be30 comp rx=0 tx=0).stop 2024-09-12T03:20:40.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:40.166+0000 7f981c5b1640 1 -- 172.21.15.6:0/216760168 shutdown_connections 2024-09-12T03:20:40.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:40.166+0000 7f981c5b1640 1 --2- 172.21.15.6:0/216760168 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f98000a2c90 0x7f98000a5150 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:40.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:40.166+0000 7f981c5b1640 1 --2- 172.21.15.6:0/216760168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9814106f10 0x7f9814074f70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:40.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:40.166+0000 7f981c5b1640 1 --2- 172.21.15.6:0/216760168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9814102790 0x7f9814074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:40.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:40.166+0000 7f981c5b1640 1 --2- 172.21.15.6:0/216760168 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9814101de0 0x7f98140744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:40.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:40.166+0000 7f981c5b1640 1 -- 172.21.15.6:0/216760168 >> 172.21.15.6:0/216760168 conn(0x7f98140fd8c0 msgr2=0x7f98140fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:40.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:40.166+0000 7f981c5b1640 1 -- 172.21.15.6:0/216760168 shutdown_connections 2024-09-12T03:20:40.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:40.166+0000 7f981c5b1640 1 -- 172.21.15.6:0/216760168 wait complete. 2024-09-12T03:20:40.812 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:20:41.131 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:41.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:40 smithi006 ceph-mon[30991]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:20:41.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:40 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/216760168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:41.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:40 smithi118 ceph-mon[35048]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:20:41.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:40 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/216760168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:41.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:40 smithi161 ceph-mon[35485]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:20:41.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:40 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/216760168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:41.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.573+0000 7f28da678640 1 -- 172.21.15.6:0/3261467725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28d4101420 msgr2=0x7f28d410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:41.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.573+0000 7f28da678640 1 --2- 172.21.15.6:0/3261467725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28d4101420 0x7f28d410d950 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009cd0 tx=0x7f28cc02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.573+0000 7f28da678640 1 -- 172.21.15.6:0/3261467725 shutdown_connections 2024-09-12T03:20:41.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.573+0000 7f28da678640 1 --2- 172.21.15.6:0/3261467725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28d4101420 0x7f28d410d950 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.573+0000 7f28da678640 1 --2- 172.21.15.6:0/3261467725 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d4100a80 0x7f28d4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.573+0000 7f28da678640 1 --2- 172.21.15.6:0/3261467725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d4106a80 0x7f28d4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.573+0000 7f28da678640 1 -- 172.21.15.6:0/3261467725 >> 172.21.15.6:0/3261467725 conn(0x7f28d40fc7a0 msgr2=0x7f28d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:41.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.573+0000 7f28da678640 1 -- 172.21.15.6:0/3261467725 shutdown_connections 2024-09-12T03:20:41.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.573+0000 7f28da678640 1 -- 172.21.15.6:0/3261467725 wait complete. 2024-09-12T03:20:41.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.574+0000 7f28da678640 1 Processor -- start 2024-09-12T03:20:41.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.574+0000 7f28da678640 1 -- start start 2024-09-12T03:20:41.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.575+0000 7f28da678640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d4100a80 0x7f28d4199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:41.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.575+0000 7f28da678640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d4101420 0x7f28d4199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:41.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.575+0000 7f28da678640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28d4106a80 0x7f28d41a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:41.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.575+0000 7f28da678640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f28d4110e80 con 0x7f28d4106a80 2024-09-12T03:20:41.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.575+0000 7f28da678640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f28d4110d00 con 0x7f28d4101420 2024-09-12T03:20:41.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.575+0000 7f28da678640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f28d4111000 con 0x7f28d4100a80 2024-09-12T03:20:41.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.575+0000 7f28d8c15640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28d4106a80 0x7f28d41a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:41.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.575+0000 7f28d8c15640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28d4106a80 0x7f28d41a4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56452/0 (socket says 172.21.15.6:56452) 2024-09-12T03:20:41.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.575+0000 7f28d8c15640 1 -- 172.21.15.6:0/2480141194 learned_addr learned my addr 172.21.15.6:0/2480141194 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:41.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.575+0000 7f28d3fff640 1 --2- 172.21.15.6:0/2480141194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d4100a80 0x7f28d4199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:41.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.575+0000 7f28cbfff640 1 --2- 172.21.15.6:0/2480141194 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d4101420 0x7f28d4199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:41.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.575+0000 7f28d8c15640 1 -- 172.21.15.6:0/2480141194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d4100a80 msgr2=0x7f28d4199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:41.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.575+0000 7f28d8c15640 1 --2- 172.21.15.6:0/2480141194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d4100a80 0x7f28d4199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.575+0000 7f28d8c15640 1 -- 172.21.15.6:0/2480141194 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d4101420 msgr2=0x7f28d4199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:41.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.575+0000 7f28d8c15640 1 --2- 172.21.15.6:0/2480141194 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d4101420 0x7f28d4199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.575+0000 7f28d8c15640 1 -- 172.21.15.6:0/2480141194 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d41a47c0 con 0x7f28d4106a80 2024-09-12T03:20:41.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.575+0000 7f28cbfff640 1 --2- 172.21.15.6:0/2480141194 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d4101420 0x7f28d4199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:41.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.575+0000 7f28d8c15640 1 --2- 172.21.15.6:0/2480141194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28d4106a80 0x7f28d41a4280 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f28cc006010 tx=0x7f28cc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:41.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.575+0000 7f28d1ffb640 1 -- 172.21.15.6:0/2480141194 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc004450 con 0x7f28d4106a80 2024-09-12T03:20:41.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.575+0000 7f28d1ffb640 1 -- 172.21.15.6:0/2480141194 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28cc003e10 con 0x7f28d4106a80 2024-09-12T03:20:41.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.576+0000 7f28da678640 1 -- 172.21.15.6:0/2480141194 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d41a4990 con 0x7f28d4106a80 2024-09-12T03:20:41.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.576+0000 7f28d1ffb640 1 -- 172.21.15.6:0/2480141194 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc0466e0 con 0x7f28d4106a80 2024-09-12T03:20:41.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.576+0000 7f28da678640 1 -- 172.21.15.6:0/2480141194 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d41a4e70 con 0x7f28d4106a80 2024-09-12T03:20:41.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.576+0000 7f28da678640 1 -- 172.21.15.6:0/2480141194 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d4100ee0 con 0x7f28d4106a80 2024-09-12T03:20:41.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.578+0000 7f28d1ffb640 1 -- 172.21.15.6:0/2480141194 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f28cc0305a0 con 0x7f28d4106a80 2024-09-12T03:20:41.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.578+0000 7f28d1ffb640 1 --2- 172.21.15.6:0/2480141194 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f28c00a2c90 0x7f28c00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:41.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.579+0000 7f28d3fff640 1 --2- 172.21.15.6:0/2480141194 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f28c00a2c90 0x7f28c00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:41.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.579+0000 7f28d1ffb640 1 -- 172.21.15.6:0/2480141194 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f28cc02e030 con 0x7f28d4106a80 2024-09-12T03:20:41.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.579+0000 7f28d3fff640 1 --2- 172.21.15.6:0/2480141194 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f28c00a2c90 0x7f28c00a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f28c40058c0 tx=0x7f28c400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:41.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.582+0000 7f28d1ffb640 1 -- 172.21.15.6:0/2480141194 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f28cc04b060 con 0x7f28d4106a80 2024-09-12T03:20:41.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.793+0000 7f28da678640 1 -- 172.21.15.6:0/2480141194 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28d41a50d0 con 0x7f28d4106a80 2024-09-12T03:20:41.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.794+0000 7f28d1ffb640 1 -- 172.21.15.6:0/2480141194 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11908 (secure 0 0 0) 0x7f28cc0bda90 con 0x7f28d4106a80 2024-09-12T03:20:41.795 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:20:41.795 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":24,"btime":"2024-09-12T03:19:21:375379+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:19:20.919190+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24463,24464,24494]},"id":1}]} 2024-09-12T03:20:41.795 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 24 2024-09-12T03:20:41.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.796+0000 7f28da678640 1 -- 172.21.15.6:0/2480141194 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f28c00a2c90 msgr2=0x7f28c00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:41.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.796+0000 7f28da678640 1 --2- 172.21.15.6:0/2480141194 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f28c00a2c90 0x7f28c00a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f28c40058c0 tx=0x7f28c400a2d0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.796+0000 7f28da678640 1 -- 172.21.15.6:0/2480141194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28d4106a80 msgr2=0x7f28d41a4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:41.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.796+0000 7f28da678640 1 --2- 172.21.15.6:0/2480141194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28d4106a80 0x7f28d41a4280 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f28cc006010 tx=0x7f28cc003c20 comp rx=0 tx=0).stop 2024-09-12T03:20:41.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.796+0000 7f28da678640 1 -- 172.21.15.6:0/2480141194 shutdown_connections 2024-09-12T03:20:41.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.796+0000 7f28da678640 1 --2- 172.21.15.6:0/2480141194 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f28c00a2c90 0x7f28c00a5150 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.796+0000 7f28da678640 1 --2- 172.21.15.6:0/2480141194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28d4106a80 0x7f28d41a4280 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.796+0000 7f28da678640 1 --2- 172.21.15.6:0/2480141194 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d4101420 0x7f28d4199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.796+0000 7f28da678640 1 --2- 172.21.15.6:0/2480141194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d4100a80 0x7f28d4199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.796+0000 7f28da678640 1 -- 172.21.15.6:0/2480141194 >> 172.21.15.6:0/2480141194 conn(0x7f28d40fc7a0 msgr2=0x7f28d40fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:41.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.796+0000 7f28da678640 1 -- 172.21.15.6:0/2480141194 shutdown_connections 2024-09-12T03:20:41.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:41.796+0000 7f28da678640 1 -- 172.21.15.6:0/2480141194 wait complete. 2024-09-12T03:20:42.094 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:41 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2480141194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:42.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:41 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2480141194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:42.357 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:20:42.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:41 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2480141194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:42.610 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:42.997 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:42 smithi006 ceph-mon[30991]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:43.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.078+0000 7fbf84415640 1 -- 172.21.15.6:0/590933940 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf7c102ef0 msgr2=0x7fbf7c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:43.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.078+0000 7fbf84415640 1 --2- 172.21.15.6:0/590933940 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf7c102ef0 0x7fbf7c10da40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009ca0 tx=0x7fbf7402a950 comp rx=0 tx=0).stop 2024-09-12T03:20:43.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.078+0000 7fbf84415640 1 -- 172.21.15.6:0/590933940 shutdown_connections 2024-09-12T03:20:43.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.078+0000 7fbf84415640 1 --2- 172.21.15.6:0/590933940 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf7c102ef0 0x7fbf7c10da40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:43.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.078+0000 7fbf84415640 1 --2- 172.21.15.6:0/590933940 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf7c102570 0x7fbf7c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:43.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.078+0000 7fbf84415640 1 --2- 172.21.15.6:0/590933940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf7c100e20 0x7fbf7c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:43.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.078+0000 7fbf84415640 1 -- 172.21.15.6:0/590933940 >> 172.21.15.6:0/590933940 conn(0x7fbf7c0fc7d0 msgr2=0x7fbf7c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:43.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.078+0000 7fbf84415640 1 -- 172.21.15.6:0/590933940 shutdown_connections 2024-09-12T03:20:43.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.078+0000 7fbf84415640 1 -- 172.21.15.6:0/590933940 wait complete. 2024-09-12T03:20:43.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.079+0000 7fbf84415640 1 Processor -- start 2024-09-12T03:20:43.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.079+0000 7fbf84415640 1 -- start start 2024-09-12T03:20:43.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.079+0000 7fbf84415640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf7c100e20 0x7fbf7c197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:43.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.079+0000 7fbf84415640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf7c102570 0x7fbf7c197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:43.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.079+0000 7fbf84415640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf7c102ef0 0x7fbf7c19bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:43.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.079+0000 7fbf84415640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7c110fa0 con 0x7fbf7c102ef0 2024-09-12T03:20:43.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.079+0000 7fbf84415640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7c110e20 con 0x7fbf7c102570 2024-09-12T03:20:43.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.079+0000 7fbf84415640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7c111120 con 0x7fbf7c100e20 2024-09-12T03:20:43.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.079+0000 7fbf829b2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf7c102ef0 0x7fbf7c19bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:43.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.079+0000 7fbf829b2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf7c102ef0 0x7fbf7c19bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56472/0 (socket says 172.21.15.6:56472) 2024-09-12T03:20:43.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.079+0000 7fbf829b2640 1 -- 172.21.15.6:0/3379338705 learned_addr learned my addr 172.21.15.6:0/3379338705 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:43.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.080+0000 7fbf819b0640 1 --2- 172.21.15.6:0/3379338705 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf7c102570 0x7fbf7c197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:43.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.080+0000 7fbf821b1640 1 --2- 172.21.15.6:0/3379338705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf7c100e20 0x7fbf7c197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:43.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.080+0000 7fbf829b2640 1 -- 172.21.15.6:0/3379338705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf7c100e20 msgr2=0x7fbf7c197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:43.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.080+0000 7fbf829b2640 1 --2- 172.21.15.6:0/3379338705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf7c100e20 0x7fbf7c197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:43.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.080+0000 7fbf829b2640 1 -- 172.21.15.6:0/3379338705 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf7c102570 msgr2=0x7fbf7c197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:43.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.080+0000 7fbf829b2640 1 --2- 172.21.15.6:0/3379338705 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf7c102570 0x7fbf7c197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:43.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.080+0000 7fbf829b2640 1 -- 172.21.15.6:0/3379338705 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7c19c2e0 con 0x7fbf7c102ef0 2024-09-12T03:20:43.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.080+0000 7fbf829b2640 1 --2- 172.21.15.6:0/3379338705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf7c102ef0 0x7fbf7c19bc00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbf74006010 tx=0x7fbf7402f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:43.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.080+0000 7fbf737fe640 1 -- 172.21.15.6:0/3379338705 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74034040 con 0x7fbf7c102ef0 2024-09-12T03:20:43.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.080+0000 7fbf737fe640 1 -- 172.21.15.6:0/3379338705 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf74003e00 con 0x7fbf7c102ef0 2024-09-12T03:20:43.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.080+0000 7fbf737fe640 1 -- 172.21.15.6:0/3379338705 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7403a5c0 con 0x7fbf7c102ef0 2024-09-12T03:20:43.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.080+0000 7fbf84415640 1 -- 172.21.15.6:0/3379338705 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7c19c570 con 0x7fbf7c102ef0 2024-09-12T03:20:43.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.080+0000 7fbf84415640 1 -- 172.21.15.6:0/3379338705 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf7c069f70 con 0x7fbf7c102ef0 2024-09-12T03:20:43.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.081+0000 7fbf84415640 1 -- 172.21.15.6:0/3379338705 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf7c1021b0 con 0x7fbf7c102ef0 2024-09-12T03:20:43.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.083+0000 7fbf737fe640 1 -- 172.21.15.6:0/3379338705 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fbf74037020 con 0x7fbf7c102ef0 2024-09-12T03:20:43.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.083+0000 7fbf737fe640 1 --2- 172.21.15.6:0/3379338705 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fbf680a2c10 0x7fbf680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:43.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.083+0000 7fbf821b1640 1 --2- 172.21.15.6:0/3379338705 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fbf680a2c10 0x7fbf680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:43.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.083+0000 7fbf737fe640 1 -- 172.21.15.6:0/3379338705 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fbf740f1a60 con 0x7fbf7c102ef0 2024-09-12T03:20:43.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.084+0000 7fbf821b1640 1 --2- 172.21.15.6:0/3379338705 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fbf680a2c10 0x7fbf680a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800abe0 tx=0x7fbf7800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:43.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.087+0000 7fbf737fe640 1 -- 172.21.15.6:0/3379338705 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbf74040070 con 0x7fbf7c102ef0 2024-09-12T03:20:43.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:42 smithi118 ceph-mon[35048]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:43.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.264+0000 7fbf84415640 1 -- 172.21.15.6:0/3379338705 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf7c06a230 con 0x7fbf7c102ef0 2024-09-12T03:20:43.267 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.266+0000 7fbf737fe640 1 -- 172.21.15.6:0/3379338705 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22607 (secure 0 0 0) 0x7fbf740be890 con 0x7fbf7c102ef0 2024-09-12T03:20:43.267 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:20:43.267 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":82,"fsid":"89d99938-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:11:29.530545+0000","modified":"2024-09-12T03:19:20.366710+0000","last_up_change":"2024-09-12T03:18:21.292572+0000","last_in_change":"2024-09-12T03:17:58.013800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:14:37.662010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:06.143398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:08.305279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f55d68b-95a0-4ee2-a46f-bc422b38471e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6803","nonce":726729286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6805","nonce":726729286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6809","nonce":726729286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6807","nonce":726729286}]},"public_addr":"172.21.15.6:6803/726729286","cluster_addr":"172.21.15.6:6805/726729286","heartbeat_back_addr":"172.21.15.6:6809/726729286","heartbeat_front_addr":"172.21.15.6:6807/726729286","state":["exists","up"]},{"osd":1,"uuid":"83afdb01-c731-4de1-96c6-3f798549b1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6801","nonce":3377555415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6803","nonce":3377555415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6807","nonce":3377555415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6805","nonce":3377555415}]},"public_addr":"172.21.15.118:6801/3377555415","cluster_addr":"172.21.15.118:6803/3377555415","heartbeat_back_addr":"172.21.15.118:6807/3377555415","heartbeat_front_addr":"172.21.15.118:6805/3377555415","state":["exists","up"]},{"osd":2,"uuid":"aed718b4-b849-403a-b81c-3a140b5c380c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6801","nonce":392541009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6803","nonce":392541009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6807","nonce":392541009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6805","nonce":392541009}]},"public_addr":"172.21.15.161:6801/392541009","cluster_addr":"172.21.15.161:6803/392541009","heartbeat_back_addr":"172.21.15.161:6807/392541009","heartbeat_front_addr":"172.21.15.161:6805/392541009","state":["exists","up"]},{"osd":3,"uuid":"25ced4c1-bf31-402d-8397-d42fd57f615f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6811","nonce":1892039151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6813","nonce":1892039151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6817","nonce":1892039151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6815","nonce":1892039151}]},"public_addr":"172.21.15.6:6811/1892039151","cluster_addr":"172.21.15.6:6813/1892039151","heartbeat_back_addr":"172.21.15.6:6817/1892039151","heartbeat_front_addr":"172.21.15.6:6815/1892039151","state":["exists","up"]},{"osd":4,"uuid":"1d884604-920b-4c35-b41e-d629ad9f9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6809","nonce":150177024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6811","nonce":150177024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6815","nonce":150177024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6813","nonce":150177024}]},"public_addr":"172.21.15.118:6809/150177024","cluster_addr":"172.21.15.118:6811/150177024","heartbeat_back_addr":"172.21.15.118:6815/150177024","heartbeat_front_addr":"172.21.15.118:6813/150177024","state":["exists","up"]},{"osd":5,"uuid":"1d40c0ef-99f6-426f-90c7-27579edc66af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6809","nonce":3758167854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6811","nonce":3758167854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6815","nonce":3758167854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6813","nonce":3758167854}]},"public_addr":"172.21.15.161:6809/3758167854","cluster_addr":"172.21.15.161:6811/3758167854","heartbeat_back_addr":"172.21.15.161:6815/3758167854","heartbeat_front_addr":"172.21.15.161:6813/3758167854","state":["exists","up"]},{"osd":6,"uuid":"9d9f9510-b107-40c2-9a98-9012715944b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6819","nonce":2129224874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6821","nonce":2129224874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6825","nonce":2129224874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6823","nonce":2129224874}]},"public_addr":"172.21.15.6:6819/2129224874","cluster_addr":"172.21.15.6:6821/2129224874","heartbeat_back_addr":"172.21.15.6:6825/2129224874","heartbeat_front_addr":"172.21.15.6:6823/2129224874","state":["exists","up"]},{"osd":7,"uuid":"84635ae1-3240-4954-af14-76bb3aa162e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6817","nonce":269980353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6819","nonce":269980353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6823","nonce":269980353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6821","nonce":269980353}]},"public_addr":"172.21.15.118:6817/269980353","cluster_addr":"172.21.15.118:6819/269980353","heartbeat_back_addr":"172.21.15.118:6823/269980353","heartbeat_front_addr":"172.21.15.118:6821/269980353","state":["exists","up"]},{"osd":8,"uuid":"6fd58e55-1492-4595-8647-823ea3e3c5d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6817","nonce":1783416134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6819","nonce":1783416134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6823","nonce":1783416134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6821","nonce":1783416134}]},"public_addr":"172.21.15.161:6817/1783416134","cluster_addr":"172.21.15.161:6819/1783416134","heartbeat_back_addr":"172.21.15.161:6823/1783416134","heartbeat_front_addr":"172.21.15.161:6821/1783416134","state":["exists","up"]},{"osd":9,"uuid":"90508a6d-a49c-43c6-a600-5bbccd8702ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6827","nonce":2987332365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6829","nonce":2987332365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6833","nonce":2987332365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6831","nonce":2987332365}]},"public_addr":"172.21.15.6:6827/2987332365","cluster_addr":"172.21.15.6:6829/2987332365","heartbeat_back_addr":"172.21.15.6:6833/2987332365","heartbeat_front_addr":"172.21.15.6:6831/2987332365","state":["exists","up"]},{"osd":10,"uuid":"911e86d1-669a-4935-82c0-7da51d141df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6825","nonce":2167146341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6827","nonce":2167146341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6831","nonce":2167146341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6829","nonce":2167146341}]},"public_addr":"172.21.15.118:6825/2167146341","cluster_addr":"172.21.15.118:6827/2167146341","heartbeat_back_addr":"172.21.15.118:6831/2167146341","heartbeat_front_addr":"172.21.15.118:6829/2167146341","state":["exists","up"]},{"osd":11,"uuid":"5a15c6a7-ff43-473c-bb48-0c129c316df5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6825","nonce":2889795796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6827","nonce":2889795796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6831","nonce":2889795796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6829","nonce":2889795796}]},"public_addr":"172.21.15.161:6825/2889795796","cluster_addr":"172.21.15.161:6827/2889795796","heartbeat_back_addr":"172.21.15.161:6831/2889795796","heartbeat_front_addr":"172.21.15.161:6829/2889795796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:42.912205+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:08.410669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:34.299134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:59.093181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:24.116635+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:49.566433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:15.719320+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:40.541017+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:05.178178+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:31.163877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:55.405110+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:19.417156+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6800/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/2620491916":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/4037656392":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/1377116573":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/3062800125":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:6800/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/4145338968":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/825701128":"2024-09-13T03:11:55.244257+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:20:43.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.268+0000 7fbf84415640 1 -- 172.21.15.6:0/3379338705 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fbf680a2c10 msgr2=0x7fbf680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:43.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.268+0000 7fbf84415640 1 --2- 172.21.15.6:0/3379338705 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fbf680a2c10 0x7fbf680a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800abe0 tx=0x7fbf7800a2d0 comp rx=0 tx=0).stop 2024-09-12T03:20:43.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.268+0000 7fbf84415640 1 -- 172.21.15.6:0/3379338705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf7c102ef0 msgr2=0x7fbf7c19bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:43.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.268+0000 7fbf84415640 1 --2- 172.21.15.6:0/3379338705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf7c102ef0 0x7fbf7c19bc00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbf74006010 tx=0x7fbf7402f6f0 comp rx=0 tx=0).stop 2024-09-12T03:20:43.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.268+0000 7fbf84415640 1 -- 172.21.15.6:0/3379338705 shutdown_connections 2024-09-12T03:20:43.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.268+0000 7fbf84415640 1 --2- 172.21.15.6:0/3379338705 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fbf680a2c10 0x7fbf680a50d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:43.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.268+0000 7fbf84415640 1 --2- 172.21.15.6:0/3379338705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf7c102ef0 0x7fbf7c19bc00 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:43.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.268+0000 7fbf84415640 1 --2- 172.21.15.6:0/3379338705 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf7c102570 0x7fbf7c197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:43.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.268+0000 7fbf84415640 1 --2- 172.21.15.6:0/3379338705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf7c100e20 0x7fbf7c197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:43.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.268+0000 7fbf84415640 1 -- 172.21.15.6:0/3379338705 >> 172.21.15.6:0/3379338705 conn(0x7fbf7c0fc7d0 msgr2=0x7fbf7c106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:43.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.268+0000 7fbf84415640 1 -- 172.21.15.6:0/3379338705 shutdown_connections 2024-09-12T03:20:43.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:43.268+0000 7fbf84415640 1 -- 172.21.15.6:0/3379338705 wait complete. 2024-09-12T03:20:43.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:42 smithi161 ceph-mon[35485]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:43.890 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T03:19:09.851402+0000', 'modified': '2024-09-12T03:19:20.919190+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24494, 'mds_1': 24464, 'mds_2': 24463}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24439': {'gid': 24439, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/955127607', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 955127607}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 955127607}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.118:6835/1389654073', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6834', 'nonce': 1389654073}, {'type': 'v1', 'addr': '172.21.15.118:6835', 'nonce': 1389654073}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6837/283024944', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 283024944}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 283024944}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.161:6839/3645555254', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 3645555254}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 3645555254}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.118:6837/3879191727', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6836', 'nonce': 3879191727}, {'type': 'v1', 'addr': '172.21.15.118:6837', 'nonce': 3879191727}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.118:6839/3035878957', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6838', 'nonce': 3035878957}, {'type': 'v1', 'addr': '172.21.15.118:6839', 'nonce': 3035878957}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24463, 24464, 24494]} 2024-09-12T03:20:43.891 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T03:20:43.891 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T03:20:44.148 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:44.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:43 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3379338705' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:20:44.269 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:43 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3379338705' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:20:44.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:43 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3379338705' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:20:44.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.562+0000 7fee2eb41640 1 -- 172.21.15.6:0/2951075394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee28103780 msgr2=0x7fee28107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:44.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.562+0000 7fee2eb41640 1 --2- 172.21.15.6:0/2951075394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee28103780 0x7fee28107bc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009930 tx=0x7fee1c02a420 comp rx=0 tx=0).stop 2024-09-12T03:20:44.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.562+0000 7fee2eb41640 1 -- 172.21.15.6:0/2951075394 shutdown_connections 2024-09-12T03:20:44.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.562+0000 7fee2eb41640 1 --2- 172.21.15.6:0/2951075394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee28108300 0x7fee2810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.562+0000 7fee2eb41640 1 --2- 172.21.15.6:0/2951075394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee28103780 0x7fee28107bc0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.562+0000 7fee2eb41640 1 --2- 172.21.15.6:0/2951075394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee28102dd0 0x7fee281031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.562+0000 7fee2eb41640 1 -- 172.21.15.6:0/2951075394 >> 172.21.15.6:0/2951075394 conn(0x7fee280fc7d0 msgr2=0x7fee280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:44.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.562+0000 7fee2eb41640 1 -- 172.21.15.6:0/2951075394 shutdown_connections 2024-09-12T03:20:44.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.562+0000 7fee2eb41640 1 -- 172.21.15.6:0/2951075394 wait complete. 2024-09-12T03:20:44.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.562+0000 7fee2eb41640 1 Processor -- start 2024-09-12T03:20:44.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.562+0000 7fee2eb41640 1 -- start start 2024-09-12T03:20:44.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.563+0000 7fee2eb41640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee28102dd0 0x7fee28100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:44.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.563+0000 7fee2eb41640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee28103780 0x7fee28100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:44.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.563+0000 7fee2eb41640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee28108300 0x7fee281014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:44.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.563+0000 7fee2eb41640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fee28112050 con 0x7fee28103780 2024-09-12T03:20:44.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.563+0000 7fee2eb41640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fee28111ed0 con 0x7fee28102dd0 2024-09-12T03:20:44.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.563+0000 7fee2eb41640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fee281121d0 con 0x7fee28108300 2024-09-12T03:20:44.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.563+0000 7fee27fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee28103780 0x7fee28100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:44.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.563+0000 7fee27fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee28103780 0x7fee28100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56478/0 (socket says 172.21.15.6:56478) 2024-09-12T03:20:44.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.563+0000 7fee27fff640 1 -- 172.21.15.6:0/3373964810 learned_addr learned my addr 172.21.15.6:0/3373964810 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:44.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.563+0000 7fee2c8dd640 1 --2- 172.21.15.6:0/3373964810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee28102dd0 0x7fee28100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:44.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.563+0000 7fee2d0de640 1 --2- 172.21.15.6:0/3373964810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee28108300 0x7fee281014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:44.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.563+0000 7fee27fff640 1 -- 172.21.15.6:0/3373964810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee28108300 msgr2=0x7fee281014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:44.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.563+0000 7fee27fff640 1 --2- 172.21.15.6:0/3373964810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee28108300 0x7fee281014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.563+0000 7fee27fff640 1 -- 172.21.15.6:0/3373964810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee28102dd0 msgr2=0x7fee28100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:44.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.563+0000 7fee27fff640 1 --2- 172.21.15.6:0/3373964810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee28102dd0 0x7fee28100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.563+0000 7fee27fff640 1 -- 172.21.15.6:0/3373964810 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee281a6a90 con 0x7fee28103780 2024-09-12T03:20:44.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.563+0000 7fee27fff640 1 --2- 172.21.15.6:0/3373964810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee28103780 0x7fee28100f70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fee1c005940 tx=0x7fee1c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:44.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.564+0000 7fee25ffb640 1 -- 172.21.15.6:0/3373964810 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c03f030 con 0x7fee28103780 2024-09-12T03:20:44.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.564+0000 7fee2eb41640 1 -- 172.21.15.6:0/3373964810 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee281a6d20 con 0x7fee28103780 2024-09-12T03:20:44.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.564+0000 7fee25ffb640 1 -- 172.21.15.6:0/3373964810 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee1c003ce0 con 0x7fee28103780 2024-09-12T03:20:44.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.564+0000 7fee25ffb640 1 -- 172.21.15.6:0/3373964810 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c035070 con 0x7fee28103780 2024-09-12T03:20:44.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.564+0000 7fee2eb41640 1 -- 172.21.15.6:0/3373964810 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee281a71f0 con 0x7fee28103780 2024-09-12T03:20:44.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.566+0000 7fee25ffb640 1 -- 172.21.15.6:0/3373964810 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fee1c042030 con 0x7fee28103780 2024-09-12T03:20:44.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.566+0000 7fee30f1a740 1 Processor -- start 2024-09-12T03:20:44.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.567+0000 7fee30f1a740 1 -- start start 2024-09-12T03:20:44.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.567+0000 7fee25ffb640 1 --2- 172.21.15.6:0/3373964810 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fee100a2c10 0x7fee100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:44.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.567+0000 7fee30f1a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f781d9d0 0x5622f781ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:44.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.567+0000 7fee30f1a740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5622f781e2f0 0x5622f7828670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:44.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.567+0000 7fee30f1a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f7828bb0 0x5622f782afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:44.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.567+0000 7fee30f1a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5622f7784090 con 0x5622f781d9d0 2024-09-12T03:20:44.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.567+0000 7fee30f1a740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5622f772ba70 con 0x5622f781e2f0 2024-09-12T03:20:44.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.567+0000 7fee30f1a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5622f7780050 con 0x5622f7828bb0 2024-09-12T03:20:44.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.567+0000 7fee2c8dd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f781d9d0 0x5622f781ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:44.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.567+0000 7fee2c8dd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f781d9d0 0x5622f781ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56488/0 (socket says 172.21.15.6:56488) 2024-09-12T03:20:44.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.567+0000 7fee2c8dd640 1 -- 172.21.15.6:0/345742836 learned_addr learned my addr 172.21.15.6:0/345742836 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:44.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.567+0000 7fee2d0de640 1 --2- 172.21.15.6:0/3373964810 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fee100a2c10 0x7fee100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:44.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.567+0000 7fee25ffb640 1 -- 172.21.15.6:0/3373964810 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fee1c0ef5d0 con 0x7fee28103780 2024-09-12T03:20:44.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.567+0000 7fee27fff640 1 --2- 172.21.15.6:0/345742836 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5622f781e2f0 0x5622f7828670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:44.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.567+0000 7fee2c8dd640 1 -- 172.21.15.6:0/345742836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f7828bb0 msgr2=0x5622f782afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:44.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.567+0000 7fee2c8dd640 1 --2- 172.21.15.6:0/345742836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f7828bb0 0x5622f782afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.567+0000 7fee2c8dd640 1 -- 172.21.15.6:0/345742836 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5622f781e2f0 msgr2=0x5622f7828670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:44.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.567+0000 7fee2c8dd640 1 --2- 172.21.15.6:0/345742836 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5622f781e2f0 0x5622f7828670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.567+0000 7fee2c8dd640 1 -- 172.21.15.6:0/345742836 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622f7738130 con 0x5622f781d9d0 2024-09-12T03:20:44.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.567+0000 7fee2d0de640 1 --2- 172.21.15.6:0/3373964810 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fee100a2c10 0x7fee100a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fee20005fd0 tx=0x7fee20010740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:44.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.567+0000 7fee2c8dd640 1 --2- 172.21.15.6:0/345742836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f781d9d0 0x5622f781ddb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fee1800bd10 tx=0x7fee180107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e551fb8f3f234c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:44.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.568+0000 7fee0effd640 1 -- 172.21.15.6:0/345742836 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18004030 con 0x5622f781d9d0 2024-09-12T03:20:44.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.568+0000 7fee0effd640 1 -- 172.21.15.6:0/345742836 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee180041d0 con 0x5622f781d9d0 2024-09-12T03:20:44.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.568+0000 7fee0effd640 1 -- 172.21.15.6:0/345742836 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee180117a0 con 0x5622f781d9d0 2024-09-12T03:20:44.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.568+0000 7fee30f1a740 1 -- 172.21.15.6:0/345742836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f781d9d0 msgr2=0x5622f781ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:44.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.568+0000 7fee30f1a740 1 --2- 172.21.15.6:0/345742836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f781d9d0 0x5622f781ddb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fee1800bd10 tx=0x7fee180107a0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.568+0000 7fee30f1a740 1 -- 172.21.15.6:0/345742836 shutdown_connections 2024-09-12T03:20:44.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.568+0000 7fee30f1a740 1 --2- 172.21.15.6:0/345742836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f7828bb0 0x5622f782afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.568+0000 7fee30f1a740 1 --2- 172.21.15.6:0/345742836 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5622f781e2f0 0x5622f7828670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.568+0000 7fee30f1a740 1 --2- 172.21.15.6:0/345742836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f781d9d0 0x5622f781ddb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.568+0000 7fee30f1a740 1 -- 172.21.15.6:0/345742836 >> 172.21.15.6:0/345742836 conn(0x5622f7819f60 msgr2=0x5622f781a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:44.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.568+0000 7fee30f1a740 1 -- 172.21.15.6:0/345742836 shutdown_connections 2024-09-12T03:20:44.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.568+0000 7fee30f1a740 1 -- 172.21.15.6:0/345742836 wait complete. 2024-09-12T03:20:44.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.569+0000 7fee30f1a740 1 -- start start 2024-09-12T03:20:44.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.569+0000 7fee30f1a740 1 Processor -- start 2024-09-12T03:20:44.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.569+0000 7fee30f1a740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5622f781d9d0 0x5622f7828360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:44.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.569+0000 7fee30f1a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f781e2f0 0x5622f78256b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:44.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.569+0000 7fee30f1a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f7828bb0 0x5622f7825bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:44.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.569+0000 7fee30f1a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5622f782b780 con 0x5622f7828bb0 2024-09-12T03:20:44.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.569+0000 7fee30f1a740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5622f773ef20 con 0x5622f781d9d0 2024-09-12T03:20:44.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.569+0000 7fee30f1a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5622f782b900 con 0x5622f781e2f0 2024-09-12T03:20:44.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.569+0000 7fee30f1a740 10 client.0 _ll_register_callbacks cb 0x5622f7708030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:44.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.569+0000 7fee2d0de640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f7828bb0 0x5622f7825bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:44.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.569+0000 7fee2d0de640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f7828bb0 0x5622f7825bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56498/0 (socket says 172.21.15.6:56498) 2024-09-12T03:20:44.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.569+0000 7fee2d0de640 1 -- 172.21.15.6:0/460373087 learned_addr learned my addr 172.21.15.6:0/460373087 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:44.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.570+0000 7fee27fff640 1 --2- 172.21.15.6:0/460373087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f781e2f0 0x5622f78256b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:44.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.570+0000 7fee2c8dd640 1 --2- 172.21.15.6:0/460373087 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5622f781d9d0 0x5622f7828360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:44.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.570+0000 7fee2d0de640 1 -- 172.21.15.6:0/460373087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f781e2f0 msgr2=0x5622f78256b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:44.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.570+0000 7fee2d0de640 1 --2- 172.21.15.6:0/460373087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f781e2f0 0x5622f78256b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.570+0000 7fee2d0de640 1 -- 172.21.15.6:0/460373087 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5622f781d9d0 msgr2=0x5622f7828360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:44.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.570+0000 7fee2d0de640 1 --2- 172.21.15.6:0/460373087 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5622f781d9d0 0x5622f7828360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.570+0000 7fee2d0de640 1 -- 172.21.15.6:0/460373087 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622f74f52b0 con 0x5622f7828bb0 2024-09-12T03:20:44.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.570+0000 7fee2d0de640 1 --2- 172.21.15.6:0/460373087 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f7828bb0 0x5622f7825bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fee20012c30 tx=0x7fee20004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31fac83346f0eff4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:44.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.570+0000 7fee07fff640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:20:44.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.570+0000 7fee07fff640 1 -- 172.21.15.6:0/460373087 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20020650 con 0x5622f7828bb0 2024-09-12T03:20:44.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.570+0000 7fee30f1a740 1 -- 172.21.15.6:0/460373087 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee180041d0 con 0x5622f7828bb0 2024-09-12T03:20:44.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.570+0000 7fee07fff640 1 -- 172.21.15.6:0/460373087 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fee200207f0 con 0x5622f7828bb0 2024-09-12T03:20:44.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.570+0000 7fee07fff640 1 -- 172.21.15.6:0/460373087 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fee200209f0 con 0x5622f7828bb0 2024-09-12T03:20:44.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.570+0000 7fee30f1a740 10 client.15063 fetch_fsmap learned FSMap version 24 2024-09-12T03:20:44.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.570+0000 7fee30f1a740 1 -- 172.21.15.6:0/460373087 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5622f7837070 con 0x5622f7828bb0 2024-09-12T03:20:44.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.570+0000 7fee07fff640 1 -- 172.21.15.6:0/460373087 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee180041d0 con 0x5622f7828bb0 2024-09-12T03:20:44.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.571+0000 7fee07fff640 1 -- 172.21.15.6:0/460373087 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fee2000f040 con 0x5622f7828bb0 2024-09-12T03:20:44.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.571+0000 7fee30f1a740 10 client.15063 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:20:44.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.571+0000 7fee30f1a740 10 client.15063 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:20:44.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.571+0000 7fee30f1a740 20 client.15063 populate_metadata read hostname 'smithi006' 2024-09-12T03:20:44.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.571+0000 7fee30f1a740 1 --2- 172.21.15.6:0/460373087 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5622f78396f0 0x5622f783ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:44.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.571+0000 7fee30f1a740 4 client.15063 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:44.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.571+0000 7fee30f1a740 1 -- 172.21.15.6:0/460373087 --> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622f7791600 con 0x5622f78396f0 2024-09-12T03:20:44.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.571+0000 7fee2c8dd640 1 --2- 172.21.15.6:0/460373087 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5622f78396f0 0x5622f783ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:44.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.571+0000 7fee2c8dd640 1 --2- 172.21.15.6:0/460373087 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5622f78396f0 0x5622f783ab60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d816ab7b1050f0b2 server_cookie=1d5296efbd1cd11b in_seq=0 out_seq=0 2024-09-12T03:20:44.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.571+0000 7fee07fff640 10 client.15063 ms_handle_connect on v2:172.21.15.161:6838/3645555254 2024-09-12T03:20:44.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.572+0000 7fee07fff640 1 -- 172.21.15.6:0/460373087 <== mds.2 v2:172.21.15.161:6838/3645555254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5622f7791600 con 0x5622f78396f0 2024-09-12T03:20:44.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.572+0000 7fee07fff640 10 client.15063 handle_command_reply: tid=0 2024-09-12T03:20:44.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.572+0000 7fee30f1a740 1 client.15063 shutdown 2024-09-12T03:20:44.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.572+0000 7fee30f1a740 1 -- 172.21.15.6:0/460373087 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f7828bb0 msgr2=0x5622f7825bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:44.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.572+0000 7fee30f1a740 1 --2- 172.21.15.6:0/460373087 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f7828bb0 0x5622f7825bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fee20012c30 tx=0x7fee20004ab0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.572+0000 7fee30f1a740 1 -- 172.21.15.6:0/460373087 shutdown_connections 2024-09-12T03:20:44.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.572+0000 7fee30f1a740 1 --2- 172.21.15.6:0/460373087 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5622f78396f0 0x5622f783ab60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.572+0000 7fee30f1a740 1 --2- 172.21.15.6:0/460373087 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f7828bb0 0x5622f7825bf0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.572+0000 7fee30f1a740 1 --2- 172.21.15.6:0/460373087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f781e2f0 0x5622f78256b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.572+0000 7fee30f1a740 1 --2- 172.21.15.6:0/460373087 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5622f781d9d0 0x5622f7828360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.572+0000 7fee07fff640 0 client.15063 ms_handle_reset on v2:172.21.15.161:6838/3645555254 2024-09-12T03:20:44.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.572+0000 7fee30f1a740 1 -- 172.21.15.6:0/460373087 >> 172.21.15.6:0/460373087 conn(0x5622f7819f60 msgr2=0x5622f781fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:44.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.573+0000 7fee30f1a740 1 -- 172.21.15.6:0/460373087 shutdown_connections 2024-09-12T03:20:44.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.573+0000 7fee30f1a740 1 -- 172.21.15.6:0/460373087 wait complete. 2024-09-12T03:20:44.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.573+0000 7fee30f1a740 20 client.15063 trim_cache size 0 max 16384 2024-09-12T03:20:44.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.591+0000 7fee30f1a740 1 Processor -- start 2024-09-12T03:20:44.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.591+0000 7fee30f1a740 1 -- start start 2024-09-12T03:20:44.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.591+0000 7fee30f1a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f781d9d0 0x5622f783a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:44.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.591+0000 7fee30f1a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f781e2f0 0x5622f7824bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:44.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.591+0000 7fee30f1a740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5622f7828bb0 0x5622f7823f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:44.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.591+0000 7fee30f1a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5622f7838ea0 con 0x5622f781d9d0 2024-09-12T03:20:44.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.591+0000 7fee30f1a740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5622f7839570 con 0x5622f7828bb0 2024-09-12T03:20:44.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.591+0000 7fee30f1a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5622f7839c40 con 0x5622f781e2f0 2024-09-12T03:20:44.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.591+0000 7fee2c8dd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f781d9d0 0x5622f783a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:44.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.591+0000 7fee2c8dd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f781d9d0 0x5622f783a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56508/0 (socket says 172.21.15.6:56508) 2024-09-12T03:20:44.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.591+0000 7fee2c8dd640 1 -- 172.21.15.6:0/2049838546 learned_addr learned my addr 172.21.15.6:0/2049838546 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:44.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.591+0000 7fee2c8dd640 1 -- 172.21.15.6:0/2049838546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f781e2f0 msgr2=0x5622f7824bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:44.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.591+0000 7fee2c8dd640 1 --2- 172.21.15.6:0/2049838546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f781e2f0 0x5622f7824bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.591+0000 7fee2c8dd640 1 -- 172.21.15.6:0/2049838546 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5622f7828bb0 msgr2=0x5622f7823f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:44.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.591+0000 7fee2c8dd640 1 --2- 172.21.15.6:0/2049838546 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5622f7828bb0 0x5622f7823f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.591+0000 7fee2c8dd640 1 -- 172.21.15.6:0/2049838546 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622f7819d90 con 0x5622f781d9d0 2024-09-12T03:20:44.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.591+0000 7fee2c8dd640 1 --2- 172.21.15.6:0/2049838546 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f781d9d0 0x5622f783a490 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fee1800bc30 tx=0x7fee18003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb78da6ee8c1a7e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:44.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.592+0000 7fee0e7fc640 1 -- 172.21.15.6:0/2049838546 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18005240 con 0x5622f781d9d0 2024-09-12T03:20:44.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.592+0000 7fee0e7fc640 1 -- 172.21.15.6:0/2049838546 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee180035f0 con 0x5622f781d9d0 2024-09-12T03:20:44.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.592+0000 7fee0e7fc640 1 -- 172.21.15.6:0/2049838546 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18011710 con 0x5622f781d9d0 2024-09-12T03:20:44.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.592+0000 7fee30f1a740 1 -- 172.21.15.6:0/2049838546 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f781d9d0 msgr2=0x5622f783a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:44.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.592+0000 7fee30f1a740 1 --2- 172.21.15.6:0/2049838546 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f781d9d0 0x5622f783a490 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fee1800bc30 tx=0x7fee18003e80 comp rx=0 tx=0).stop 2024-09-12T03:20:44.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.592+0000 7fee30f1a740 1 -- 172.21.15.6:0/2049838546 shutdown_connections 2024-09-12T03:20:44.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.592+0000 7fee30f1a740 1 --2- 172.21.15.6:0/2049838546 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5622f7828bb0 0x5622f7823f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.592+0000 7fee30f1a740 1 --2- 172.21.15.6:0/2049838546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f781e2f0 0x5622f7824bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.592+0000 7fee30f1a740 1 --2- 172.21.15.6:0/2049838546 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f781d9d0 0x5622f783a490 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.593+0000 7fee30f1a740 1 -- 172.21.15.6:0/2049838546 >> 172.21.15.6:0/2049838546 conn(0x5622f7819f60 msgr2=0x5622f7825670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:44.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.593+0000 7fee30f1a740 1 -- 172.21.15.6:0/2049838546 shutdown_connections 2024-09-12T03:20:44.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.593+0000 7fee30f1a740 1 -- 172.21.15.6:0/2049838546 wait complete. 2024-09-12T03:20:44.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.593+0000 7fee30f1a740 1 -- start start 2024-09-12T03:20:44.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.593+0000 7fee30f1a740 1 Processor -- start 2024-09-12T03:20:44.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.594+0000 7fee30f1a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f781d9d0 0x5622f783e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:44.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.594+0000 7fee30f1a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f781e2f0 0x5622f783e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:44.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.594+0000 7fee30f1a740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5622f7828bb0 0x5622f789be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:44.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.594+0000 7fee30f1a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5622f783a9e0 con 0x5622f781e2f0 2024-09-12T03:20:44.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.594+0000 7fee30f1a740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5622f783a310 con 0x5622f7828bb0 2024-09-12T03:20:44.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.594+0000 7fee30f1a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5622f783b0b0 con 0x5622f781d9d0 2024-09-12T03:20:44.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.594+0000 7fee30f1a740 10 client.0 _ll_register_callbacks cb 0x7fee20009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:44.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.594+0000 7fee27fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f781e2f0 0x5622f783e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:44.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.594+0000 7fee2c8dd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f781d9d0 0x5622f783e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:44.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.594+0000 7fee2c8dd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f781d9d0 0x5622f783e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:42452/0 (socket says 172.21.15.6:42452) 2024-09-12T03:20:44.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.594+0000 7fee27fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f781e2f0 0x5622f783e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56524/0 (socket says 172.21.15.6:56524) 2024-09-12T03:20:44.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.594+0000 7fee2c8dd640 1 -- 172.21.15.6:0/2073879480 learned_addr learned my addr 172.21.15.6:0/2073879480 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:44.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.594+0000 7fee2d0de640 1 --2- 172.21.15.6:0/2073879480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5622f7828bb0 0x5622f789be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:44.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.594+0000 7fee2c8dd640 1 -- 172.21.15.6:0/2073879480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5622f7828bb0 msgr2=0x5622f789be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:44.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.594+0000 7fee2c8dd640 1 --2- 172.21.15.6:0/2073879480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5622f7828bb0 0x5622f789be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.594+0000 7fee2c8dd640 1 -- 172.21.15.6:0/2073879480 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f781e2f0 msgr2=0x5622f783e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:44.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.594+0000 7fee2c8dd640 1 --2- 172.21.15.6:0/2073879480 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f781e2f0 0x5622f783e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.594+0000 7fee2c8dd640 1 -- 172.21.15.6:0/2073879480 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622f789c4d0 con 0x5622f781d9d0 2024-09-12T03:20:44.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.594+0000 7fee27fff640 1 --2- 172.21.15.6:0/2073879480 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f781e2f0 0x5622f783e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:44.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.594+0000 7fee2c8dd640 1 --2- 172.21.15.6:0/2073879480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f781d9d0 0x5622f783e470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fee18003820 tx=0x7fee18003850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3262b0bb1124a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:44.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.594+0000 7fee07fff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:20:44.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.595+0000 7fee07fff640 1 -- 172.21.15.6:0/2073879480 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1803d060 con 0x5622f781d9d0 2024-09-12T03:20:44.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.595+0000 7fee07fff640 1 -- 172.21.15.6:0/2073879480 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fee18003c80 con 0x5622f781d9d0 2024-09-12T03:20:44.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.595+0000 7fee30f1a740 1 -- 172.21.15.6:0/2073879480 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee180035f0 con 0x5622f781d9d0 2024-09-12T03:20:44.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.595+0000 7fee07fff640 1 -- 172.21.15.6:0/2073879480 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fee18005130 con 0x5622f781d9d0 2024-09-12T03:20:44.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.595+0000 7fee30f1a740 10 client.24745 fetch_fsmap learned FSMap version 24 2024-09-12T03:20:44.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.595+0000 7fee30f1a740 1 -- 172.21.15.6:0/2073879480 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5622f78a3860 con 0x5622f781d9d0 2024-09-12T03:20:44.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.595+0000 7fee07fff640 1 -- 172.21.15.6:0/2073879480 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee180035f0 con 0x5622f781d9d0 2024-09-12T03:20:44.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.595+0000 7fee07fff640 1 -- 172.21.15.6:0/2073879480 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fee18011cd0 con 0x5622f781d9d0 2024-09-12T03:20:44.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.595+0000 7fee30f1a740 10 client.24745 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:20:44.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.595+0000 7fee30f1a740 10 client.24745 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:20:44.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.595+0000 7fee30f1a740 20 client.24745 populate_metadata read hostname 'smithi006' 2024-09-12T03:20:44.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.595+0000 7fee30f1a740 1 --2- 172.21.15.6:0/2073879480 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5622f78a7ce0 0x5622f78a69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:44.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.596+0000 7fee30f1a740 4 client.24745 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:20:44.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.596+0000 7fee30f1a740 1 -- 172.21.15.6:0/2073879480 --> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] -- command(tid 0: {"prefix": "status"}) -- 0x5622f778bb10 con 0x5622f78a7ce0 2024-09-12T03:20:44.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.596+0000 7fee27fff640 1 --2- 172.21.15.6:0/2073879480 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5622f78a7ce0 0x5622f78a69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:44.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.596+0000 7fee27fff640 1 --2- 172.21.15.6:0/2073879480 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5622f78a7ce0 0x5622f78a69f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e64d23a40b9c9c6 server_cookie=47f0cbd503eabe5 in_seq=0 out_seq=0 2024-09-12T03:20:44.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.596+0000 7fee07fff640 10 client.24745 ms_handle_connect on v2:172.21.15.161:6838/3645555254 2024-09-12T03:20:44.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.596+0000 7fee07fff640 1 -- 172.21.15.6:0/2073879480 <== mds.2 v2:172.21.15.161:6838/3645555254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5622f778bb10 con 0x5622f78a7ce0 2024-09-12T03:20:44.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.596+0000 7fee07fff640 10 client.24745 handle_command_reply: tid=0 2024-09-12T03:20:44.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.596+0000 7fee30f1a740 1 client.24745 shutdown 2024-09-12T03:20:44.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.596+0000 7fee30f1a740 1 -- 172.21.15.6:0/2073879480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f781d9d0 msgr2=0x5622f783e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:44.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.596+0000 7fee30f1a740 1 --2- 172.21.15.6:0/2073879480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f781d9d0 0x5622f783e470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fee18003820 tx=0x7fee18003850 comp rx=0 tx=0).stop 2024-09-12T03:20:44.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.597+0000 7fee30f1a740 1 -- 172.21.15.6:0/2073879480 shutdown_connections 2024-09-12T03:20:44.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.597+0000 7fee30f1a740 1 --2- 172.21.15.6:0/2073879480 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5622f78a7ce0 0x5622f78a69f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.597+0000 7fee30f1a740 1 --2- 172.21.15.6:0/2073879480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5622f7828bb0 0x5622f789be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.597+0000 7fee30f1a740 1 --2- 172.21.15.6:0/2073879480 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622f781e2f0 0x5622f783e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.597+0000 7fee07fff640 0 client.24745 ms_handle_reset on v2:172.21.15.161:6838/3645555254 2024-09-12T03:20:44.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.597+0000 7fee30f1a740 1 --2- 172.21.15.6:0/2073879480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f781d9d0 0x5622f783e470 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.597+0000 7fee30f1a740 1 -- 172.21.15.6:0/2073879480 >> 172.21.15.6:0/2073879480 conn(0x5622f7819f60 msgr2=0x5622f7823b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:44.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.597+0000 7fee30f1a740 1 -- 172.21.15.6:0/2073879480 shutdown_connections 2024-09-12T03:20:44.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.597+0000 7fee30f1a740 1 -- 172.21.15.6:0/2073879480 wait complete. 2024-09-12T03:20:44.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.597+0000 7fee30f1a740 20 client.24745 trim_cache size 0 max 16384 2024-09-12T03:20:44.625 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:20:44.625 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "89d99938-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:20:44.625 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 2, 2024-09-12T03:20:44.626 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24463, 2024-09-12T03:20:44.626 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-09-12T03:20:44.626 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-09-12T03:20:44.626 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-09-12T03:20:44.626 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 86.235703462000004, 2024-09-12T03:20:44.626 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 24, 2024-09-12T03:20:44.626 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 82, 2024-09-12T03:20:44.626 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T03:20:44.626 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 111.862502734 2024-09-12T03:20:44.626 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:20:44.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.598+0000 7fee2eb41640 1 -- 172.21.15.6:0/3373964810 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fee100a2c10 msgr2=0x7fee100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:44.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.598+0000 7fee2eb41640 1 --2- 172.21.15.6:0/3373964810 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fee100a2c10 0x7fee100a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fee20005fd0 tx=0x7fee20010740 comp rx=0 tx=0).stop 2024-09-12T03:20:44.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.598+0000 7fee2eb41640 1 -- 172.21.15.6:0/3373964810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee28103780 msgr2=0x7fee28100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:44.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.598+0000 7fee2eb41640 1 --2- 172.21.15.6:0/3373964810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee28103780 0x7fee28100f70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fee1c005940 tx=0x7fee1c0042f0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.598+0000 7fee2eb41640 1 -- 172.21.15.6:0/3373964810 shutdown_connections 2024-09-12T03:20:44.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.598+0000 7fee2eb41640 1 --2- 172.21.15.6:0/3373964810 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fee100a2c10 0x7fee100a50d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.598+0000 7fee2eb41640 1 --2- 172.21.15.6:0/3373964810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee28108300 0x7fee281014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.598+0000 7fee2eb41640 1 --2- 172.21.15.6:0/3373964810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee28103780 0x7fee28100f70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.598+0000 7fee2eb41640 1 --2- 172.21.15.6:0/3373964810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee28102dd0 0x7fee28100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:44.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.598+0000 7fee2eb41640 1 -- 172.21.15.6:0/3373964810 >> 172.21.15.6:0/3373964810 conn(0x7fee280fc7d0 msgr2=0x7fee28105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:44.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.598+0000 7fee2eb41640 1 -- 172.21.15.6:0/3373964810 shutdown_connections 2024-09-12T03:20:44.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:44.598+0000 7fee2eb41640 1 -- 172.21.15.6:0/3373964810 wait complete. 2024-09-12T03:20:45.175 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:44 smithi006 ceph-mon[30991]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:20:45.204 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-12T03:20:45.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:44 smithi118 ceph-mon[35048]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:20:45.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:44 smithi161 ceph-mon[35485]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:20:45.464 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:45.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.962+0000 7fb65cc3e640 1 -- 172.21.15.6:0/2023410290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6581082a0 msgr2=0x7fb65810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:45.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.962+0000 7fb65cc3e640 1 --2- 172.21.15.6:0/2023410290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6581082a0 0x7fb65810ec40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb650009cd0 tx=0x7fb65002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.962+0000 7fb65cc3e640 1 -- 172.21.15.6:0/2023410290 shutdown_connections 2024-09-12T03:20:45.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.962+0000 7fb65cc3e640 1 --2- 172.21.15.6:0/2023410290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6581082a0 0x7fb65810ec40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.962+0000 7fb65cc3e640 1 --2- 172.21.15.6:0/2023410290 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb6581036e0 0x7fb658107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.962+0000 7fb65cc3e640 1 --2- 172.21.15.6:0/2023410290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb658102d30 0x7fb658103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.962+0000 7fb65cc3e640 1 -- 172.21.15.6:0/2023410290 >> 172.21.15.6:0/2023410290 conn(0x7fb6580fc7d0 msgr2=0x7fb6580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:45.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.962+0000 7fb65cc3e640 1 -- 172.21.15.6:0/2023410290 shutdown_connections 2024-09-12T03:20:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.962+0000 7fb65cc3e640 1 -- 172.21.15.6:0/2023410290 wait complete. 2024-09-12T03:20:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.963+0000 7fb65cc3e640 1 Processor -- start 2024-09-12T03:20:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.963+0000 7fb65cc3e640 1 -- start start 2024-09-12T03:20:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.963+0000 7fb65cc3e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb658102d30 0x7fb658198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.963+0000 7fb65cc3e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6581036e0 0x7fb658198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.963+0000 7fb65cc3e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb6581082a0 0x7fb65819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.963+0000 7fb65cc3e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581120f0 con 0x7fb6581036e0 2024-09-12T03:20:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.963+0000 7fb65cc3e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fb658111f70 con 0x7fb6581082a0 2024-09-12T03:20:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.963+0000 7fb65cc3e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb658112270 con 0x7fb658102d30 2024-09-12T03:20:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.963+0000 7fb64ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6581036e0 0x7fb658198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.963+0000 7fb64ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6581036e0 0x7fb658198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56544/0 (socket says 172.21.15.6:56544) 2024-09-12T03:20:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.963+0000 7fb64ffff640 1 -- 172.21.15.6:0/1282015476 learned_addr learned my addr 172.21.15.6:0/1282015476 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.964+0000 7fb65659c640 1 --2- 172.21.15.6:0/1282015476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb658102d30 0x7fb658198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.964+0000 7fb656d9d640 1 --2- 172.21.15.6:0/1282015476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb6581082a0 0x7fb65819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.964+0000 7fb64ffff640 1 -- 172.21.15.6:0/1282015476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb658102d30 msgr2=0x7fb658198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.964+0000 7fb64ffff640 1 --2- 172.21.15.6:0/1282015476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb658102d30 0x7fb658198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.964+0000 7fb64ffff640 1 -- 172.21.15.6:0/1282015476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb6581082a0 msgr2=0x7fb65819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.964+0000 7fb64ffff640 1 --2- 172.21.15.6:0/1282015476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb6581082a0 0x7fb65819ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.964+0000 7fb64ffff640 1 -- 172.21.15.6:0/1282015476 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65819d540 con 0x7fb6581036e0 2024-09-12T03:20:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.964+0000 7fb64ffff640 1 --2- 172.21.15.6:0/1282015476 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6581036e0 0x7fb658198ad0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb64000d740 tx=0x7fb64000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.964+0000 7fb64f7fe640 1 -- 172.21.15.6:0/1282015476 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640017070 con 0x7fb6581036e0 2024-09-12T03:20:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.964+0000 7fb64f7fe640 1 -- 172.21.15.6:0/1282015476 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb64000de50 con 0x7fb6581036e0 2024-09-12T03:20:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.964+0000 7fb65cc3e640 1 -- 172.21.15.6:0/1282015476 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb65819d830 con 0x7fb6581036e0 2024-09-12T03:20:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.964+0000 7fb64f7fe640 1 -- 172.21.15.6:0/1282015476 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64000f6b0 con 0x7fb6581036e0 2024-09-12T03:20:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.964+0000 7fb65cc3e640 1 -- 172.21.15.6:0/1282015476 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6581a5110 con 0x7fb6581036e0 2024-09-12T03:20:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.966+0000 7fb64f7fe640 1 -- 172.21.15.6:0/1282015476 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb64001a070 con 0x7fb6581036e0 2024-09-12T03:20:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.967+0000 7fb65f017740 1 Processor -- start 2024-09-12T03:20:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.967+0000 7fb65f017740 1 -- start start 2024-09-12T03:20:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.967+0000 7fb65f017740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e103099d0 0x559e10309db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.967+0000 7fb65f017740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e1030a2f0 0x559e10314670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.967+0000 7fb65f017740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e10314bb0 0x559e10316fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.967+0000 7fb65f017740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x559e10270090 con 0x559e10314bb0 2024-09-12T03:20:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.967+0000 7fb65f017740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x559e10217a70 con 0x559e1030a2f0 2024-09-12T03:20:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.967+0000 7fb65f017740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559e1026c050 con 0x559e103099d0 2024-09-12T03:20:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.967+0000 7fb64f7fe640 1 --2- 172.21.15.6:0/1282015476 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb6440a2c90 0x7fb6440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.967+0000 7fb64ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e10314bb0 0x559e10316fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.967+0000 7fb64ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e10314bb0 0x559e10316fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56556/0 (socket says 172.21.15.6:56556) 2024-09-12T03:20:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.967+0000 7fb64ffff640 1 -- 172.21.15.6:0/1116602273 learned_addr learned my addr 172.21.15.6:0/1116602273 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:45.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.967+0000 7fb65659c640 1 --2- 172.21.15.6:0/1116602273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e1030a2f0 0x559e10314670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:45.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.967+0000 7fb656d9d640 1 --2- 172.21.15.6:0/1116602273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e103099d0 0x559e10309db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:45.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.967+0000 7fb656d9d640 1 --2- 172.21.15.6:0/1282015476 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb6440a2c90 0x7fb6440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:45.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.968+0000 7fb64f7fe640 1 -- 172.21.15.6:0/1282015476 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fb6400cea30 con 0x7fb6581036e0 2024-09-12T03:20:45.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.968+0000 7fb64ffff640 1 -- 172.21.15.6:0/1116602273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e103099d0 msgr2=0x559e10309db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:45.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.968+0000 7fb64ffff640 1 --2- 172.21.15.6:0/1116602273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e103099d0 0x559e10309db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.968+0000 7fb64ffff640 1 -- 172.21.15.6:0/1116602273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e1030a2f0 msgr2=0x559e10314670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:45.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.968+0000 7fb64ffff640 1 --2- 172.21.15.6:0/1116602273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e1030a2f0 0x559e10314670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.968+0000 7fb64ffff640 1 -- 172.21.15.6:0/1116602273 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e10224130 con 0x559e10314bb0 2024-09-12T03:20:45.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.968+0000 7fb65659c640 1 --2- 172.21.15.6:0/1116602273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e1030a2f0 0x559e10314670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:45.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.968+0000 7fb656d9d640 1 --2- 172.21.15.6:0/1282015476 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb6440a2c90 0x7fb6440a5150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb650005e30 tx=0x7fb650009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:45.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.968+0000 7fb64ffff640 1 --2- 172.21.15.6:0/1116602273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e10314bb0 0x559e10316fa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb6400a38a0 tx=0x7fb640010d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890072ecd4f295ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:45.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.968+0000 7fb64cff9640 1 -- 172.21.15.6:0/1116602273 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640021070 con 0x559e10314bb0 2024-09-12T03:20:45.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.968+0000 7fb64cff9640 1 -- 172.21.15.6:0/1116602273 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb64009b210 con 0x559e10314bb0 2024-09-12T03:20:45.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.968+0000 7fb64cff9640 1 -- 172.21.15.6:0/1116602273 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6400bb260 con 0x559e10314bb0 2024-09-12T03:20:45.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.969+0000 7fb65f017740 1 -- 172.21.15.6:0/1116602273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e10314bb0 msgr2=0x559e10316fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:45.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.969+0000 7fb65f017740 1 --2- 172.21.15.6:0/1116602273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e10314bb0 0x559e10316fa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb6400a38a0 tx=0x7fb640010d40 comp rx=0 tx=0).stop 2024-09-12T03:20:45.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.969+0000 7fb65f017740 1 -- 172.21.15.6:0/1116602273 shutdown_connections 2024-09-12T03:20:45.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.969+0000 7fb65f017740 1 --2- 172.21.15.6:0/1116602273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e10314bb0 0x559e10316fa0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.969+0000 7fb65f017740 1 --2- 172.21.15.6:0/1116602273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e1030a2f0 0x559e10314670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.969+0000 7fb65f017740 1 --2- 172.21.15.6:0/1116602273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e103099d0 0x559e10309db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.969+0000 7fb65f017740 1 -- 172.21.15.6:0/1116602273 >> 172.21.15.6:0/1116602273 conn(0x559e10305f60 msgr2=0x559e10306340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:45.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.969+0000 7fb65f017740 1 -- 172.21.15.6:0/1116602273 shutdown_connections 2024-09-12T03:20:45.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.969+0000 7fb65f017740 1 -- 172.21.15.6:0/1116602273 wait complete. 2024-09-12T03:20:45.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.969+0000 7fb65f017740 1 -- start start 2024-09-12T03:20:45.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb65f017740 1 Processor -- start 2024-09-12T03:20:45.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb65f017740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e103099d0 0x559e103144d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:45.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb65f017740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e1030a2f0 0x559e10311820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:45.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb65f017740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e10314bb0 0x559e10311d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:45.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb65f017740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x559e10317780 con 0x559e103099d0 2024-09-12T03:20:45.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb65f017740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x559e1022af20 con 0x559e1030a2f0 2024-09-12T03:20:45.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb65f017740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559e10317900 con 0x559e10314bb0 2024-09-12T03:20:45.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb65f017740 10 client.0 _ll_register_callbacks cb 0x559e101f4030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:45.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb65659c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e103099d0 0x559e103144d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:45.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb65659c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e103099d0 0x559e103144d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56570/0 (socket says 172.21.15.6:56570) 2024-09-12T03:20:45.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb65659c640 1 -- 172.21.15.6:0/973719667 learned_addr learned my addr 172.21.15.6:0/973719667 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:45.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb64ffff640 1 --2- 172.21.15.6:0/973719667 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e1030a2f0 0x559e10311820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb656d9d640 1 --2- 172.21.15.6:0/973719667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e10314bb0 0x559e10311d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb65659c640 1 -- 172.21.15.6:0/973719667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e10314bb0 msgr2=0x559e10311d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb65659c640 1 --2- 172.21.15.6:0/973719667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e10314bb0 0x559e10311d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb65659c640 1 -- 172.21.15.6:0/973719667 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e1030a2f0 msgr2=0x559e10311820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb65659c640 1 --2- 172.21.15.6:0/973719667 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e1030a2f0 0x559e10311820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb65659c640 1 -- 172.21.15.6:0/973719667 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e0ffe12b0 con 0x559e103099d0 2024-09-12T03:20:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb64ffff640 1 --2- 172.21.15.6:0/973719667 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e1030a2f0 0x559e10311820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb656d9d640 1 --2- 172.21.15.6:0/973719667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e10314bb0 0x559e10311d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb65659c640 1 --2- 172.21.15.6:0/973719667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e103099d0 0x559e103144d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb64800ba10 tx=0x7fb64800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6645f9addeffa5cd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.970+0000 7fb63dffb640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:20:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.971+0000 7fb63dffb640 1 -- 172.21.15.6:0/973719667 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648003e80 con 0x559e103099d0 2024-09-12T03:20:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.971+0000 7fb63dffb640 1 -- 172.21.15.6:0/973719667 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fb648011440 con 0x559e103099d0 2024-09-12T03:20:45.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.971+0000 7fb65f017740 1 -- 172.21.15.6:0/973719667 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb64009b210 con 0x559e103099d0 2024-09-12T03:20:45.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.971+0000 7fb63dffb640 1 -- 172.21.15.6:0/973719667 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb648011640 con 0x559e103099d0 2024-09-12T03:20:45.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.971+0000 7fb65f017740 10 client.15081 fetch_fsmap learned FSMap version 24 2024-09-12T03:20:45.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.971+0000 7fb65f017740 1 -- 172.21.15.6:0/973719667 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559e103231b0 con 0x559e103099d0 2024-09-12T03:20:45.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.971+0000 7fb63dffb640 1 -- 172.21.15.6:0/973719667 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb64009b210 con 0x559e103099d0 2024-09-12T03:20:45.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.971+0000 7fb63dffb640 1 -- 172.21.15.6:0/973719667 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb648026ab0 con 0x559e103099d0 2024-09-12T03:20:45.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.971+0000 7fb65f017740 10 client.15081 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:20:45.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.971+0000 7fb65f017740 10 client.15081 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T03:20:45.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.971+0000 7fb65f017740 20 client.15081 populate_metadata read hostname 'smithi006' 2024-09-12T03:20:45.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.971+0000 7fb65f017740 1 --2- 172.21.15.6:0/973719667 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x559e10325840 0x559e10326cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:45.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.971+0000 7fb65f017740 4 client.15081 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:45.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.971+0000 7fb65f017740 1 -- 172.21.15.6:0/973719667 --> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e1027d600 con 0x559e10325840 2024-09-12T03:20:45.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.972+0000 7fb64ffff640 1 --2- 172.21.15.6:0/973719667 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x559e10325840 0x559e10326cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:45.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.972+0000 7fb64ffff640 1 --2- 172.21.15.6:0/973719667 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x559e10325840 0x559e10326cb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad516180be6daf9d server_cookie=cf00652609e313ce in_seq=0 out_seq=0 2024-09-12T03:20:45.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.972+0000 7fb63dffb640 10 client.15081 ms_handle_connect on v2:172.21.15.118:6836/3879191727 2024-09-12T03:20:45.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.973+0000 7fb63dffb640 1 -- 172.21.15.6:0/973719667 <== mds.1 v2:172.21.15.118:6836/3879191727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e1027d600 con 0x559e10325840 2024-09-12T03:20:45.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.973+0000 7fb63dffb640 10 client.15081 handle_command_reply: tid=0 2024-09-12T03:20:45.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.973+0000 7fb65f017740 1 client.15081 shutdown 2024-09-12T03:20:45.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.973+0000 7fb65f017740 1 -- 172.21.15.6:0/973719667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e103099d0 msgr2=0x559e103144d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:45.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.973+0000 7fb65f017740 1 --2- 172.21.15.6:0/973719667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e103099d0 0x559e103144d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb64800ba10 tx=0x7fb64800bed0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.973+0000 7fb65f017740 1 -- 172.21.15.6:0/973719667 shutdown_connections 2024-09-12T03:20:45.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.973+0000 7fb65f017740 1 --2- 172.21.15.6:0/973719667 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x559e10325840 0x559e10326cb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.973+0000 7fb63dffb640 0 client.15081 ms_handle_reset on v2:172.21.15.118:6836/3879191727 2024-09-12T03:20:45.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.973+0000 7fb65f017740 1 --2- 172.21.15.6:0/973719667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e10314bb0 0x559e10311d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.973+0000 7fb65f017740 1 --2- 172.21.15.6:0/973719667 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e1030a2f0 0x559e10311820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.973+0000 7fb65f017740 1 --2- 172.21.15.6:0/973719667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e103099d0 0x559e103144d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.973+0000 7fb65f017740 1 -- 172.21.15.6:0/973719667 >> 172.21.15.6:0/973719667 conn(0x559e10305f60 msgr2=0x559e1030bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:45.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.973+0000 7fb65f017740 1 -- 172.21.15.6:0/973719667 shutdown_connections 2024-09-12T03:20:45.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.973+0000 7fb65f017740 1 -- 172.21.15.6:0/973719667 wait complete. 2024-09-12T03:20:45.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.973+0000 7fb65f017740 20 client.15081 trim_cache size 0 max 16384 2024-09-12T03:20:45.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.991+0000 7fb65f017740 1 Processor -- start 2024-09-12T03:20:45.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.991+0000 7fb65f017740 1 -- start start 2024-09-12T03:20:45.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.991+0000 7fb65f017740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e103099d0 0x559e10306aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:45.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.991+0000 7fb65f017740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e1030a2f0 0x559e1032f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:45.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.991+0000 7fb65f017740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e10314bb0 0x559e1030ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:45.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.991+0000 7fb65f017740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x559e10324ff0 con 0x559e10314bb0 2024-09-12T03:20:45.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.991+0000 7fb65f017740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x559e103256c0 con 0x559e1030a2f0 2024-09-12T03:20:45.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.991+0000 7fb65f017740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559e10325d90 con 0x559e103099d0 2024-09-12T03:20:45.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.991+0000 7fb656d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e10314bb0 0x559e1030ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:45.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.992+0000 7fb656d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e10314bb0 0x559e1030ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56574/0 (socket says 172.21.15.6:56574) 2024-09-12T03:20:45.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.992+0000 7fb64ffff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e1030a2f0 0x559e1032f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:45.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.992+0000 7fb656d9d640 1 -- 172.21.15.6:0/381271841 learned_addr learned my addr 172.21.15.6:0/381271841 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:45.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.992+0000 7fb65659c640 1 --2- 172.21.15.6:0/381271841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e103099d0 0x559e10306aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:45.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.992+0000 7fb656d9d640 1 -- 172.21.15.6:0/381271841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e103099d0 msgr2=0x559e10306aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:45.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.992+0000 7fb656d9d640 1 --2- 172.21.15.6:0/381271841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e103099d0 0x559e10306aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.992+0000 7fb656d9d640 1 -- 172.21.15.6:0/381271841 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e1030a2f0 msgr2=0x559e1032f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:45.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.992+0000 7fb656d9d640 1 --2- 172.21.15.6:0/381271841 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e1030a2f0 0x559e1032f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.992+0000 7fb656d9d640 1 -- 172.21.15.6:0/381271841 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e10309800 con 0x559e10314bb0 2024-09-12T03:20:45.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.992+0000 7fb64ffff640 1 --2- 172.21.15.6:0/381271841 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e1030a2f0 0x559e1032f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:45.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.992+0000 7fb65659c640 1 --2- 172.21.15.6:0/381271841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e103099d0 0x559e10306aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:45.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.992+0000 7fb656d9d640 1 --2- 172.21.15.6:0/381271841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e10314bb0 0x559e1030ea80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb650009cd0 tx=0x7fb65003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c2566452637208 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:45.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.992+0000 7fb63ffff640 1 -- 172.21.15.6:0/381271841 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650041070 con 0x559e10314bb0 2024-09-12T03:20:45.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.992+0000 7fb63ffff640 1 -- 172.21.15.6:0/381271841 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb650036050 con 0x559e10314bb0 2024-09-12T03:20:45.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.992+0000 7fb63ffff640 1 -- 172.21.15.6:0/381271841 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650044040 con 0x559e10314bb0 2024-09-12T03:20:45.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.993+0000 7fb65f017740 1 -- 172.21.15.6:0/381271841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e10314bb0 msgr2=0x559e1030ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.993+0000 7fb65f017740 1 --2- 172.21.15.6:0/381271841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e10314bb0 0x559e1030ea80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb650009cd0 tx=0x7fb65003e000 comp rx=0 tx=0).stop 2024-09-12T03:20:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.993+0000 7fb65f017740 1 -- 172.21.15.6:0/381271841 shutdown_connections 2024-09-12T03:20:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.993+0000 7fb65f017740 1 --2- 172.21.15.6:0/381271841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e10314bb0 0x559e1030ea80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.993+0000 7fb65f017740 1 --2- 172.21.15.6:0/381271841 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e1030a2f0 0x559e1032f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.993+0000 7fb65f017740 1 --2- 172.21.15.6:0/381271841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e103099d0 0x559e10306aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.993+0000 7fb65f017740 1 -- 172.21.15.6:0/381271841 >> 172.21.15.6:0/381271841 conn(0x559e10305f60 msgr2=0x559e1030cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.993+0000 7fb65f017740 1 -- 172.21.15.6:0/381271841 shutdown_connections 2024-09-12T03:20:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.993+0000 7fb65f017740 1 -- 172.21.15.6:0/381271841 wait complete. 2024-09-12T03:20:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.993+0000 7fb65f017740 1 -- start start 2024-09-12T03:20:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.993+0000 7fb65f017740 1 Processor -- start 2024-09-12T03:20:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.994+0000 7fb65f017740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e103099d0 0x559e1038b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.994+0000 7fb65f017740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e1030a2f0 0x559e1038bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.994+0000 7fb65f017740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e10314bb0 0x559e1032e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.994+0000 7fb65f017740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x559e10326b30 con 0x559e1030a2f0 2024-09-12T03:20:46.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.994+0000 7fb65f017740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x559e10326460 con 0x559e103099d0 2024-09-12T03:20:46.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.994+0000 7fb65f017740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559e10327200 con 0x559e10314bb0 2024-09-12T03:20:46.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.994+0000 7fb65f017740 10 client.0 _ll_register_callbacks cb 0x7fb6480042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:46.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.994+0000 7fb64ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e1030a2f0 0x559e1038bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.994+0000 7fb64ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e1030a2f0 0x559e1038bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56582/0 (socket says 172.21.15.6:56582) 2024-09-12T03:20:46.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.994+0000 7fb64ffff640 1 -- 172.21.15.6:0/3616843433 learned_addr learned my addr 172.21.15.6:0/3616843433 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:46.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.994+0000 7fb656d9d640 1 --2- 172.21.15.6:0/3616843433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e10314bb0 0x559e1032e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.994+0000 7fb65659c640 1 --2- 172.21.15.6:0/3616843433 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e103099d0 0x559e1038b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.994+0000 7fb64ffff640 1 -- 172.21.15.6:0/3616843433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e10314bb0 msgr2=0x559e1032e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:46.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.994+0000 7fb64ffff640 1 --2- 172.21.15.6:0/3616843433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e10314bb0 0x559e1032e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.994+0000 7fb64ffff640 1 -- 172.21.15.6:0/3616843433 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e103099d0 msgr2=0x559e1038b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:46.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.994+0000 7fb64ffff640 1 --2- 172.21.15.6:0/3616843433 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e103099d0 0x559e1038b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.994+0000 7fb64ffff640 1 -- 172.21.15.6:0/3616843433 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e1032f1e0 con 0x559e1030a2f0 2024-09-12T03:20:46.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.994+0000 7fb64ffff640 1 --2- 172.21.15.6:0/3616843433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e1030a2f0 0x559e1038bb70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb640097be0 tx=0x7fb6400975d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a935298d1dbf134 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:46.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.994+0000 7fb63dffb640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:20:46.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.994+0000 7fb63dffb640 1 -- 172.21.15.6:0/3616843433 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640021040 con 0x559e1030a2f0 2024-09-12T03:20:46.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.995+0000 7fb63dffb640 1 -- 172.21.15.6:0/3616843433 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fb6400bb150 con 0x559e1030a2f0 2024-09-12T03:20:46.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.995+0000 7fb65f017740 1 -- 172.21.15.6:0/3616843433 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb650036050 con 0x559e1030a2f0 2024-09-12T03:20:46.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.995+0000 7fb63dffb640 1 -- 172.21.15.6:0/3616843433 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb64009f510 con 0x559e1030a2f0 2024-09-12T03:20:46.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.995+0000 7fb65f017740 10 client.15087 fetch_fsmap learned FSMap version 24 2024-09-12T03:20:46.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.995+0000 7fb65f017740 1 -- 172.21.15.6:0/3616843433 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559e10393f30 con 0x559e1030a2f0 2024-09-12T03:20:46.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.995+0000 7fb63dffb640 1 -- 172.21.15.6:0/3616843433 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb650036050 con 0x559e1030a2f0 2024-09-12T03:20:46.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.995+0000 7fb63dffb640 1 -- 172.21.15.6:0/3616843433 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb6400a3940 con 0x559e1030a2f0 2024-09-12T03:20:46.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.995+0000 7fb65f017740 10 client.15087 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:20:46.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.995+0000 7fb65f017740 10 client.15087 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T03:20:46.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.995+0000 7fb65f017740 20 client.15087 populate_metadata read hostname 'smithi006' 2024-09-12T03:20:46.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.995+0000 7fb65f017740 1 --2- 172.21.15.6:0/3616843433 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x559e10325840 0x559e10397260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.995+0000 7fb65f017740 4 client.15087 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:20:46.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.995+0000 7fb65f017740 1 -- 172.21.15.6:0/3616843433 --> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] -- command(tid 0: {"prefix": "status"}) -- 0x559e10277b10 con 0x559e10325840 2024-09-12T03:20:46.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.996+0000 7fb65659c640 1 --2- 172.21.15.6:0/3616843433 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x559e10325840 0x559e10397260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.996+0000 7fb65659c640 1 --2- 172.21.15.6:0/3616843433 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x559e10325840 0x559e10397260 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af12f189cd1630aa server_cookie=4974fb6a18793c96 in_seq=0 out_seq=0 2024-09-12T03:20:46.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.996+0000 7fb63dffb640 10 client.15087 ms_handle_connect on v2:172.21.15.118:6836/3879191727 2024-09-12T03:20:46.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.996+0000 7fb63dffb640 1 -- 172.21.15.6:0/3616843433 <== mds.1 v2:172.21.15.118:6836/3879191727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x559e10277b10 con 0x559e10325840 2024-09-12T03:20:46.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.996+0000 7fb63dffb640 10 client.15087 handle_command_reply: tid=0 2024-09-12T03:20:46.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.996+0000 7fb65f017740 1 client.15087 shutdown 2024-09-12T03:20:46.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.996+0000 7fb65f017740 1 -- 172.21.15.6:0/3616843433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e1030a2f0 msgr2=0x559e1038bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.996+0000 7fb65f017740 1 --2- 172.21.15.6:0/3616843433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e1030a2f0 0x559e1038bb70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb640097be0 tx=0x7fb6400975d0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.996+0000 7fb65f017740 1 -- 172.21.15.6:0/3616843433 shutdown_connections 2024-09-12T03:20:46.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.996+0000 7fb65f017740 1 --2- 172.21.15.6:0/3616843433 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x559e10325840 0x559e10397260 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.996+0000 7fb65f017740 1 --2- 172.21.15.6:0/3616843433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e10314bb0 0x559e1032e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.996+0000 7fb65f017740 1 --2- 172.21.15.6:0/3616843433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e1030a2f0 0x559e1038bb70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.996+0000 7fb63dffb640 0 client.15087 ms_handle_reset on v2:172.21.15.118:6836/3879191727 2024-09-12T03:20:46.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.996+0000 7fb65f017740 1 --2- 172.21.15.6:0/3616843433 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559e103099d0 0x559e1038b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.996+0000 7fb65f017740 1 -- 172.21.15.6:0/3616843433 >> 172.21.15.6:0/3616843433 conn(0x559e10305f60 msgr2=0x559e1030b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:46.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.996+0000 7fb65f017740 1 -- 172.21.15.6:0/3616843433 shutdown_connections 2024-09-12T03:20:46.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.996+0000 7fb65f017740 1 -- 172.21.15.6:0/3616843433 wait complete. 2024-09-12T03:20:46.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.997+0000 7fb65f017740 20 client.15087 trim_cache size 0 max 16384 2024-09-12T03:20:46.004 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:20:46.004 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "89d99938-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:20:46.004 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 1, 2024-09-12T03:20:46.004 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24464, 2024-09-12T03:20:46.004 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-09-12T03:20:46.004 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-09-12T03:20:46.004 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-09-12T03:20:46.004 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 88.641055408, 2024-09-12T03:20:46.004 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 24, 2024-09-12T03:20:46.004 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 82, 2024-09-12T03:20:46.004 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T03:20:46.004 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 115.90929973599999 2024-09-12T03:20:46.005 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:20:46.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.997+0000 7fb65cc3e640 1 -- 172.21.15.6:0/1282015476 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb6440a2c90 msgr2=0x7fb6440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.997+0000 7fb65cc3e640 1 --2- 172.21.15.6:0/1282015476 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb6440a2c90 0x7fb6440a5150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb650005e30 tx=0x7fb650009360 comp rx=0 tx=0).stop 2024-09-12T03:20:46.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.997+0000 7fb65cc3e640 1 -- 172.21.15.6:0/1282015476 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6581036e0 msgr2=0x7fb658198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.997+0000 7fb65cc3e640 1 --2- 172.21.15.6:0/1282015476 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6581036e0 0x7fb658198ad0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb64000d740 tx=0x7fb64000dc00 comp rx=0 tx=0).stop 2024-09-12T03:20:46.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.997+0000 7fb65cc3e640 1 -- 172.21.15.6:0/1282015476 shutdown_connections 2024-09-12T03:20:46.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.997+0000 7fb65cc3e640 1 --2- 172.21.15.6:0/1282015476 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb6440a2c90 0x7fb6440a5150 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.997+0000 7fb65cc3e640 1 --2- 172.21.15.6:0/1282015476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb6581082a0 0x7fb65819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.997+0000 7fb65cc3e640 1 --2- 172.21.15.6:0/1282015476 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6581036e0 0x7fb658198ad0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.997+0000 7fb65cc3e640 1 --2- 172.21.15.6:0/1282015476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb658102d30 0x7fb658198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.997+0000 7fb65cc3e640 1 -- 172.21.15.6:0/1282015476 >> 172.21.15.6:0/1282015476 conn(0x7fb6580fc7d0 msgr2=0x7fb658106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:46.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.997+0000 7fb65cc3e640 1 -- 172.21.15.6:0/1282015476 shutdown_connections 2024-09-12T03:20:46.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:45.997+0000 7fb65cc3e640 1 -- 172.21.15.6:0/1282015476 wait complete. 2024-09-12T03:20:46.676 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T03:20:46.932 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:47.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:46 smithi118 ceph-mon[35048]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:20:47.220 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:46 smithi006 ceph-mon[30991]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:20:47.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:46 smithi161 ceph-mon[35485]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:20:47.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.410+0000 7fdd6278c640 1 -- 172.21.15.6:0/55455933 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd5c106a80 msgr2=0x7fdd5c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.410+0000 7fdd6278c640 1 --2- 172.21.15.6:0/55455933 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd5c106a80 0x7fdd5c106e60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009930 tx=0x7fdd4c02a420 comp rx=0 tx=0).stop 2024-09-12T03:20:47.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.410+0000 7fdd6278c640 1 -- 172.21.15.6:0/55455933 shutdown_connections 2024-09-12T03:20:47.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.410+0000 7fdd6278c640 1 --2- 172.21.15.6:0/55455933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd5c101420 0x7fdd5c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.410+0000 7fdd6278c640 1 --2- 172.21.15.6:0/55455933 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdd5c100a80 0x7fdd5c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.410+0000 7fdd6278c640 1 --2- 172.21.15.6:0/55455933 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd5c106a80 0x7fdd5c106e60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.410+0000 7fdd6278c640 1 -- 172.21.15.6:0/55455933 >> 172.21.15.6:0/55455933 conn(0x7fdd5c0fc7a0 msgr2=0x7fdd5c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:47.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.410+0000 7fdd6278c640 1 -- 172.21.15.6:0/55455933 shutdown_connections 2024-09-12T03:20:47.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.410+0000 7fdd6278c640 1 -- 172.21.15.6:0/55455933 wait complete. 2024-09-12T03:20:47.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.411+0000 7fdd6278c640 1 Processor -- start 2024-09-12T03:20:47.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.411+0000 7fdd6278c640 1 -- start start 2024-09-12T03:20:47.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.411+0000 7fdd6278c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd5c100a80 0x7fdd5c102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.411+0000 7fdd6278c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdd5c101420 0x7fdd5c102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.411+0000 7fdd6278c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd5c106a80 0x7fdd5c1030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.411+0000 7fdd6278c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c110ea0 con 0x7fdd5c106a80 2024-09-12T03:20:47.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.411+0000 7fdd6278c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c110d20 con 0x7fdd5c101420 2024-09-12T03:20:47.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.411+0000 7fdd6278c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c111020 con 0x7fdd5c100a80 2024-09-12T03:20:47.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.411+0000 7fdd60d29640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd5c106a80 0x7fdd5c1030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.411+0000 7fdd60d29640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd5c106a80 0x7fdd5c1030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56608/0 (socket says 172.21.15.6:56608) 2024-09-12T03:20:47.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.411+0000 7fdd60d29640 1 -- 172.21.15.6:0/3411190254 learned_addr learned my addr 172.21.15.6:0/3411190254 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:47.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.411+0000 7fdd5bfff640 1 --2- 172.21.15.6:0/3411190254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd5c100a80 0x7fdd5c102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.411+0000 7fdd5b7fe640 1 --2- 172.21.15.6:0/3411190254 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdd5c101420 0x7fdd5c102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.411+0000 7fdd60d29640 1 -- 172.21.15.6:0/3411190254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd5c100a80 msgr2=0x7fdd5c102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.411+0000 7fdd60d29640 1 --2- 172.21.15.6:0/3411190254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd5c100a80 0x7fdd5c102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.412+0000 7fdd60d29640 1 -- 172.21.15.6:0/3411190254 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdd5c101420 msgr2=0x7fdd5c102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.412+0000 7fdd60d29640 1 --2- 172.21.15.6:0/3411190254 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdd5c101420 0x7fdd5c102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.412+0000 7fdd60d29640 1 -- 172.21.15.6:0/3411190254 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd5c1a6960 con 0x7fdd5c106a80 2024-09-12T03:20:47.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.412+0000 7fdd5b7fe640 1 --2- 172.21.15.6:0/3411190254 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdd5c101420 0x7fdd5c102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:47.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.412+0000 7fdd60d29640 1 --2- 172.21.15.6:0/3411190254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd5c106a80 0x7fdd5c1030d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400b970 tx=0x7fdd5400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:47.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.412+0000 7fdd597fa640 1 -- 172.21.15.6:0/3411190254 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54019070 con 0x7fdd5c106a80 2024-09-12T03:20:47.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.412+0000 7fdd597fa640 1 -- 172.21.15.6:0/3411190254 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd5400cb40 con 0x7fdd5c106a80 2024-09-12T03:20:47.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.412+0000 7fdd6278c640 1 -- 172.21.15.6:0/3411190254 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd5c1a6c50 con 0x7fdd5c106a80 2024-09-12T03:20:47.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.412+0000 7fdd597fa640 1 -- 172.21.15.6:0/3411190254 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54007650 con 0x7fdd5c106a80 2024-09-12T03:20:47.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.412+0000 7fdd6278c640 1 -- 172.21.15.6:0/3411190254 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd5c1a71e0 con 0x7fdd5c106a80 2024-09-12T03:20:47.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.414+0000 7fdd597fa640 1 -- 172.21.15.6:0/3411190254 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fdd5401c070 con 0x7fdd5c106a80 2024-09-12T03:20:47.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.414+0000 7fdd64b65740 1 Processor -- start 2024-09-12T03:20:47.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.414+0000 7fdd64b65740 1 -- start start 2024-09-12T03:20:47.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.415+0000 7fdd64b65740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b03cbbb9d0 0x55b03cbbbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.415+0000 7fdd64b65740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbbc2f0 0x55b03cbc6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.415+0000 7fdd64b65740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbc6bb0 0x55b03cbc8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.415+0000 7fdd64b65740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b03cb22090 con 0x55b03cbbc2f0 2024-09-12T03:20:47.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.415+0000 7fdd64b65740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55b03cac9a70 con 0x55b03cbc6bb0 2024-09-12T03:20:47.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.415+0000 7fdd64b65740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b03cb1e050 con 0x55b03cbbb9d0 2024-09-12T03:20:47.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.415+0000 7fdd5bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbbc2f0 0x55b03cbc6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.415+0000 7fdd5bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbbc2f0 0x55b03cbc6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56614/0 (socket says 172.21.15.6:56614) 2024-09-12T03:20:47.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.415+0000 7fdd5bfff640 1 -- 172.21.15.6:0/2743064184 learned_addr learned my addr 172.21.15.6:0/2743064184 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:47.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.415+0000 7fdd597fa640 1 --2- 172.21.15.6:0/3411190254 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fdd440a2c90 0x7fdd440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.415+0000 7fdd5b7fe640 1 --2- 172.21.15.6:0/2743064184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b03cbbb9d0 0x55b03cbbbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.415+0000 7fdd5b7fe640 1 --2- 172.21.15.6:0/2743064184 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbc6bb0 0x55b03cbc8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.415+0000 7fdd5bfff640 1 -- 172.21.15.6:0/2743064184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b03cbbb9d0 msgr2=0x55b03cbbbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.415+0000 7fdd5bfff640 1 --2- 172.21.15.6:0/2743064184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b03cbbb9d0 0x55b03cbbbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.415+0000 7fdd5bfff640 1 -- 172.21.15.6:0/2743064184 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbc6bb0 msgr2=0x55b03cbc8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.415+0000 7fdd5bfff640 1 --2- 172.21.15.6:0/2743064184 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbc6bb0 0x55b03cbc8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.415+0000 7fdd5bfff640 1 -- 172.21.15.6:0/2743064184 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b03cad6130 con 0x55b03cbbc2f0 2024-09-12T03:20:47.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.415+0000 7fdd60d29640 1 --2- 172.21.15.6:0/3411190254 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fdd440a2c90 0x7fdd440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.415+0000 7fdd5b7fe640 1 --2- 172.21.15.6:0/2743064184 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbc6bb0 0x55b03cbc8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:47.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.415+0000 7fdd597fa640 1 -- 172.21.15.6:0/3411190254 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fdd540ce5a0 con 0x7fdd5c106a80 2024-09-12T03:20:47.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.415+0000 7fdd5bfff640 1 --2- 172.21.15.6:0/2743064184 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbbc2f0 0x55b03cbc6670 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c02a940 tx=0x7fdd4c0316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9fb8578b804f010 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:47.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.416+0000 7fdd427fc640 1 -- 172.21.15.6:0/2743064184 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c040070 con 0x55b03cbbc2f0 2024-09-12T03:20:47.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.416+0000 7fdd427fc640 1 -- 172.21.15.6:0/2743064184 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd4c005020 con 0x55b03cbbc2f0 2024-09-12T03:20:47.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.416+0000 7fdd60d29640 1 --2- 172.21.15.6:0/3411190254 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fdd440a2c90 0x7fdd440a5150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdd54097860 tx=0x7fdd54065b10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:47.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.416+0000 7fdd427fc640 1 -- 172.21.15.6:0/2743064184 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c035040 con 0x55b03cbbc2f0 2024-09-12T03:20:47.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.416+0000 7fdd64b65740 1 -- 172.21.15.6:0/2743064184 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbbc2f0 msgr2=0x55b03cbc6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.416+0000 7fdd64b65740 1 --2- 172.21.15.6:0/2743064184 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbbc2f0 0x55b03cbc6670 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c02a940 tx=0x7fdd4c0316e0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.416+0000 7fdd64b65740 1 -- 172.21.15.6:0/2743064184 shutdown_connections 2024-09-12T03:20:47.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.416+0000 7fdd64b65740 1 --2- 172.21.15.6:0/2743064184 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbc6bb0 0x55b03cbc8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.416+0000 7fdd64b65740 1 --2- 172.21.15.6:0/2743064184 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbbc2f0 0x55b03cbc6670 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.416+0000 7fdd64b65740 1 --2- 172.21.15.6:0/2743064184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b03cbbb9d0 0x55b03cbbbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.416+0000 7fdd64b65740 1 -- 172.21.15.6:0/2743064184 >> 172.21.15.6:0/2743064184 conn(0x55b03cbb7f60 msgr2=0x55b03cbb8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:47.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.416+0000 7fdd64b65740 1 -- 172.21.15.6:0/2743064184 shutdown_connections 2024-09-12T03:20:47.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.416+0000 7fdd64b65740 1 -- 172.21.15.6:0/2743064184 wait complete. 2024-09-12T03:20:47.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.417+0000 7fdd64b65740 1 -- start start 2024-09-12T03:20:47.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.417+0000 7fdd64b65740 1 Processor -- start 2024-09-12T03:20:47.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.417+0000 7fdd64b65740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbbb9d0 0x55b03cbc6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.417+0000 7fdd64b65740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b03cbbc2f0 0x55b03cbc36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.417+0000 7fdd64b65740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbc6bb0 0x55b03cbc3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.417+0000 7fdd64b65740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b03cbc9780 con 0x55b03cbbb9d0 2024-09-12T03:20:47.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.417+0000 7fdd64b65740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55b03cadcf20 con 0x55b03cbc6bb0 2024-09-12T03:20:47.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.417+0000 7fdd64b65740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b03cbc9900 con 0x55b03cbbc2f0 2024-09-12T03:20:47.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.417+0000 7fdd64b65740 10 client.0 _ll_register_callbacks cb 0x55b03caa6030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:47.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.417+0000 7fdd5b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbbb9d0 0x55b03cbc6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.417+0000 7fdd5b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbbb9d0 0x55b03cbc6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56616/0 (socket says 172.21.15.6:56616) 2024-09-12T03:20:47.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.417+0000 7fdd5b7fe640 1 -- 172.21.15.6:0/337891113 learned_addr learned my addr 172.21.15.6:0/337891113 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:47.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.418+0000 7fdd5bfff640 1 --2- 172.21.15.6:0/337891113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b03cbbc2f0 0x55b03cbc36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.418+0000 7fdd5b7fe640 1 --2- 172.21.15.6:0/337891113 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbc6bb0 0x55b03cbc3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.418+0000 7fdd5b7fe640 1 -- 172.21.15.6:0/337891113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b03cbbc2f0 msgr2=0x55b03cbc36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.418+0000 7fdd5b7fe640 1 --2- 172.21.15.6:0/337891113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b03cbbc2f0 0x55b03cbc36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.418+0000 7fdd5b7fe640 1 -- 172.21.15.6:0/337891113 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbc6bb0 msgr2=0x55b03cbc3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.418+0000 7fdd5b7fe640 1 --2- 172.21.15.6:0/337891113 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbc6bb0 0x55b03cbc3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.418+0000 7fdd5b7fe640 1 -- 172.21.15.6:0/337891113 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b03c8932b0 con 0x55b03cbbb9d0 2024-09-12T03:20:47.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.418+0000 7fdd5bfff640 1 --2- 172.21.15.6:0/337891113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b03cbbc2f0 0x55b03cbc36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:47.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.418+0000 7fdd5b7fe640 1 --2- 172.21.15.6:0/337891113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbbb9d0 0x55b03cbc6350 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdd50005fa0 tx=0x7fdd500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5c45917e47fe8a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:47.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.418+0000 7fdd3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:20:47.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.418+0000 7fdd3b7fe640 1 -- 172.21.15.6:0/337891113 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd50018de0 con 0x55b03cbbb9d0 2024-09-12T03:20:47.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.418+0000 7fdd3b7fe640 1 -- 172.21.15.6:0/337891113 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fdd5000ea70 con 0x55b03cbbb9d0 2024-09-12T03:20:47.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.418+0000 7fdd64b65740 1 -- 172.21.15.6:0/337891113 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd4c005020 con 0x55b03cbbb9d0 2024-09-12T03:20:47.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.418+0000 7fdd3b7fe640 1 -- 172.21.15.6:0/337891113 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdd5000ec70 con 0x55b03cbbb9d0 2024-09-12T03:20:47.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.418+0000 7fdd64b65740 10 client.15099 fetch_fsmap learned FSMap version 24 2024-09-12T03:20:47.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.418+0000 7fdd64b65740 1 -- 172.21.15.6:0/337891113 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b03cbd4f90 con 0x55b03cbbb9d0 2024-09-12T03:20:47.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.419+0000 7fdd3b7fe640 1 -- 172.21.15.6:0/337891113 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd4c005020 con 0x55b03cbbb9d0 2024-09-12T03:20:47.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.419+0000 7fdd3b7fe640 1 -- 172.21.15.6:0/337891113 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fdd50026480 con 0x55b03cbbb9d0 2024-09-12T03:20:47.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.419+0000 7fdd64b65740 10 client.15099 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:20:47.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.419+0000 7fdd64b65740 10 client.15099 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:20:47.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.419+0000 7fdd64b65740 20 client.15099 populate_metadata read hostname 'smithi006' 2024-09-12T03:20:47.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.419+0000 7fdd64b65740 1 --2- 172.21.15.6:0/337891113 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b03cbd7580 0x55b03cbd89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.419+0000 7fdd64b65740 4 client.15099 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:47.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.419+0000 7fdd64b65740 1 -- 172.21.15.6:0/337891113 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b03cb2f600 con 0x55b03cbd7580 2024-09-12T03:20:47.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.419+0000 7fdd5bfff640 1 --2- 172.21.15.6:0/337891113 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b03cbd7580 0x55b03cbd89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.420+0000 7fdd5bfff640 1 --2- 172.21.15.6:0/337891113 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b03cbd7580 0x55b03cbd89f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15ac3fe1687b3391 server_cookie=4d207b6c6a8d8893 in_seq=0 out_seq=0 2024-09-12T03:20:47.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.420+0000 7fdd3b7fe640 10 client.15099 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:20:47.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.420+0000 7fdd3b7fe640 1 -- 172.21.15.6:0/337891113 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b03cb2f600 con 0x55b03cbd7580 2024-09-12T03:20:47.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.420+0000 7fdd3b7fe640 10 client.15099 handle_command_reply: tid=0 2024-09-12T03:20:47.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.420+0000 7fdd64b65740 1 client.15099 shutdown 2024-09-12T03:20:47.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.421+0000 7fdd64b65740 1 -- 172.21.15.6:0/337891113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbbb9d0 msgr2=0x55b03cbc6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.421+0000 7fdd64b65740 1 --2- 172.21.15.6:0/337891113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbbb9d0 0x55b03cbc6350 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdd50005fa0 tx=0x7fdd500046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.421+0000 7fdd64b65740 1 -- 172.21.15.6:0/337891113 shutdown_connections 2024-09-12T03:20:47.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.421+0000 7fdd64b65740 1 --2- 172.21.15.6:0/337891113 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b03cbd7580 0x55b03cbd89f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.421+0000 7fdd64b65740 1 --2- 172.21.15.6:0/337891113 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbc6bb0 0x55b03cbc3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.421+0000 7fdd64b65740 1 --2- 172.21.15.6:0/337891113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b03cbbc2f0 0x55b03cbc36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.421+0000 7fdd64b65740 1 --2- 172.21.15.6:0/337891113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbbb9d0 0x55b03cbc6350 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.421+0000 7fdd64b65740 1 -- 172.21.15.6:0/337891113 >> 172.21.15.6:0/337891113 conn(0x55b03cbb7f60 msgr2=0x55b03cbbdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:47.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.421+0000 7fdd3b7fe640 0 client.15099 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:20:47.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.421+0000 7fdd64b65740 1 -- 172.21.15.6:0/337891113 shutdown_connections 2024-09-12T03:20:47.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.421+0000 7fdd64b65740 1 -- 172.21.15.6:0/337891113 wait complete. 2024-09-12T03:20:47.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.421+0000 7fdd64b65740 20 client.15099 trim_cache size 0 max 16384 2024-09-12T03:20:47.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.439+0000 7fdd64b65740 1 Processor -- start 2024-09-12T03:20:47.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.439+0000 7fdd64b65740 1 -- start start 2024-09-12T03:20:47.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.439+0000 7fdd64b65740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbbb9d0 0x55b03cbd7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.439+0000 7fdd64b65740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbbc2f0 0x55b03cbc3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.439+0000 7fdd64b65740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b03cbc6bb0 0x55b03cbe0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.439+0000 7fdd64b65740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b03cbd6d30 con 0x55b03cbbc2f0 2024-09-12T03:20:47.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.439+0000 7fdd64b65740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55b03cbd7400 con 0x55b03cbbb9d0 2024-09-12T03:20:47.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.439+0000 7fdd64b65740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b03cbd7ad0 con 0x55b03cbc6bb0 2024-09-12T03:20:47.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.439+0000 7fdd5bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbbc2f0 0x55b03cbc3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.439+0000 7fdd5bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbbc2f0 0x55b03cbc3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56618/0 (socket says 172.21.15.6:56618) 2024-09-12T03:20:47.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.439+0000 7fdd5bfff640 1 -- 172.21.15.6:0/1180252845 learned_addr learned my addr 172.21.15.6:0/1180252845 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:47.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.439+0000 7fdd5b7fe640 1 --2- 172.21.15.6:0/1180252845 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbbb9d0 0x55b03cbd7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.439+0000 7fdd5b7fe640 1 --2- 172.21.15.6:0/1180252845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b03cbc6bb0 0x55b03cbe0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.439+0000 7fdd5bfff640 1 -- 172.21.15.6:0/1180252845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b03cbc6bb0 msgr2=0x55b03cbe0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.439+0000 7fdd5bfff640 1 --2- 172.21.15.6:0/1180252845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b03cbc6bb0 0x55b03cbe0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.439+0000 7fdd5bfff640 1 -- 172.21.15.6:0/1180252845 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbbb9d0 msgr2=0x55b03cbd7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.439+0000 7fdd5bfff640 1 --2- 172.21.15.6:0/1180252845 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbbb9d0 0x55b03cbd7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.439+0000 7fdd5bfff640 1 -- 172.21.15.6:0/1180252845 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b03cbb7d90 con 0x55b03cbbc2f0 2024-09-12T03:20:47.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.439+0000 7fdd5b7fe640 1 --2- 172.21.15.6:0/1180252845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b03cbc6bb0 0x55b03cbe0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:47.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.439+0000 7fdd5bfff640 1 --2- 172.21.15.6:0/1180252845 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbbc2f0 0x55b03cbc3bf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009930 tx=0x7fdd4c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af06d6fe1800e025 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:47.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.440+0000 7fdd41ffb640 1 -- 172.21.15.6:0/1180252845 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c040070 con 0x55b03cbbc2f0 2024-09-12T03:20:47.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.440+0000 7fdd41ffb640 1 -- 172.21.15.6:0/1180252845 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd4c003ad0 con 0x55b03cbbc2f0 2024-09-12T03:20:47.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.440+0000 7fdd41ffb640 1 -- 172.21.15.6:0/1180252845 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c035040 con 0x55b03cbbc2f0 2024-09-12T03:20:47.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.440+0000 7fdd64b65740 1 -- 172.21.15.6:0/1180252845 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbbc2f0 msgr2=0x55b03cbc3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.440+0000 7fdd64b65740 1 --2- 172.21.15.6:0/1180252845 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbbc2f0 0x55b03cbc3bf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009930 tx=0x7fdd4c000bc0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.440+0000 7fdd64b65740 1 -- 172.21.15.6:0/1180252845 shutdown_connections 2024-09-12T03:20:47.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.440+0000 7fdd64b65740 1 --2- 172.21.15.6:0/1180252845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b03cbc6bb0 0x55b03cbe0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.440+0000 7fdd64b65740 1 --2- 172.21.15.6:0/1180252845 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbbc2f0 0x55b03cbc3bf0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.440+0000 7fdd64b65740 1 --2- 172.21.15.6:0/1180252845 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbbb9d0 0x55b03cbd7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.440+0000 7fdd64b65740 1 -- 172.21.15.6:0/1180252845 >> 172.21.15.6:0/1180252845 conn(0x55b03cbb7f60 msgr2=0x55b03cbc1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:47.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.440+0000 7fdd64b65740 1 -- 172.21.15.6:0/1180252845 shutdown_connections 2024-09-12T03:20:47.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.440+0000 7fdd64b65740 1 -- 172.21.15.6:0/1180252845 wait complete. 2024-09-12T03:20:47.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.441+0000 7fdd64b65740 1 -- start start 2024-09-12T03:20:47.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.441+0000 7fdd64b65740 1 Processor -- start 2024-09-12T03:20:47.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.441+0000 7fdd64b65740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b03cbbb9d0 0x55b03cbdbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.441+0000 7fdd64b65740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbbc2f0 0x55b03cbdc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.441+0000 7fdd64b65740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbc6bb0 0x55b03cbdc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.441+0000 7fdd64b65740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b03cbd8870 con 0x55b03cbc6bb0 2024-09-12T03:20:47.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.441+0000 7fdd64b65740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55b03cbd81a0 con 0x55b03cbbc2f0 2024-09-12T03:20:47.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.441+0000 7fdd64b65740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b03cbd8f40 con 0x55b03cbbb9d0 2024-09-12T03:20:47.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.441+0000 7fdd64b65740 10 client.0 _ll_register_callbacks cb 0x7fdd5000d620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:47.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.441+0000 7fdd5b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbc6bb0 0x55b03cbdc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.441+0000 7fdd5b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbc6bb0 0x55b03cbdc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56630/0 (socket says 172.21.15.6:56630) 2024-09-12T03:20:47.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.441+0000 7fdd5b7fe640 1 -- 172.21.15.6:0/4189769912 learned_addr learned my addr 172.21.15.6:0/4189769912 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:47.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.442+0000 7fdd5bfff640 1 --2- 172.21.15.6:0/4189769912 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbbc2f0 0x55b03cbdc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.442+0000 7fdd5b7fe640 1 -- 172.21.15.6:0/4189769912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b03cbbb9d0 msgr2=0x55b03cbdbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.442+0000 7fdd5b7fe640 1 --2- 172.21.15.6:0/4189769912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b03cbbb9d0 0x55b03cbdbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.442+0000 7fdd5b7fe640 1 -- 172.21.15.6:0/4189769912 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbbc2f0 msgr2=0x55b03cbdc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.442+0000 7fdd5b7fe640 1 --2- 172.21.15.6:0/4189769912 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbbc2f0 0x55b03cbdc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.442+0000 7fdd5b7fe640 1 -- 172.21.15.6:0/4189769912 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b03cc3fd80 con 0x55b03cbc6bb0 2024-09-12T03:20:47.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.442+0000 7fdd5bfff640 1 --2- 172.21.15.6:0/4189769912 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbbc2f0 0x55b03cbdc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:47.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.442+0000 7fdd5b7fe640 1 --2- 172.21.15.6:0/4189769912 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbc6bb0 0x55b03cbdc610 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdd500058c0 tx=0x7fdd5000dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b81e89bed977c26 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:47.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.442+0000 7fdd3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:20:47.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.442+0000 7fdd3b7fe640 1 -- 172.21.15.6:0/4189769912 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5000ee10 con 0x55b03cbc6bb0 2024-09-12T03:20:47.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.442+0000 7fdd3b7fe640 1 -- 172.21.15.6:0/4189769912 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fdd50005040 con 0x55b03cbc6bb0 2024-09-12T03:20:47.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.442+0000 7fdd64b65740 1 -- 172.21.15.6:0/4189769912 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd4c003ad0 con 0x55b03cbc6bb0 2024-09-12T03:20:47.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.442+0000 7fdd3b7fe640 1 -- 172.21.15.6:0/4189769912 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdd50026a40 con 0x55b03cbc6bb0 2024-09-12T03:20:47.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.442+0000 7fdd64b65740 10 client.15105 fetch_fsmap learned FSMap version 24 2024-09-12T03:20:47.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.442+0000 7fdd64b65740 1 -- 172.21.15.6:0/4189769912 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b03cc40fb0 con 0x55b03cbc6bb0 2024-09-12T03:20:47.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.442+0000 7fdd3b7fe640 1 -- 172.21.15.6:0/4189769912 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd4c003ad0 con 0x55b03cbc6bb0 2024-09-12T03:20:47.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.443+0000 7fdd3b7fe640 1 -- 172.21.15.6:0/4189769912 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fdd50004a20 con 0x55b03cbc6bb0 2024-09-12T03:20:47.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.443+0000 7fdd64b65740 10 client.15105 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:20:47.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.443+0000 7fdd64b65740 10 client.15105 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:20:47.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.443+0000 7fdd64b65740 20 client.15105 populate_metadata read hostname 'smithi006' 2024-09-12T03:20:47.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.443+0000 7fdd64b65740 1 --2- 172.21.15.6:0/4189769912 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b03cc45c40 0x55b03cc44950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.443+0000 7fdd64b65740 4 client.15105 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:20:47.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.443+0000 7fdd64b65740 1 -- 172.21.15.6:0/4189769912 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "status"}) -- 0x55b03cb29b10 con 0x55b03cc45c40 2024-09-12T03:20:47.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.443+0000 7fdd5bfff640 1 --2- 172.21.15.6:0/4189769912 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b03cc45c40 0x55b03cc44950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.443+0000 7fdd5bfff640 1 --2- 172.21.15.6:0/4189769912 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b03cc45c40 0x55b03cc44950 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2befe214740887cb server_cookie=59bb67d1d64ddb58 in_seq=0 out_seq=0 2024-09-12T03:20:47.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.443+0000 7fdd3b7fe640 10 client.15105 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:20:47.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.443+0000 7fdd3b7fe640 1 -- 172.21.15.6:0/4189769912 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b03cb29b10 con 0x55b03cc45c40 2024-09-12T03:20:47.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.444+0000 7fdd3b7fe640 10 client.15105 handle_command_reply: tid=0 2024-09-12T03:20:47.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.444+0000 7fdd64b65740 1 client.15105 shutdown 2024-09-12T03:20:47.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.444+0000 7fdd64b65740 1 -- 172.21.15.6:0/4189769912 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbc6bb0 msgr2=0x55b03cbdc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.444+0000 7fdd64b65740 1 --2- 172.21.15.6:0/4189769912 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbc6bb0 0x55b03cbdc610 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdd500058c0 tx=0x7fdd5000dce0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.444+0000 7fdd64b65740 1 -- 172.21.15.6:0/4189769912 shutdown_connections 2024-09-12T03:20:47.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.444+0000 7fdd64b65740 1 --2- 172.21.15.6:0/4189769912 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b03cc45c40 0x55b03cc44950 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.444+0000 7fdd64b65740 1 --2- 172.21.15.6:0/4189769912 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b03cbc6bb0 0x55b03cbdc610 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.444+0000 7fdd3b7fe640 0 client.15105 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:20:47.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.444+0000 7fdd64b65740 1 --2- 172.21.15.6:0/4189769912 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b03cbbc2f0 0x55b03cbdc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.444+0000 7fdd64b65740 1 --2- 172.21.15.6:0/4189769912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b03cbbb9d0 0x55b03cbdbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.444+0000 7fdd64b65740 1 -- 172.21.15.6:0/4189769912 >> 172.21.15.6:0/4189769912 conn(0x55b03cbb7f60 msgr2=0x55b03cbb67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:47.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.444+0000 7fdd64b65740 1 -- 172.21.15.6:0/4189769912 shutdown_connections 2024-09-12T03:20:47.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.444+0000 7fdd64b65740 1 -- 172.21.15.6:0/4189769912 wait complete. 2024-09-12T03:20:47.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.444+0000 7fdd64b65740 20 client.15105 trim_cache size 0 max 16384 2024-09-12T03:20:47.451 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:20:47.451 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "89d99938-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:20:47.451 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 0, 2024-09-12T03:20:47.451 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24494, 2024-09-12T03:20:47.451 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-09-12T03:20:47.451 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-09-12T03:20:47.451 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-09-12T03:20:47.452 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 97.122368508999998, 2024-09-12T03:20:47.452 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 24, 2024-09-12T03:20:47.452 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 82, 2024-09-12T03:20:47.452 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T03:20:47.452 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 106.38763806999999 2024-09-12T03:20:47.452 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:20:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.445+0000 7fdd6278c640 1 -- 172.21.15.6:0/3411190254 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fdd440a2c90 msgr2=0x7fdd440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.445+0000 7fdd6278c640 1 --2- 172.21.15.6:0/3411190254 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fdd440a2c90 0x7fdd440a5150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdd54097860 tx=0x7fdd54065b10 comp rx=0 tx=0).stop 2024-09-12T03:20:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.445+0000 7fdd6278c640 1 -- 172.21.15.6:0/3411190254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd5c106a80 msgr2=0x7fdd5c1030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.445+0000 7fdd6278c640 1 --2- 172.21.15.6:0/3411190254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd5c106a80 0x7fdd5c1030d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400b970 tx=0x7fdd5400be30 comp rx=0 tx=0).stop 2024-09-12T03:20:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.445+0000 7fdd6278c640 1 -- 172.21.15.6:0/3411190254 shutdown_connections 2024-09-12T03:20:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.445+0000 7fdd6278c640 1 --2- 172.21.15.6:0/3411190254 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fdd440a2c90 0x7fdd440a5150 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.445+0000 7fdd6278c640 1 --2- 172.21.15.6:0/3411190254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd5c106a80 0x7fdd5c1030d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.445+0000 7fdd6278c640 1 --2- 172.21.15.6:0/3411190254 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdd5c101420 0x7fdd5c102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.445+0000 7fdd6278c640 1 --2- 172.21.15.6:0/3411190254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd5c100a80 0x7fdd5c102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.445+0000 7fdd6278c640 1 -- 172.21.15.6:0/3411190254 >> 172.21.15.6:0/3411190254 conn(0x7fdd5c0fc7a0 msgr2=0x7fdd5c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.445+0000 7fdd6278c640 1 -- 172.21.15.6:0/3411190254 shutdown_connections 2024-09-12T03:20:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:47.445+0000 7fdd6278c640 1 -- 172.21.15.6:0/3411190254 wait complete. 2024-09-12T03:20:48.211 INFO:teuthology.run_tasks:Running task sequential... 2024-09-12T03:20:48.221 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-12T03:20:48.221 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-12T03:20:48.222 INFO:teuthology.task.print:Enabling mgr modules 2024-09-12T03:20:48.222 INFO:teuthology.run_tasks:Running task kclient... 2024-09-12T03:20:48.316 INFO:tasks.kclient:Mounting kernel clients... 2024-09-12T03:20:48.316 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-12T03:20:48.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:48.317 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2024-09-12T03:20:48.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:48.335 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link delete ceph-brx 2024-09-12T03:20:48.405 INFO:teuthology.orchestra.run.smithi006.stderr:Cannot find device "ceph-brx" 2024-09-12T03:20:48.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:20:48.407 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-12T03:20:48.407 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T03:20:48.407 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:20:48.408 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-12T03:20:48.704 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:49.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:48 smithi006 ceph-mon[30991]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:49.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.230+0000 7f0bc6841640 1 -- 172.21.15.6:0/3891559876 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0bc011b140 msgr2=0x7f0bc011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:49.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.230+0000 7f0bc6841640 1 --2- 172.21.15.6:0/3891559876 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0bc011b140 0x7f0bc011d530 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009900 tx=0x7f0bb002a440 comp rx=0 tx=0).stop 2024-09-12T03:20:49.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.230+0000 7f0bc6841640 1 -- 172.21.15.6:0/3891559876 shutdown_connections 2024-09-12T03:20:49.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.230+0000 7f0bc6841640 1 --2- 172.21.15.6:0/3891559876 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0bc011b140 0x7f0bc011d530 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.230+0000 7f0bc6841640 1 --2- 172.21.15.6:0/3891559876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0bc0109830 0x7f0bc0109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.230+0000 7f0bc6841640 1 --2- 172.21.15.6:0/3891559876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bc0109450 0x7f0bc0110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.230+0000 7f0bc6841640 1 -- 172.21.15.6:0/3891559876 >> 172.21.15.6:0/3891559876 conn(0x7f0bc006e140 msgr2=0x7f0bc006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:49.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.231+0000 7f0bc6841640 1 -- 172.21.15.6:0/3891559876 shutdown_connections 2024-09-12T03:20:49.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.232+0000 7f0bc6841640 1 -- 172.21.15.6:0/3891559876 wait complete. 2024-09-12T03:20:49.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.233+0000 7f0bc6841640 1 Processor -- start 2024-09-12T03:20:49.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.233+0000 7f0bc6841640 1 -- start start 2024-09-12T03:20:49.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.233+0000 7f0bc6841640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bc0109830 0x7f0bc01a7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:49.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.233+0000 7f0bc6841640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0bc011b140 0x7f0bc01a7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:49.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.233+0000 7f0bc6841640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0bc01abd10 0x7f0bc01ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:49.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.233+0000 7f0bc6841640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc0120e80 con 0x7f0bc01abd10 2024-09-12T03:20:49.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.233+0000 7f0bc6841640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc0120d00 con 0x7f0bc011b140 2024-09-12T03:20:49.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.233+0000 7f0bc6841640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc0121000 con 0x7f0bc0109830 2024-09-12T03:20:49.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.234+0000 7f0bc4dde640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0bc01abd10 0x7f0bc01ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:49.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.234+0000 7f0bc4dde640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0bc01abd10 0x7f0bc01ac1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36280/0 (socket says 172.21.15.6:36280) 2024-09-12T03:20:49.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.234+0000 7f0bc4dde640 1 -- 172.21.15.6:0/1314611585 learned_addr learned my addr 172.21.15.6:0/1314611585 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:49.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.234+0000 7f0bbf7fe640 1 --2- 172.21.15.6:0/1314611585 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0bc011b140 0x7f0bc01a7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:49.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.234+0000 7f0bc4dde640 1 -- 172.21.15.6:0/1314611585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bc0109830 msgr2=0x7f0bc01a7440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:20:49.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.234+0000 7f0bc4dde640 1 --2- 172.21.15.6:0/1314611585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bc0109830 0x7f0bc01a7440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.234+0000 7f0bc4dde640 1 -- 172.21.15.6:0/1314611585 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0bc011b140 msgr2=0x7f0bc01a7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:49.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.234+0000 7f0bc4dde640 1 --2- 172.21.15.6:0/1314611585 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0bc011b140 0x7f0bc01a7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.234+0000 7f0bc4dde640 1 -- 172.21.15.6:0/1314611585 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc01ac7d0 con 0x7f0bc01abd10 2024-09-12T03:20:49.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.241+0000 7f0bc4dde640 1 --2- 172.21.15.6:0/1314611585 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0bc01abd10 0x7f0bc01ac1c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0005ab0 tx=0x7f0bb00317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:49.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.243+0000 7f0bbd7fa640 1 -- 172.21.15.6:0/1314611585 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb003f030 con 0x7f0bc01abd10 2024-09-12T03:20:49.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.243+0000 7f0bbd7fa640 1 -- 172.21.15.6:0/1314611585 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb0003ad0 con 0x7f0bc01abd10 2024-09-12T03:20:49.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.243+0000 7f0bbd7fa640 1 -- 172.21.15.6:0/1314611585 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0035070 con 0x7f0bc01abd10 2024-09-12T03:20:49.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.243+0000 7f0bc6841640 1 -- 172.21.15.6:0/1314611585 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc01aca60 con 0x7f0bc01abd10 2024-09-12T03:20:49.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.243+0000 7f0bc6841640 1 -- 172.21.15.6:0/1314611585 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc0110af0 con 0x7f0bc01abd10 2024-09-12T03:20:49.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.245+0000 7f0bc6841640 1 -- 172.21.15.6:0/1314611585 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc01a87a0 con 0x7f0bc01abd10 2024-09-12T03:20:49.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.250+0000 7f0bbd7fa640 1 -- 172.21.15.6:0/1314611585 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0bb0042030 con 0x7f0bc01abd10 2024-09-12T03:20:49.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.251+0000 7f0bbd7fa640 1 --2- 172.21.15.6:0/1314611585 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0ba80a2da0 0x7f0ba80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:49.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.251+0000 7f0bbd7fa640 1 -- 172.21.15.6:0/1314611585 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f0bb00ba200 con 0x7f0bc01abd10 2024-09-12T03:20:49.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.251+0000 7f0bbffff640 1 --2- 172.21.15.6:0/1314611585 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0ba80a2da0 0x7f0ba80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:49.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.252+0000 7f0bbffff640 1 --2- 172.21.15.6:0/1314611585 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0ba80a2da0 0x7f0ba80a5260 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0bc006f290 tx=0x7f0bb8009250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:49.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.254+0000 7f0bbd7fa640 1 -- 172.21.15.6:0/1314611585 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0bb0049070 con 0x7f0bc01abd10 2024-09-12T03:20:49.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:48 smithi161 ceph-mon[35485]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:49.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:48 smithi118 ceph-mon[35048]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:20:49.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.469+0000 7f0bc6841640 1 -- 172.21.15.6:0/1314611585 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0bc002d980 con 0x7f0bc01abd10 2024-09-12T03:20:49.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.470+0000 7f0bbd7fa640 1 -- 172.21.15.6:0/1314611585 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f0bb00bc860 con 0x7f0bc01abd10 2024-09-12T03:20:49.471 INFO:teuthology.orchestra.run.smithi006.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:20:49.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.473+0000 7f0bc6841640 1 -- 172.21.15.6:0/1314611585 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0ba80a2da0 msgr2=0x7f0ba80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:49.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.473+0000 7f0bc6841640 1 --2- 172.21.15.6:0/1314611585 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0ba80a2da0 0x7f0ba80a5260 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0bc006f290 tx=0x7f0bb8009250 comp rx=0 tx=0).stop 2024-09-12T03:20:49.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.473+0000 7f0bc6841640 1 -- 172.21.15.6:0/1314611585 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0bc01abd10 msgr2=0x7f0bc01ac1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:49.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.473+0000 7f0bc6841640 1 --2- 172.21.15.6:0/1314611585 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0bc01abd10 0x7f0bc01ac1c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0005ab0 tx=0x7f0bb00317e0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.473+0000 7f0bc6841640 1 -- 172.21.15.6:0/1314611585 shutdown_connections 2024-09-12T03:20:49.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.473+0000 7f0bc6841640 1 --2- 172.21.15.6:0/1314611585 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0ba80a2da0 0x7f0ba80a5260 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.473+0000 7f0bc6841640 1 --2- 172.21.15.6:0/1314611585 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0bc01abd10 0x7f0bc01ac1c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.473+0000 7f0bc6841640 1 --2- 172.21.15.6:0/1314611585 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0bc011b140 0x7f0bc01a7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.473+0000 7f0bc6841640 1 --2- 172.21.15.6:0/1314611585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bc0109830 0x7f0bc01a7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.473+0000 7f0bc6841640 1 -- 172.21.15.6:0/1314611585 >> 172.21.15.6:0/1314611585 conn(0x7f0bc006e140 msgr2=0x7f0bc011bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:49.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.473+0000 7f0bc6841640 1 -- 172.21.15.6:0/1314611585 shutdown_connections 2024-09-12T03:20:49.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:49.473+0000 7f0bc6841640 1 -- 172.21.15.6:0/1314611585 wait complete. 2024-09-12T03:20:50.147 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:49 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1314611585' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:20:50.181 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T03:20:50.181 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:20:50.181 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi006.front.sepia.ceph.com 2024-09-12T03:20:50.181 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T03:20:50.181 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:20:50.181 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T03:20:50.181 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T03:20:50.181 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:20:50.182 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:20:50.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:50.182 DEBUG:teuthology.orchestra.run.smithi006:> ip addr 2024-09-12T03:20:50.200 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:20:50.201 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:20:50.201 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:20:50.201 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:50.201 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2024-09-12T03:20:50.201 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:50.201 INFO:teuthology.orchestra.run.smithi006.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:20:50.201 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e6 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:50.201 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f0 2024-09-12T03:20:50.201 INFO:teuthology.orchestra.run.smithi006.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:20:50.201 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e7 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:50.201 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f1 2024-09-12T03:20:50.202 INFO:teuthology.orchestra.run.smithi006.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:20:50.202 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e4 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:50.202 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f0 2024-09-12T03:20:50.202 INFO:teuthology.orchestra.run.smithi006.stdout: inet 172.21.15.6/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:20:50.202 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft 42176sec preferred_lft 42176sec 2024-09-12T03:20:50.202 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::ec4:7aff:febd:15e4/64 scope link 2024-09-12T03:20:50.202 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:50.202 INFO:teuthology.orchestra.run.smithi006.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:20:50.202 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e5 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:50.202 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f1 2024-09-12T03:20:50.203 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T03:20:50.203 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:20:50.203 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-09-12T03:20:50.203 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add name ceph-brx type bridge 2024-09-12T03:20:50.203 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr flush dev ceph-brx 2024-09-12T03:20:50.203 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set ceph-brx up 2024-09-12T03:20:50.203 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T03:20:50.203 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-09-12T03:20:50.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:20:50.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:49 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1314611585' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:20:50.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:20:50.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:50.425 DEBUG:teuthology.orchestra.run.smithi006:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:20:50.451 INFO:teuthology.orchestra.run.smithi006.stdout:1 2024-09-12T03:20:50.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:50.453 DEBUG:teuthology.orchestra.run.smithi006:> route 2024-09-12T03:20:50.461 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:49 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1314611585' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:20:50.524 INFO:teuthology.orchestra.run.smithi006.stdout:Kernel IP routing table 2024-09-12T03:20:50.524 INFO:teuthology.orchestra.run.smithi006.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:20:50.525 INFO:teuthology.orchestra.run.smithi006.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:20:50.525 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:20:50.525 INFO:teuthology.orchestra.run.smithi006.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:20:50.525 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:20:50.525 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-09-12T03:20:50.525 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:20:50.525 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:20:50.525 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:20:50.526 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-09-12T03:20:50.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:20:50.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:20:50.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:50.820 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2024-09-12T03:20:50.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:50.837 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list-id 2024-09-12T03:20:50.893 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:20:50.893 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-09-12T03:20:50.894 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:20:50.894 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-12T03:20:50.894 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-09-12T03:20:50.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:20:50.992 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:50 smithi006 ceph-mon[30991]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:20:50.992 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:50 smithi006 ceph-mon[30991]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T03:20:51.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:20:51.014 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-12T03:20:51.014 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:20:51.015 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-09-12T03:20:51.015 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-12T03:20:51.015 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T03:20:51.015 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-12T03:20:51.015 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-12T03:20:51.015 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-12T03:20:51.015 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-09-12T03:20:51.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:20:51.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:20:51.152 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:20:51.152 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-09-12T03:20:51.152 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set brx.0 up 2024-09-12T03:20:51.152 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T03:20:51.152 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-09-12T03:20:51.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:20:51.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:20:51.231 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:20:51.231 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T03:20:51.250 INFO:teuthology.orchestra.run.smithi006.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:20:51.250 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:20:51.250 DEBUG:teuthology.orchestra.run.smithi006:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:20:51.307 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-12T03:20:51.307 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-12T03:20:51.307 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/43932019-504c-4abf-b8ca-9aefdc54d27f 2024-09-12T03:20:51.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:51.307 DEBUG:teuthology.orchestra.run.smithi006:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/43932019-504c-4abf-b8ca-9aefdc54d27f /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-12T03:20:51.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:50 smithi161 ceph-mon[35485]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:20:51.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:50 smithi161 ceph-mon[35485]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T03:20:51.406 INFO:teuthology.orchestra.run.smithi006.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-12T03:20:51.406 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-09-12T03:20:51.406 INFO:teuthology.orchestra.run.smithi006.stdout:invalid new device string format 2024-09-12T03:20:51.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:50 smithi118 ceph-mon[35048]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:20:51.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:50 smithi118 ceph-mon[35048]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T03:20:51.533 INFO:teuthology.orchestra.run.smithi006.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-12T03:20:51.534 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-09-12T03:20:51.534 INFO:teuthology.orchestra.run.smithi006.stdout:invalid new device string format 2024-09-12T03:20:51.534 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: resolved to: "172.21.15.6:3300,172.21.15.118:3300,172.21.15.161:3300" 2024-09-12T03:20:51.534 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.6:3300,172.21.15.118:3300,172.21.15.161:3300:/volumes/qa/sv_1/43932019-504c-4abf-b8ca-9aefdc54d27f 2024-09-12T03:20:51.534 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=89d99938-70b4-11ef-bcea-c7b262605968" will pass to kernel 2024-09-12T03:20:51.535 INFO:teuthology.orchestra.run.smithi006.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T03:20:51.535 INFO:teuthology.orchestra.run.smithi006.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:20:51.535 INFO:teuthology.orchestra.run.smithi006.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:20:51.535 INFO:teuthology.orchestra.run.smithi006.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:20:51.535 INFO:teuthology.orchestra.run.smithi006.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:20:51.537 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:20:51.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:51.538 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:20:51.565 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:20:51.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:20:51.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:20:51.667 INFO:teuthology.orchestra.run.smithi006.stdout:client24772 2024-09-12T03:20:51.667 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-09-12T03:20:51.667 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:51.685 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:20:51.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:20:51.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:20:51.783 INFO:teuthology.orchestra.run.smithi006.stdout:client24772 2024-09-12T03:20:51.783 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:20:51.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:20:51.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:20:51.852 INFO:teuthology.orchestra.run.smithi006.stdout:client24772 2024-09-12T03:20:51.853 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/89d99938-70b4-11ef-bcea-c7b262605968.client24772/status') 2024-09-12T03:20:51.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:20:51.914 INFO:teuthology.orchestra.run.smithi006.stdout:instance: client.24772 (3)172.21.15.6:0/1667746712 2024-09-12T03:20:51.914 INFO:teuthology.orchestra.run.smithi006.stdout:blocklisted: false 2024-09-12T03:20:51.915 INFO:teuthology.orchestra.run.smithi006.stderr:0+1 records in 2024-09-12T03:20:51.915 INFO:teuthology.orchestra.run.smithi006.stderr:0+1 records out 2024-09-12T03:20:51.915 INFO:teuthology.orchestra.run.smithi006.stderr:70 bytes copied, 2.4019e-05 s, 2.9 MB/s 2024-09-12T03:20:51.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:20:51.922 INFO:teuthology.run_tasks:Running task exec... 2024-09-12T03:20:51.933 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T03:20:51.933 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi006.front.sepia.ceph.com 2024-09-12T03:20:51.933 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-12T03:20:52.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.037+0000 7f8400423640 1 -- 172.21.15.6:0/1703261303 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83f8101c30 msgr2=0x7f83f81020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.037+0000 7f8400423640 1 --2- 172.21.15.6:0/1703261303 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83f8101c30 0x7f83f81020b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009900 tx=0x7f83e802a410 comp rx=0 tx=0).stop 2024-09-12T03:20:52.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.037+0000 7f8400423640 1 -- 172.21.15.6:0/1703261303 shutdown_connections 2024-09-12T03:20:52.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.037+0000 7f8400423640 1 --2- 172.21.15.6:0/1703261303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f81025f0 0x7f83f8108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.037+0000 7f8400423640 1 --2- 172.21.15.6:0/1703261303 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83f8101c30 0x7f83f81020b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.037+0000 7f8400423640 1 --2- 172.21.15.6:0/1703261303 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f83f8100a40 0x7f83f8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.037+0000 7f8400423640 1 -- 172.21.15.6:0/1703261303 >> 172.21.15.6:0/1703261303 conn(0x7f83f80fc1e0 msgr2=0x7f83f80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:52.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.037+0000 7f8400423640 1 -- 172.21.15.6:0/1703261303 shutdown_connections 2024-09-12T03:20:52.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.037+0000 7f8400423640 1 -- 172.21.15.6:0/1703261303 wait complete. 2024-09-12T03:20:52.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.038+0000 7f8400423640 1 Processor -- start 2024-09-12T03:20:52.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.038+0000 7f8400423640 1 -- start start 2024-09-12T03:20:52.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.038+0000 7f8400423640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f83f8100a40 0x7f83f806a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.038+0000 7f8400423640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83f8101c30 0x7f83f819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.038+0000 7f8400423640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f81025f0 0x7f83f819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.038+0000 7f8400423640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f83f810c6f0 con 0x7f83f8101c30 2024-09-12T03:20:52.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.038+0000 7f8400423640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f83f810c570 con 0x7f83f8100a40 2024-09-12T03:20:52.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.038+0000 7f8400423640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f83f810c870 con 0x7f83f81025f0 2024-09-12T03:20:52.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.038+0000 7f83fd9be640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83f8101c30 0x7f83f819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.038+0000 7f83fd9be640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83f8101c30 0x7f83f819cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36298/0 (socket says 172.21.15.6:36298) 2024-09-12T03:20:52.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.038+0000 7f83fd9be640 1 -- 172.21.15.6:0/743909518 learned_addr learned my addr 172.21.15.6:0/743909518 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:52.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.040+0000 7f83fe1bf640 1 --2- 172.21.15.6:0/743909518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f83f8100a40 0x7f83f806a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.040+0000 7f83fe9c0640 1 --2- 172.21.15.6:0/743909518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f81025f0 0x7f83f819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.040+0000 7f83fd9be640 1 -- 172.21.15.6:0/743909518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f81025f0 msgr2=0x7f83f819f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.040+0000 7f83fd9be640 1 --2- 172.21.15.6:0/743909518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f81025f0 0x7f83f819f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.040+0000 7f83fd9be640 1 -- 172.21.15.6:0/743909518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f83f8100a40 msgr2=0x7f83f806a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.040+0000 7f83fd9be640 1 --2- 172.21.15.6:0/743909518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f83f8100a40 0x7f83f806a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.040+0000 7f83fd9be640 1 -- 172.21.15.6:0/743909518 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f819fa30 con 0x7f83f8101c30 2024-09-12T03:20:52.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.040+0000 7f83fd9be640 1 --2- 172.21.15.6:0/743909518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83f8101c30 0x7f83f819cb00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f83e8031440 tx=0x7f83e8032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:52.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.040+0000 7f83ef7fe640 1 -- 172.21.15.6:0/743909518 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e803f030 con 0x7f83f8101c30 2024-09-12T03:20:52.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.040+0000 7f8400423640 1 -- 172.21.15.6:0/743909518 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f819fc60 con 0x7f83f8101c30 2024-09-12T03:20:52.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.040+0000 7f83ef7fe640 1 -- 172.21.15.6:0/743909518 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e8003ce0 con 0x7f83f8101c30 2024-09-12T03:20:52.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.040+0000 7f83ef7fe640 1 -- 172.21.15.6:0/743909518 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8035070 con 0x7f83f8101c30 2024-09-12T03:20:52.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.040+0000 7f8400423640 1 -- 172.21.15.6:0/743909518 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f81a01a0 con 0x7f83f8101c30 2024-09-12T03:20:52.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.040+0000 7f8400423640 1 -- 172.21.15.6:0/743909518 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83f8107d60 con 0x7f83f8101c30 2024-09-12T03:20:52.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.041+0000 7f83ef7fe640 1 -- 172.21.15.6:0/743909518 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f83e8042030 con 0x7f83f8101c30 2024-09-12T03:20:52.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.042+0000 7f83ef7fe640 1 --2- 172.21.15.6:0/743909518 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f83e40a2c30 0x7f83e40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.042+0000 7f83fe1bf640 1 --2- 172.21.15.6:0/743909518 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f83e40a2c30 0x7f83e40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.042+0000 7f83ef7fe640 1 -- 172.21.15.6:0/743909518 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f83e80efa90 con 0x7f83f8101c30 2024-09-12T03:20:52.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.043+0000 7f83fe1bf640 1 --2- 172.21.15.6:0/743909518 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f83e40a2c30 0x7f83e40a50f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f83f8101a90 tx=0x7f83f400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:52.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.045+0000 7f83ef7fe640 1 -- 172.21.15.6:0/743909518 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f83e80bca20 con 0x7f83f8101c30 2024-09-12T03:20:52.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.260+0000 7f8400423640 1 -- 172.21.15.6:0/743909518 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f83f802d7c0 con 0x7f83f8101c30 2024-09-12T03:20:52.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.975+0000 7f83ef7fe640 1 -- 172.21.15.6:0/743909518 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f83e80bc320 con 0x7f83f8101c30 2024-09-12T03:20:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.977+0000 7f8400423640 1 -- 172.21.15.6:0/743909518 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f83e40a2c30 msgr2=0x7f83e40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.977+0000 7f8400423640 1 --2- 172.21.15.6:0/743909518 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f83e40a2c30 0x7f83e40a50f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f83f8101a90 tx=0x7f83f400b390 comp rx=0 tx=0).stop 2024-09-12T03:20:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.977+0000 7f8400423640 1 -- 172.21.15.6:0/743909518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83f8101c30 msgr2=0x7f83f819cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.977+0000 7f8400423640 1 --2- 172.21.15.6:0/743909518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83f8101c30 0x7f83f819cb00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f83e8031440 tx=0x7f83e8032e20 comp rx=0 tx=0).stop 2024-09-12T03:20:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.977+0000 7f8400423640 1 -- 172.21.15.6:0/743909518 shutdown_connections 2024-09-12T03:20:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.977+0000 7f8400423640 1 --2- 172.21.15.6:0/743909518 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f83e40a2c30 0x7f83e40a50f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.977+0000 7f8400423640 1 --2- 172.21.15.6:0/743909518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f81025f0 0x7f83f819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.977+0000 7f8400423640 1 --2- 172.21.15.6:0/743909518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83f8101c30 0x7f83f819cb00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.977+0000 7f8400423640 1 --2- 172.21.15.6:0/743909518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f83f8100a40 0x7f83f806a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.977+0000 7f8400423640 1 -- 172.21.15.6:0/743909518 >> 172.21.15.6:0/743909518 conn(0x7f83f80fc1e0 msgr2=0x7f83f8104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.977+0000 7f8400423640 1 -- 172.21.15.6:0/743909518 shutdown_connections 2024-09-12T03:20:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:52.978+0000 7f8400423640 1 -- 172.21.15.6:0/743909518 wait complete. 2024-09-12T03:20:52.989 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-12T03:20:53.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.097+0000 7f3f8c277640 1 -- 172.21.15.6:0/2784016965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f841025f0 msgr2=0x7f3f84108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.097+0000 7f3f8c277640 1 --2- 172.21.15.6:0/2784016965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f841025f0 0x7f3f84108e80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009900 tx=0x7f3f7c02a410 comp rx=0 tx=0).stop 2024-09-12T03:20:53.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.098+0000 7f3f8c277640 1 -- 172.21.15.6:0/2784016965 shutdown_connections 2024-09-12T03:20:53.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.098+0000 7f3f8c277640 1 --2- 172.21.15.6:0/2784016965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f841025f0 0x7f3f84108e80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.098+0000 7f3f8c277640 1 --2- 172.21.15.6:0/2784016965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f84101c30 0x7f3f841020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.098+0000 7f3f8c277640 1 --2- 172.21.15.6:0/2784016965 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f84100a40 0x7f3f84100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.098+0000 7f3f8c277640 1 -- 172.21.15.6:0/2784016965 >> 172.21.15.6:0/2784016965 conn(0x7f3f840fc1e0 msgr2=0x7f3f840fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:53.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.098+0000 7f3f8c277640 1 -- 172.21.15.6:0/2784016965 shutdown_connections 2024-09-12T03:20:53.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.098+0000 7f3f8c277640 1 -- 172.21.15.6:0/2784016965 wait complete. 2024-09-12T03:20:53.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.098+0000 7f3f8c277640 1 Processor -- start 2024-09-12T03:20:53.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.098+0000 7f3f8c277640 1 -- start start 2024-09-12T03:20:53.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.098+0000 7f3f8c277640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f84100a40 0x7f3f84197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.098+0000 7f3f8c277640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f84101c30 0x7f3f84198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.098+0000 7f3f8c277640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f841025f0 0x7f3f8419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.098+0000 7f3f8c277640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8410c6f0 con 0x7f3f84101c30 2024-09-12T03:20:53.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.098+0000 7f3f8c277640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8410c570 con 0x7f3f841025f0 2024-09-12T03:20:53.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.098+0000 7f3f8c277640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8410c870 con 0x7f3f84100a40 2024-09-12T03:20:53.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.099+0000 7f3f89812640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f84101c30 0x7f3f84198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.099+0000 7f3f89812640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f84101c30 0x7f3f84198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36312/0 (socket says 172.21.15.6:36312) 2024-09-12T03:20:53.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.099+0000 7f3f89812640 1 -- 172.21.15.6:0/3320066178 learned_addr learned my addr 172.21.15.6:0/3320066178 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:53.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.099+0000 7f3f8a013640 1 --2- 172.21.15.6:0/3320066178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f84100a40 0x7f3f84197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.099+0000 7f3f89812640 1 -- 172.21.15.6:0/3320066178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f84100a40 msgr2=0x7f3f84197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.099+0000 7f3f89812640 1 --2- 172.21.15.6:0/3320066178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f84100a40 0x7f3f84197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.099+0000 7f3f89812640 1 -- 172.21.15.6:0/3320066178 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f841025f0 msgr2=0x7f3f8419f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.099+0000 7f3f8a814640 1 --2- 172.21.15.6:0/3320066178 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f841025f0 0x7f3f8419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.099+0000 7f3f89812640 1 --2- 172.21.15.6:0/3320066178 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f841025f0 0x7f3f8419f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.099+0000 7f3f89812640 1 -- 172.21.15.6:0/3320066178 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8419f960 con 0x7f3f84101c30 2024-09-12T03:20:53.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.099+0000 7f3f8a013640 1 --2- 172.21.15.6:0/3320066178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f84100a40 0x7f3f84197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:53.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.099+0000 7f3f89812640 1 --2- 172.21.15.6:0/3320066178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f84101c30 0x7f3f84198290 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3f7400d950 tx=0x7f3f7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:53.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.099+0000 7f3f7b7fe640 1 -- 172.21.15.6:0/3320066178 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74017030 con 0x7f3f84101c30 2024-09-12T03:20:53.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.099+0000 7f3f8c277640 1 -- 172.21.15.6:0/3320066178 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8419fc50 con 0x7f3f84101c30 2024-09-12T03:20:53.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.099+0000 7f3f7b7fe640 1 -- 172.21.15.6:0/3320066178 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f74004020 con 0x7f3f84101c30 2024-09-12T03:20:53.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.099+0000 7f3f7b7fe640 1 -- 172.21.15.6:0/3320066178 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f740047a0 con 0x7f3f84101c30 2024-09-12T03:20:53.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.099+0000 7f3f8c277640 1 -- 172.21.15.6:0/3320066178 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f841a01a0 con 0x7f3f84101c30 2024-09-12T03:20:53.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.100+0000 7f3f8c277640 1 -- 172.21.15.6:0/3320066178 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f841a03d0 con 0x7f3f84101c30 2024-09-12T03:20:53.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.101+0000 7f3f7b7fe640 1 -- 172.21.15.6:0/3320066178 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3f740041d0 con 0x7f3f84101c30 2024-09-12T03:20:53.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.102+0000 7f3f7b7fe640 1 --2- 172.21.15.6:0/3320066178 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3f700a2c30 0x7f3f700a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.103+0000 7f3f8a013640 1 --2- 172.21.15.6:0/3320066178 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3f700a2c30 0x7f3f700a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.103+0000 7f3f7b7fe640 1 -- 172.21.15.6:0/3320066178 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f3f740cf240 con 0x7f3f84101c30 2024-09-12T03:20:53.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.103+0000 7f3f8a013640 1 --2- 172.21.15.6:0/3320066178 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3f700a2c30 0x7f3f700a50f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3f84199220 tx=0x7f3f80009420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:53.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.106+0000 7f3f7b7fe640 1 -- 172.21.15.6:0/3320066178 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3f7409c170 con 0x7f3f84101c30 2024-09-12T03:20:53.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.309+0000 7f3f8c277640 1 -- 172.21.15.6:0/3320066178 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f3f84062630 con 0x7f3f700a2c30 2024-09-12T03:20:53.315 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:52 smithi006 ceph-mon[30991]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:20:53.315 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:52 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/743909518' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T03:20:53.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.329+0000 7f3f7b7fe640 1 -- 172.21.15.6:0/3320066178 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f3f84062630 con 0x7f3f700a2c30 2024-09-12T03:20:53.330 INFO:teuthology.orchestra.run.smithi006.stdout:{} 2024-09-12T03:20:53.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.331+0000 7f3f8c277640 1 -- 172.21.15.6:0/3320066178 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3f700a2c30 msgr2=0x7f3f700a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.331+0000 7f3f8c277640 1 --2- 172.21.15.6:0/3320066178 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3f700a2c30 0x7f3f700a50f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3f84199220 tx=0x7f3f80009420 comp rx=0 tx=0).stop 2024-09-12T03:20:53.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.331+0000 7f3f8c277640 1 -- 172.21.15.6:0/3320066178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f84101c30 msgr2=0x7f3f84198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.331+0000 7f3f8c277640 1 --2- 172.21.15.6:0/3320066178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f84101c30 0x7f3f84198290 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3f7400d950 tx=0x7f3f7400de10 comp rx=0 tx=0).stop 2024-09-12T03:20:53.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.331+0000 7f3f8c277640 1 -- 172.21.15.6:0/3320066178 shutdown_connections 2024-09-12T03:20:53.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.331+0000 7f3f8c277640 1 --2- 172.21.15.6:0/3320066178 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3f700a2c30 0x7f3f700a50f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.331+0000 7f3f8c277640 1 --2- 172.21.15.6:0/3320066178 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f841025f0 0x7f3f8419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.331+0000 7f3f8c277640 1 --2- 172.21.15.6:0/3320066178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f84101c30 0x7f3f84198290 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.331+0000 7f3f8c277640 1 --2- 172.21.15.6:0/3320066178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f84100a40 0x7f3f84197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.331+0000 7f3f8c277640 1 -- 172.21.15.6:0/3320066178 >> 172.21.15.6:0/3320066178 conn(0x7f3f840fc1e0 msgr2=0x7f3f8406a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:53.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.332+0000 7f3f8c277640 1 -- 172.21.15.6:0/3320066178 shutdown_connections 2024-09-12T03:20:53.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.332+0000 7f3f8c277640 1 -- 172.21.15.6:0/3320066178 wait complete. 2024-09-12T03:20:53.342 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-12T03:20:53.353 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-12T03:20:53.847 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:20:53.848 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:52 smithi118 ceph-mon[35048]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:20:53.848 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:52 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/743909518' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T03:20:53.849 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:52 smithi161 ceph-mon[35485]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:20:53.849 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:52 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/743909518' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T03:20:53.849 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:20:53 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:20:53.314+0000 7f8aa508c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:20:53.849 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:20:53 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:20:53.314+0000 7f8aa508c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:20:53.849 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:20:53 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:20:53.314+0000 7f8aa508c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:20:53.849 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:20:53 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:20:53.314+0000 7f8aa508c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:20:53.849 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:20:53 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:20:53.314+0000 7f8aa508c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:20:53.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.958+0000 7fe982c8d640 1 -- 172.21.15.6:0/2277824117 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe97c101c20 msgr2=0x7fe97c1020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.958+0000 7fe982c8d640 1 --2- 172.21.15.6:0/2277824117 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe97c101c20 0x7fe97c1020a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe964009900 tx=0x7fe96402a410 comp rx=0 tx=0).stop 2024-09-12T03:20:53.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.958+0000 7fe982c8d640 1 -- 172.21.15.6:0/2277824117 shutdown_connections 2024-09-12T03:20:53.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.958+0000 7fe982c8d640 1 --2- 172.21.15.6:0/2277824117 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe97c1025e0 0x7fe97c108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.958+0000 7fe982c8d640 1 --2- 172.21.15.6:0/2277824117 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe97c101c20 0x7fe97c1020a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.958+0000 7fe982c8d640 1 --2- 172.21.15.6:0/2277824117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe97c100a30 0x7fe97c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.958+0000 7fe982c8d640 1 -- 172.21.15.6:0/2277824117 >> 172.21.15.6:0/2277824117 conn(0x7fe97c0fc1b0 msgr2=0x7fe97c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:53.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.958+0000 7fe982c8d640 1 -- 172.21.15.6:0/2277824117 shutdown_connections 2024-09-12T03:20:53.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.958+0000 7fe982c8d640 1 -- 172.21.15.6:0/2277824117 wait complete. 2024-09-12T03:20:53.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.959+0000 7fe982c8d640 1 Processor -- start 2024-09-12T03:20:53.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.959+0000 7fe982c8d640 1 -- start start 2024-09-12T03:20:53.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.959+0000 7fe982c8d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe97c100a30 0x7fe97c197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.959+0000 7fe982c8d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe97c101c20 0x7fe97c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.959+0000 7fe982c8d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe97c1025e0 0x7fe97c19f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.959+0000 7fe982c8d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c10d800 con 0x7fe97c100a30 2024-09-12T03:20:53.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.959+0000 7fe982c8d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c10d680 con 0x7fe97c1025e0 2024-09-12T03:20:53.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.959+0000 7fe982c8d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c10d980 con 0x7fe97c101c20 2024-09-12T03:20:53.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.959+0000 7fe980a29640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe97c100a30 0x7fe97c197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.959+0000 7fe980a29640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe97c100a30 0x7fe97c197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36348/0 (socket says 172.21.15.6:36348) 2024-09-12T03:20:53.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.959+0000 7fe980a29640 1 -- 172.21.15.6:0/683310881 learned_addr learned my addr 172.21.15.6:0/683310881 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:53.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.959+0000 7fe97bfff640 1 --2- 172.21.15.6:0/683310881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe97c101c20 0x7fe97c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.959+0000 7fe980a29640 1 -- 172.21.15.6:0/683310881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe97c101c20 msgr2=0x7fe97c198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.959+0000 7fe980a29640 1 --2- 172.21.15.6:0/683310881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe97c101c20 0x7fe97c198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.959+0000 7fe980a29640 1 -- 172.21.15.6:0/683310881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe97c1025e0 msgr2=0x7fe97c19f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.959+0000 7fe980a29640 1 --2- 172.21.15.6:0/683310881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe97c1025e0 0x7fe97c19f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.959+0000 7fe980a29640 1 -- 172.21.15.6:0/683310881 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97c19f9e0 con 0x7fe97c100a30 2024-09-12T03:20:53.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.960+0000 7fe980a29640 1 --2- 172.21.15.6:0/683310881 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe97c100a30 0x7fe97c197dc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00c970 tx=0x7fe96c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:53.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.960+0000 7fe979ffb640 1 -- 172.21.15.6:0/683310881 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c007c10 con 0x7fe97c100a30 2024-09-12T03:20:53.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.960+0000 7fe979ffb640 1 -- 172.21.15.6:0/683310881 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe96c007db0 con 0x7fe97c100a30 2024-09-12T03:20:53.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.960+0000 7fe982c8d640 1 -- 172.21.15.6:0/683310881 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97c19fcd0 con 0x7fe97c100a30 2024-09-12T03:20:53.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.960+0000 7fe979ffb640 1 -- 172.21.15.6:0/683310881 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c00f940 con 0x7fe97c100a30 2024-09-12T03:20:53.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.960+0000 7fe982c8d640 1 -- 172.21.15.6:0/683310881 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe97c1a01a0 con 0x7fe97c100a30 2024-09-12T03:20:53.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.961+0000 7fe982c8d640 1 -- 172.21.15.6:0/683310881 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe97c107d70 con 0x7fe97c100a30 2024-09-12T03:20:53.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.962+0000 7fe979ffb640 1 -- 172.21.15.6:0/683310881 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe96c00fae0 con 0x7fe97c100a30 2024-09-12T03:20:53.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.963+0000 7fe979ffb640 1 --2- 172.21.15.6:0/683310881 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe9680a2b90 0x7fe9680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.963+0000 7fe97bfff640 1 --2- 172.21.15.6:0/683310881 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe9680a2b90 0x7fe9680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.963+0000 7fe979ffb640 1 -- 172.21.15.6:0/683310881 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe96c0cf520 con 0x7fe97c100a30 2024-09-12T03:20:53.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.964+0000 7fe97bfff640 1 --2- 172.21.15.6:0/683310881 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe9680a2b90 0x7fe9680a5050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe964000c00 tx=0x7fe9640315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:53.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:53.968+0000 7fe979ffb640 1 -- 172.21.15.6:0/683310881 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe96c09c4e0 con 0x7fe97c100a30 2024-09-12T03:20:54.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.144+0000 7fe982c8d640 1 -- 172.21.15.6:0/683310881 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe97c062e90 con 0x7fe97c100a30 2024-09-12T03:20:54.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.146+0000 7fe979ffb640 1 -- 172.21.15.6:0/683310881 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22607 (secure 0 0 0) 0x7fe96c09bde0 con 0x7fe97c100a30 2024-09-12T03:20:54.148 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:20:54.148 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":82,"fsid":"89d99938-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:11:29.530545+0000","modified":"2024-09-12T03:19:20.366710+0000","last_up_change":"2024-09-12T03:18:21.292572+0000","last_in_change":"2024-09-12T03:17:58.013800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:14:37.662010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:06.143398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:08.305279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f55d68b-95a0-4ee2-a46f-bc422b38471e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6803","nonce":726729286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6805","nonce":726729286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6809","nonce":726729286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6807","nonce":726729286}]},"public_addr":"172.21.15.6:6803/726729286","cluster_addr":"172.21.15.6:6805/726729286","heartbeat_back_addr":"172.21.15.6:6809/726729286","heartbeat_front_addr":"172.21.15.6:6807/726729286","state":["exists","up"]},{"osd":1,"uuid":"83afdb01-c731-4de1-96c6-3f798549b1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6801","nonce":3377555415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6803","nonce":3377555415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6807","nonce":3377555415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6805","nonce":3377555415}]},"public_addr":"172.21.15.118:6801/3377555415","cluster_addr":"172.21.15.118:6803/3377555415","heartbeat_back_addr":"172.21.15.118:6807/3377555415","heartbeat_front_addr":"172.21.15.118:6805/3377555415","state":["exists","up"]},{"osd":2,"uuid":"aed718b4-b849-403a-b81c-3a140b5c380c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6801","nonce":392541009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6803","nonce":392541009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6807","nonce":392541009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6805","nonce":392541009}]},"public_addr":"172.21.15.161:6801/392541009","cluster_addr":"172.21.15.161:6803/392541009","heartbeat_back_addr":"172.21.15.161:6807/392541009","heartbeat_front_addr":"172.21.15.161:6805/392541009","state":["exists","up"]},{"osd":3,"uuid":"25ced4c1-bf31-402d-8397-d42fd57f615f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6811","nonce":1892039151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6813","nonce":1892039151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6817","nonce":1892039151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6815","nonce":1892039151}]},"public_addr":"172.21.15.6:6811/1892039151","cluster_addr":"172.21.15.6:6813/1892039151","heartbeat_back_addr":"172.21.15.6:6817/1892039151","heartbeat_front_addr":"172.21.15.6:6815/1892039151","state":["exists","up"]},{"osd":4,"uuid":"1d884604-920b-4c35-b41e-d629ad9f9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6809","nonce":150177024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6811","nonce":150177024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6815","nonce":150177024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6813","nonce":150177024}]},"public_addr":"172.21.15.118:6809/150177024","cluster_addr":"172.21.15.118:6811/150177024","heartbeat_back_addr":"172.21.15.118:6815/150177024","heartbeat_front_addr":"172.21.15.118:6813/150177024","state":["exists","up"]},{"osd":5,"uuid":"1d40c0ef-99f6-426f-90c7-27579edc66af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6809","nonce":3758167854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6811","nonce":3758167854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6815","nonce":3758167854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6813","nonce":3758167854}]},"public_addr":"172.21.15.161:6809/3758167854","cluster_addr":"172.21.15.161:6811/3758167854","heartbeat_back_addr":"172.21.15.161:6815/3758167854","heartbeat_front_addr":"172.21.15.161:6813/3758167854","state":["exists","up"]},{"osd":6,"uuid":"9d9f9510-b107-40c2-9a98-9012715944b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6819","nonce":2129224874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6821","nonce":2129224874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6825","nonce":2129224874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6823","nonce":2129224874}]},"public_addr":"172.21.15.6:6819/2129224874","cluster_addr":"172.21.15.6:6821/2129224874","heartbeat_back_addr":"172.21.15.6:6825/2129224874","heartbeat_front_addr":"172.21.15.6:6823/2129224874","state":["exists","up"]},{"osd":7,"uuid":"84635ae1-3240-4954-af14-76bb3aa162e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6817","nonce":269980353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6819","nonce":269980353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6823","nonce":269980353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6821","nonce":269980353}]},"public_addr":"172.21.15.118:6817/269980353","cluster_addr":"172.21.15.118:6819/269980353","heartbeat_back_addr":"172.21.15.118:6823/269980353","heartbeat_front_addr":"172.21.15.118:6821/269980353","state":["exists","up"]},{"osd":8,"uuid":"6fd58e55-1492-4595-8647-823ea3e3c5d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6817","nonce":1783416134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6819","nonce":1783416134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6823","nonce":1783416134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6821","nonce":1783416134}]},"public_addr":"172.21.15.161:6817/1783416134","cluster_addr":"172.21.15.161:6819/1783416134","heartbeat_back_addr":"172.21.15.161:6823/1783416134","heartbeat_front_addr":"172.21.15.161:6821/1783416134","state":["exists","up"]},{"osd":9,"uuid":"90508a6d-a49c-43c6-a600-5bbccd8702ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6827","nonce":2987332365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6829","nonce":2987332365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6833","nonce":2987332365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6831","nonce":2987332365}]},"public_addr":"172.21.15.6:6827/2987332365","cluster_addr":"172.21.15.6:6829/2987332365","heartbeat_back_addr":"172.21.15.6:6833/2987332365","heartbeat_front_addr":"172.21.15.6:6831/2987332365","state":["exists","up"]},{"osd":10,"uuid":"911e86d1-669a-4935-82c0-7da51d141df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6825","nonce":2167146341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6827","nonce":2167146341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6831","nonce":2167146341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6829","nonce":2167146341}]},"public_addr":"172.21.15.118:6825/2167146341","cluster_addr":"172.21.15.118:6827/2167146341","heartbeat_back_addr":"172.21.15.118:6831/2167146341","heartbeat_front_addr":"172.21.15.118:6829/2167146341","state":["exists","up"]},{"osd":11,"uuid":"5a15c6a7-ff43-473c-bb48-0c129c316df5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6825","nonce":2889795796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6827","nonce":2889795796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6831","nonce":2889795796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6829","nonce":2889795796}]},"public_addr":"172.21.15.161:6825/2889795796","cluster_addr":"172.21.15.161:6827/2889795796","heartbeat_back_addr":"172.21.15.161:6831/2889795796","heartbeat_front_addr":"172.21.15.161:6829/2889795796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:42.912205+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:08.410669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:34.299134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:59.093181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:24.116635+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:49.566433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:15.719320+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:40.541017+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:05.178178+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:31.163877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:55.405110+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:19.417156+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6800/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/2620491916":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/4037656392":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/1377116573":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/3062800125":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:6800/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/4145338968":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/825701128":"2024-09-13T03:11:55.244257+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:20:54.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.148+0000 7fe982c8d640 1 -- 172.21.15.6:0/683310881 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe9680a2b90 msgr2=0x7fe9680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.148+0000 7fe982c8d640 1 --2- 172.21.15.6:0/683310881 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe9680a2b90 0x7fe9680a5050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe964000c00 tx=0x7fe9640315d0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.148+0000 7fe982c8d640 1 -- 172.21.15.6:0/683310881 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe97c100a30 msgr2=0x7fe97c197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.148+0000 7fe982c8d640 1 --2- 172.21.15.6:0/683310881 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe97c100a30 0x7fe97c197dc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00c970 tx=0x7fe96c00ce30 comp rx=0 tx=0).stop 2024-09-12T03:20:54.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.148+0000 7fe982c8d640 1 -- 172.21.15.6:0/683310881 shutdown_connections 2024-09-12T03:20:54.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.148+0000 7fe982c8d640 1 --2- 172.21.15.6:0/683310881 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe9680a2b90 0x7fe9680a5050 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.148+0000 7fe982c8d640 1 --2- 172.21.15.6:0/683310881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe97c1025e0 0x7fe97c19f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.148+0000 7fe982c8d640 1 --2- 172.21.15.6:0/683310881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe97c101c20 0x7fe97c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.148+0000 7fe982c8d640 1 --2- 172.21.15.6:0/683310881 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe97c100a30 0x7fe97c197dc0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.148+0000 7fe982c8d640 1 -- 172.21.15.6:0/683310881 >> 172.21.15.6:0/683310881 conn(0x7fe97c0fc1b0 msgr2=0x7fe97c104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:54.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.148+0000 7fe982c8d640 1 -- 172.21.15.6:0/683310881 shutdown_connections 2024-09-12T03:20:54.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.148+0000 7fe982c8d640 1 -- 172.21.15.6:0/683310881 wait complete. 2024-09-12T03:20:54.158 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:53 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/743909518' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T03:20:54.158 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:53 smithi006 ceph-mon[30991]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:20:54.160 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T03:14:37.662010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T03:19:06.143398+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T03:19:08.305279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:20:54.161 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:20:54.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.268+0000 7f732c79f640 1 -- 172.21.15.6:0/88342971 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7324100a00 msgr2=0x7f7324100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.268+0000 7f732c79f640 1 --2- 172.21.15.6:0/88342971 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7324100a00 0x7f7324100e00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f731c009900 tx=0x7f731c02a410 comp rx=0 tx=0).stop 2024-09-12T03:20:54.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.268+0000 7f732c79f640 1 -- 172.21.15.6:0/88342971 shutdown_connections 2024-09-12T03:20:54.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.268+0000 7f732c79f640 1 --2- 172.21.15.6:0/88342971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73241025b0 0x7f7324108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.268+0000 7f732c79f640 1 --2- 172.21.15.6:0/88342971 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7324101bf0 0x7f7324102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.268+0000 7f732c79f640 1 --2- 172.21.15.6:0/88342971 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7324100a00 0x7f7324100e00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.268+0000 7f732c79f640 1 -- 172.21.15.6:0/88342971 >> 172.21.15.6:0/88342971 conn(0x7f73240fc140 msgr2=0x7f73240fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:54.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.268+0000 7f732c79f640 1 -- 172.21.15.6:0/88342971 shutdown_connections 2024-09-12T03:20:54.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.268+0000 7f732c79f640 1 -- 172.21.15.6:0/88342971 wait complete. 2024-09-12T03:20:54.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.268+0000 7f732c79f640 1 Processor -- start 2024-09-12T03:20:54.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.268+0000 7f732c79f640 1 -- start start 2024-09-12T03:20:54.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.268+0000 7f732c79f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7324100a00 0x7f7324197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.268+0000 7f732c79f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7324101bf0 0x7f7324198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f732c79f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73241025b0 0x7f732419f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f732c79f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f732410c660 con 0x7f73241025b0 2024-09-12T03:20:54.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f732c79f640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f732410c4e0 con 0x7f7324101bf0 2024-09-12T03:20:54.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f732c79f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f732410c7e0 con 0x7f7324100a00 2024-09-12T03:20:54.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f732ad3c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73241025b0 0x7f732419f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f732ad3c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73241025b0 0x7f732419f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36374/0 (socket says 172.21.15.6:36374) 2024-09-12T03:20:54.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f732ad3c640 1 -- 172.21.15.6:0/570529672 learned_addr learned my addr 172.21.15.6:0/570529672 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:54.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f732a53b640 1 --2- 172.21.15.6:0/570529672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7324100a00 0x7f7324197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f7329d3a640 1 --2- 172.21.15.6:0/570529672 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7324101bf0 0x7f7324198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f732ad3c640 1 -- 172.21.15.6:0/570529672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7324100a00 msgr2=0x7f7324197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f732ad3c640 1 --2- 172.21.15.6:0/570529672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7324100a00 0x7f7324197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f732ad3c640 1 -- 172.21.15.6:0/570529672 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7324101bf0 msgr2=0x7f7324198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f732ad3c640 1 --2- 172.21.15.6:0/570529672 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7324101bf0 0x7f7324198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f732ad3c640 1 -- 172.21.15.6:0/570529672 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732419faf0 con 0x7f73241025b0 2024-09-12T03:20:54.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f732a53b640 1 --2- 172.21.15.6:0/570529672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7324100a00 0x7f7324197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:54.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f7329d3a640 1 --2- 172.21.15.6:0/570529672 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7324101bf0 0x7f7324198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:54.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f732ad3c640 1 --2- 172.21.15.6:0/570529672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73241025b0 0x7f732419f480 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7314009fd0 tx=0x7f731400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:54.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f73237fe640 1 -- 172.21.15.6:0/570529672 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314017070 con 0x7f73241025b0 2024-09-12T03:20:54.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f73237fe640 1 -- 172.21.15.6:0/570529672 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7314003e50 con 0x7f73241025b0 2024-09-12T03:20:54.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f732c79f640 1 -- 172.21.15.6:0/570529672 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732419fde0 con 0x7f73241025b0 2024-09-12T03:20:54.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f73237fe640 1 -- 172.21.15.6:0/570529672 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73140106c0 con 0x7f73241025b0 2024-09-12T03:20:54.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.269+0000 7f732c79f640 1 -- 172.21.15.6:0/570529672 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73241a0250 con 0x7f73241025b0 2024-09-12T03:20:54.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.270+0000 7f732c79f640 1 -- 172.21.15.6:0/570529672 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73241a04b0 con 0x7f73241025b0 2024-09-12T03:20:54.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.272+0000 7f73237fe640 1 -- 172.21.15.6:0/570529672 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7314003970 con 0x7f73241025b0 2024-09-12T03:20:54.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.272+0000 7f73237fe640 1 --2- 172.21.15.6:0/570529672 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f730c0a2b90 0x7f730c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.273+0000 7f732a53b640 1 --2- 172.21.15.6:0/570529672 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f730c0a2b90 0x7f730c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.273+0000 7f73237fe640 1 -- 172.21.15.6:0/570529672 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f73140ceec0 con 0x7f73241025b0 2024-09-12T03:20:54.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.273+0000 7f732a53b640 1 --2- 172.21.15.6:0/570529672 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f730c0a2b90 0x7f730c0a5050 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f731c006010 tx=0x7f731c032cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:54.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.275+0000 7f73237fe640 1 -- 172.21.15.6:0/570529672 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f731409bdf0 con 0x7f73241025b0 2024-09-12T03:20:54.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:53 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/743909518' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T03:20:54.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:53 smithi161 ceph-mon[35485]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:20:54.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.449+0000 7f732c79f640 1 -- 172.21.15.6:0/570529672 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f73241a0680 con 0x7f73241025b0 2024-09-12T03:20:54.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.450+0000 7f73237fe640 1 -- 172.21.15.6:0/570529672 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f731409b6f0 con 0x7f73241025b0 2024-09-12T03:20:54.450 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-09-12T03:20:54.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.452+0000 7f732c79f640 1 -- 172.21.15.6:0/570529672 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f730c0a2b90 msgr2=0x7f730c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.452+0000 7f732c79f640 1 --2- 172.21.15.6:0/570529672 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f730c0a2b90 0x7f730c0a5050 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f731c006010 tx=0x7f731c032cf0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.452+0000 7f732c79f640 1 -- 172.21.15.6:0/570529672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73241025b0 msgr2=0x7f732419f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.452+0000 7f732c79f640 1 --2- 172.21.15.6:0/570529672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73241025b0 0x7f732419f480 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7314009fd0 tx=0x7f731400b260 comp rx=0 tx=0).stop 2024-09-12T03:20:54.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.452+0000 7f732c79f640 1 -- 172.21.15.6:0/570529672 shutdown_connections 2024-09-12T03:20:54.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.452+0000 7f732c79f640 1 --2- 172.21.15.6:0/570529672 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f730c0a2b90 0x7f730c0a5050 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.452+0000 7f732c79f640 1 --2- 172.21.15.6:0/570529672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73241025b0 0x7f732419f480 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.452+0000 7f732c79f640 1 --2- 172.21.15.6:0/570529672 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7324101bf0 0x7f7324198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.452+0000 7f732c79f640 1 --2- 172.21.15.6:0/570529672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7324100a00 0x7f7324197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.452+0000 7f732c79f640 1 -- 172.21.15.6:0/570529672 >> 172.21.15.6:0/570529672 conn(0x7f73240fc140 msgr2=0x7f7324069330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:54.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.452+0000 7f732c79f640 1 -- 172.21.15.6:0/570529672 shutdown_connections 2024-09-12T03:20:54.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.452+0000 7f732c79f640 1 -- 172.21.15.6:0/570529672 wait complete. 2024-09-12T03:20:54.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:53 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/743909518' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T03:20:54.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:53 smithi118 ceph-mon[35048]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:20:54.463 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T03:20:54.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.569+0000 7fc69cad7640 1 -- 172.21.15.6:0/3853061766 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc698100a10 msgr2=0x7fc698100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.569+0000 7fc69cad7640 1 --2- 172.21.15.6:0/3853061766 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc698100a10 0x7fc698100e10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc690009900 tx=0x7fc69002a410 comp rx=0 tx=0).stop 2024-09-12T03:20:54.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.569+0000 7fc69cad7640 1 -- 172.21.15.6:0/3853061766 shutdown_connections 2024-09-12T03:20:54.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.569+0000 7fc69cad7640 1 --2- 172.21.15.6:0/3853061766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6981025c0 0x7fc698108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.569+0000 7fc69cad7640 1 --2- 172.21.15.6:0/3853061766 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc698101c00 0x7fc698102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.569+0000 7fc69cad7640 1 --2- 172.21.15.6:0/3853061766 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc698100a10 0x7fc698100e10 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.569+0000 7fc69cad7640 1 -- 172.21.15.6:0/3853061766 >> 172.21.15.6:0/3853061766 conn(0x7fc6980fc190 msgr2=0x7fc6980fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:54.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.569+0000 7fc69cad7640 1 -- 172.21.15.6:0/3853061766 shutdown_connections 2024-09-12T03:20:54.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.569+0000 7fc69cad7640 1 -- 172.21.15.6:0/3853061766 wait complete. 2024-09-12T03:20:54.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.570+0000 7fc69cad7640 1 Processor -- start 2024-09-12T03:20:54.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.570+0000 7fc69cad7640 1 -- start start 2024-09-12T03:20:54.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.570+0000 7fc69cad7640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc698100a10 0x7fc698197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.570+0000 7fc69cad7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc698101c00 0x7fc698198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.570+0000 7fc69cad7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6981025c0 0x7fc69819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.570+0000 7fc69cad7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc69810c670 con 0x7fc698101c00 2024-09-12T03:20:54.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.570+0000 7fc69cad7640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fc69810c4f0 con 0x7fc698100a10 2024-09-12T03:20:54.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.570+0000 7fc69cad7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc69810c7f0 con 0x7fc6981025c0 2024-09-12T03:20:54.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.570+0000 7fc695d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc698101c00 0x7fc698198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.571+0000 7fc695d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc698101c00 0x7fc698198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36404/0 (socket says 172.21.15.6:36404) 2024-09-12T03:20:54.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.571+0000 7fc695d9b640 1 -- 172.21.15.6:0/3869151118 learned_addr learned my addr 172.21.15.6:0/3869151118 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:54.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.571+0000 7fc69659c640 1 --2- 172.21.15.6:0/3869151118 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc698100a10 0x7fc698197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.571+0000 7fc696d9d640 1 --2- 172.21.15.6:0/3869151118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6981025c0 0x7fc69819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.571+0000 7fc695d9b640 1 -- 172.21.15.6:0/3869151118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6981025c0 msgr2=0x7fc69819f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.571+0000 7fc695d9b640 1 --2- 172.21.15.6:0/3869151118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6981025c0 0x7fc69819f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.571+0000 7fc695d9b640 1 -- 172.21.15.6:0/3869151118 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc698100a10 msgr2=0x7fc698197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.571+0000 7fc695d9b640 1 --2- 172.21.15.6:0/3869151118 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc698100a10 0x7fc698197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.571+0000 7fc695d9b640 1 -- 172.21.15.6:0/3869151118 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69819fa60 con 0x7fc698101c00 2024-09-12T03:20:54.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.571+0000 7fc695d9b640 1 --2- 172.21.15.6:0/3869151118 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc698101c00 0x7fc698198380 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc68000d950 tx=0x7fc68000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:54.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.571+0000 7fc68f7fe640 1 -- 172.21.15.6:0/3869151118 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680017030 con 0x7fc698101c00 2024-09-12T03:20:54.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.571+0000 7fc68f7fe640 1 -- 172.21.15.6:0/3869151118 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc680004020 con 0x7fc698101c00 2024-09-12T03:20:54.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.571+0000 7fc69cad7640 1 -- 172.21.15.6:0/3869151118 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69819fd50 con 0x7fc698101c00 2024-09-12T03:20:54.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.571+0000 7fc68f7fe640 1 -- 172.21.15.6:0/3869151118 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6800047a0 con 0x7fc698101c00 2024-09-12T03:20:54.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.571+0000 7fc69cad7640 1 -- 172.21.15.6:0/3869151118 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6981a0240 con 0x7fc698101c00 2024-09-12T03:20:54.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.572+0000 7fc69cad7640 1 -- 172.21.15.6:0/3869151118 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6981a04d0 con 0x7fc698101c00 2024-09-12T03:20:54.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.573+0000 7fc68f7fe640 1 -- 172.21.15.6:0/3869151118 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc6800041d0 con 0x7fc698101c00 2024-09-12T03:20:54.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.574+0000 7fc68f7fe640 1 --2- 172.21.15.6:0/3869151118 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc6840a2b90 0x7fc6840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.574+0000 7fc68f7fe640 1 -- 172.21.15.6:0/3869151118 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fc6800ceae0 con 0x7fc698101c00 2024-09-12T03:20:54.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.574+0000 7fc69659c640 1 --2- 172.21.15.6:0/3869151118 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc6840a2b90 0x7fc6840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.575+0000 7fc69659c640 1 --2- 172.21.15.6:0/3869151118 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc6840a2b90 0x7fc6840a5050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc690005ab0 tx=0x7fc690000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:54.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.578+0000 7fc68f7fe640 1 -- 172.21.15.6:0/3869151118 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc68009ba10 con 0x7fc698101c00 2024-09-12T03:20:54.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.753+0000 7fc69cad7640 1 -- 172.21.15.6:0/3869151118 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc6981a06a0 con 0x7fc698101c00 2024-09-12T03:20:54.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.753+0000 7fc68f7fe640 1 -- 172.21.15.6:0/3869151118 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7fc68009b310 con 0x7fc698101c00 2024-09-12T03:20:54.753 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2024-09-12T03:20:54.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.755+0000 7fc69cad7640 1 -- 172.21.15.6:0/3869151118 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc6840a2b90 msgr2=0x7fc6840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.755+0000 7fc69cad7640 1 --2- 172.21.15.6:0/3869151118 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc6840a2b90 0x7fc6840a5050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc690005ab0 tx=0x7fc690000bc0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.755+0000 7fc69cad7640 1 -- 172.21.15.6:0/3869151118 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc698101c00 msgr2=0x7fc698198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.755+0000 7fc69cad7640 1 --2- 172.21.15.6:0/3869151118 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc698101c00 0x7fc698198380 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc68000d950 tx=0x7fc68000de10 comp rx=0 tx=0).stop 2024-09-12T03:20:54.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.755+0000 7fc69cad7640 1 -- 172.21.15.6:0/3869151118 shutdown_connections 2024-09-12T03:20:54.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.755+0000 7fc69cad7640 1 --2- 172.21.15.6:0/3869151118 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc6840a2b90 0x7fc6840a5050 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.755+0000 7fc69cad7640 1 --2- 172.21.15.6:0/3869151118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6981025c0 0x7fc69819f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.755+0000 7fc69cad7640 1 --2- 172.21.15.6:0/3869151118 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc698101c00 0x7fc698198380 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.755+0000 7fc69cad7640 1 --2- 172.21.15.6:0/3869151118 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc698100a10 0x7fc698197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.755+0000 7fc69cad7640 1 -- 172.21.15.6:0/3869151118 >> 172.21.15.6:0/3869151118 conn(0x7fc6980fc190 msgr2=0x7fc69806a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:54.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.755+0000 7fc69cad7640 1 -- 172.21.15.6:0/3869151118 shutdown_connections 2024-09-12T03:20:54.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.755+0000 7fc69cad7640 1 -- 172.21.15.6:0/3869151118 wait complete. 2024-09-12T03:20:54.766 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T03:20:54.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.872+0000 7f5286850640 1 -- 172.21.15.6:0/351551398 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5280102d10 msgr2=0x7f5280103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.872+0000 7f5286850640 1 --2- 172.21.15.6:0/351551398 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5280102d10 0x7f5280103190 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5274009900 tx=0x7f527402a410 comp rx=0 tx=0).stop 2024-09-12T03:20:54.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.873+0000 7f5286850640 1 -- 172.21.15.6:0/351551398 shutdown_connections 2024-09-12T03:20:54.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.873+0000 7f5286850640 1 --2- 172.21.15.6:0/351551398 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f52801036d0 0x7f5280109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.873+0000 7f5286850640 1 --2- 172.21.15.6:0/351551398 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5280102d10 0x7f5280103190 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.873+0000 7f5286850640 1 --2- 172.21.15.6:0/351551398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5280101b20 0x7f5280101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.873+0000 7f5286850640 1 -- 172.21.15.6:0/351551398 >> 172.21.15.6:0/351551398 conn(0x7f52800fd2a0 msgr2=0x7f52800ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:54.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.873+0000 7f5286850640 1 -- 172.21.15.6:0/351551398 shutdown_connections 2024-09-12T03:20:54.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.873+0000 7f5286850640 1 -- 172.21.15.6:0/351551398 wait complete. 2024-09-12T03:20:54.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.873+0000 7f5286850640 1 Processor -- start 2024-09-12T03:20:54.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.873+0000 7f5286850640 1 -- start start 2024-09-12T03:20:54.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.873+0000 7f5286850640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5280101b20 0x7f5280197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.873+0000 7f5286850640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5280102d10 0x7f52801983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.874+0000 7f5286850640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52801036d0 0x7f528019f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.874+0000 7f5286850640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f528010d7a0 con 0x7f52801036d0 2024-09-12T03:20:54.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.874+0000 7f5286850640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f528010d620 con 0x7f5280101b20 2024-09-12T03:20:54.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.874+0000 7f5286850640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f528010d920 con 0x7f5280102d10 2024-09-12T03:20:54.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.874+0000 7f5284ded640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52801036d0 0x7f528019f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.874+0000 7f5284ded640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52801036d0 0x7f528019f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36412/0 (socket says 172.21.15.6:36412) 2024-09-12T03:20:54.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.874+0000 7f5284ded640 1 -- 172.21.15.6:0/1761521627 learned_addr learned my addr 172.21.15.6:0/1761521627 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:54.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.874+0000 7f527f7fe640 1 --2- 172.21.15.6:0/1761521627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5280102d10 0x7f52801983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.874+0000 7f527ffff640 1 --2- 172.21.15.6:0/1761521627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5280101b20 0x7f5280197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.874+0000 7f5284ded640 1 -- 172.21.15.6:0/1761521627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5280102d10 msgr2=0x7f52801983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.874+0000 7f5284ded640 1 --2- 172.21.15.6:0/1761521627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5280102d10 0x7f52801983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.874+0000 7f5284ded640 1 -- 172.21.15.6:0/1761521627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5280101b20 msgr2=0x7f5280197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.874+0000 7f5284ded640 1 --2- 172.21.15.6:0/1761521627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5280101b20 0x7f5280197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.874+0000 7f5284ded640 1 -- 172.21.15.6:0/1761521627 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528019faa0 con 0x7f52801036d0 2024-09-12T03:20:54.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.874+0000 7f5284ded640 1 --2- 172.21.15.6:0/1761521627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52801036d0 0x7f528019f430 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f527800eb10 tx=0x7f527800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:54.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.874+0000 7f527d7fa640 1 -- 172.21.15.6:0/1761521627 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5278004b20 con 0x7f52801036d0 2024-09-12T03:20:54.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.874+0000 7f527d7fa640 1 -- 172.21.15.6:0/1761521627 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5278004cc0 con 0x7f52801036d0 2024-09-12T03:20:54.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.874+0000 7f527d7fa640 1 -- 172.21.15.6:0/1761521627 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5278010760 con 0x7f52801036d0 2024-09-12T03:20:54.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.874+0000 7f5286850640 1 -- 172.21.15.6:0/1761521627 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f528019fd90 con 0x7f52801036d0 2024-09-12T03:20:54.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.874+0000 7f5286850640 1 -- 172.21.15.6:0/1761521627 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52801a0200 con 0x7f52801036d0 2024-09-12T03:20:54.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.875+0000 7f5286850640 1 -- 172.21.15.6:0/1761521627 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5280108f80 con 0x7f52801036d0 2024-09-12T03:20:54.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.876+0000 7f527d7fa640 1 -- 172.21.15.6:0/1761521627 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5278005020 con 0x7f52801036d0 2024-09-12T03:20:54.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.877+0000 7f527d7fa640 1 --2- 172.21.15.6:0/1761521627 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f52680a2b90 0x7f52680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.877+0000 7f527ffff640 1 --2- 172.21.15.6:0/1761521627 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f52680a2b90 0x7f52680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.877+0000 7f527d7fa640 1 -- 172.21.15.6:0/1761521627 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f52780cf6f0 con 0x7f52801036d0 2024-09-12T03:20:54.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.878+0000 7f527ffff640 1 --2- 172.21.15.6:0/1761521627 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f52680a2b90 0x7f52680a5050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5280102b70 tx=0x7f5270005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:54.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:54.881+0000 7f527d7fa640 1 -- 172.21.15.6:0/1761521627 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f527809c680 con 0x7f52801036d0 2024-09-12T03:20:55.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.056+0000 7f5286850640 1 -- 172.21.15.6:0/1761521627 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5280062df0 con 0x7f52801036d0 2024-09-12T03:20:55.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.056+0000 7f527d7fa640 1 -- 172.21.15.6:0/1761521627 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7f527809bf80 con 0x7f52801036d0 2024-09-12T03:20:55.056 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2024-09-12T03:20:55.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.058+0000 7f5286850640 1 -- 172.21.15.6:0/1761521627 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f52680a2b90 msgr2=0x7f52680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.058+0000 7f5286850640 1 --2- 172.21.15.6:0/1761521627 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f52680a2b90 0x7f52680a5050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5280102b70 tx=0x7f5270005f90 comp rx=0 tx=0).stop 2024-09-12T03:20:55.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.058+0000 7f5286850640 1 -- 172.21.15.6:0/1761521627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52801036d0 msgr2=0x7f528019f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.058+0000 7f5286850640 1 --2- 172.21.15.6:0/1761521627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52801036d0 0x7f528019f430 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f527800eb10 tx=0x7f527800c2d0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.058+0000 7f5286850640 1 -- 172.21.15.6:0/1761521627 shutdown_connections 2024-09-12T03:20:55.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.058+0000 7f5286850640 1 --2- 172.21.15.6:0/1761521627 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f52680a2b90 0x7f52680a5050 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.058+0000 7f5286850640 1 --2- 172.21.15.6:0/1761521627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52801036d0 0x7f528019f430 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.058+0000 7f5286850640 1 --2- 172.21.15.6:0/1761521627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5280102d10 0x7f52801983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.058+0000 7f5286850640 1 --2- 172.21.15.6:0/1761521627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5280101b20 0x7f5280197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.058+0000 7f5286850640 1 -- 172.21.15.6:0/1761521627 >> 172.21.15.6:0/1761521627 conn(0x7f52800fd2a0 msgr2=0x7f5280106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:55.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.058+0000 7f5286850640 1 -- 172.21.15.6:0/1761521627 shutdown_connections 2024-09-12T03:20:55.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.058+0000 7f5286850640 1 -- 172.21.15.6:0/1761521627 wait complete. 2024-09-12T03:20:55.068 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:54 smithi006 ceph-mon[30991]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-12T03:20:55.068 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:54 smithi006 ceph-mon[30991]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:55.068 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:54 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/683310881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:20:55.068 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:54 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/570529672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:20:55.068 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:54 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3869151118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:20:55.069 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-12T03:20:55.069 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:20:55.327 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:55.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:54 smithi161 ceph-mon[35485]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-12T03:20:55.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:54 smithi161 ceph-mon[35485]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:55.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:54 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/683310881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:20:55.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:54 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/570529672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:20:55.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:54 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3869151118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:20:55.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:54 smithi118 ceph-mon[35048]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-12T03:20:55.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:54 smithi118 ceph-mon[35048]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:55.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:54 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/683310881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:20:55.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:54 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/570529672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:20:55.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:54 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3869151118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:20:55.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.896+0000 7efff9b6a640 1 -- 172.21.15.6:0/920008733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efff4102790 msgr2=0x7efff41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.896+0000 7efff9b6a640 1 --2- 172.21.15.6:0/920008733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efff4102790 0x7efff41067e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7effdc009900 tx=0x7effdc02a410 comp rx=0 tx=0).stop 2024-09-12T03:20:55.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.896+0000 7efff9b6a640 1 -- 172.21.15.6:0/920008733 shutdown_connections 2024-09-12T03:20:55.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.896+0000 7efff9b6a640 1 --2- 172.21.15.6:0/920008733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efff4106f10 0x7efff410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.896+0000 7efff9b6a640 1 --2- 172.21.15.6:0/920008733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efff4102790 0x7efff41067e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.896+0000 7efff9b6a640 1 --2- 172.21.15.6:0/920008733 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efff4101de0 0x7efff41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.896+0000 7efff9b6a640 1 -- 172.21.15.6:0/920008733 >> 172.21.15.6:0/920008733 conn(0x7efff40fd8c0 msgr2=0x7efff40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:55.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.896+0000 7efff9b6a640 1 -- 172.21.15.6:0/920008733 shutdown_connections 2024-09-12T03:20:55.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.896+0000 7efff9b6a640 1 -- 172.21.15.6:0/920008733 wait complete. 2024-09-12T03:20:55.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.897+0000 7efff9b6a640 1 Processor -- start 2024-09-12T03:20:55.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.897+0000 7efff9b6a640 1 -- start start 2024-09-12T03:20:55.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff9b6a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efff4101de0 0x7efff4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff9b6a640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efff4102790 0x7efff4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff9b6a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efff4106f10 0x7efff419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff9b6a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7efff4111f70 con 0x7efff4106f10 2024-09-12T03:20:55.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff9b6a640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7efff4111df0 con 0x7efff4102790 2024-09-12T03:20:55.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff9b6a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efff41120f0 con 0x7efff4101de0 2024-09-12T03:20:55.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff3fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efff4106f10 0x7efff419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff3fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efff4106f10 0x7efff419cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36426/0 (socket says 172.21.15.6:36426) 2024-09-12T03:20:55.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff3fff640 1 -- 172.21.15.6:0/21357866 learned_addr learned my addr 172.21.15.6:0/21357866 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:55.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff37fe640 1 --2- 172.21.15.6:0/21357866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efff4101de0 0x7efff4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff2ffd640 1 --2- 172.21.15.6:0/21357866 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efff4102790 0x7efff4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff3fff640 1 -- 172.21.15.6:0/21357866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efff4101de0 msgr2=0x7efff4198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff3fff640 1 --2- 172.21.15.6:0/21357866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efff4101de0 0x7efff4198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff3fff640 1 -- 172.21.15.6:0/21357866 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efff4102790 msgr2=0x7efff4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff3fff640 1 --2- 172.21.15.6:0/21357866 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efff4102790 0x7efff4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff3fff640 1 -- 172.21.15.6:0/21357866 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff419d3e0 con 0x7efff4106f10 2024-09-12T03:20:55.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff3fff640 1 --2- 172.21.15.6:0/21357866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efff4106f10 0x7efff419cd00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7effec00cad0 tx=0x7effec00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:55.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff0ff9640 1 -- 172.21.15.6:0/21357866 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec00ae20 con 0x7efff4106f10 2024-09-12T03:20:55.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff0ff9640 1 -- 172.21.15.6:0/21357866 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effec005020 con 0x7efff4106f10 2024-09-12T03:20:55.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff0ff9640 1 -- 172.21.15.6:0/21357866 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec00f6f0 con 0x7efff4106f10 2024-09-12T03:20:55.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff9b6a640 1 -- 172.21.15.6:0/21357866 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff419d6d0 con 0x7efff4106f10 2024-09-12T03:20:55.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.898+0000 7efff9b6a640 1 -- 172.21.15.6:0/21357866 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff41a4fb0 con 0x7efff4106f10 2024-09-12T03:20:55.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.900+0000 7efff9b6a640 1 -- 172.21.15.6:0/21357866 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efff41021c0 con 0x7efff4106f10 2024-09-12T03:20:55.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.901+0000 7efff0ff9640 1 -- 172.21.15.6:0/21357866 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7effec0051d0 con 0x7efff4106f10 2024-09-12T03:20:55.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.902+0000 7efff0ff9640 1 --2- 172.21.15.6:0/21357866 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7effe00a2c00 0x7effe00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.902+0000 7efff37fe640 1 --2- 172.21.15.6:0/21357866 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7effe00a2c00 0x7effe00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.902+0000 7efff0ff9640 1 -- 172.21.15.6:0/21357866 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7effec0cf360 con 0x7efff4106f10 2024-09-12T03:20:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.902+0000 7efff37fe640 1 --2- 172.21.15.6:0/21357866 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7effe00a2c00 0x7effe00a50c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7effe4005fd0 tx=0x7effe400b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:55.906+0000 7efff0ff9640 1 -- 172.21.15.6:0/21357866 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7effec014030 con 0x7efff4106f10 2024-09-12T03:20:56.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:56.010+0000 7efff0ff9640 1 -- 172.21.15.6:0/21357866 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7effec098260 con 0x7efff4106f10 2024-09-12T03:20:56.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:55 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1761521627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:20:56.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:56.123+0000 7efff9b6a640 1 -- 172.21.15.6:0/21357866 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efff410a150 con 0x7efff4106f10 2024-09-12T03:20:56.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:56.124+0000 7efff0ff9640 1 -- 172.21.15.6:0/21357866 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11908 (secure 0 0 0) 0x7effec0c1bb0 con 0x7efff4106f10 2024-09-12T03:20:56.124 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:20:56.125 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"btime":"2024-09-12T03:20:52:974598+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:20:52.261388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24463]},"id":1}]} 2024-09-12T03:20:56.125 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 25 2024-09-12T03:20:56.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:56.126+0000 7efff9b6a640 1 -- 172.21.15.6:0/21357866 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7effe00a2c00 msgr2=0x7effe00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:56.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:56.126+0000 7efff9b6a640 1 --2- 172.21.15.6:0/21357866 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7effe00a2c00 0x7effe00a50c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7effe4005fd0 tx=0x7effe400b590 comp rx=0 tx=0).stop 2024-09-12T03:20:56.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:56.126+0000 7efff9b6a640 1 -- 172.21.15.6:0/21357866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efff4106f10 msgr2=0x7efff419cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:56.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:56.126+0000 7efff9b6a640 1 --2- 172.21.15.6:0/21357866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efff4106f10 0x7efff419cd00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7effec00cad0 tx=0x7effec00cf90 comp rx=0 tx=0).stop 2024-09-12T03:20:56.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:56.126+0000 7efff9b6a640 1 -- 172.21.15.6:0/21357866 shutdown_connections 2024-09-12T03:20:56.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:56.126+0000 7efff9b6a640 1 --2- 172.21.15.6:0/21357866 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7effe00a2c00 0x7effe00a50c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:56.126+0000 7efff9b6a640 1 --2- 172.21.15.6:0/21357866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efff4106f10 0x7efff419cd00 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:56.126+0000 7efff9b6a640 1 --2- 172.21.15.6:0/21357866 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efff4102790 0x7efff4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:56.126+0000 7efff9b6a640 1 --2- 172.21.15.6:0/21357866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efff4101de0 0x7efff4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:56.126+0000 7efff9b6a640 1 -- 172.21.15.6:0/21357866 >> 172.21.15.6:0/21357866 conn(0x7efff40fd8c0 msgr2=0x7efff40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:56.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:56.127+0000 7efff9b6a640 1 -- 172.21.15.6:0/21357866 shutdown_connections 2024-09-12T03:20:56.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:56.127+0000 7efff9b6a640 1 -- 172.21.15.6:0/21357866 wait complete. 2024-09-12T03:20:56.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:56 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1761521627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:20:56.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:55 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1761521627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:20:57.013 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-12T03:20:57.013 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-12T03:20:57.013 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:20:57.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.120+0000 7f05618dd640 1 -- 172.21.15.6:0/1200223736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f055c1078b0 msgr2=0x7f055c109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.120+0000 7f05618dd640 1 --2- 172.21.15.6:0/1200223736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f055c1078b0 0x7f055c109ca0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0554009900 tx=0x7f055402a410 comp rx=0 tx=0).stop 2024-09-12T03:20:57.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.120+0000 7f05618dd640 1 -- 172.21.15.6:0/1200223736 shutdown_connections 2024-09-12T03:20:57.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.120+0000 7f05618dd640 1 --2- 172.21.15.6:0/1200223736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f055c1078b0 0x7f055c109ca0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.120+0000 7f05618dd640 1 --2- 172.21.15.6:0/1200223736 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f055c104f80 0x7f055c107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.120+0000 7f05618dd640 1 --2- 172.21.15.6:0/1200223736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f055c102650 0x7f055c104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.120+0000 7f05618dd640 1 -- 172.21.15.6:0/1200223736 >> 172.21.15.6:0/1200223736 conn(0x7f055c0fc190 msgr2=0x7f055c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:57.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.120+0000 7f05618dd640 1 -- 172.21.15.6:0/1200223736 shutdown_connections 2024-09-12T03:20:57.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.120+0000 7f05618dd640 1 -- 172.21.15.6:0/1200223736 wait complete. 2024-09-12T03:20:57.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.121+0000 7f05618dd640 1 Processor -- start 2024-09-12T03:20:57.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.121+0000 7f05618dd640 1 -- start start 2024-09-12T03:20:57.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.121+0000 7f05618dd640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f055c102650 0x7f055c101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.121+0000 7f05618dd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f055c104f80 0x7f055c100600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.121+0000 7f05618dd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f055c1078b0 0x7f055c100b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.121+0000 7f05618dd640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f055c10d7d0 con 0x7f055c104f80 2024-09-12T03:20:57.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.121+0000 7f05618dd640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f055c10d650 con 0x7f055c102650 2024-09-12T03:20:57.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.121+0000 7f05618dd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f055c10d950 con 0x7f055c1078b0 2024-09-12T03:20:57.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.121+0000 7f055a7fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f055c104f80 0x7f055c100600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.121+0000 7f055a7fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f055c104f80 0x7f055c100600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36452/0 (socket says 172.21.15.6:36452) 2024-09-12T03:20:57.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.121+0000 7f055a7fc640 1 -- 172.21.15.6:0/360631215 learned_addr learned my addr 172.21.15.6:0/360631215 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:57.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.121+0000 7f055b7fe640 1 --2- 172.21.15.6:0/360631215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f055c1078b0 0x7f055c100b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.122+0000 7f055affd640 1 --2- 172.21.15.6:0/360631215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f055c102650 0x7f055c101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.122+0000 7f055a7fc640 1 -- 172.21.15.6:0/360631215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f055c1078b0 msgr2=0x7f055c100b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.122+0000 7f055a7fc640 1 --2- 172.21.15.6:0/360631215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f055c1078b0 0x7f055c100b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.122+0000 7f055a7fc640 1 -- 172.21.15.6:0/360631215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f055c102650 msgr2=0x7f055c101fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.122+0000 7f055a7fc640 1 --2- 172.21.15.6:0/360631215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f055c102650 0x7f055c101fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.122+0000 7f055a7fc640 1 -- 172.21.15.6:0/360631215 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055c1013d0 con 0x7f055c104f80 2024-09-12T03:20:57.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.122+0000 7f055b7fe640 1 --2- 172.21.15.6:0/360631215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f055c1078b0 0x7f055c100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:57.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.122+0000 7f055a7fc640 1 --2- 172.21.15.6:0/360631215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f055c104f80 0x7f055c100600 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f055000c970 tx=0x7f055000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:57.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.122+0000 7f05608db640 1 -- 172.21.15.6:0/360631215 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550007bb0 con 0x7f055c104f80 2024-09-12T03:20:57.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.122+0000 7f05608db640 1 -- 172.21.15.6:0/360631215 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0550007d50 con 0x7f055c104f80 2024-09-12T03:20:57.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.122+0000 7f05618dd640 1 -- 172.21.15.6:0/360631215 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055c1a1fa0 con 0x7f055c104f80 2024-09-12T03:20:57.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.122+0000 7f05608db640 1 -- 172.21.15.6:0/360631215 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055000f450 con 0x7f055c104f80 2024-09-12T03:20:57.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.122+0000 7f05618dd640 1 -- 172.21.15.6:0/360631215 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f055c1a24b0 con 0x7f055c104f80 2024-09-12T03:20:57.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.123+0000 7f05618dd640 1 -- 172.21.15.6:0/360631215 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f055c105360 con 0x7f055c104f80 2024-09-12T03:20:57.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.124+0000 7f05608db640 1 -- 172.21.15.6:0/360631215 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f05500046c0 con 0x7f055c104f80 2024-09-12T03:20:57.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.125+0000 7f05608db640 1 --2- 172.21.15.6:0/360631215 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f05440a2ca0 0x7f05440a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.125+0000 7f05608db640 1 -- 172.21.15.6:0/360631215 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f05500cf290 con 0x7f055c104f80 2024-09-12T03:20:57.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.125+0000 7f055affd640 1 --2- 172.21.15.6:0/360631215 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f05440a2ca0 0x7f05440a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.125+0000 7f055affd640 1 --2- 172.21.15.6:0/360631215 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f05440a2ca0 0x7f05440a5160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f054c005fd0 tx=0x7f054c009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:57.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.129+0000 7f05608db640 1 -- 172.21.15.6:0/360631215 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0550014030 con 0x7f055c104f80 2024-09-12T03:20:57.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.303+0000 7f05618dd640 1 -- 172.21.15.6:0/360631215 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f055c0626f0 con 0x7f05440a2ca0 2024-09-12T03:20:57.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.310+0000 7f05608db640 1 -- 172.21.15.6:0/360631215 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258918 (secure 0 0 0) 0x7f055c0626f0 con 0x7f05440a2ca0 2024-09-12T03:20:57.313 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:20:57.314 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-09-12T03:20:57.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.313+0000 7f05618dd640 1 -- 172.21.15.6:0/360631215 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f05440a2ca0 msgr2=0x7f05440a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.313+0000 7f05618dd640 1 --2- 172.21.15.6:0/360631215 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f05440a2ca0 0x7f05440a5160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f054c005fd0 tx=0x7f054c009870 comp rx=0 tx=0).stop 2024-09-12T03:20:57.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.313+0000 7f05618dd640 1 -- 172.21.15.6:0/360631215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f055c104f80 msgr2=0x7f055c100600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.313+0000 7f05618dd640 1 --2- 172.21.15.6:0/360631215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f055c104f80 0x7f055c100600 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f055000c970 tx=0x7f055000ce30 comp rx=0 tx=0).stop 2024-09-12T03:20:57.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.313+0000 7f05618dd640 1 -- 172.21.15.6:0/360631215 shutdown_connections 2024-09-12T03:20:57.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.313+0000 7f05618dd640 1 --2- 172.21.15.6:0/360631215 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f05440a2ca0 0x7f05440a5160 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.313+0000 7f05618dd640 1 --2- 172.21.15.6:0/360631215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f055c1078b0 0x7f055c100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.313+0000 7f05618dd640 1 --2- 172.21.15.6:0/360631215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f055c104f80 0x7f055c100600 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.313+0000 7f05618dd640 1 --2- 172.21.15.6:0/360631215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f055c102650 0x7f055c101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.313+0000 7f05618dd640 1 -- 172.21.15.6:0/360631215 >> 172.21.15.6:0/360631215 conn(0x7f055c0fc190 msgr2=0x7f055c102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:57.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.313+0000 7f05618dd640 1 -- 172.21.15.6:0/360631215 shutdown_connections 2024-09-12T03:20:57.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.313+0000 7f05618dd640 1 -- 172.21.15.6:0/360631215 wait complete. 2024-09-12T03:20:57.323 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:57 smithi006 ceph-mon[30991]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:20:57.323 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:57 smithi006 ceph-mon[30991]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-12T03:20:57.323 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:57 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/21357866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:57.324 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-12T03:20:57.264063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656161,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":727,"num_read_kb":476,"num_write":362,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":790,"ondisk_log_size":790,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382596,"kb_used_data":12548,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559228,"statfs":{"total":1151940427776,"available":1151548649472,"internally_reserved":0,"allocated":12849152,"data_stored":7172088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1792,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":36,"num_write":15,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002802"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265565+0000","last_change":"2024-09-12T03:19:09.343829+0000","last_active":"2024-09-12T03:20:49.265565+0000","last_peered":"2024-09-12T03:20:49.265565+0000","last_clean":"2024-09-12T03:20:49.265565+0000","last_became_active":"2024-09-12T03:19:09.343228+0000","last_became_peered":"2024-09-12T03:19:09.343228+0000","last_unstale":"2024-09-12T03:20:49.265565+0000","last_undegraded":"2024-09-12T03:20:49.265565+0000","last_fullsized":"2024-09-12T03:20:49.265565+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:08.720934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.237141+0000","last_change":"2024-09-12T03:19:09.331829+0000","last_active":"2024-09-12T03:20:49.237141+0000","last_peered":"2024-09-12T03:20:49.237141+0000","last_clean":"2024-09-12T03:20:49.237141+0000","last_became_active":"2024-09-12T03:19:09.331276+0000","last_became_peered":"2024-09-12T03:19:09.331276+0000","last_unstale":"2024-09-12T03:20:49.237141+0000","last_undegraded":"2024-09-12T03:20:49.237141+0000","last_fullsized":"2024-09-12T03:20:49.237141+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:48.672085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.263951+0000","last_change":"2024-09-12T03:19:09.338120+0000","last_active":"2024-09-12T03:20:49.263951+0000","last_peered":"2024-09-12T03:20:49.263951+0000","last_clean":"2024-09-12T03:20:49.263951+0000","last_became_active":"2024-09-12T03:19:09.337773+0000","last_became_peered":"2024-09-12T03:19:09.337773+0000","last_unstale":"2024-09-12T03:20:49.263951+0000","last_undegraded":"2024-09-12T03:20:49.263951+0000","last_fullsized":"2024-09-12T03:20:49.263951+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:13.716047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254592+0000","last_change":"2024-09-12T03:19:09.339404+0000","last_active":"2024-09-12T03:20:49.254592+0000","last_peered":"2024-09-12T03:20:49.254592+0000","last_clean":"2024-09-12T03:20:49.254592+0000","last_became_active":"2024-09-12T03:19:09.339105+0000","last_became_peered":"2024-09-12T03:19:09.339105+0000","last_unstale":"2024-09-12T03:20:49.254592+0000","last_undegraded":"2024-09-12T03:20:49.254592+0000","last_fullsized":"2024-09-12T03:20:49.254592+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:37.930963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.242261+0000","last_change":"2024-09-12T03:19:09.326285+0000","last_active":"2024-09-12T03:20:49.242261+0000","last_peered":"2024-09-12T03:20:49.242261+0000","last_clean":"2024-09-12T03:20:49.242261+0000","last_became_active":"2024-09-12T03:19:09.325852+0000","last_became_peered":"2024-09-12T03:19:09.325852+0000","last_unstale":"2024-09-12T03:20:49.242261+0000","last_undegraded":"2024-09-12T03:20:49.242261+0000","last_fullsized":"2024-09-12T03:20:49.242261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:44:25.387014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.249918+0000","last_change":"2024-09-12T03:19:20.376197+0000","last_active":"2024-09-12T03:20:49.249918+0000","last_peered":"2024-09-12T03:20:49.249918+0000","last_clean":"2024-09-12T03:20:49.249918+0000","last_became_active":"2024-09-12T03:19:20.375744+0000","last_became_peered":"2024-09-12T03:19:20.375744+0000","last_unstale":"2024-09-12T03:20:49.249918+0000","last_undegraded":"2024-09-12T03:20:49.249918+0000","last_fullsized":"2024-09-12T03:20:49.249918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:08.663121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.245988+0000","last_change":"2024-09-12T03:19:09.335284+0000","last_active":"2024-09-12T03:20:49.245988+0000","last_peered":"2024-09-12T03:20:49.245988+0000","last_clean":"2024-09-12T03:20:49.245988+0000","last_became_active":"2024-09-12T03:19:09.334948+0000","last_became_peered":"2024-09-12T03:19:09.334948+0000","last_unstale":"2024-09-12T03:20:49.245988+0000","last_undegraded":"2024-09-12T03:20:49.245988+0000","last_fullsized":"2024-09-12T03:20:49.245988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:32.778433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250337+0000","last_change":"2024-09-12T03:19:09.338398+0000","last_active":"2024-09-12T03:20:49.250337+0000","last_peered":"2024-09-12T03:20:49.250337+0000","last_clean":"2024-09-12T03:20:49.250337+0000","last_became_active":"2024-09-12T03:19:09.337750+0000","last_became_peered":"2024-09-12T03:19:09.337750+0000","last_unstale":"2024-09-12T03:20:49.250337+0000","last_undegraded":"2024-09-12T03:20:49.250337+0000","last_fullsized":"2024-09-12T03:20:49.250337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:07.542322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265597+0000","last_change":"2024-09-12T03:19:09.340703+0000","last_active":"2024-09-12T03:20:49.265597+0000","last_peered":"2024-09-12T03:20:49.265597+0000","last_clean":"2024-09-12T03:20:49.265597+0000","last_became_active":"2024-09-12T03:19:09.340386+0000","last_became_peered":"2024-09-12T03:19:09.340386+0000","last_unstale":"2024-09-12T03:20:49.265597+0000","last_undegraded":"2024-09-12T03:20:49.265597+0000","last_fullsized":"2024-09-12T03:20:49.265597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:19.990894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248198+0000","last_change":"2024-09-12T03:19:09.341489+0000","last_active":"2024-09-12T03:20:49.248198+0000","last_peered":"2024-09-12T03:20:49.248198+0000","last_clean":"2024-09-12T03:20:49.248198+0000","last_became_active":"2024-09-12T03:19:09.341183+0000","last_became_peered":"2024-09-12T03:19:09.341183+0000","last_unstale":"2024-09-12T03:20:49.248198+0000","last_undegraded":"2024-09-12T03:20:49.248198+0000","last_fullsized":"2024-09-12T03:20:49.248198+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:05.688402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.255447+0000","last_change":"2024-09-12T03:19:09.339289+0000","last_active":"2024-09-12T03:20:49.255447+0000","last_peered":"2024-09-12T03:20:49.255447+0000","last_clean":"2024-09-12T03:20:49.255447+0000","last_became_active":"2024-09-12T03:19:09.338466+0000","last_became_peered":"2024-09-12T03:19:09.338466+0000","last_unstale":"2024-09-12T03:20:49.255447+0000","last_undegraded":"2024-09-12T03:20:49.255447+0000","last_fullsized":"2024-09-12T03:20:49.255447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:47:13.070057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254364+0000","last_change":"2024-09-12T03:19:09.337754+0000","last_active":"2024-09-12T03:20:49.254364+0000","last_peered":"2024-09-12T03:20:49.254364+0000","last_clean":"2024-09-12T03:20:49.254364+0000","last_became_active":"2024-09-12T03:19:09.332789+0000","last_became_peered":"2024-09-12T03:19:09.332789+0000","last_unstale":"2024-09-12T03:20:49.254364+0000","last_undegraded":"2024-09-12T03:20:49.254364+0000","last_fullsized":"2024-09-12T03:20:49.254364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:34.622136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250957+0000","last_change":"2024-09-12T03:19:09.333577+0000","last_active":"2024-09-12T03:20:49.250957+0000","last_peered":"2024-09-12T03:20:49.250957+0000","last_clean":"2024-09-12T03:20:49.250957+0000","last_became_active":"2024-09-12T03:19:09.333234+0000","last_became_peered":"2024-09-12T03:19:09.333234+0000","last_unstale":"2024-09-12T03:20:49.250957+0000","last_undegraded":"2024-09-12T03:20:49.250957+0000","last_fullsized":"2024-09-12T03:20:49.250957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:22.604640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.237367+0000","last_change":"2024-09-12T03:19:09.331607+0000","last_active":"2024-09-12T03:20:49.237367+0000","last_peered":"2024-09-12T03:20:49.237367+0000","last_clean":"2024-09-12T03:20:49.237367+0000","last_became_active":"2024-09-12T03:19:09.331253+0000","last_became_peered":"2024-09-12T03:19:09.331253+0000","last_unstale":"2024-09-12T03:20:49.237367+0000","last_undegraded":"2024-09-12T03:20:49.237367+0000","last_fullsized":"2024-09-12T03:20:49.237367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:11.969265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256379+0000","last_change":"2024-09-12T03:19:09.340768+0000","last_active":"2024-09-12T03:20:49.256379+0000","last_peered":"2024-09-12T03:20:49.256379+0000","last_clean":"2024-09-12T03:20:49.256379+0000","last_became_active":"2024-09-12T03:19:09.340437+0000","last_became_peered":"2024-09-12T03:19:09.340437+0000","last_unstale":"2024-09-12T03:20:49.256379+0000","last_undegraded":"2024-09-12T03:20:49.256379+0000","last_fullsized":"2024-09-12T03:20:49.256379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:32.645691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.263546+0000","last_change":"2024-09-12T03:19:09.333829+0000","last_active":"2024-09-12T03:20:49.263546+0000","last_peered":"2024-09-12T03:20:49.263546+0000","last_clean":"2024-09-12T03:20:49.263546+0000","last_became_active":"2024-09-12T03:19:09.333462+0000","last_became_peered":"2024-09-12T03:19:09.333462+0000","last_unstale":"2024-09-12T03:20:49.263546+0000","last_undegraded":"2024-09-12T03:20:49.263546+0000","last_fullsized":"2024-09-12T03:20:49.263546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:18:57.586836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265659+0000","last_change":"2024-09-12T03:19:09.344068+0000","last_active":"2024-09-12T03:20:49.265659+0000","last_peered":"2024-09-12T03:20:49.265659+0000","last_clean":"2024-09-12T03:20:49.265659+0000","last_became_active":"2024-09-12T03:19:09.343751+0000","last_became_peered":"2024-09-12T03:19:09.343751+0000","last_unstale":"2024-09-12T03:20:49.265659+0000","last_undegraded":"2024-09-12T03:20:49.265659+0000","last_fullsized":"2024-09-12T03:20:49.265659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:52.425547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.249413+0000","last_change":"2024-09-12T03:19:09.330828+0000","last_active":"2024-09-12T03:20:49.249413+0000","last_peered":"2024-09-12T03:20:49.249413+0000","last_clean":"2024-09-12T03:20:49.249413+0000","last_became_active":"2024-09-12T03:19:09.330497+0000","last_became_peered":"2024-09-12T03:19:09.330497+0000","last_unstale":"2024-09-12T03:20:49.249413+0000","last_undegraded":"2024-09-12T03:20:49.249413+0000","last_fullsized":"2024-09-12T03:20:49.249413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:21:20.064177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.245548+0000","last_change":"2024-09-12T03:19:09.344403+0000","last_active":"2024-09-12T03:20:49.245548+0000","last_peered":"2024-09-12T03:20:49.245548+0000","last_clean":"2024-09-12T03:20:49.245548+0000","last_became_active":"2024-09-12T03:19:09.343946+0000","last_became_peered":"2024-09-12T03:19:09.343946+0000","last_unstale":"2024-09-12T03:20:49.245548+0000","last_undegraded":"2024-09-12T03:20:49.245548+0000","last_fullsized":"2024-09-12T03:20:49.245548+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:48.386117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248484+0000","last_change":"2024-09-12T03:19:09.341481+0000","last_active":"2024-09-12T03:20:49.248484+0000","last_peered":"2024-09-12T03:20:49.248484+0000","last_clean":"2024-09-12T03:20:49.248484+0000","last_became_active":"2024-09-12T03:19:09.341142+0000","last_became_peered":"2024-09-12T03:19:09.341142+0000","last_unstale":"2024-09-12T03:20:49.248484+0000","last_undegraded":"2024-09-12T03:20:49.248484+0000","last_fullsized":"2024-09-12T03:20:49.248484+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:42.371221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.249677+0000","last_change":"2024-09-12T03:19:09.330621+0000","last_active":"2024-09-12T03:20:49.249677+0000","last_peered":"2024-09-12T03:20:49.249677+0000","last_clean":"2024-09-12T03:20:49.249677+0000","last_became_active":"2024-09-12T03:19:09.330169+0000","last_became_peered":"2024-09-12T03:19:09.330169+0000","last_unstale":"2024-09-12T03:20:49.249677+0000","last_undegraded":"2024-09-12T03:20:49.249677+0000","last_fullsized":"2024-09-12T03:20:49.249677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:16.634141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.264220+0000","last_change":"2024-09-12T03:19:09.332817+0000","last_active":"2024-09-12T03:20:49.264220+0000","last_peered":"2024-09-12T03:20:49.264220+0000","last_clean":"2024-09-12T03:20:49.264220+0000","last_became_active":"2024-09-12T03:19:09.332236+0000","last_became_peered":"2024-09-12T03:19:09.332236+0000","last_unstale":"2024-09-12T03:20:49.264220+0000","last_undegraded":"2024-09-12T03:20:49.264220+0000","last_fullsized":"2024-09-12T03:20:49.264220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:28.024294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252477+0000","last_change":"2024-09-12T03:19:09.337910+0000","last_active":"2024-09-12T03:20:49.252477+0000","last_peered":"2024-09-12T03:20:49.252477+0000","last_clean":"2024-09-12T03:20:49.252477+0000","last_became_active":"2024-09-12T03:19:09.337176+0000","last_became_peered":"2024-09-12T03:19:09.337176+0000","last_unstale":"2024-09-12T03:20:49.252477+0000","last_undegraded":"2024-09-12T03:20:49.252477+0000","last_fullsized":"2024-09-12T03:20:49.252477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:53.795898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.239957+0000","last_change":"2024-09-12T03:19:09.337902+0000","last_active":"2024-09-12T03:20:49.239957+0000","last_peered":"2024-09-12T03:20:49.239957+0000","last_clean":"2024-09-12T03:20:49.239957+0000","last_became_active":"2024-09-12T03:19:09.337567+0000","last_became_peered":"2024-09-12T03:19:09.337567+0000","last_unstale":"2024-09-12T03:20:49.239957+0000","last_undegraded":"2024-09-12T03:20:49.239957+0000","last_fullsized":"2024-09-12T03:20:49.239957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:08.471313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248557+0000","last_change":"2024-09-12T03:19:09.339850+0000","last_active":"2024-09-12T03:20:49.248557+0000","last_peered":"2024-09-12T03:20:49.248557+0000","last_clean":"2024-09-12T03:20:49.248557+0000","last_became_active":"2024-09-12T03:19:09.339563+0000","last_became_peered":"2024-09-12T03:19:09.339563+0000","last_unstale":"2024-09-12T03:20:49.248557+0000","last_undegraded":"2024-09-12T03:20:49.248557+0000","last_fullsized":"2024-09-12T03:20:49.248557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:27.473351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.247769+0000","last_change":"2024-09-12T03:19:09.330555+0000","last_active":"2024-09-12T03:20:49.247769+0000","last_peered":"2024-09-12T03:20:49.247769+0000","last_clean":"2024-09-12T03:20:49.247769+0000","last_became_active":"2024-09-12T03:19:09.330183+0000","last_became_peered":"2024-09-12T03:19:09.330183+0000","last_unstale":"2024-09-12T03:20:49.247769+0000","last_undegraded":"2024-09-12T03:20:49.247769+0000","last_fullsized":"2024-09-12T03:20:49.247769+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:39.334116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252190+0000","last_change":"2024-09-12T03:19:09.338601+0000","last_active":"2024-09-12T03:20:49.252190+0000","last_peered":"2024-09-12T03:20:49.252190+0000","last_clean":"2024-09-12T03:20:49.252190+0000","last_became_active":"2024-09-12T03:19:09.337903+0000","last_became_peered":"2024-09-12T03:19:09.337903+0000","last_unstale":"2024-09-12T03:20:49.252190+0000","last_undegraded":"2024-09-12T03:20:49.252190+0000","last_fullsized":"2024-09-12T03:20:49.252190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:17.810139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250990+0000","last_change":"2024-09-12T03:19:09.326629+0000","last_active":"2024-09-12T03:20:49.250990+0000","last_peered":"2024-09-12T03:20:49.250990+0000","last_clean":"2024-09-12T03:20:49.250990+0000","last_became_active":"2024-09-12T03:19:09.326333+0000","last_became_peered":"2024-09-12T03:19:09.326333+0000","last_unstale":"2024-09-12T03:20:49.250990+0000","last_undegraded":"2024-09-12T03:20:49.250990+0000","last_fullsized":"2024-09-12T03:20:49.250990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:20.807593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256412+0000","last_change":"2024-09-12T03:19:09.339919+0000","last_active":"2024-09-12T03:20:49.256412+0000","last_peered":"2024-09-12T03:20:49.256412+0000","last_clean":"2024-09-12T03:20:49.256412+0000","last_became_active":"2024-09-12T03:19:09.339618+0000","last_became_peered":"2024-09-12T03:19:09.339618+0000","last_unstale":"2024-09-12T03:20:49.256412+0000","last_undegraded":"2024-09-12T03:20:49.256412+0000","last_fullsized":"2024-09-12T03:20:49.256412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:11:03.740079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.246446+0000","last_change":"2024-09-12T03:19:09.336148+0000","last_active":"2024-09-12T03:20:49.246446+0000","last_peered":"2024-09-12T03:20:49.246446+0000","last_clean":"2024-09-12T03:20:49.246446+0000","last_became_active":"2024-09-12T03:19:09.335779+0000","last_became_peered":"2024-09-12T03:19:09.335779+0000","last_unstale":"2024-09-12T03:20:49.246446+0000","last_undegraded":"2024-09-12T03:20:49.246446+0000","last_fullsized":"2024-09-12T03:20:49.246446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:18.234320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.247971+0000","last_change":"2024-09-12T03:19:20.382536+0000","last_active":"2024-09-12T03:20:49.247971+0000","last_peered":"2024-09-12T03:20:49.247971+0000","last_clean":"2024-09-12T03:20:49.247971+0000","last_became_active":"2024-09-12T03:19:20.382118+0000","last_became_peered":"2024-09-12T03:19:20.382118+0000","last_unstale":"2024-09-12T03:20:49.247971+0000","last_undegraded":"2024-09-12T03:20:49.247971+0000","last_fullsized":"2024-09-12T03:20:49.247971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:55.836646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252455+0000","last_change":"2024-09-12T03:19:09.345064+0000","last_active":"2024-09-12T03:20:49.252455+0000","last_peered":"2024-09-12T03:20:49.252455+0000","last_clean":"2024-09-12T03:20:49.252455+0000","last_became_active":"2024-09-12T03:19:09.344628+0000","last_became_peered":"2024-09-12T03:19:09.344628+0000","last_unstale":"2024-09-12T03:20:49.252455+0000","last_undegraded":"2024-09-12T03:20:49.252455+0000","last_fullsized":"2024-09-12T03:20:49.252455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:50.011751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.244081+0000","last_change":"2024-09-12T03:19:09.338052+0000","last_active":"2024-09-12T03:20:49.244081+0000","last_peered":"2024-09-12T03:20:49.244081+0000","last_clean":"2024-09-12T03:20:49.244081+0000","last_became_active":"2024-09-12T03:19:09.337710+0000","last_became_peered":"2024-09-12T03:19:09.337710+0000","last_unstale":"2024-09-12T03:20:49.244081+0000","last_undegraded":"2024-09-12T03:20:49.244081+0000","last_fullsized":"2024-09-12T03:20:49.244081+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:36.521269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'9","reported_seq":134,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:54.518339+0000","last_change":"2024-09-12T03:19:07.208341+0000","last_active":"2024-09-12T03:20:54.518339+0000","last_peered":"2024-09-12T03:20:54.518339+0000","last_clean":"2024-09-12T03:20:54.518339+0000","last_became_active":"2024-09-12T03:19:07.207385+0000","last_became_peered":"2024-09-12T03:19:07.207385+0000","last_unstale":"2024-09-12T03:20:54.518339+0000","last_undegraded":"2024-09-12T03:20:54.518339+0000","last_fullsized":"2024-09-12T03:20:54.518339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:18.476795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1857,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254115+0000","last_change":"2024-09-12T03:19:09.339997+0000","last_active":"2024-09-12T03:20:49.254115+0000","last_peered":"2024-09-12T03:20:49.254115+0000","last_clean":"2024-09-12T03:20:49.254115+0000","last_became_active":"2024-09-12T03:19:09.339261+0000","last_became_peered":"2024-09-12T03:19:09.339261+0000","last_unstale":"2024-09-12T03:20:49.254115+0000","last_undegraded":"2024-09-12T03:20:49.254115+0000","last_fullsized":"2024-09-12T03:20:49.254115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:07.670345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374030+0000","last_change":"2024-09-12T03:19:07.200222+0000","last_active":"2024-09-12T03:19:20.374030+0000","last_peered":"2024-09-12T03:19:20.374030+0000","last_clean":"2024-09-12T03:19:20.374030+0000","last_became_active":"2024-09-12T03:19:07.199819+0000","last_became_peered":"2024-09-12T03:19:07.199819+0000","last_unstale":"2024-09-12T03:19:20.374030+0000","last_undegraded":"2024-09-12T03:19:20.374030+0000","last_fullsized":"2024-09-12T03:19:20.374030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:39:28.933609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375686+0000","last_change":"2024-09-12T03:19:07.207005+0000","last_active":"2024-09-12T03:19:20.375686+0000","last_peered":"2024-09-12T03:19:20.375686+0000","last_clean":"2024-09-12T03:19:20.375686+0000","last_became_active":"2024-09-12T03:19:07.206685+0000","last_became_peered":"2024-09-12T03:19:07.206685+0000","last_unstale":"2024-09-12T03:19:20.375686+0000","last_undegraded":"2024-09-12T03:19:20.375686+0000","last_fullsized":"2024-09-12T03:19:20.375686+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:31.111706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252939+0000","last_change":"2024-09-12T03:19:09.335871+0000","last_active":"2024-09-12T03:20:49.252939+0000","last_peered":"2024-09-12T03:20:49.252939+0000","last_clean":"2024-09-12T03:20:49.252939+0000","last_became_active":"2024-09-12T03:19:09.335568+0000","last_became_peered":"2024-09-12T03:19:09.335568+0000","last_unstale":"2024-09-12T03:20:49.252939+0000","last_undegraded":"2024-09-12T03:20:49.252939+0000","last_fullsized":"2024-09-12T03:20:49.252939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:52:34.471512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386049+0000","last_change":"2024-09-12T03:19:07.208320+0000","last_active":"2024-09-12T03:19:20.386049+0000","last_peered":"2024-09-12T03:19:20.386049+0000","last_clean":"2024-09-12T03:19:20.386049+0000","last_became_active":"2024-09-12T03:19:07.207873+0000","last_became_peered":"2024-09-12T03:19:07.207873+0000","last_unstale":"2024-09-12T03:19:20.386049+0000","last_undegraded":"2024-09-12T03:19:20.386049+0000","last_fullsized":"2024-09-12T03:19:20.386049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:00.128595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250364+0000","last_change":"2024-09-12T03:19:09.331930+0000","last_active":"2024-09-12T03:20:49.250364+0000","last_peered":"2024-09-12T03:20:49.250364+0000","last_clean":"2024-09-12T03:20:49.250364+0000","last_became_active":"2024-09-12T03:19:09.331558+0000","last_became_peered":"2024-09-12T03:19:09.331558+0000","last_unstale":"2024-09-12T03:20:49.250364+0000","last_undegraded":"2024-09-12T03:20:49.250364+0000","last_fullsized":"2024-09-12T03:20:49.250364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:00:27.935717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":120,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:51.511645+0000","last_change":"2024-09-12T03:19:07.205473+0000","last_active":"2024-09-12T03:20:51.511645+0000","last_peered":"2024-09-12T03:20:51.511645+0000","last_clean":"2024-09-12T03:20:51.511645+0000","last_became_active":"2024-09-12T03:19:07.205134+0000","last_became_peered":"2024-09-12T03:19:07.205134+0000","last_unstale":"2024-09-12T03:20:51.511645+0000","last_undegraded":"2024-09-12T03:20:51.511645+0000","last_fullsized":"2024-09-12T03:20:51.511645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:23.443957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265317+0000","last_change":"2024-09-12T03:19:09.333868+0000","last_active":"2024-09-12T03:20:49.265317+0000","last_peered":"2024-09-12T03:20:49.265317+0000","last_clean":"2024-09-12T03:20:49.265317+0000","last_became_active":"2024-09-12T03:19:09.333484+0000","last_became_peered":"2024-09-12T03:19:09.333484+0000","last_unstale":"2024-09-12T03:20:49.265317+0000","last_undegraded":"2024-09-12T03:20:49.265317+0000","last_fullsized":"2024-09-12T03:20:49.265317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:21:10.226821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386303+0000","last_change":"2024-09-12T03:19:20.386303+0000","last_active":"2024-09-12T03:19:20.386303+0000","last_peered":"2024-09-12T03:19:20.386303+0000","last_clean":"2024-09-12T03:19:20.386303+0000","last_became_active":"2024-09-12T03:19:20.383198+0000","last_became_peered":"2024-09-12T03:19:20.383198+0000","last_unstale":"2024-09-12T03:19:20.386303+0000","last_undegraded":"2024-09-12T03:19:20.386303+0000","last_fullsized":"2024-09-12T03:19:20.386303+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:42.224849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.257056+0000","last_change":"2024-09-12T03:19:09.340849+0000","last_active":"2024-09-12T03:20:49.257056+0000","last_peered":"2024-09-12T03:20:49.257056+0000","last_clean":"2024-09-12T03:20:49.257056+0000","last_became_active":"2024-09-12T03:19:09.340517+0000","last_became_peered":"2024-09-12T03:19:09.340517+0000","last_unstale":"2024-09-12T03:20:49.257056+0000","last_undegraded":"2024-09-12T03:20:49.257056+0000","last_fullsized":"2024-09-12T03:20:49.257056+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:42.018269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386240+0000","last_change":"2024-09-12T03:19:07.389147+0000","last_active":"2024-09-12T03:19:20.386240+0000","last_peered":"2024-09-12T03:19:20.386240+0000","last_clean":"2024-09-12T03:19:20.386240+0000","last_became_active":"2024-09-12T03:19:07.388744+0000","last_became_peered":"2024-09-12T03:19:07.388744+0000","last_unstale":"2024-09-12T03:19:20.386240+0000","last_undegraded":"2024-09-12T03:19:20.386240+0000","last_fullsized":"2024-09-12T03:19:20.386240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:16.700644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.264622+0000","last_change":"2024-09-12T03:19:09.329243+0000","last_active":"2024-09-12T03:20:49.264622+0000","last_peered":"2024-09-12T03:20:49.264622+0000","last_clean":"2024-09-12T03:20:49.264622+0000","last_became_active":"2024-09-12T03:19:09.328610+0000","last_became_peered":"2024-09-12T03:19:09.328610+0000","last_unstale":"2024-09-12T03:20:49.264622+0000","last_undegraded":"2024-09-12T03:20:49.264622+0000","last_fullsized":"2024-09-12T03:20:49.264622+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:49.452942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'32","reported_seq":251,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:52.513814+0000","last_change":"2024-09-12T03:19:07.207804+0000","last_active":"2024-09-12T03:20:52.513814+0000","last_peered":"2024-09-12T03:20:52.513814+0000","last_clean":"2024-09-12T03:20:52.513814+0000","last_became_active":"2024-09-12T03:19:07.207409+0000","last_became_peered":"2024-09-12T03:19:07.207409+0000","last_unstale":"2024-09-12T03:20:52.513814+0000","last_undegraded":"2024-09-12T03:20:52.513814+0000","last_fullsized":"2024-09-12T03:20:52.513814+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:10.732423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60321,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":197,"num_read_kb":157,"num_write":33,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.255978+0000","last_change":"2024-09-12T03:19:09.333102+0000","last_active":"2024-09-12T03:20:49.255978+0000","last_peered":"2024-09-12T03:20:49.255978+0000","last_clean":"2024-09-12T03:20:49.255978+0000","last_became_active":"2024-09-12T03:19:09.332712+0000","last_became_peered":"2024-09-12T03:19:09.332712+0000","last_unstale":"2024-09-12T03:20:49.255978+0000","last_undegraded":"2024-09-12T03:20:49.255978+0000","last_fullsized":"2024-09-12T03:20:49.255978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:22.912213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382902+0000","last_change":"2024-09-12T03:19:20.382348+0000","last_active":"2024-09-12T03:19:20.382902+0000","last_peered":"2024-09-12T03:19:20.382902+0000","last_clean":"2024-09-12T03:19:20.382902+0000","last_became_active":"2024-09-12T03:19:20.375142+0000","last_became_peered":"2024-09-12T03:19:20.375142+0000","last_unstale":"2024-09-12T03:19:20.382902+0000","last_undegraded":"2024-09-12T03:19:20.382902+0000","last_fullsized":"2024-09-12T03:19:20.382902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:04.573599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.246879+0000","last_change":"2024-09-12T03:19:09.333194+0000","last_active":"2024-09-12T03:20:49.246879+0000","last_peered":"2024-09-12T03:20:49.246879+0000","last_clean":"2024-09-12T03:20:49.246879+0000","last_became_active":"2024-09-12T03:19:09.332865+0000","last_became_peered":"2024-09-12T03:19:09.332865+0000","last_unstale":"2024-09-12T03:20:49.246879+0000","last_undegraded":"2024-09-12T03:20:49.246879+0000","last_fullsized":"2024-09-12T03:20:49.246879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:57.096138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386176+0000","last_change":"2024-09-12T03:19:07.209289+0000","last_active":"2024-09-12T03:19:20.386176+0000","last_peered":"2024-09-12T03:19:20.386176+0000","last_clean":"2024-09-12T03:19:20.386176+0000","last_became_active":"2024-09-12T03:19:07.208863+0000","last_became_peered":"2024-09-12T03:19:07.208863+0000","last_unstale":"2024-09-12T03:19:20.386176+0000","last_undegraded":"2024-09-12T03:19:20.386176+0000","last_fullsized":"2024-09-12T03:19:20.386176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:26.168148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256973+0000","last_change":"2024-09-12T03:19:09.343620+0000","last_active":"2024-09-12T03:20:49.256973+0000","last_peered":"2024-09-12T03:20:49.256973+0000","last_clean":"2024-09-12T03:20:49.256973+0000","last_became_active":"2024-09-12T03:19:09.343299+0000","last_became_peered":"2024-09-12T03:19:09.343299+0000","last_unstale":"2024-09-12T03:20:49.256973+0000","last_undegraded":"2024-09-12T03:20:49.256973+0000","last_fullsized":"2024-09-12T03:20:49.256973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:34.268886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":126,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:55.551460+0000","last_change":"2024-09-12T03:19:07.206160+0000","last_active":"2024-09-12T03:20:55.551460+0000","last_peered":"2024-09-12T03:20:55.551460+0000","last_clean":"2024-09-12T03:20:55.551460+0000","last_became_active":"2024-09-12T03:19:07.205509+0000","last_became_peered":"2024-09-12T03:19:07.205509+0000","last_unstale":"2024-09-12T03:20:55.551460+0000","last_undegraded":"2024-09-12T03:20:55.551460+0000","last_fullsized":"2024-09-12T03:20:55.551460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:35.362831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250896+0000","last_change":"2024-09-12T03:19:09.337595+0000","last_active":"2024-09-12T03:20:49.250896+0000","last_peered":"2024-09-12T03:20:49.250896+0000","last_clean":"2024-09-12T03:20:49.250896+0000","last_became_active":"2024-09-12T03:19:09.337281+0000","last_became_peered":"2024-09-12T03:19:09.337281+0000","last_unstale":"2024-09-12T03:20:49.250896+0000","last_undegraded":"2024-09-12T03:20:49.250896+0000","last_fullsized":"2024-09-12T03:20:49.250896+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:35.761679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373995+0000","last_change":"2024-09-12T03:19:07.389336+0000","last_active":"2024-09-12T03:19:20.373995+0000","last_peered":"2024-09-12T03:19:20.373995+0000","last_clean":"2024-09-12T03:19:20.373995+0000","last_became_active":"2024-09-12T03:19:07.388649+0000","last_became_peered":"2024-09-12T03:19:07.388649+0000","last_unstale":"2024-09-12T03:19:20.373995+0000","last_undegraded":"2024-09-12T03:19:20.373995+0000","last_fullsized":"2024-09-12T03:19:20.373995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:33.312700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.253821+0000","last_change":"2024-09-12T03:19:09.340674+0000","last_active":"2024-09-12T03:20:49.253821+0000","last_peered":"2024-09-12T03:20:49.253821+0000","last_clean":"2024-09-12T03:20:49.253821+0000","last_became_active":"2024-09-12T03:19:09.340144+0000","last_became_peered":"2024-09-12T03:19:09.340144+0000","last_unstale":"2024-09-12T03:20:49.253821+0000","last_undegraded":"2024-09-12T03:20:49.253821+0000","last_fullsized":"2024-09-12T03:20:49.253821+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:21:15.605086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373837+0000","last_change":"2024-09-12T03:19:07.206513+0000","last_active":"2024-09-12T03:19:20.373837+0000","last_peered":"2024-09-12T03:19:20.373837+0000","last_clean":"2024-09-12T03:19:20.373837+0000","last_became_active":"2024-09-12T03:19:07.205974+0000","last_became_peered":"2024-09-12T03:19:07.205974+0000","last_unstale":"2024-09-12T03:19:20.373837+0000","last_undegraded":"2024-09-12T03:19:20.373837+0000","last_fullsized":"2024-09-12T03:19:20.373837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:43:10.002123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252033+0000","last_change":"2024-09-12T03:19:09.342200+0000","last_active":"2024-09-12T03:20:49.252033+0000","last_peered":"2024-09-12T03:20:49.252033+0000","last_clean":"2024-09-12T03:20:49.252033+0000","last_became_active":"2024-09-12T03:19:09.341901+0000","last_became_peered":"2024-09-12T03:19:09.341901+0000","last_unstale":"2024-09-12T03:20:49.252033+0000","last_undegraded":"2024-09-12T03:20:49.252033+0000","last_fullsized":"2024-09-12T03:20:49.252033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:27.486214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375373+0000","last_change":"2024-09-12T03:19:07.207514+0000","last_active":"2024-09-12T03:19:20.375373+0000","last_peered":"2024-09-12T03:19:20.375373+0000","last_clean":"2024-09-12T03:19:20.375373+0000","last_became_active":"2024-09-12T03:19:07.205925+0000","last_became_peered":"2024-09-12T03:19:07.205925+0000","last_unstale":"2024-09-12T03:19:20.375373+0000","last_undegraded":"2024-09-12T03:19:20.375373+0000","last_fullsized":"2024-09-12T03:19:20.375373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:20.244703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256005+0000","last_change":"2024-09-12T03:19:09.337837+0000","last_active":"2024-09-12T03:20:49.256005+0000","last_peered":"2024-09-12T03:20:49.256005+0000","last_clean":"2024-09-12T03:20:49.256005+0000","last_became_active":"2024-09-12T03:19:09.337502+0000","last_became_peered":"2024-09-12T03:19:09.337502+0000","last_unstale":"2024-09-12T03:20:49.256005+0000","last_undegraded":"2024-09-12T03:20:49.256005+0000","last_fullsized":"2024-09-12T03:20:49.256005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:30.167139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386356+0000","last_change":"2024-09-12T03:19:07.209130+0000","last_active":"2024-09-12T03:19:20.386356+0000","last_peered":"2024-09-12T03:19:20.386356+0000","last_clean":"2024-09-12T03:19:20.386356+0000","last_became_active":"2024-09-12T03:19:07.208753+0000","last_became_peered":"2024-09-12T03:19:07.208753+0000","last_unstale":"2024-09-12T03:19:20.386356+0000","last_undegraded":"2024-09-12T03:19:20.386356+0000","last_fullsized":"2024-09-12T03:19:20.386356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:34.719292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250623+0000","last_change":"2024-09-12T03:19:09.336382+0000","last_active":"2024-09-12T03:20:49.250623+0000","last_peered":"2024-09-12T03:20:49.250623+0000","last_clean":"2024-09-12T03:20:49.250623+0000","last_became_active":"2024-09-12T03:19:09.336015+0000","last_became_peered":"2024-09-12T03:19:09.336015+0000","last_unstale":"2024-09-12T03:20:49.250623+0000","last_undegraded":"2024-09-12T03:20:49.250623+0000","last_fullsized":"2024-09-12T03:20:49.250623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:19.430849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375044+0000","last_change":"2024-09-12T03:19:07.528567+0000","last_active":"2024-09-12T03:19:20.375044+0000","last_peered":"2024-09-12T03:19:20.375044+0000","last_clean":"2024-09-12T03:19:20.375044+0000","last_became_active":"2024-09-12T03:19:07.528123+0000","last_became_peered":"2024-09-12T03:19:07.528123+0000","last_unstale":"2024-09-12T03:19:20.375044+0000","last_undegraded":"2024-09-12T03:19:20.375044+0000","last_fullsized":"2024-09-12T03:19:20.375044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:34.416714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256025+0000","last_change":"2024-09-12T03:19:09.338836+0000","last_active":"2024-09-12T03:20:49.256025+0000","last_peered":"2024-09-12T03:20:49.256025+0000","last_clean":"2024-09-12T03:20:49.256025+0000","last_became_active":"2024-09-12T03:19:09.337571+0000","last_became_peered":"2024-09-12T03:19:09.337571+0000","last_unstale":"2024-09-12T03:20:49.256025+0000","last_undegraded":"2024-09-12T03:20:49.256025+0000","last_fullsized":"2024-09-12T03:20:49.256025+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:39.039878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":330,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372792+0000","last_change":"2024-09-12T03:18:01.051311+0000","last_active":"2024-09-12T03:19:20.372792+0000","last_peered":"2024-09-12T03:19:20.372792+0000","last_clean":"2024-09-12T03:19:20.372792+0000","last_became_active":"2024-09-12T03:18:01.050849+0000","last_became_peered":"2024-09-12T03:18:01.050849+0000","last_unstale":"2024-09-12T03:19:20.372792+0000","last_undegraded":"2024-09-12T03:19:20.372792+0000","last_fullsized":"2024-09-12T03:19:20.372792+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:14:38.096222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:14:38.096222+0000","last_clean_scrub_stamp":"2024-09-12T03:14:38.096222+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:33.067603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382035+0000","last_change":"2024-09-12T03:19:07.206045+0000","last_active":"2024-09-12T03:19:20.382035+0000","last_peered":"2024-09-12T03:19:20.382035+0000","last_clean":"2024-09-12T03:19:20.382035+0000","last_became_active":"2024-09-12T03:19:07.205594+0000","last_became_peered":"2024-09-12T03:19:07.205594+0000","last_unstale":"2024-09-12T03:19:20.382035+0000","last_undegraded":"2024-09-12T03:19:20.382035+0000","last_fullsized":"2024-09-12T03:19:20.382035+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:12.811906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.244733+0000","last_change":"2024-09-12T03:19:20.387274+0000","last_active":"2024-09-12T03:20:49.244733+0000","last_peered":"2024-09-12T03:20:49.244733+0000","last_clean":"2024-09-12T03:20:49.244733+0000","last_became_active":"2024-09-12T03:19:20.386878+0000","last_became_peered":"2024-09-12T03:19:20.386878+0000","last_unstale":"2024-09-12T03:20:49.244733+0000","last_undegraded":"2024-09-12T03:20:49.244733+0000","last_fullsized":"2024-09-12T03:20:49.244733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:26.674032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372302+0000","last_change":"2024-09-12T03:19:07.206212+0000","last_active":"2024-09-12T03:19:20.372302+0000","last_peered":"2024-09-12T03:19:20.372302+0000","last_clean":"2024-09-12T03:19:20.372302+0000","last_became_active":"2024-09-12T03:19:07.205899+0000","last_became_peered":"2024-09-12T03:19:07.205899+0000","last_unstale":"2024-09-12T03:19:20.372302+0000","last_undegraded":"2024-09-12T03:19:20.372302+0000","last_fullsized":"2024-09-12T03:19:20.372302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:02.169436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250795+0000","last_change":"2024-09-12T03:19:20.377494+0000","last_active":"2024-09-12T03:20:49.250795+0000","last_peered":"2024-09-12T03:20:49.250795+0000","last_clean":"2024-09-12T03:20:49.250795+0000","last_became_active":"2024-09-12T03:19:20.377091+0000","last_became_peered":"2024-09-12T03:19:20.377091+0000","last_unstale":"2024-09-12T03:20:49.250795+0000","last_undegraded":"2024-09-12T03:20:49.250795+0000","last_fullsized":"2024-09-12T03:20:49.250795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:30.926853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":119,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:51.542783+0000","last_change":"2024-09-12T03:19:07.388486+0000","last_active":"2024-09-12T03:20:51.542783+0000","last_peered":"2024-09-12T03:20:51.542783+0000","last_clean":"2024-09-12T03:20:51.542783+0000","last_became_active":"2024-09-12T03:19:07.388125+0000","last_became_peered":"2024-09-12T03:19:07.388125+0000","last_unstale":"2024-09-12T03:20:51.542783+0000","last_undegraded":"2024-09-12T03:20:51.542783+0000","last_fullsized":"2024-09-12T03:20:51.542783+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:46.218601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254069+0000","last_change":"2024-09-12T03:19:09.344308+0000","last_active":"2024-09-12T03:20:49.254069+0000","last_peered":"2024-09-12T03:20:49.254069+0000","last_clean":"2024-09-12T03:20:49.254069+0000","last_became_active":"2024-09-12T03:19:09.343987+0000","last_became_peered":"2024-09-12T03:19:09.343987+0000","last_unstale":"2024-09-12T03:20:49.254069+0000","last_undegraded":"2024-09-12T03:20:49.254069+0000","last_fullsized":"2024-09-12T03:20:49.254069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:16:11.616555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372560+0000","last_change":"2024-09-12T03:19:07.208619+0000","last_active":"2024-09-12T03:19:20.372560+0000","last_peered":"2024-09-12T03:19:20.372560+0000","last_clean":"2024-09-12T03:19:20.372560+0000","last_became_active":"2024-09-12T03:19:07.208254+0000","last_became_peered":"2024-09-12T03:19:07.208254+0000","last_unstale":"2024-09-12T03:19:20.372560+0000","last_undegraded":"2024-09-12T03:19:20.372560+0000","last_fullsized":"2024-09-12T03:19:20.372560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:56.177948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.238148+0000","last_change":"2024-09-12T03:19:09.341837+0000","last_active":"2024-09-12T03:20:49.238148+0000","last_peered":"2024-09-12T03:20:49.238148+0000","last_clean":"2024-09-12T03:20:49.238148+0000","last_became_active":"2024-09-12T03:19:09.341069+0000","last_became_peered":"2024-09-12T03:19:09.341069+0000","last_unstale":"2024-09-12T03:20:49.238148+0000","last_undegraded":"2024-09-12T03:20:49.238148+0000","last_fullsized":"2024-09-12T03:20:49.238148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:04.238986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372460+0000","last_change":"2024-09-12T03:19:07.388870+0000","last_active":"2024-09-12T03:19:20.372460+0000","last_peered":"2024-09-12T03:19:20.372460+0000","last_clean":"2024-09-12T03:19:20.372460+0000","last_became_active":"2024-09-12T03:19:07.388555+0000","last_became_peered":"2024-09-12T03:19:07.388555+0000","last_unstale":"2024-09-12T03:19:20.372460+0000","last_undegraded":"2024-09-12T03:19:20.372460+0000","last_fullsized":"2024-09-12T03:19:20.372460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:29.789454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.247542+0000","last_change":"2024-09-12T03:19:09.334714+0000","last_active":"2024-09-12T03:20:49.247542+0000","last_peered":"2024-09-12T03:20:49.247542+0000","last_clean":"2024-09-12T03:20:49.247542+0000","last_became_active":"2024-09-12T03:19:09.334413+0000","last_became_peered":"2024-09-12T03:19:09.334413+0000","last_unstale":"2024-09-12T03:20:49.247542+0000","last_undegraded":"2024-09-12T03:20:49.247542+0000","last_fullsized":"2024-09-12T03:20:49.247542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:52.222340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373797+0000","last_change":"2024-09-12T03:19:07.389055+0000","last_active":"2024-09-12T03:19:20.373797+0000","last_peered":"2024-09-12T03:19:20.373797+0000","last_clean":"2024-09-12T03:19:20.373797+0000","last_became_active":"2024-09-12T03:19:07.388673+0000","last_became_peered":"2024-09-12T03:19:07.388673+0000","last_unstale":"2024-09-12T03:19:20.373797+0000","last_undegraded":"2024-09-12T03:19:20.373797+0000","last_fullsized":"2024-09-12T03:19:20.373797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:20.974533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248932+0000","last_change":"2024-09-12T03:19:09.340105+0000","last_active":"2024-09-12T03:20:49.248932+0000","last_peered":"2024-09-12T03:20:49.248932+0000","last_clean":"2024-09-12T03:20:49.248932+0000","last_became_active":"2024-09-12T03:19:09.339642+0000","last_became_peered":"2024-09-12T03:19:09.339642+0000","last_unstale":"2024-09-12T03:20:49.248932+0000","last_undegraded":"2024-09-12T03:20:49.248932+0000","last_fullsized":"2024-09-12T03:20:49.248932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:01.624236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'15","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:52.343269+0000","last_change":"2024-09-12T03:19:07.207232+0000","last_active":"2024-09-12T03:20:52.343269+0000","last_peered":"2024-09-12T03:20:52.343269+0000","last_clean":"2024-09-12T03:20:52.343269+0000","last_became_active":"2024-09-12T03:19:07.206921+0000","last_became_peered":"2024-09-12T03:19:07.206921+0000","last_unstale":"2024-09-12T03:20:52.343269+0000","last_undegraded":"2024-09-12T03:20:52.343269+0000","last_fullsized":"2024-09-12T03:20:52.343269+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:25.457853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252490+0000","last_change":"2024-09-12T03:19:09.330442+0000","last_active":"2024-09-12T03:20:49.252490+0000","last_peered":"2024-09-12T03:20:49.252490+0000","last_clean":"2024-09-12T03:20:49.252490+0000","last_became_active":"2024-09-12T03:19:09.330081+0000","last_became_peered":"2024-09-12T03:19:09.330081+0000","last_unstale":"2024-09-12T03:20:49.252490+0000","last_undegraded":"2024-09-12T03:20:49.252490+0000","last_fullsized":"2024-09-12T03:20:49.252490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:56.152562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382392+0000","last_change":"2024-09-12T03:19:07.528646+0000","last_active":"2024-09-12T03:19:20.382392+0000","last_peered":"2024-09-12T03:19:20.382392+0000","last_clean":"2024-09-12T03:19:20.382392+0000","last_became_active":"2024-09-12T03:19:07.528334+0000","last_became_peered":"2024-09-12T03:19:07.528334+0000","last_unstale":"2024-09-12T03:19:20.382392+0000","last_undegraded":"2024-09-12T03:19:20.382392+0000","last_fullsized":"2024-09-12T03:19:20.382392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:40.425579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256685+0000","last_change":"2024-09-12T03:19:09.331993+0000","last_active":"2024-09-12T03:20:49.256685+0000","last_peered":"2024-09-12T03:20:49.256685+0000","last_clean":"2024-09-12T03:20:49.256685+0000","last_became_active":"2024-09-12T03:19:09.331698+0000","last_became_peered":"2024-09-12T03:19:09.331698+0000","last_unstale":"2024-09-12T03:20:49.256685+0000","last_undegraded":"2024-09-12T03:20:49.256685+0000","last_fullsized":"2024-09-12T03:20:49.256685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:03:54.182833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370179+0000","last_change":"2024-09-12T03:19:07.388297+0000","last_active":"2024-09-12T03:19:20.370179+0000","last_peered":"2024-09-12T03:19:20.370179+0000","last_clean":"2024-09-12T03:19:20.370179+0000","last_became_active":"2024-09-12T03:19:07.388038+0000","last_became_peered":"2024-09-12T03:19:07.388038+0000","last_unstale":"2024-09-12T03:19:20.370179+0000","last_undegraded":"2024-09-12T03:19:20.370179+0000","last_fullsized":"2024-09-12T03:19:20.370179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:45.874195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.212080+0000","last_change":"2024-09-12T03:19:09.331525+0000","last_active":"2024-09-12T03:20:49.212080+0000","last_peered":"2024-09-12T03:20:49.212080+0000","last_clean":"2024-09-12T03:20:49.212080+0000","last_became_active":"2024-09-12T03:19:09.330903+0000","last_became_peered":"2024-09-12T03:19:09.330903+0000","last_unstale":"2024-09-12T03:20:49.212080+0000","last_undegraded":"2024-09-12T03:20:49.212080+0000","last_fullsized":"2024-09-12T03:20:49.212080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:02.455132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372235+0000","last_change":"2024-09-12T03:19:07.205181+0000","last_active":"2024-09-12T03:19:20.372235+0000","last_peered":"2024-09-12T03:19:20.372235+0000","last_clean":"2024-09-12T03:19:20.372235+0000","last_became_active":"2024-09-12T03:19:07.204726+0000","last_became_peered":"2024-09-12T03:19:07.204726+0000","last_unstale":"2024-09-12T03:19:20.372235+0000","last_undegraded":"2024-09-12T03:19:20.372235+0000","last_fullsized":"2024-09-12T03:19:20.372235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:26.112800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250768+0000","last_change":"2024-09-12T03:19:09.345338+0000","last_active":"2024-09-12T03:20:49.250768+0000","last_peered":"2024-09-12T03:20:49.250768+0000","last_clean":"2024-09-12T03:20:49.250768+0000","last_became_active":"2024-09-12T03:19:09.345023+0000","last_became_peered":"2024-09-12T03:19:09.345023+0000","last_unstale":"2024-09-12T03:20:49.250768+0000","last_undegraded":"2024-09-12T03:20:49.250768+0000","last_fullsized":"2024-09-12T03:20:49.250768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:39.184065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374938+0000","last_change":"2024-09-12T03:19:07.211954+0000","last_active":"2024-09-12T03:19:20.374938+0000","last_peered":"2024-09-12T03:19:20.374938+0000","last_clean":"2024-09-12T03:19:20.374938+0000","last_became_active":"2024-09-12T03:19:07.211509+0000","last_became_peered":"2024-09-12T03:19:07.211509+0000","last_unstale":"2024-09-12T03:19:20.374938+0000","last_undegraded":"2024-09-12T03:19:20.374938+0000","last_fullsized":"2024-09-12T03:19:20.374938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:27.461924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.244962+0000","last_change":"2024-09-12T03:19:09.331168+0000","last_active":"2024-09-12T03:20:49.244962+0000","last_peered":"2024-09-12T03:20:49.244962+0000","last_clean":"2024-09-12T03:20:49.244962+0000","last_became_active":"2024-09-12T03:19:09.329592+0000","last_became_peered":"2024-09-12T03:19:09.329592+0000","last_unstale":"2024-09-12T03:20:49.244962+0000","last_undegraded":"2024-09-12T03:20:49.244962+0000","last_fullsized":"2024-09-12T03:20:49.244962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:46.127861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373572+0000","last_change":"2024-09-12T03:19:07.207802+0000","last_active":"2024-09-12T03:19:20.373572+0000","last_peered":"2024-09-12T03:19:20.373572+0000","last_clean":"2024-09-12T03:19:20.373572+0000","last_became_active":"2024-09-12T03:19:07.207428+0000","last_became_peered":"2024-09-12T03:19:07.207428+0000","last_unstale":"2024-09-12T03:19:20.373572+0000","last_undegraded":"2024-09-12T03:19:20.373572+0000","last_fullsized":"2024-09-12T03:19:20.373572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:45.135019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.255394+0000","last_change":"2024-09-12T03:19:09.336094+0000","last_active":"2024-09-12T03:20:49.255394+0000","last_peered":"2024-09-12T03:20:49.255394+0000","last_clean":"2024-09-12T03:20:49.255394+0000","last_became_active":"2024-09-12T03:19:09.335779+0000","last_became_peered":"2024-09-12T03:19:09.335779+0000","last_unstale":"2024-09-12T03:20:49.255394+0000","last_undegraded":"2024-09-12T03:20:49.255394+0000","last_fullsized":"2024-09-12T03:20:49.255394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:37.019082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254523+0000","last_change":"2024-09-12T03:19:09.344537+0000","last_active":"2024-09-12T03:20:49.254523+0000","last_peered":"2024-09-12T03:20:49.254523+0000","last_clean":"2024-09-12T03:20:49.254523+0000","last_became_active":"2024-09-12T03:19:09.343879+0000","last_became_peered":"2024-09-12T03:19:09.343879+0000","last_unstale":"2024-09-12T03:20:49.254523+0000","last_undegraded":"2024-09-12T03:20:49.254523+0000","last_fullsized":"2024-09-12T03:20:49.254523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:40.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373973+0000","last_change":"2024-09-12T03:19:07.208040+0000","last_active":"2024-09-12T03:19:20.373973+0000","last_peered":"2024-09-12T03:19:20.373973+0000","last_clean":"2024-09-12T03:19:20.373973+0000","last_became_active":"2024-09-12T03:19:07.207658+0000","last_became_peered":"2024-09-12T03:19:07.207658+0000","last_unstale":"2024-09-12T03:19:20.373973+0000","last_undegraded":"2024-09-12T03:19:20.373973+0000","last_fullsized":"2024-09-12T03:19:20.373973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:48.620707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265759+0000","last_change":"2024-09-12T03:19:09.333837+0000","last_active":"2024-09-12T03:20:49.265759+0000","last_peered":"2024-09-12T03:20:49.265759+0000","last_clean":"2024-09-12T03:20:49.265759+0000","last_became_active":"2024-09-12T03:19:09.333194+0000","last_became_peered":"2024-09-12T03:19:09.333194+0000","last_unstale":"2024-09-12T03:20:49.265759+0000","last_undegraded":"2024-09-12T03:20:49.265759+0000","last_fullsized":"2024-09-12T03:20:49.265759+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:18.766181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.385455+0000","last_change":"2024-09-12T03:19:20.384978+0000","last_active":"2024-09-12T03:19:20.385455+0000","last_peered":"2024-09-12T03:19:20.385455+0000","last_clean":"2024-09-12T03:19:20.385455+0000","last_became_active":"2024-09-12T03:19:20.376883+0000","last_became_peered":"2024-09-12T03:19:20.376883+0000","last_unstale":"2024-09-12T03:19:20.385455+0000","last_undegraded":"2024-09-12T03:19:20.385455+0000","last_fullsized":"2024-09-12T03:19:20.385455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:32.542334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265022+0000","last_change":"2024-09-12T03:19:09.334684+0000","last_active":"2024-09-12T03:20:49.265022+0000","last_peered":"2024-09-12T03:20:49.265022+0000","last_clean":"2024-09-12T03:20:49.265022+0000","last_became_active":"2024-09-12T03:19:09.334281+0000","last_became_peered":"2024-09-12T03:19:09.334281+0000","last_unstale":"2024-09-12T03:20:49.265022+0000","last_undegraded":"2024-09-12T03:20:49.265022+0000","last_fullsized":"2024-09-12T03:20:49.265022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:44.837067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372829+0000","last_change":"2024-09-12T03:19:07.388555+0000","last_active":"2024-09-12T03:19:20.372829+0000","last_peered":"2024-09-12T03:19:20.372829+0000","last_clean":"2024-09-12T03:19:20.372829+0000","last_became_active":"2024-09-12T03:19:07.388206+0000","last_became_peered":"2024-09-12T03:19:07.388206+0000","last_unstale":"2024-09-12T03:19:20.372829+0000","last_undegraded":"2024-09-12T03:19:20.372829+0000","last_fullsized":"2024-09-12T03:19:20.372829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:33.628294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248105+0000","last_change":"2024-09-12T03:19:09.339108+0000","last_active":"2024-09-12T03:20:49.248105+0000","last_peered":"2024-09-12T03:20:49.248105+0000","last_clean":"2024-09-12T03:20:49.248105+0000","last_became_active":"2024-09-12T03:19:09.338819+0000","last_became_peered":"2024-09-12T03:19:09.338819+0000","last_unstale":"2024-09-12T03:20:49.248105+0000","last_undegraded":"2024-09-12T03:20:49.248105+0000","last_fullsized":"2024-09-12T03:20:49.248105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:32.864900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372798+0000","last_change":"2024-09-12T03:19:07.207197+0000","last_active":"2024-09-12T03:19:20.372798+0000","last_peered":"2024-09-12T03:19:20.372798+0000","last_clean":"2024-09-12T03:19:20.372798+0000","last_became_active":"2024-09-12T03:19:07.206879+0000","last_became_peered":"2024-09-12T03:19:07.206879+0000","last_unstale":"2024-09-12T03:19:20.372798+0000","last_undegraded":"2024-09-12T03:19:20.372798+0000","last_fullsized":"2024-09-12T03:19:20.372798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:00:46.269636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65559,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":601,"num_read_kb":367,"num_write":179,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":209550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579938,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":433387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36892,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708260,"statfs":{"total":95995035648,"available":95957258240,"internally_reserved":0,"allocated":1585152,"data_stored":1085155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972395,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":433589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201329,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":433509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430262,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1499136,"data_stored":1025890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593787,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":432978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822720,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1511424,"data_stored":1025967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":897024,"data_stored":433808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":433289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":501409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40440,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704712,"statfs":{"total":95995035648,"available":95953625088,"internally_reserved":0,"allocated":958464,"data_stored":498858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":434249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":66087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:20:57.329 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:20:57.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:57 smithi161 ceph-mon[35485]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:20:57.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:57 smithi161 ceph-mon[35485]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-12T03:20:57.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:57 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/21357866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:57.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:57 smithi118 ceph-mon[35048]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:20:57.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:57 smithi118 ceph-mon[35048]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-12T03:20:57.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:57 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/21357866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:57.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.468+0000 7faaeb34b640 1 -- 172.21.15.6:0/389041331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faae4107810 msgr2=0x7faae4109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.468+0000 7faaeb34b640 1 --2- 172.21.15.6:0/389041331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faae4107810 0x7faae4109c80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7faadc009ca0 tx=0x7faadc02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.468+0000 7faaeb34b640 1 -- 172.21.15.6:0/389041331 shutdown_connections 2024-09-12T03:20:57.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.468+0000 7faaeb34b640 1 --2- 172.21.15.6:0/389041331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faae4107810 0x7faae4109c80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.468+0000 7faaeb34b640 1 --2- 172.21.15.6:0/389041331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faae4104ee0 0x7faae41072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.468+0000 7faaeb34b640 1 --2- 172.21.15.6:0/389041331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faae41025b0 0x7faae41049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.468+0000 7faaeb34b640 1 -- 172.21.15.6:0/389041331 >> 172.21.15.6:0/389041331 conn(0x7faae40fc1b0 msgr2=0x7faae40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:57.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.468+0000 7faaeb34b640 1 -- 172.21.15.6:0/389041331 shutdown_connections 2024-09-12T03:20:57.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.468+0000 7faaeb34b640 1 -- 172.21.15.6:0/389041331 wait complete. 2024-09-12T03:20:57.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.468+0000 7faaeb34b640 1 Processor -- start 2024-09-12T03:20:57.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.468+0000 7faaeb34b640 1 -- start start 2024-09-12T03:20:57.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.469+0000 7faaeb34b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faae41025b0 0x7faae4101f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.469+0000 7faaeb34b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faae4104ee0 0x7faae4100560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.469+0000 7faaeb34b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faae4107810 0x7faae4100aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.469+0000 7faaeb34b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7faae410d820 con 0x7faae4104ee0 2024-09-12T03:20:57.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.469+0000 7faaeb34b640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7faae410d6a0 con 0x7faae41025b0 2024-09-12T03:20:57.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.469+0000 7faaeb34b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faae410d9a0 con 0x7faae4107810 2024-09-12T03:20:57.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.469+0000 7faae88e6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faae4104ee0 0x7faae4100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.469+0000 7faae88e6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faae4104ee0 0x7faae4100560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36470/0 (socket says 172.21.15.6:36470) 2024-09-12T03:20:57.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.469+0000 7faae88e6640 1 -- 172.21.15.6:0/3605550021 learned_addr learned my addr 172.21.15.6:0/3605550021 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:57.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.469+0000 7faae90e7640 1 --2- 172.21.15.6:0/3605550021 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faae41025b0 0x7faae4101f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.469+0000 7faae98e8640 1 --2- 172.21.15.6:0/3605550021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faae4107810 0x7faae4100aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.469+0000 7faae88e6640 1 -- 172.21.15.6:0/3605550021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faae4107810 msgr2=0x7faae4100aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.469+0000 7faae88e6640 1 --2- 172.21.15.6:0/3605550021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faae4107810 0x7faae4100aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.469+0000 7faae88e6640 1 -- 172.21.15.6:0/3605550021 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faae41025b0 msgr2=0x7faae4101f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.469+0000 7faae88e6640 1 --2- 172.21.15.6:0/3605550021 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faae41025b0 0x7faae4101f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.469+0000 7faae88e6640 1 -- 172.21.15.6:0/3605550021 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae4101330 con 0x7faae4104ee0 2024-09-12T03:20:57.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.469+0000 7faae88e6640 1 --2- 172.21.15.6:0/3605550021 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faae4104ee0 0x7faae4100560 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7faad4005fd0 tx=0x7faad400dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:57.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.470+0000 7faada7fc640 1 -- 172.21.15.6:0/3605550021 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4017070 con 0x7faae4104ee0 2024-09-12T03:20:57.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.470+0000 7faada7fc640 1 -- 172.21.15.6:0/3605550021 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faad400de50 con 0x7faae4104ee0 2024-09-12T03:20:57.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.470+0000 7faaeb34b640 1 -- 172.21.15.6:0/3605550021 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae41a1f10 con 0x7faae4104ee0 2024-09-12T03:20:57.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.470+0000 7faada7fc640 1 -- 172.21.15.6:0/3605550021 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4003e00 con 0x7faae4104ee0 2024-09-12T03:20:57.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.470+0000 7faaeb34b640 1 -- 172.21.15.6:0/3605550021 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae41a2380 con 0x7faae4104ee0 2024-09-12T03:20:57.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.471+0000 7faaeb34b640 1 -- 172.21.15.6:0/3605550021 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae40fdca0 con 0x7faae4104ee0 2024-09-12T03:20:57.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.472+0000 7faada7fc640 1 -- 172.21.15.6:0/3605550021 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7faad4003fa0 con 0x7faae4104ee0 2024-09-12T03:20:57.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.473+0000 7faada7fc640 1 --2- 172.21.15.6:0/3605550021 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7faad00a2ca0 0x7faad00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.473+0000 7faae90e7640 1 --2- 172.21.15.6:0/3605550021 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7faad00a2ca0 0x7faad00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.473+0000 7faada7fc640 1 -- 172.21.15.6:0/3605550021 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7faad40ced00 con 0x7faae4104ee0 2024-09-12T03:20:57.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.473+0000 7faae90e7640 1 --2- 172.21.15.6:0/3605550021 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7faad00a2ca0 0x7faad00a5160 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faae000abe0 tx=0x7faae000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:57.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.477+0000 7faada7fc640 1 -- 172.21.15.6:0/3605550021 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faad4020030 con 0x7faae4104ee0 2024-09-12T03:20:57.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.650+0000 7faaeb34b640 1 -- 172.21.15.6:0/3605550021 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faae4062630 con 0x7faad00a2ca0 2024-09-12T03:20:57.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.656+0000 7faada7fc640 1 -- 172.21.15.6:0/3605550021 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258918 (secure 0 0 0) 0x7faae4062630 con 0x7faad00a2ca0 2024-09-12T03:20:57.659 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:20:57.660 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-09-12T03:20:57.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.659+0000 7faaeb34b640 1 -- 172.21.15.6:0/3605550021 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7faad00a2ca0 msgr2=0x7faad00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.659+0000 7faaeb34b640 1 --2- 172.21.15.6:0/3605550021 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7faad00a2ca0 0x7faad00a5160 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faae000abe0 tx=0x7faae000a2d0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.659+0000 7faaeb34b640 1 -- 172.21.15.6:0/3605550021 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faae4104ee0 msgr2=0x7faae4100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.659+0000 7faaeb34b640 1 --2- 172.21.15.6:0/3605550021 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faae4104ee0 0x7faae4100560 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7faad4005fd0 tx=0x7faad400dd20 comp rx=0 tx=0).stop 2024-09-12T03:20:57.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.659+0000 7faaeb34b640 1 -- 172.21.15.6:0/3605550021 shutdown_connections 2024-09-12T03:20:57.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.659+0000 7faaeb34b640 1 --2- 172.21.15.6:0/3605550021 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7faad00a2ca0 0x7faad00a5160 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.659+0000 7faaeb34b640 1 --2- 172.21.15.6:0/3605550021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faae4107810 0x7faae4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.659+0000 7faaeb34b640 1 --2- 172.21.15.6:0/3605550021 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faae4104ee0 0x7faae4100560 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.659+0000 7faaeb34b640 1 --2- 172.21.15.6:0/3605550021 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faae41025b0 0x7faae4101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.659+0000 7faaeb34b640 1 -- 172.21.15.6:0/3605550021 >> 172.21.15.6:0/3605550021 conn(0x7faae40fc1b0 msgr2=0x7faae4102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:57.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.659+0000 7faaeb34b640 1 -- 172.21.15.6:0/3605550021 shutdown_connections 2024-09-12T03:20:57.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.659+0000 7faaeb34b640 1 -- 172.21.15.6:0/3605550021 wait complete. 2024-09-12T03:20:57.670 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-12T03:20:57.264063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656161,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":727,"num_read_kb":476,"num_write":362,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":790,"ondisk_log_size":790,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382596,"kb_used_data":12548,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559228,"statfs":{"total":1151940427776,"available":1151548649472,"internally_reserved":0,"allocated":12849152,"data_stored":7172088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1792,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":36,"num_write":15,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002802"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265565+0000","last_change":"2024-09-12T03:19:09.343829+0000","last_active":"2024-09-12T03:20:49.265565+0000","last_peered":"2024-09-12T03:20:49.265565+0000","last_clean":"2024-09-12T03:20:49.265565+0000","last_became_active":"2024-09-12T03:19:09.343228+0000","last_became_peered":"2024-09-12T03:19:09.343228+0000","last_unstale":"2024-09-12T03:20:49.265565+0000","last_undegraded":"2024-09-12T03:20:49.265565+0000","last_fullsized":"2024-09-12T03:20:49.265565+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:08.720934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.237141+0000","last_change":"2024-09-12T03:19:09.331829+0000","last_active":"2024-09-12T03:20:49.237141+0000","last_peered":"2024-09-12T03:20:49.237141+0000","last_clean":"2024-09-12T03:20:49.237141+0000","last_became_active":"2024-09-12T03:19:09.331276+0000","last_became_peered":"2024-09-12T03:19:09.331276+0000","last_unstale":"2024-09-12T03:20:49.237141+0000","last_undegraded":"2024-09-12T03:20:49.237141+0000","last_fullsized":"2024-09-12T03:20:49.237141+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:48.672085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.263951+0000","last_change":"2024-09-12T03:19:09.338120+0000","last_active":"2024-09-12T03:20:49.263951+0000","last_peered":"2024-09-12T03:20:49.263951+0000","last_clean":"2024-09-12T03:20:49.263951+0000","last_became_active":"2024-09-12T03:19:09.337773+0000","last_became_peered":"2024-09-12T03:19:09.337773+0000","last_unstale":"2024-09-12T03:20:49.263951+0000","last_undegraded":"2024-09-12T03:20:49.263951+0000","last_fullsized":"2024-09-12T03:20:49.263951+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:13.716047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254592+0000","last_change":"2024-09-12T03:19:09.339404+0000","last_active":"2024-09-12T03:20:49.254592+0000","last_peered":"2024-09-12T03:20:49.254592+0000","last_clean":"2024-09-12T03:20:49.254592+0000","last_became_active":"2024-09-12T03:19:09.339105+0000","last_became_peered":"2024-09-12T03:19:09.339105+0000","last_unstale":"2024-09-12T03:20:49.254592+0000","last_undegraded":"2024-09-12T03:20:49.254592+0000","last_fullsized":"2024-09-12T03:20:49.254592+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:37.930963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.242261+0000","last_change":"2024-09-12T03:19:09.326285+0000","last_active":"2024-09-12T03:20:49.242261+0000","last_peered":"2024-09-12T03:20:49.242261+0000","last_clean":"2024-09-12T03:20:49.242261+0000","last_became_active":"2024-09-12T03:19:09.325852+0000","last_became_peered":"2024-09-12T03:19:09.325852+0000","last_unstale":"2024-09-12T03:20:49.242261+0000","last_undegraded":"2024-09-12T03:20:49.242261+0000","last_fullsized":"2024-09-12T03:20:49.242261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:44:25.387014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.249918+0000","last_change":"2024-09-12T03:19:20.376197+0000","last_active":"2024-09-12T03:20:49.249918+0000","last_peered":"2024-09-12T03:20:49.249918+0000","last_clean":"2024-09-12T03:20:49.249918+0000","last_became_active":"2024-09-12T03:19:20.375744+0000","last_became_peered":"2024-09-12T03:19:20.375744+0000","last_unstale":"2024-09-12T03:20:49.249918+0000","last_undegraded":"2024-09-12T03:20:49.249918+0000","last_fullsized":"2024-09-12T03:20:49.249918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:08.663121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.245988+0000","last_change":"2024-09-12T03:19:09.335284+0000","last_active":"2024-09-12T03:20:49.245988+0000","last_peered":"2024-09-12T03:20:49.245988+0000","last_clean":"2024-09-12T03:20:49.245988+0000","last_became_active":"2024-09-12T03:19:09.334948+0000","last_became_peered":"2024-09-12T03:19:09.334948+0000","last_unstale":"2024-09-12T03:20:49.245988+0000","last_undegraded":"2024-09-12T03:20:49.245988+0000","last_fullsized":"2024-09-12T03:20:49.245988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:32.778433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250337+0000","last_change":"2024-09-12T03:19:09.338398+0000","last_active":"2024-09-12T03:20:49.250337+0000","last_peered":"2024-09-12T03:20:49.250337+0000","last_clean":"2024-09-12T03:20:49.250337+0000","last_became_active":"2024-09-12T03:19:09.337750+0000","last_became_peered":"2024-09-12T03:19:09.337750+0000","last_unstale":"2024-09-12T03:20:49.250337+0000","last_undegraded":"2024-09-12T03:20:49.250337+0000","last_fullsized":"2024-09-12T03:20:49.250337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:07.542322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265597+0000","last_change":"2024-09-12T03:19:09.340703+0000","last_active":"2024-09-12T03:20:49.265597+0000","last_peered":"2024-09-12T03:20:49.265597+0000","last_clean":"2024-09-12T03:20:49.265597+0000","last_became_active":"2024-09-12T03:19:09.340386+0000","last_became_peered":"2024-09-12T03:19:09.340386+0000","last_unstale":"2024-09-12T03:20:49.265597+0000","last_undegraded":"2024-09-12T03:20:49.265597+0000","last_fullsized":"2024-09-12T03:20:49.265597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:19.990894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248198+0000","last_change":"2024-09-12T03:19:09.341489+0000","last_active":"2024-09-12T03:20:49.248198+0000","last_peered":"2024-09-12T03:20:49.248198+0000","last_clean":"2024-09-12T03:20:49.248198+0000","last_became_active":"2024-09-12T03:19:09.341183+0000","last_became_peered":"2024-09-12T03:19:09.341183+0000","last_unstale":"2024-09-12T03:20:49.248198+0000","last_undegraded":"2024-09-12T03:20:49.248198+0000","last_fullsized":"2024-09-12T03:20:49.248198+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:05.688402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.255447+0000","last_change":"2024-09-12T03:19:09.339289+0000","last_active":"2024-09-12T03:20:49.255447+0000","last_peered":"2024-09-12T03:20:49.255447+0000","last_clean":"2024-09-12T03:20:49.255447+0000","last_became_active":"2024-09-12T03:19:09.338466+0000","last_became_peered":"2024-09-12T03:19:09.338466+0000","last_unstale":"2024-09-12T03:20:49.255447+0000","last_undegraded":"2024-09-12T03:20:49.255447+0000","last_fullsized":"2024-09-12T03:20:49.255447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:47:13.070057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254364+0000","last_change":"2024-09-12T03:19:09.337754+0000","last_active":"2024-09-12T03:20:49.254364+0000","last_peered":"2024-09-12T03:20:49.254364+0000","last_clean":"2024-09-12T03:20:49.254364+0000","last_became_active":"2024-09-12T03:19:09.332789+0000","last_became_peered":"2024-09-12T03:19:09.332789+0000","last_unstale":"2024-09-12T03:20:49.254364+0000","last_undegraded":"2024-09-12T03:20:49.254364+0000","last_fullsized":"2024-09-12T03:20:49.254364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:34.622136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250957+0000","last_change":"2024-09-12T03:19:09.333577+0000","last_active":"2024-09-12T03:20:49.250957+0000","last_peered":"2024-09-12T03:20:49.250957+0000","last_clean":"2024-09-12T03:20:49.250957+0000","last_became_active":"2024-09-12T03:19:09.333234+0000","last_became_peered":"2024-09-12T03:19:09.333234+0000","last_unstale":"2024-09-12T03:20:49.250957+0000","last_undegraded":"2024-09-12T03:20:49.250957+0000","last_fullsized":"2024-09-12T03:20:49.250957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:22.604640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.237367+0000","last_change":"2024-09-12T03:19:09.331607+0000","last_active":"2024-09-12T03:20:49.237367+0000","last_peered":"2024-09-12T03:20:49.237367+0000","last_clean":"2024-09-12T03:20:49.237367+0000","last_became_active":"2024-09-12T03:19:09.331253+0000","last_became_peered":"2024-09-12T03:19:09.331253+0000","last_unstale":"2024-09-12T03:20:49.237367+0000","last_undegraded":"2024-09-12T03:20:49.237367+0000","last_fullsized":"2024-09-12T03:20:49.237367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:11.969265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256379+0000","last_change":"2024-09-12T03:19:09.340768+0000","last_active":"2024-09-12T03:20:49.256379+0000","last_peered":"2024-09-12T03:20:49.256379+0000","last_clean":"2024-09-12T03:20:49.256379+0000","last_became_active":"2024-09-12T03:19:09.340437+0000","last_became_peered":"2024-09-12T03:19:09.340437+0000","last_unstale":"2024-09-12T03:20:49.256379+0000","last_undegraded":"2024-09-12T03:20:49.256379+0000","last_fullsized":"2024-09-12T03:20:49.256379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:32.645691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.263546+0000","last_change":"2024-09-12T03:19:09.333829+0000","last_active":"2024-09-12T03:20:49.263546+0000","last_peered":"2024-09-12T03:20:49.263546+0000","last_clean":"2024-09-12T03:20:49.263546+0000","last_became_active":"2024-09-12T03:19:09.333462+0000","last_became_peered":"2024-09-12T03:19:09.333462+0000","last_unstale":"2024-09-12T03:20:49.263546+0000","last_undegraded":"2024-09-12T03:20:49.263546+0000","last_fullsized":"2024-09-12T03:20:49.263546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:18:57.586836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265659+0000","last_change":"2024-09-12T03:19:09.344068+0000","last_active":"2024-09-12T03:20:49.265659+0000","last_peered":"2024-09-12T03:20:49.265659+0000","last_clean":"2024-09-12T03:20:49.265659+0000","last_became_active":"2024-09-12T03:19:09.343751+0000","last_became_peered":"2024-09-12T03:19:09.343751+0000","last_unstale":"2024-09-12T03:20:49.265659+0000","last_undegraded":"2024-09-12T03:20:49.265659+0000","last_fullsized":"2024-09-12T03:20:49.265659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:52.425547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.249413+0000","last_change":"2024-09-12T03:19:09.330828+0000","last_active":"2024-09-12T03:20:49.249413+0000","last_peered":"2024-09-12T03:20:49.249413+0000","last_clean":"2024-09-12T03:20:49.249413+0000","last_became_active":"2024-09-12T03:19:09.330497+0000","last_became_peered":"2024-09-12T03:19:09.330497+0000","last_unstale":"2024-09-12T03:20:49.249413+0000","last_undegraded":"2024-09-12T03:20:49.249413+0000","last_fullsized":"2024-09-12T03:20:49.249413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:21:20.064177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.245548+0000","last_change":"2024-09-12T03:19:09.344403+0000","last_active":"2024-09-12T03:20:49.245548+0000","last_peered":"2024-09-12T03:20:49.245548+0000","last_clean":"2024-09-12T03:20:49.245548+0000","last_became_active":"2024-09-12T03:19:09.343946+0000","last_became_peered":"2024-09-12T03:19:09.343946+0000","last_unstale":"2024-09-12T03:20:49.245548+0000","last_undegraded":"2024-09-12T03:20:49.245548+0000","last_fullsized":"2024-09-12T03:20:49.245548+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:48.386117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248484+0000","last_change":"2024-09-12T03:19:09.341481+0000","last_active":"2024-09-12T03:20:49.248484+0000","last_peered":"2024-09-12T03:20:49.248484+0000","last_clean":"2024-09-12T03:20:49.248484+0000","last_became_active":"2024-09-12T03:19:09.341142+0000","last_became_peered":"2024-09-12T03:19:09.341142+0000","last_unstale":"2024-09-12T03:20:49.248484+0000","last_undegraded":"2024-09-12T03:20:49.248484+0000","last_fullsized":"2024-09-12T03:20:49.248484+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:42.371221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.249677+0000","last_change":"2024-09-12T03:19:09.330621+0000","last_active":"2024-09-12T03:20:49.249677+0000","last_peered":"2024-09-12T03:20:49.249677+0000","last_clean":"2024-09-12T03:20:49.249677+0000","last_became_active":"2024-09-12T03:19:09.330169+0000","last_became_peered":"2024-09-12T03:19:09.330169+0000","last_unstale":"2024-09-12T03:20:49.249677+0000","last_undegraded":"2024-09-12T03:20:49.249677+0000","last_fullsized":"2024-09-12T03:20:49.249677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:16.634141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.264220+0000","last_change":"2024-09-12T03:19:09.332817+0000","last_active":"2024-09-12T03:20:49.264220+0000","last_peered":"2024-09-12T03:20:49.264220+0000","last_clean":"2024-09-12T03:20:49.264220+0000","last_became_active":"2024-09-12T03:19:09.332236+0000","last_became_peered":"2024-09-12T03:19:09.332236+0000","last_unstale":"2024-09-12T03:20:49.264220+0000","last_undegraded":"2024-09-12T03:20:49.264220+0000","last_fullsized":"2024-09-12T03:20:49.264220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:28.024294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252477+0000","last_change":"2024-09-12T03:19:09.337910+0000","last_active":"2024-09-12T03:20:49.252477+0000","last_peered":"2024-09-12T03:20:49.252477+0000","last_clean":"2024-09-12T03:20:49.252477+0000","last_became_active":"2024-09-12T03:19:09.337176+0000","last_became_peered":"2024-09-12T03:19:09.337176+0000","last_unstale":"2024-09-12T03:20:49.252477+0000","last_undegraded":"2024-09-12T03:20:49.252477+0000","last_fullsized":"2024-09-12T03:20:49.252477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:53.795898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.239957+0000","last_change":"2024-09-12T03:19:09.337902+0000","last_active":"2024-09-12T03:20:49.239957+0000","last_peered":"2024-09-12T03:20:49.239957+0000","last_clean":"2024-09-12T03:20:49.239957+0000","last_became_active":"2024-09-12T03:19:09.337567+0000","last_became_peered":"2024-09-12T03:19:09.337567+0000","last_unstale":"2024-09-12T03:20:49.239957+0000","last_undegraded":"2024-09-12T03:20:49.239957+0000","last_fullsized":"2024-09-12T03:20:49.239957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:08.471313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248557+0000","last_change":"2024-09-12T03:19:09.339850+0000","last_active":"2024-09-12T03:20:49.248557+0000","last_peered":"2024-09-12T03:20:49.248557+0000","last_clean":"2024-09-12T03:20:49.248557+0000","last_became_active":"2024-09-12T03:19:09.339563+0000","last_became_peered":"2024-09-12T03:19:09.339563+0000","last_unstale":"2024-09-12T03:20:49.248557+0000","last_undegraded":"2024-09-12T03:20:49.248557+0000","last_fullsized":"2024-09-12T03:20:49.248557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:27.473351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.247769+0000","last_change":"2024-09-12T03:19:09.330555+0000","last_active":"2024-09-12T03:20:49.247769+0000","last_peered":"2024-09-12T03:20:49.247769+0000","last_clean":"2024-09-12T03:20:49.247769+0000","last_became_active":"2024-09-12T03:19:09.330183+0000","last_became_peered":"2024-09-12T03:19:09.330183+0000","last_unstale":"2024-09-12T03:20:49.247769+0000","last_undegraded":"2024-09-12T03:20:49.247769+0000","last_fullsized":"2024-09-12T03:20:49.247769+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:39.334116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252190+0000","last_change":"2024-09-12T03:19:09.338601+0000","last_active":"2024-09-12T03:20:49.252190+0000","last_peered":"2024-09-12T03:20:49.252190+0000","last_clean":"2024-09-12T03:20:49.252190+0000","last_became_active":"2024-09-12T03:19:09.337903+0000","last_became_peered":"2024-09-12T03:19:09.337903+0000","last_unstale":"2024-09-12T03:20:49.252190+0000","last_undegraded":"2024-09-12T03:20:49.252190+0000","last_fullsized":"2024-09-12T03:20:49.252190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:17.810139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250990+0000","last_change":"2024-09-12T03:19:09.326629+0000","last_active":"2024-09-12T03:20:49.250990+0000","last_peered":"2024-09-12T03:20:49.250990+0000","last_clean":"2024-09-12T03:20:49.250990+0000","last_became_active":"2024-09-12T03:19:09.326333+0000","last_became_peered":"2024-09-12T03:19:09.326333+0000","last_unstale":"2024-09-12T03:20:49.250990+0000","last_undegraded":"2024-09-12T03:20:49.250990+0000","last_fullsized":"2024-09-12T03:20:49.250990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:20.807593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256412+0000","last_change":"2024-09-12T03:19:09.339919+0000","last_active":"2024-09-12T03:20:49.256412+0000","last_peered":"2024-09-12T03:20:49.256412+0000","last_clean":"2024-09-12T03:20:49.256412+0000","last_became_active":"2024-09-12T03:19:09.339618+0000","last_became_peered":"2024-09-12T03:19:09.339618+0000","last_unstale":"2024-09-12T03:20:49.256412+0000","last_undegraded":"2024-09-12T03:20:49.256412+0000","last_fullsized":"2024-09-12T03:20:49.256412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:11:03.740079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.246446+0000","last_change":"2024-09-12T03:19:09.336148+0000","last_active":"2024-09-12T03:20:49.246446+0000","last_peered":"2024-09-12T03:20:49.246446+0000","last_clean":"2024-09-12T03:20:49.246446+0000","last_became_active":"2024-09-12T03:19:09.335779+0000","last_became_peered":"2024-09-12T03:19:09.335779+0000","last_unstale":"2024-09-12T03:20:49.246446+0000","last_undegraded":"2024-09-12T03:20:49.246446+0000","last_fullsized":"2024-09-12T03:20:49.246446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:18.234320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.247971+0000","last_change":"2024-09-12T03:19:20.382536+0000","last_active":"2024-09-12T03:20:49.247971+0000","last_peered":"2024-09-12T03:20:49.247971+0000","last_clean":"2024-09-12T03:20:49.247971+0000","last_became_active":"2024-09-12T03:19:20.382118+0000","last_became_peered":"2024-09-12T03:19:20.382118+0000","last_unstale":"2024-09-12T03:20:49.247971+0000","last_undegraded":"2024-09-12T03:20:49.247971+0000","last_fullsized":"2024-09-12T03:20:49.247971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:55.836646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252455+0000","last_change":"2024-09-12T03:19:09.345064+0000","last_active":"2024-09-12T03:20:49.252455+0000","last_peered":"2024-09-12T03:20:49.252455+0000","last_clean":"2024-09-12T03:20:49.252455+0000","last_became_active":"2024-09-12T03:19:09.344628+0000","last_became_peered":"2024-09-12T03:19:09.344628+0000","last_unstale":"2024-09-12T03:20:49.252455+0000","last_undegraded":"2024-09-12T03:20:49.252455+0000","last_fullsized":"2024-09-12T03:20:49.252455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:50.011751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.244081+0000","last_change":"2024-09-12T03:19:09.338052+0000","last_active":"2024-09-12T03:20:49.244081+0000","last_peered":"2024-09-12T03:20:49.244081+0000","last_clean":"2024-09-12T03:20:49.244081+0000","last_became_active":"2024-09-12T03:19:09.337710+0000","last_became_peered":"2024-09-12T03:19:09.337710+0000","last_unstale":"2024-09-12T03:20:49.244081+0000","last_undegraded":"2024-09-12T03:20:49.244081+0000","last_fullsized":"2024-09-12T03:20:49.244081+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:36.521269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'9","reported_seq":134,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:54.518339+0000","last_change":"2024-09-12T03:19:07.208341+0000","last_active":"2024-09-12T03:20:54.518339+0000","last_peered":"2024-09-12T03:20:54.518339+0000","last_clean":"2024-09-12T03:20:54.518339+0000","last_became_active":"2024-09-12T03:19:07.207385+0000","last_became_peered":"2024-09-12T03:19:07.207385+0000","last_unstale":"2024-09-12T03:20:54.518339+0000","last_undegraded":"2024-09-12T03:20:54.518339+0000","last_fullsized":"2024-09-12T03:20:54.518339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:18.476795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1857,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254115+0000","last_change":"2024-09-12T03:19:09.339997+0000","last_active":"2024-09-12T03:20:49.254115+0000","last_peered":"2024-09-12T03:20:49.254115+0000","last_clean":"2024-09-12T03:20:49.254115+0000","last_became_active":"2024-09-12T03:19:09.339261+0000","last_became_peered":"2024-09-12T03:19:09.339261+0000","last_unstale":"2024-09-12T03:20:49.254115+0000","last_undegraded":"2024-09-12T03:20:49.254115+0000","last_fullsized":"2024-09-12T03:20:49.254115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:07.670345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374030+0000","last_change":"2024-09-12T03:19:07.200222+0000","last_active":"2024-09-12T03:19:20.374030+0000","last_peered":"2024-09-12T03:19:20.374030+0000","last_clean":"2024-09-12T03:19:20.374030+0000","last_became_active":"2024-09-12T03:19:07.199819+0000","last_became_peered":"2024-09-12T03:19:07.199819+0000","last_unstale":"2024-09-12T03:19:20.374030+0000","last_undegraded":"2024-09-12T03:19:20.374030+0000","last_fullsized":"2024-09-12T03:19:20.374030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:39:28.933609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375686+0000","last_change":"2024-09-12T03:19:07.207005+0000","last_active":"2024-09-12T03:19:20.375686+0000","last_peered":"2024-09-12T03:19:20.375686+0000","last_clean":"2024-09-12T03:19:20.375686+0000","last_became_active":"2024-09-12T03:19:07.206685+0000","last_became_peered":"2024-09-12T03:19:07.206685+0000","last_unstale":"2024-09-12T03:19:20.375686+0000","last_undegraded":"2024-09-12T03:19:20.375686+0000","last_fullsized":"2024-09-12T03:19:20.375686+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:31.111706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252939+0000","last_change":"2024-09-12T03:19:09.335871+0000","last_active":"2024-09-12T03:20:49.252939+0000","last_peered":"2024-09-12T03:20:49.252939+0000","last_clean":"2024-09-12T03:20:49.252939+0000","last_became_active":"2024-09-12T03:19:09.335568+0000","last_became_peered":"2024-09-12T03:19:09.335568+0000","last_unstale":"2024-09-12T03:20:49.252939+0000","last_undegraded":"2024-09-12T03:20:49.252939+0000","last_fullsized":"2024-09-12T03:20:49.252939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:52:34.471512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386049+0000","last_change":"2024-09-12T03:19:07.208320+0000","last_active":"2024-09-12T03:19:20.386049+0000","last_peered":"2024-09-12T03:19:20.386049+0000","last_clean":"2024-09-12T03:19:20.386049+0000","last_became_active":"2024-09-12T03:19:07.207873+0000","last_became_peered":"2024-09-12T03:19:07.207873+0000","last_unstale":"2024-09-12T03:19:20.386049+0000","last_undegraded":"2024-09-12T03:19:20.386049+0000","last_fullsized":"2024-09-12T03:19:20.386049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:00.128595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250364+0000","last_change":"2024-09-12T03:19:09.331930+0000","last_active":"2024-09-12T03:20:49.250364+0000","last_peered":"2024-09-12T03:20:49.250364+0000","last_clean":"2024-09-12T03:20:49.250364+0000","last_became_active":"2024-09-12T03:19:09.331558+0000","last_became_peered":"2024-09-12T03:19:09.331558+0000","last_unstale":"2024-09-12T03:20:49.250364+0000","last_undegraded":"2024-09-12T03:20:49.250364+0000","last_fullsized":"2024-09-12T03:20:49.250364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:00:27.935717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":120,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:51.511645+0000","last_change":"2024-09-12T03:19:07.205473+0000","last_active":"2024-09-12T03:20:51.511645+0000","last_peered":"2024-09-12T03:20:51.511645+0000","last_clean":"2024-09-12T03:20:51.511645+0000","last_became_active":"2024-09-12T03:19:07.205134+0000","last_became_peered":"2024-09-12T03:19:07.205134+0000","last_unstale":"2024-09-12T03:20:51.511645+0000","last_undegraded":"2024-09-12T03:20:51.511645+0000","last_fullsized":"2024-09-12T03:20:51.511645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:23.443957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265317+0000","last_change":"2024-09-12T03:19:09.333868+0000","last_active":"2024-09-12T03:20:49.265317+0000","last_peered":"2024-09-12T03:20:49.265317+0000","last_clean":"2024-09-12T03:20:49.265317+0000","last_became_active":"2024-09-12T03:19:09.333484+0000","last_became_peered":"2024-09-12T03:19:09.333484+0000","last_unstale":"2024-09-12T03:20:49.265317+0000","last_undegraded":"2024-09-12T03:20:49.265317+0000","last_fullsized":"2024-09-12T03:20:49.265317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:21:10.226821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386303+0000","last_change":"2024-09-12T03:19:20.386303+0000","last_active":"2024-09-12T03:19:20.386303+0000","last_peered":"2024-09-12T03:19:20.386303+0000","last_clean":"2024-09-12T03:19:20.386303+0000","last_became_active":"2024-09-12T03:19:20.383198+0000","last_became_peered":"2024-09-12T03:19:20.383198+0000","last_unstale":"2024-09-12T03:19:20.386303+0000","last_undegraded":"2024-09-12T03:19:20.386303+0000","last_fullsized":"2024-09-12T03:19:20.386303+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:42.224849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.257056+0000","last_change":"2024-09-12T03:19:09.340849+0000","last_active":"2024-09-12T03:20:49.257056+0000","last_peered":"2024-09-12T03:20:49.257056+0000","last_clean":"2024-09-12T03:20:49.257056+0000","last_became_active":"2024-09-12T03:19:09.340517+0000","last_became_peered":"2024-09-12T03:19:09.340517+0000","last_unstale":"2024-09-12T03:20:49.257056+0000","last_undegraded":"2024-09-12T03:20:49.257056+0000","last_fullsized":"2024-09-12T03:20:49.257056+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:42.018269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386240+0000","last_change":"2024-09-12T03:19:07.389147+0000","last_active":"2024-09-12T03:19:20.386240+0000","last_peered":"2024-09-12T03:19:20.386240+0000","last_clean":"2024-09-12T03:19:20.386240+0000","last_became_active":"2024-09-12T03:19:07.388744+0000","last_became_peered":"2024-09-12T03:19:07.388744+0000","last_unstale":"2024-09-12T03:19:20.386240+0000","last_undegraded":"2024-09-12T03:19:20.386240+0000","last_fullsized":"2024-09-12T03:19:20.386240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:16.700644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.264622+0000","last_change":"2024-09-12T03:19:09.329243+0000","last_active":"2024-09-12T03:20:49.264622+0000","last_peered":"2024-09-12T03:20:49.264622+0000","last_clean":"2024-09-12T03:20:49.264622+0000","last_became_active":"2024-09-12T03:19:09.328610+0000","last_became_peered":"2024-09-12T03:19:09.328610+0000","last_unstale":"2024-09-12T03:20:49.264622+0000","last_undegraded":"2024-09-12T03:20:49.264622+0000","last_fullsized":"2024-09-12T03:20:49.264622+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:49.452942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'32","reported_seq":251,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:52.513814+0000","last_change":"2024-09-12T03:19:07.207804+0000","last_active":"2024-09-12T03:20:52.513814+0000","last_peered":"2024-09-12T03:20:52.513814+0000","last_clean":"2024-09-12T03:20:52.513814+0000","last_became_active":"2024-09-12T03:19:07.207409+0000","last_became_peered":"2024-09-12T03:19:07.207409+0000","last_unstale":"2024-09-12T03:20:52.513814+0000","last_undegraded":"2024-09-12T03:20:52.513814+0000","last_fullsized":"2024-09-12T03:20:52.513814+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:10.732423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60321,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":197,"num_read_kb":157,"num_write":33,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.255978+0000","last_change":"2024-09-12T03:19:09.333102+0000","last_active":"2024-09-12T03:20:49.255978+0000","last_peered":"2024-09-12T03:20:49.255978+0000","last_clean":"2024-09-12T03:20:49.255978+0000","last_became_active":"2024-09-12T03:19:09.332712+0000","last_became_peered":"2024-09-12T03:19:09.332712+0000","last_unstale":"2024-09-12T03:20:49.255978+0000","last_undegraded":"2024-09-12T03:20:49.255978+0000","last_fullsized":"2024-09-12T03:20:49.255978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:22.912213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382902+0000","last_change":"2024-09-12T03:19:20.382348+0000","last_active":"2024-09-12T03:19:20.382902+0000","last_peered":"2024-09-12T03:19:20.382902+0000","last_clean":"2024-09-12T03:19:20.382902+0000","last_became_active":"2024-09-12T03:19:20.375142+0000","last_became_peered":"2024-09-12T03:19:20.375142+0000","last_unstale":"2024-09-12T03:19:20.382902+0000","last_undegraded":"2024-09-12T03:19:20.382902+0000","last_fullsized":"2024-09-12T03:19:20.382902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:04.573599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.246879+0000","last_change":"2024-09-12T03:19:09.333194+0000","last_active":"2024-09-12T03:20:49.246879+0000","last_peered":"2024-09-12T03:20:49.246879+0000","last_clean":"2024-09-12T03:20:49.246879+0000","last_became_active":"2024-09-12T03:19:09.332865+0000","last_became_peered":"2024-09-12T03:19:09.332865+0000","last_unstale":"2024-09-12T03:20:49.246879+0000","last_undegraded":"2024-09-12T03:20:49.246879+0000","last_fullsized":"2024-09-12T03:20:49.246879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:57.096138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386176+0000","last_change":"2024-09-12T03:19:07.209289+0000","last_active":"2024-09-12T03:19:20.386176+0000","last_peered":"2024-09-12T03:19:20.386176+0000","last_clean":"2024-09-12T03:19:20.386176+0000","last_became_active":"2024-09-12T03:19:07.208863+0000","last_became_peered":"2024-09-12T03:19:07.208863+0000","last_unstale":"2024-09-12T03:19:20.386176+0000","last_undegraded":"2024-09-12T03:19:20.386176+0000","last_fullsized":"2024-09-12T03:19:20.386176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:26.168148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256973+0000","last_change":"2024-09-12T03:19:09.343620+0000","last_active":"2024-09-12T03:20:49.256973+0000","last_peered":"2024-09-12T03:20:49.256973+0000","last_clean":"2024-09-12T03:20:49.256973+0000","last_became_active":"2024-09-12T03:19:09.343299+0000","last_became_peered":"2024-09-12T03:19:09.343299+0000","last_unstale":"2024-09-12T03:20:49.256973+0000","last_undegraded":"2024-09-12T03:20:49.256973+0000","last_fullsized":"2024-09-12T03:20:49.256973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:34.268886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":126,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:55.551460+0000","last_change":"2024-09-12T03:19:07.206160+0000","last_active":"2024-09-12T03:20:55.551460+0000","last_peered":"2024-09-12T03:20:55.551460+0000","last_clean":"2024-09-12T03:20:55.551460+0000","last_became_active":"2024-09-12T03:19:07.205509+0000","last_became_peered":"2024-09-12T03:19:07.205509+0000","last_unstale":"2024-09-12T03:20:55.551460+0000","last_undegraded":"2024-09-12T03:20:55.551460+0000","last_fullsized":"2024-09-12T03:20:55.551460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:35.362831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250896+0000","last_change":"2024-09-12T03:19:09.337595+0000","last_active":"2024-09-12T03:20:49.250896+0000","last_peered":"2024-09-12T03:20:49.250896+0000","last_clean":"2024-09-12T03:20:49.250896+0000","last_became_active":"2024-09-12T03:19:09.337281+0000","last_became_peered":"2024-09-12T03:19:09.337281+0000","last_unstale":"2024-09-12T03:20:49.250896+0000","last_undegraded":"2024-09-12T03:20:49.250896+0000","last_fullsized":"2024-09-12T03:20:49.250896+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:35.761679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373995+0000","last_change":"2024-09-12T03:19:07.389336+0000","last_active":"2024-09-12T03:19:20.373995+0000","last_peered":"2024-09-12T03:19:20.373995+0000","last_clean":"2024-09-12T03:19:20.373995+0000","last_became_active":"2024-09-12T03:19:07.388649+0000","last_became_peered":"2024-09-12T03:19:07.388649+0000","last_unstale":"2024-09-12T03:19:20.373995+0000","last_undegraded":"2024-09-12T03:19:20.373995+0000","last_fullsized":"2024-09-12T03:19:20.373995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:33.312700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.253821+0000","last_change":"2024-09-12T03:19:09.340674+0000","last_active":"2024-09-12T03:20:49.253821+0000","last_peered":"2024-09-12T03:20:49.253821+0000","last_clean":"2024-09-12T03:20:49.253821+0000","last_became_active":"2024-09-12T03:19:09.340144+0000","last_became_peered":"2024-09-12T03:19:09.340144+0000","last_unstale":"2024-09-12T03:20:49.253821+0000","last_undegraded":"2024-09-12T03:20:49.253821+0000","last_fullsized":"2024-09-12T03:20:49.253821+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:21:15.605086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373837+0000","last_change":"2024-09-12T03:19:07.206513+0000","last_active":"2024-09-12T03:19:20.373837+0000","last_peered":"2024-09-12T03:19:20.373837+0000","last_clean":"2024-09-12T03:19:20.373837+0000","last_became_active":"2024-09-12T03:19:07.205974+0000","last_became_peered":"2024-09-12T03:19:07.205974+0000","last_unstale":"2024-09-12T03:19:20.373837+0000","last_undegraded":"2024-09-12T03:19:20.373837+0000","last_fullsized":"2024-09-12T03:19:20.373837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:43:10.002123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252033+0000","last_change":"2024-09-12T03:19:09.342200+0000","last_active":"2024-09-12T03:20:49.252033+0000","last_peered":"2024-09-12T03:20:49.252033+0000","last_clean":"2024-09-12T03:20:49.252033+0000","last_became_active":"2024-09-12T03:19:09.341901+0000","last_became_peered":"2024-09-12T03:19:09.341901+0000","last_unstale":"2024-09-12T03:20:49.252033+0000","last_undegraded":"2024-09-12T03:20:49.252033+0000","last_fullsized":"2024-09-12T03:20:49.252033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:27.486214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375373+0000","last_change":"2024-09-12T03:19:07.207514+0000","last_active":"2024-09-12T03:19:20.375373+0000","last_peered":"2024-09-12T03:19:20.375373+0000","last_clean":"2024-09-12T03:19:20.375373+0000","last_became_active":"2024-09-12T03:19:07.205925+0000","last_became_peered":"2024-09-12T03:19:07.205925+0000","last_unstale":"2024-09-12T03:19:20.375373+0000","last_undegraded":"2024-09-12T03:19:20.375373+0000","last_fullsized":"2024-09-12T03:19:20.375373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:20.244703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256005+0000","last_change":"2024-09-12T03:19:09.337837+0000","last_active":"2024-09-12T03:20:49.256005+0000","last_peered":"2024-09-12T03:20:49.256005+0000","last_clean":"2024-09-12T03:20:49.256005+0000","last_became_active":"2024-09-12T03:19:09.337502+0000","last_became_peered":"2024-09-12T03:19:09.337502+0000","last_unstale":"2024-09-12T03:20:49.256005+0000","last_undegraded":"2024-09-12T03:20:49.256005+0000","last_fullsized":"2024-09-12T03:20:49.256005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:30.167139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386356+0000","last_change":"2024-09-12T03:19:07.209130+0000","last_active":"2024-09-12T03:19:20.386356+0000","last_peered":"2024-09-12T03:19:20.386356+0000","last_clean":"2024-09-12T03:19:20.386356+0000","last_became_active":"2024-09-12T03:19:07.208753+0000","last_became_peered":"2024-09-12T03:19:07.208753+0000","last_unstale":"2024-09-12T03:19:20.386356+0000","last_undegraded":"2024-09-12T03:19:20.386356+0000","last_fullsized":"2024-09-12T03:19:20.386356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:34.719292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250623+0000","last_change":"2024-09-12T03:19:09.336382+0000","last_active":"2024-09-12T03:20:49.250623+0000","last_peered":"2024-09-12T03:20:49.250623+0000","last_clean":"2024-09-12T03:20:49.250623+0000","last_became_active":"2024-09-12T03:19:09.336015+0000","last_became_peered":"2024-09-12T03:19:09.336015+0000","last_unstale":"2024-09-12T03:20:49.250623+0000","last_undegraded":"2024-09-12T03:20:49.250623+0000","last_fullsized":"2024-09-12T03:20:49.250623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:19.430849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375044+0000","last_change":"2024-09-12T03:19:07.528567+0000","last_active":"2024-09-12T03:19:20.375044+0000","last_peered":"2024-09-12T03:19:20.375044+0000","last_clean":"2024-09-12T03:19:20.375044+0000","last_became_active":"2024-09-12T03:19:07.528123+0000","last_became_peered":"2024-09-12T03:19:07.528123+0000","last_unstale":"2024-09-12T03:19:20.375044+0000","last_undegraded":"2024-09-12T03:19:20.375044+0000","last_fullsized":"2024-09-12T03:19:20.375044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:34.416714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256025+0000","last_change":"2024-09-12T03:19:09.338836+0000","last_active":"2024-09-12T03:20:49.256025+0000","last_peered":"2024-09-12T03:20:49.256025+0000","last_clean":"2024-09-12T03:20:49.256025+0000","last_became_active":"2024-09-12T03:19:09.337571+0000","last_became_peered":"2024-09-12T03:19:09.337571+0000","last_unstale":"2024-09-12T03:20:49.256025+0000","last_undegraded":"2024-09-12T03:20:49.256025+0000","last_fullsized":"2024-09-12T03:20:49.256025+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:39.039878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":330,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372792+0000","last_change":"2024-09-12T03:18:01.051311+0000","last_active":"2024-09-12T03:19:20.372792+0000","last_peered":"2024-09-12T03:19:20.372792+0000","last_clean":"2024-09-12T03:19:20.372792+0000","last_became_active":"2024-09-12T03:18:01.050849+0000","last_became_peered":"2024-09-12T03:18:01.050849+0000","last_unstale":"2024-09-12T03:19:20.372792+0000","last_undegraded":"2024-09-12T03:19:20.372792+0000","last_fullsized":"2024-09-12T03:19:20.372792+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:14:38.096222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:14:38.096222+0000","last_clean_scrub_stamp":"2024-09-12T03:14:38.096222+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:33.067603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382035+0000","last_change":"2024-09-12T03:19:07.206045+0000","last_active":"2024-09-12T03:19:20.382035+0000","last_peered":"2024-09-12T03:19:20.382035+0000","last_clean":"2024-09-12T03:19:20.382035+0000","last_became_active":"2024-09-12T03:19:07.205594+0000","last_became_peered":"2024-09-12T03:19:07.205594+0000","last_unstale":"2024-09-12T03:19:20.382035+0000","last_undegraded":"2024-09-12T03:19:20.382035+0000","last_fullsized":"2024-09-12T03:19:20.382035+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:12.811906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.244733+0000","last_change":"2024-09-12T03:19:20.387274+0000","last_active":"2024-09-12T03:20:49.244733+0000","last_peered":"2024-09-12T03:20:49.244733+0000","last_clean":"2024-09-12T03:20:49.244733+0000","last_became_active":"2024-09-12T03:19:20.386878+0000","last_became_peered":"2024-09-12T03:19:20.386878+0000","last_unstale":"2024-09-12T03:20:49.244733+0000","last_undegraded":"2024-09-12T03:20:49.244733+0000","last_fullsized":"2024-09-12T03:20:49.244733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:26.674032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372302+0000","last_change":"2024-09-12T03:19:07.206212+0000","last_active":"2024-09-12T03:19:20.372302+0000","last_peered":"2024-09-12T03:19:20.372302+0000","last_clean":"2024-09-12T03:19:20.372302+0000","last_became_active":"2024-09-12T03:19:07.205899+0000","last_became_peered":"2024-09-12T03:19:07.205899+0000","last_unstale":"2024-09-12T03:19:20.372302+0000","last_undegraded":"2024-09-12T03:19:20.372302+0000","last_fullsized":"2024-09-12T03:19:20.372302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:02.169436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250795+0000","last_change":"2024-09-12T03:19:20.377494+0000","last_active":"2024-09-12T03:20:49.250795+0000","last_peered":"2024-09-12T03:20:49.250795+0000","last_clean":"2024-09-12T03:20:49.250795+0000","last_became_active":"2024-09-12T03:19:20.377091+0000","last_became_peered":"2024-09-12T03:19:20.377091+0000","last_unstale":"2024-09-12T03:20:49.250795+0000","last_undegraded":"2024-09-12T03:20:49.250795+0000","last_fullsized":"2024-09-12T03:20:49.250795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:30.926853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":119,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:51.542783+0000","last_change":"2024-09-12T03:19:07.388486+0000","last_active":"2024-09-12T03:20:51.542783+0000","last_peered":"2024-09-12T03:20:51.542783+0000","last_clean":"2024-09-12T03:20:51.542783+0000","last_became_active":"2024-09-12T03:19:07.388125+0000","last_became_peered":"2024-09-12T03:19:07.388125+0000","last_unstale":"2024-09-12T03:20:51.542783+0000","last_undegraded":"2024-09-12T03:20:51.542783+0000","last_fullsized":"2024-09-12T03:20:51.542783+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:46.218601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254069+0000","last_change":"2024-09-12T03:19:09.344308+0000","last_active":"2024-09-12T03:20:49.254069+0000","last_peered":"2024-09-12T03:20:49.254069+0000","last_clean":"2024-09-12T03:20:49.254069+0000","last_became_active":"2024-09-12T03:19:09.343987+0000","last_became_peered":"2024-09-12T03:19:09.343987+0000","last_unstale":"2024-09-12T03:20:49.254069+0000","last_undegraded":"2024-09-12T03:20:49.254069+0000","last_fullsized":"2024-09-12T03:20:49.254069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:16:11.616555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372560+0000","last_change":"2024-09-12T03:19:07.208619+0000","last_active":"2024-09-12T03:19:20.372560+0000","last_peered":"2024-09-12T03:19:20.372560+0000","last_clean":"2024-09-12T03:19:20.372560+0000","last_became_active":"2024-09-12T03:19:07.208254+0000","last_became_peered":"2024-09-12T03:19:07.208254+0000","last_unstale":"2024-09-12T03:19:20.372560+0000","last_undegraded":"2024-09-12T03:19:20.372560+0000","last_fullsized":"2024-09-12T03:19:20.372560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:56.177948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.238148+0000","last_change":"2024-09-12T03:19:09.341837+0000","last_active":"2024-09-12T03:20:49.238148+0000","last_peered":"2024-09-12T03:20:49.238148+0000","last_clean":"2024-09-12T03:20:49.238148+0000","last_became_active":"2024-09-12T03:19:09.341069+0000","last_became_peered":"2024-09-12T03:19:09.341069+0000","last_unstale":"2024-09-12T03:20:49.238148+0000","last_undegraded":"2024-09-12T03:20:49.238148+0000","last_fullsized":"2024-09-12T03:20:49.238148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:04.238986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372460+0000","last_change":"2024-09-12T03:19:07.388870+0000","last_active":"2024-09-12T03:19:20.372460+0000","last_peered":"2024-09-12T03:19:20.372460+0000","last_clean":"2024-09-12T03:19:20.372460+0000","last_became_active":"2024-09-12T03:19:07.388555+0000","last_became_peered":"2024-09-12T03:19:07.388555+0000","last_unstale":"2024-09-12T03:19:20.372460+0000","last_undegraded":"2024-09-12T03:19:20.372460+0000","last_fullsized":"2024-09-12T03:19:20.372460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:29.789454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.247542+0000","last_change":"2024-09-12T03:19:09.334714+0000","last_active":"2024-09-12T03:20:49.247542+0000","last_peered":"2024-09-12T03:20:49.247542+0000","last_clean":"2024-09-12T03:20:49.247542+0000","last_became_active":"2024-09-12T03:19:09.334413+0000","last_became_peered":"2024-09-12T03:19:09.334413+0000","last_unstale":"2024-09-12T03:20:49.247542+0000","last_undegraded":"2024-09-12T03:20:49.247542+0000","last_fullsized":"2024-09-12T03:20:49.247542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:52.222340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373797+0000","last_change":"2024-09-12T03:19:07.389055+0000","last_active":"2024-09-12T03:19:20.373797+0000","last_peered":"2024-09-12T03:19:20.373797+0000","last_clean":"2024-09-12T03:19:20.373797+0000","last_became_active":"2024-09-12T03:19:07.388673+0000","last_became_peered":"2024-09-12T03:19:07.388673+0000","last_unstale":"2024-09-12T03:19:20.373797+0000","last_undegraded":"2024-09-12T03:19:20.373797+0000","last_fullsized":"2024-09-12T03:19:20.373797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:20.974533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248932+0000","last_change":"2024-09-12T03:19:09.340105+0000","last_active":"2024-09-12T03:20:49.248932+0000","last_peered":"2024-09-12T03:20:49.248932+0000","last_clean":"2024-09-12T03:20:49.248932+0000","last_became_active":"2024-09-12T03:19:09.339642+0000","last_became_peered":"2024-09-12T03:19:09.339642+0000","last_unstale":"2024-09-12T03:20:49.248932+0000","last_undegraded":"2024-09-12T03:20:49.248932+0000","last_fullsized":"2024-09-12T03:20:49.248932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:01.624236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'15","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:52.343269+0000","last_change":"2024-09-12T03:19:07.207232+0000","last_active":"2024-09-12T03:20:52.343269+0000","last_peered":"2024-09-12T03:20:52.343269+0000","last_clean":"2024-09-12T03:20:52.343269+0000","last_became_active":"2024-09-12T03:19:07.206921+0000","last_became_peered":"2024-09-12T03:19:07.206921+0000","last_unstale":"2024-09-12T03:20:52.343269+0000","last_undegraded":"2024-09-12T03:20:52.343269+0000","last_fullsized":"2024-09-12T03:20:52.343269+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:25.457853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252490+0000","last_change":"2024-09-12T03:19:09.330442+0000","last_active":"2024-09-12T03:20:49.252490+0000","last_peered":"2024-09-12T03:20:49.252490+0000","last_clean":"2024-09-12T03:20:49.252490+0000","last_became_active":"2024-09-12T03:19:09.330081+0000","last_became_peered":"2024-09-12T03:19:09.330081+0000","last_unstale":"2024-09-12T03:20:49.252490+0000","last_undegraded":"2024-09-12T03:20:49.252490+0000","last_fullsized":"2024-09-12T03:20:49.252490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:56.152562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382392+0000","last_change":"2024-09-12T03:19:07.528646+0000","last_active":"2024-09-12T03:19:20.382392+0000","last_peered":"2024-09-12T03:19:20.382392+0000","last_clean":"2024-09-12T03:19:20.382392+0000","last_became_active":"2024-09-12T03:19:07.528334+0000","last_became_peered":"2024-09-12T03:19:07.528334+0000","last_unstale":"2024-09-12T03:19:20.382392+0000","last_undegraded":"2024-09-12T03:19:20.382392+0000","last_fullsized":"2024-09-12T03:19:20.382392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:40.425579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256685+0000","last_change":"2024-09-12T03:19:09.331993+0000","last_active":"2024-09-12T03:20:49.256685+0000","last_peered":"2024-09-12T03:20:49.256685+0000","last_clean":"2024-09-12T03:20:49.256685+0000","last_became_active":"2024-09-12T03:19:09.331698+0000","last_became_peered":"2024-09-12T03:19:09.331698+0000","last_unstale":"2024-09-12T03:20:49.256685+0000","last_undegraded":"2024-09-12T03:20:49.256685+0000","last_fullsized":"2024-09-12T03:20:49.256685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:03:54.182833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370179+0000","last_change":"2024-09-12T03:19:07.388297+0000","last_active":"2024-09-12T03:19:20.370179+0000","last_peered":"2024-09-12T03:19:20.370179+0000","last_clean":"2024-09-12T03:19:20.370179+0000","last_became_active":"2024-09-12T03:19:07.388038+0000","last_became_peered":"2024-09-12T03:19:07.388038+0000","last_unstale":"2024-09-12T03:19:20.370179+0000","last_undegraded":"2024-09-12T03:19:20.370179+0000","last_fullsized":"2024-09-12T03:19:20.370179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:45.874195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.212080+0000","last_change":"2024-09-12T03:19:09.331525+0000","last_active":"2024-09-12T03:20:49.212080+0000","last_peered":"2024-09-12T03:20:49.212080+0000","last_clean":"2024-09-12T03:20:49.212080+0000","last_became_active":"2024-09-12T03:19:09.330903+0000","last_became_peered":"2024-09-12T03:19:09.330903+0000","last_unstale":"2024-09-12T03:20:49.212080+0000","last_undegraded":"2024-09-12T03:20:49.212080+0000","last_fullsized":"2024-09-12T03:20:49.212080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:02.455132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372235+0000","last_change":"2024-09-12T03:19:07.205181+0000","last_active":"2024-09-12T03:19:20.372235+0000","last_peered":"2024-09-12T03:19:20.372235+0000","last_clean":"2024-09-12T03:19:20.372235+0000","last_became_active":"2024-09-12T03:19:07.204726+0000","last_became_peered":"2024-09-12T03:19:07.204726+0000","last_unstale":"2024-09-12T03:19:20.372235+0000","last_undegraded":"2024-09-12T03:19:20.372235+0000","last_fullsized":"2024-09-12T03:19:20.372235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:26.112800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250768+0000","last_change":"2024-09-12T03:19:09.345338+0000","last_active":"2024-09-12T03:20:49.250768+0000","last_peered":"2024-09-12T03:20:49.250768+0000","last_clean":"2024-09-12T03:20:49.250768+0000","last_became_active":"2024-09-12T03:19:09.345023+0000","last_became_peered":"2024-09-12T03:19:09.345023+0000","last_unstale":"2024-09-12T03:20:49.250768+0000","last_undegraded":"2024-09-12T03:20:49.250768+0000","last_fullsized":"2024-09-12T03:20:49.250768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:39.184065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374938+0000","last_change":"2024-09-12T03:19:07.211954+0000","last_active":"2024-09-12T03:19:20.374938+0000","last_peered":"2024-09-12T03:19:20.374938+0000","last_clean":"2024-09-12T03:19:20.374938+0000","last_became_active":"2024-09-12T03:19:07.211509+0000","last_became_peered":"2024-09-12T03:19:07.211509+0000","last_unstale":"2024-09-12T03:19:20.374938+0000","last_undegraded":"2024-09-12T03:19:20.374938+0000","last_fullsized":"2024-09-12T03:19:20.374938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:27.461924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.244962+0000","last_change":"2024-09-12T03:19:09.331168+0000","last_active":"2024-09-12T03:20:49.244962+0000","last_peered":"2024-09-12T03:20:49.244962+0000","last_clean":"2024-09-12T03:20:49.244962+0000","last_became_active":"2024-09-12T03:19:09.329592+0000","last_became_peered":"2024-09-12T03:19:09.329592+0000","last_unstale":"2024-09-12T03:20:49.244962+0000","last_undegraded":"2024-09-12T03:20:49.244962+0000","last_fullsized":"2024-09-12T03:20:49.244962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:46.127861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373572+0000","last_change":"2024-09-12T03:19:07.207802+0000","last_active":"2024-09-12T03:19:20.373572+0000","last_peered":"2024-09-12T03:19:20.373572+0000","last_clean":"2024-09-12T03:19:20.373572+0000","last_became_active":"2024-09-12T03:19:07.207428+0000","last_became_peered":"2024-09-12T03:19:07.207428+0000","last_unstale":"2024-09-12T03:19:20.373572+0000","last_undegraded":"2024-09-12T03:19:20.373572+0000","last_fullsized":"2024-09-12T03:19:20.373572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:45.135019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.255394+0000","last_change":"2024-09-12T03:19:09.336094+0000","last_active":"2024-09-12T03:20:49.255394+0000","last_peered":"2024-09-12T03:20:49.255394+0000","last_clean":"2024-09-12T03:20:49.255394+0000","last_became_active":"2024-09-12T03:19:09.335779+0000","last_became_peered":"2024-09-12T03:19:09.335779+0000","last_unstale":"2024-09-12T03:20:49.255394+0000","last_undegraded":"2024-09-12T03:20:49.255394+0000","last_fullsized":"2024-09-12T03:20:49.255394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:37.019082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254523+0000","last_change":"2024-09-12T03:19:09.344537+0000","last_active":"2024-09-12T03:20:49.254523+0000","last_peered":"2024-09-12T03:20:49.254523+0000","last_clean":"2024-09-12T03:20:49.254523+0000","last_became_active":"2024-09-12T03:19:09.343879+0000","last_became_peered":"2024-09-12T03:19:09.343879+0000","last_unstale":"2024-09-12T03:20:49.254523+0000","last_undegraded":"2024-09-12T03:20:49.254523+0000","last_fullsized":"2024-09-12T03:20:49.254523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:40.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373973+0000","last_change":"2024-09-12T03:19:07.208040+0000","last_active":"2024-09-12T03:19:20.373973+0000","last_peered":"2024-09-12T03:19:20.373973+0000","last_clean":"2024-09-12T03:19:20.373973+0000","last_became_active":"2024-09-12T03:19:07.207658+0000","last_became_peered":"2024-09-12T03:19:07.207658+0000","last_unstale":"2024-09-12T03:19:20.373973+0000","last_undegraded":"2024-09-12T03:19:20.373973+0000","last_fullsized":"2024-09-12T03:19:20.373973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:48.620707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265759+0000","last_change":"2024-09-12T03:19:09.333837+0000","last_active":"2024-09-12T03:20:49.265759+0000","last_peered":"2024-09-12T03:20:49.265759+0000","last_clean":"2024-09-12T03:20:49.265759+0000","last_became_active":"2024-09-12T03:19:09.333194+0000","last_became_peered":"2024-09-12T03:19:09.333194+0000","last_unstale":"2024-09-12T03:20:49.265759+0000","last_undegraded":"2024-09-12T03:20:49.265759+0000","last_fullsized":"2024-09-12T03:20:49.265759+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:18.766181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.385455+0000","last_change":"2024-09-12T03:19:20.384978+0000","last_active":"2024-09-12T03:19:20.385455+0000","last_peered":"2024-09-12T03:19:20.385455+0000","last_clean":"2024-09-12T03:19:20.385455+0000","last_became_active":"2024-09-12T03:19:20.376883+0000","last_became_peered":"2024-09-12T03:19:20.376883+0000","last_unstale":"2024-09-12T03:19:20.385455+0000","last_undegraded":"2024-09-12T03:19:20.385455+0000","last_fullsized":"2024-09-12T03:19:20.385455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:32.542334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265022+0000","last_change":"2024-09-12T03:19:09.334684+0000","last_active":"2024-09-12T03:20:49.265022+0000","last_peered":"2024-09-12T03:20:49.265022+0000","last_clean":"2024-09-12T03:20:49.265022+0000","last_became_active":"2024-09-12T03:19:09.334281+0000","last_became_peered":"2024-09-12T03:19:09.334281+0000","last_unstale":"2024-09-12T03:20:49.265022+0000","last_undegraded":"2024-09-12T03:20:49.265022+0000","last_fullsized":"2024-09-12T03:20:49.265022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:44.837067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372829+0000","last_change":"2024-09-12T03:19:07.388555+0000","last_active":"2024-09-12T03:19:20.372829+0000","last_peered":"2024-09-12T03:19:20.372829+0000","last_clean":"2024-09-12T03:19:20.372829+0000","last_became_active":"2024-09-12T03:19:07.388206+0000","last_became_peered":"2024-09-12T03:19:07.388206+0000","last_unstale":"2024-09-12T03:19:20.372829+0000","last_undegraded":"2024-09-12T03:19:20.372829+0000","last_fullsized":"2024-09-12T03:19:20.372829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:33.628294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248105+0000","last_change":"2024-09-12T03:19:09.339108+0000","last_active":"2024-09-12T03:20:49.248105+0000","last_peered":"2024-09-12T03:20:49.248105+0000","last_clean":"2024-09-12T03:20:49.248105+0000","last_became_active":"2024-09-12T03:19:09.338819+0000","last_became_peered":"2024-09-12T03:19:09.338819+0000","last_unstale":"2024-09-12T03:20:49.248105+0000","last_undegraded":"2024-09-12T03:20:49.248105+0000","last_fullsized":"2024-09-12T03:20:49.248105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:32.864900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372798+0000","last_change":"2024-09-12T03:19:07.207197+0000","last_active":"2024-09-12T03:19:20.372798+0000","last_peered":"2024-09-12T03:19:20.372798+0000","last_clean":"2024-09-12T03:19:20.372798+0000","last_became_active":"2024-09-12T03:19:07.206879+0000","last_became_peered":"2024-09-12T03:19:07.206879+0000","last_unstale":"2024-09-12T03:19:20.372798+0000","last_undegraded":"2024-09-12T03:19:20.372798+0000","last_fullsized":"2024-09-12T03:19:20.372798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:00:46.269636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65559,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":601,"num_read_kb":367,"num_write":179,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":209550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579938,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":433387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36892,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708260,"statfs":{"total":95995035648,"available":95957258240,"internally_reserved":0,"allocated":1585152,"data_stored":1085155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972395,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":433589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201329,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":433509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430262,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1499136,"data_stored":1025890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593787,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":432978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822720,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1511424,"data_stored":1025967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":897024,"data_stored":433808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":433289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":501409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40440,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704712,"statfs":{"total":95995035648,"available":95953625088,"internally_reserved":0,"allocated":958464,"data_stored":498858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":434249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":66087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:20:57.674 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-12T03:20:57.674 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:20:57.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.813+0000 7fa02a8d0640 1 -- 172.21.15.6:0/110152581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa024101c00 msgr2=0x7fa024102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.813+0000 7fa02a8d0640 1 --2- 172.21.15.6:0/110152581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa024101c00 0x7fa024102080 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa018009900 tx=0x7fa01802a410 comp rx=0 tx=0).stop 2024-09-12T03:20:57.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.813+0000 7fa02a8d0640 1 -- 172.21.15.6:0/110152581 shutdown_connections 2024-09-12T03:20:57.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.813+0000 7fa02a8d0640 1 --2- 172.21.15.6:0/110152581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0241025c0 0x7fa024108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.813+0000 7fa02a8d0640 1 --2- 172.21.15.6:0/110152581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa024101c00 0x7fa024102080 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.813+0000 7fa02a8d0640 1 --2- 172.21.15.6:0/110152581 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa024100a10 0x7fa024100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.813+0000 7fa02a8d0640 1 -- 172.21.15.6:0/110152581 >> 172.21.15.6:0/110152581 conn(0x7fa0240fc170 msgr2=0x7fa0240fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:57.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.813+0000 7fa02a8d0640 1 -- 172.21.15.6:0/110152581 shutdown_connections 2024-09-12T03:20:57.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.813+0000 7fa02a8d0640 1 -- 172.21.15.6:0/110152581 wait complete. 2024-09-12T03:20:57.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.813+0000 7fa02a8d0640 1 Processor -- start 2024-09-12T03:20:57.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.813+0000 7fa02a8d0640 1 -- start start 2024-09-12T03:20:57.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa02a8d0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa024100a10 0x7fa024196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa02a8d0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa024101c00 0x7fa024197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa02a8d0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0241025c0 0x7fa02419e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa02a8d0640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa02410c690 con 0x7fa024101c00 2024-09-12T03:20:57.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa02a8d0640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa02410c510 con 0x7fa0241025c0 2024-09-12T03:20:57.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa02a8d0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa02410c810 con 0x7fa024100a10 2024-09-12T03:20:57.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa0237fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa024101c00 0x7fa024197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa0237fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa024101c00 0x7fa024197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36496/0 (socket says 172.21.15.6:36496) 2024-09-12T03:20:57.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa0237fe640 1 -- 172.21.15.6:0/1362042169 learned_addr learned my addr 172.21.15.6:0/1362042169 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:57.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa023fff640 1 --2- 172.21.15.6:0/1362042169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa024100a10 0x7fa024196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa028e6d640 1 --2- 172.21.15.6:0/1362042169 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0241025c0 0x7fa02419e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa0237fe640 1 -- 172.21.15.6:0/1362042169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa024100a10 msgr2=0x7fa024196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa0237fe640 1 --2- 172.21.15.6:0/1362042169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa024100a10 0x7fa024196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa0237fe640 1 -- 172.21.15.6:0/1362042169 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0241025c0 msgr2=0x7fa02419e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa0237fe640 1 --2- 172.21.15.6:0/1362042169 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0241025c0 0x7fa02419e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa0237fe640 1 -- 172.21.15.6:0/1362042169 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa02419e960 con 0x7fa024101c00 2024-09-12T03:20:57.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa028e6d640 1 --2- 172.21.15.6:0/1362042169 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0241025c0 0x7fa02419e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:57.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa0237fe640 1 --2- 172.21.15.6:0/1362042169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa024101c00 0x7fa024197280 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa018009230 tx=0x7fa018004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:57.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa0217fa640 1 -- 172.21.15.6:0/1362042169 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa018035070 con 0x7fa024101c00 2024-09-12T03:20:57.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa0217fa640 1 -- 172.21.15.6:0/1362042169 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa018003cd0 con 0x7fa024101c00 2024-09-12T03:20:57.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa02a8d0640 1 -- 172.21.15.6:0/1362042169 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa024069380 con 0x7fa024101c00 2024-09-12T03:20:57.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.814+0000 7fa0217fa640 1 -- 172.21.15.6:0/1362042169 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0180433d0 con 0x7fa024101c00 2024-09-12T03:20:57.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.815+0000 7fa02a8d0640 1 -- 172.21.15.6:0/1362042169 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0240698c0 con 0x7fa024101c00 2024-09-12T03:20:57.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.816+0000 7fa02a8d0640 1 -- 172.21.15.6:0/1362042169 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa024100e10 con 0x7fa024101c00 2024-09-12T03:20:57.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.817+0000 7fa0217fa640 1 -- 172.21.15.6:0/1362042169 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fa01803b560 con 0x7fa024101c00 2024-09-12T03:20:57.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.817+0000 7fa0217fa640 1 --2- 172.21.15.6:0/1362042169 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa00c0a2ca0 0x7fa00c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.817+0000 7fa023fff640 1 --2- 172.21.15.6:0/1362042169 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa00c0a2ca0 0x7fa00c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.817+0000 7fa0217fa640 1 -- 172.21.15.6:0/1362042169 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa0180f1d00 con 0x7fa024101c00 2024-09-12T03:20:57.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.818+0000 7fa023fff640 1 --2- 172.21.15.6:0/1362042169 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa00c0a2ca0 0x7fa00c0a5160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa024101a60 tx=0x7fa01400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:57.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.821+0000 7fa0217fa640 1 -- 172.21.15.6:0/1362042169 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa0180bec40 con 0x7fa024101c00 2024-09-12T03:20:57.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:57.993+0000 7fa02a8d0640 1 -- 172.21.15.6:0/1362042169 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa024069b20 con 0x7fa00c0a2ca0 2024-09-12T03:20:58.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.000+0000 7fa0217fa640 1 -- 172.21.15.6:0/1362042169 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258918 (secure 0 0 0) 0x7fa024069b20 con 0x7fa00c0a2ca0 2024-09-12T03:20:58.003 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:20:58.004 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-09-12T03:20:58.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.003+0000 7fa02a8d0640 1 -- 172.21.15.6:0/1362042169 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa00c0a2ca0 msgr2=0x7fa00c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:58.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.003+0000 7fa02a8d0640 1 --2- 172.21.15.6:0/1362042169 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa00c0a2ca0 0x7fa00c0a5160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa024101a60 tx=0x7fa01400b390 comp rx=0 tx=0).stop 2024-09-12T03:20:58.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.003+0000 7fa02a8d0640 1 -- 172.21.15.6:0/1362042169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa024101c00 msgr2=0x7fa024197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:58.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.003+0000 7fa02a8d0640 1 --2- 172.21.15.6:0/1362042169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa024101c00 0x7fa024197280 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa018009230 tx=0x7fa018004360 comp rx=0 tx=0).stop 2024-09-12T03:20:58.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.003+0000 7fa02a8d0640 1 -- 172.21.15.6:0/1362042169 shutdown_connections 2024-09-12T03:20:58.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.003+0000 7fa02a8d0640 1 --2- 172.21.15.6:0/1362042169 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa00c0a2ca0 0x7fa00c0a5160 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:58.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.003+0000 7fa02a8d0640 1 --2- 172.21.15.6:0/1362042169 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0241025c0 0x7fa02419e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:58.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.003+0000 7fa02a8d0640 1 --2- 172.21.15.6:0/1362042169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa024101c00 0x7fa024197280 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:58.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.003+0000 7fa02a8d0640 1 --2- 172.21.15.6:0/1362042169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa024100a10 0x7fa024196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:58.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.003+0000 7fa02a8d0640 1 -- 172.21.15.6:0/1362042169 >> 172.21.15.6:0/1362042169 conn(0x7fa0240fc170 msgr2=0x7fa024105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:58.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.003+0000 7fa02a8d0640 1 -- 172.21.15.6:0/1362042169 shutdown_connections 2024-09-12T03:20:58.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.003+0000 7fa02a8d0640 1 -- 172.21.15.6:0/1362042169 wait complete. 2024-09-12T03:20:58.014 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-12T03:20:57.264063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656161,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":727,"num_read_kb":476,"num_write":362,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":790,"ondisk_log_size":790,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382596,"kb_used_data":12548,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559228,"statfs":{"total":1151940427776,"available":1151548649472,"internally_reserved":0,"allocated":12849152,"data_stored":7172088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1792,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":36,"num_write":15,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002802"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265565+0000","last_change":"2024-09-12T03:19:09.343829+0000","last_active":"2024-09-12T03:20:49.265565+0000","last_peered":"2024-09-12T03:20:49.265565+0000","last_clean":"2024-09-12T03:20:49.265565+0000","last_became_active":"2024-09-12T03:19:09.343228+0000","last_became_peered":"2024-09-12T03:19:09.343228+0000","last_unstale":"2024-09-12T03:20:49.265565+0000","last_undegraded":"2024-09-12T03:20:49.265565+0000","last_fullsized":"2024-09-12T03:20:49.265565+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:08.720934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.237141+0000","last_change":"2024-09-12T03:19:09.331829+0000","last_active":"2024-09-12T03:20:49.237141+0000","last_peered":"2024-09-12T03:20:49.237141+0000","last_clean":"2024-09-12T03:20:49.237141+0000","last_became_active":"2024-09-12T03:19:09.331276+0000","last_became_peered":"2024-09-12T03:19:09.331276+0000","last_unstale":"2024-09-12T03:20:49.237141+0000","last_undegraded":"2024-09-12T03:20:49.237141+0000","last_fullsized":"2024-09-12T03:20:49.237141+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:48.672085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.263951+0000","last_change":"2024-09-12T03:19:09.338120+0000","last_active":"2024-09-12T03:20:49.263951+0000","last_peered":"2024-09-12T03:20:49.263951+0000","last_clean":"2024-09-12T03:20:49.263951+0000","last_became_active":"2024-09-12T03:19:09.337773+0000","last_became_peered":"2024-09-12T03:19:09.337773+0000","last_unstale":"2024-09-12T03:20:49.263951+0000","last_undegraded":"2024-09-12T03:20:49.263951+0000","last_fullsized":"2024-09-12T03:20:49.263951+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:13.716047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254592+0000","last_change":"2024-09-12T03:19:09.339404+0000","last_active":"2024-09-12T03:20:49.254592+0000","last_peered":"2024-09-12T03:20:49.254592+0000","last_clean":"2024-09-12T03:20:49.254592+0000","last_became_active":"2024-09-12T03:19:09.339105+0000","last_became_peered":"2024-09-12T03:19:09.339105+0000","last_unstale":"2024-09-12T03:20:49.254592+0000","last_undegraded":"2024-09-12T03:20:49.254592+0000","last_fullsized":"2024-09-12T03:20:49.254592+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:37.930963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.242261+0000","last_change":"2024-09-12T03:19:09.326285+0000","last_active":"2024-09-12T03:20:49.242261+0000","last_peered":"2024-09-12T03:20:49.242261+0000","last_clean":"2024-09-12T03:20:49.242261+0000","last_became_active":"2024-09-12T03:19:09.325852+0000","last_became_peered":"2024-09-12T03:19:09.325852+0000","last_unstale":"2024-09-12T03:20:49.242261+0000","last_undegraded":"2024-09-12T03:20:49.242261+0000","last_fullsized":"2024-09-12T03:20:49.242261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:44:25.387014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.249918+0000","last_change":"2024-09-12T03:19:20.376197+0000","last_active":"2024-09-12T03:20:49.249918+0000","last_peered":"2024-09-12T03:20:49.249918+0000","last_clean":"2024-09-12T03:20:49.249918+0000","last_became_active":"2024-09-12T03:19:20.375744+0000","last_became_peered":"2024-09-12T03:19:20.375744+0000","last_unstale":"2024-09-12T03:20:49.249918+0000","last_undegraded":"2024-09-12T03:20:49.249918+0000","last_fullsized":"2024-09-12T03:20:49.249918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:08.663121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.245988+0000","last_change":"2024-09-12T03:19:09.335284+0000","last_active":"2024-09-12T03:20:49.245988+0000","last_peered":"2024-09-12T03:20:49.245988+0000","last_clean":"2024-09-12T03:20:49.245988+0000","last_became_active":"2024-09-12T03:19:09.334948+0000","last_became_peered":"2024-09-12T03:19:09.334948+0000","last_unstale":"2024-09-12T03:20:49.245988+0000","last_undegraded":"2024-09-12T03:20:49.245988+0000","last_fullsized":"2024-09-12T03:20:49.245988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:32.778433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250337+0000","last_change":"2024-09-12T03:19:09.338398+0000","last_active":"2024-09-12T03:20:49.250337+0000","last_peered":"2024-09-12T03:20:49.250337+0000","last_clean":"2024-09-12T03:20:49.250337+0000","last_became_active":"2024-09-12T03:19:09.337750+0000","last_became_peered":"2024-09-12T03:19:09.337750+0000","last_unstale":"2024-09-12T03:20:49.250337+0000","last_undegraded":"2024-09-12T03:20:49.250337+0000","last_fullsized":"2024-09-12T03:20:49.250337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:07.542322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265597+0000","last_change":"2024-09-12T03:19:09.340703+0000","last_active":"2024-09-12T03:20:49.265597+0000","last_peered":"2024-09-12T03:20:49.265597+0000","last_clean":"2024-09-12T03:20:49.265597+0000","last_became_active":"2024-09-12T03:19:09.340386+0000","last_became_peered":"2024-09-12T03:19:09.340386+0000","last_unstale":"2024-09-12T03:20:49.265597+0000","last_undegraded":"2024-09-12T03:20:49.265597+0000","last_fullsized":"2024-09-12T03:20:49.265597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:19.990894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248198+0000","last_change":"2024-09-12T03:19:09.341489+0000","last_active":"2024-09-12T03:20:49.248198+0000","last_peered":"2024-09-12T03:20:49.248198+0000","last_clean":"2024-09-12T03:20:49.248198+0000","last_became_active":"2024-09-12T03:19:09.341183+0000","last_became_peered":"2024-09-12T03:19:09.341183+0000","last_unstale":"2024-09-12T03:20:49.248198+0000","last_undegraded":"2024-09-12T03:20:49.248198+0000","last_fullsized":"2024-09-12T03:20:49.248198+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:05.688402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.255447+0000","last_change":"2024-09-12T03:19:09.339289+0000","last_active":"2024-09-12T03:20:49.255447+0000","last_peered":"2024-09-12T03:20:49.255447+0000","last_clean":"2024-09-12T03:20:49.255447+0000","last_became_active":"2024-09-12T03:19:09.338466+0000","last_became_peered":"2024-09-12T03:19:09.338466+0000","last_unstale":"2024-09-12T03:20:49.255447+0000","last_undegraded":"2024-09-12T03:20:49.255447+0000","last_fullsized":"2024-09-12T03:20:49.255447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:47:13.070057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254364+0000","last_change":"2024-09-12T03:19:09.337754+0000","last_active":"2024-09-12T03:20:49.254364+0000","last_peered":"2024-09-12T03:20:49.254364+0000","last_clean":"2024-09-12T03:20:49.254364+0000","last_became_active":"2024-09-12T03:19:09.332789+0000","last_became_peered":"2024-09-12T03:19:09.332789+0000","last_unstale":"2024-09-12T03:20:49.254364+0000","last_undegraded":"2024-09-12T03:20:49.254364+0000","last_fullsized":"2024-09-12T03:20:49.254364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:34.622136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250957+0000","last_change":"2024-09-12T03:19:09.333577+0000","last_active":"2024-09-12T03:20:49.250957+0000","last_peered":"2024-09-12T03:20:49.250957+0000","last_clean":"2024-09-12T03:20:49.250957+0000","last_became_active":"2024-09-12T03:19:09.333234+0000","last_became_peered":"2024-09-12T03:19:09.333234+0000","last_unstale":"2024-09-12T03:20:49.250957+0000","last_undegraded":"2024-09-12T03:20:49.250957+0000","last_fullsized":"2024-09-12T03:20:49.250957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:22.604640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.237367+0000","last_change":"2024-09-12T03:19:09.331607+0000","last_active":"2024-09-12T03:20:49.237367+0000","last_peered":"2024-09-12T03:20:49.237367+0000","last_clean":"2024-09-12T03:20:49.237367+0000","last_became_active":"2024-09-12T03:19:09.331253+0000","last_became_peered":"2024-09-12T03:19:09.331253+0000","last_unstale":"2024-09-12T03:20:49.237367+0000","last_undegraded":"2024-09-12T03:20:49.237367+0000","last_fullsized":"2024-09-12T03:20:49.237367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:11.969265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256379+0000","last_change":"2024-09-12T03:19:09.340768+0000","last_active":"2024-09-12T03:20:49.256379+0000","last_peered":"2024-09-12T03:20:49.256379+0000","last_clean":"2024-09-12T03:20:49.256379+0000","last_became_active":"2024-09-12T03:19:09.340437+0000","last_became_peered":"2024-09-12T03:19:09.340437+0000","last_unstale":"2024-09-12T03:20:49.256379+0000","last_undegraded":"2024-09-12T03:20:49.256379+0000","last_fullsized":"2024-09-12T03:20:49.256379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:32.645691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.263546+0000","last_change":"2024-09-12T03:19:09.333829+0000","last_active":"2024-09-12T03:20:49.263546+0000","last_peered":"2024-09-12T03:20:49.263546+0000","last_clean":"2024-09-12T03:20:49.263546+0000","last_became_active":"2024-09-12T03:19:09.333462+0000","last_became_peered":"2024-09-12T03:19:09.333462+0000","last_unstale":"2024-09-12T03:20:49.263546+0000","last_undegraded":"2024-09-12T03:20:49.263546+0000","last_fullsized":"2024-09-12T03:20:49.263546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:18:57.586836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265659+0000","last_change":"2024-09-12T03:19:09.344068+0000","last_active":"2024-09-12T03:20:49.265659+0000","last_peered":"2024-09-12T03:20:49.265659+0000","last_clean":"2024-09-12T03:20:49.265659+0000","last_became_active":"2024-09-12T03:19:09.343751+0000","last_became_peered":"2024-09-12T03:19:09.343751+0000","last_unstale":"2024-09-12T03:20:49.265659+0000","last_undegraded":"2024-09-12T03:20:49.265659+0000","last_fullsized":"2024-09-12T03:20:49.265659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:52.425547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.249413+0000","last_change":"2024-09-12T03:19:09.330828+0000","last_active":"2024-09-12T03:20:49.249413+0000","last_peered":"2024-09-12T03:20:49.249413+0000","last_clean":"2024-09-12T03:20:49.249413+0000","last_became_active":"2024-09-12T03:19:09.330497+0000","last_became_peered":"2024-09-12T03:19:09.330497+0000","last_unstale":"2024-09-12T03:20:49.249413+0000","last_undegraded":"2024-09-12T03:20:49.249413+0000","last_fullsized":"2024-09-12T03:20:49.249413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:21:20.064177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.245548+0000","last_change":"2024-09-12T03:19:09.344403+0000","last_active":"2024-09-12T03:20:49.245548+0000","last_peered":"2024-09-12T03:20:49.245548+0000","last_clean":"2024-09-12T03:20:49.245548+0000","last_became_active":"2024-09-12T03:19:09.343946+0000","last_became_peered":"2024-09-12T03:19:09.343946+0000","last_unstale":"2024-09-12T03:20:49.245548+0000","last_undegraded":"2024-09-12T03:20:49.245548+0000","last_fullsized":"2024-09-12T03:20:49.245548+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:48.386117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248484+0000","last_change":"2024-09-12T03:19:09.341481+0000","last_active":"2024-09-12T03:20:49.248484+0000","last_peered":"2024-09-12T03:20:49.248484+0000","last_clean":"2024-09-12T03:20:49.248484+0000","last_became_active":"2024-09-12T03:19:09.341142+0000","last_became_peered":"2024-09-12T03:19:09.341142+0000","last_unstale":"2024-09-12T03:20:49.248484+0000","last_undegraded":"2024-09-12T03:20:49.248484+0000","last_fullsized":"2024-09-12T03:20:49.248484+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:42.371221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.249677+0000","last_change":"2024-09-12T03:19:09.330621+0000","last_active":"2024-09-12T03:20:49.249677+0000","last_peered":"2024-09-12T03:20:49.249677+0000","last_clean":"2024-09-12T03:20:49.249677+0000","last_became_active":"2024-09-12T03:19:09.330169+0000","last_became_peered":"2024-09-12T03:19:09.330169+0000","last_unstale":"2024-09-12T03:20:49.249677+0000","last_undegraded":"2024-09-12T03:20:49.249677+0000","last_fullsized":"2024-09-12T03:20:49.249677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:16.634141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.264220+0000","last_change":"2024-09-12T03:19:09.332817+0000","last_active":"2024-09-12T03:20:49.264220+0000","last_peered":"2024-09-12T03:20:49.264220+0000","last_clean":"2024-09-12T03:20:49.264220+0000","last_became_active":"2024-09-12T03:19:09.332236+0000","last_became_peered":"2024-09-12T03:19:09.332236+0000","last_unstale":"2024-09-12T03:20:49.264220+0000","last_undegraded":"2024-09-12T03:20:49.264220+0000","last_fullsized":"2024-09-12T03:20:49.264220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:28.024294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252477+0000","last_change":"2024-09-12T03:19:09.337910+0000","last_active":"2024-09-12T03:20:49.252477+0000","last_peered":"2024-09-12T03:20:49.252477+0000","last_clean":"2024-09-12T03:20:49.252477+0000","last_became_active":"2024-09-12T03:19:09.337176+0000","last_became_peered":"2024-09-12T03:19:09.337176+0000","last_unstale":"2024-09-12T03:20:49.252477+0000","last_undegraded":"2024-09-12T03:20:49.252477+0000","last_fullsized":"2024-09-12T03:20:49.252477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:53.795898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.239957+0000","last_change":"2024-09-12T03:19:09.337902+0000","last_active":"2024-09-12T03:20:49.239957+0000","last_peered":"2024-09-12T03:20:49.239957+0000","last_clean":"2024-09-12T03:20:49.239957+0000","last_became_active":"2024-09-12T03:19:09.337567+0000","last_became_peered":"2024-09-12T03:19:09.337567+0000","last_unstale":"2024-09-12T03:20:49.239957+0000","last_undegraded":"2024-09-12T03:20:49.239957+0000","last_fullsized":"2024-09-12T03:20:49.239957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:08.471313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248557+0000","last_change":"2024-09-12T03:19:09.339850+0000","last_active":"2024-09-12T03:20:49.248557+0000","last_peered":"2024-09-12T03:20:49.248557+0000","last_clean":"2024-09-12T03:20:49.248557+0000","last_became_active":"2024-09-12T03:19:09.339563+0000","last_became_peered":"2024-09-12T03:19:09.339563+0000","last_unstale":"2024-09-12T03:20:49.248557+0000","last_undegraded":"2024-09-12T03:20:49.248557+0000","last_fullsized":"2024-09-12T03:20:49.248557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:27.473351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.247769+0000","last_change":"2024-09-12T03:19:09.330555+0000","last_active":"2024-09-12T03:20:49.247769+0000","last_peered":"2024-09-12T03:20:49.247769+0000","last_clean":"2024-09-12T03:20:49.247769+0000","last_became_active":"2024-09-12T03:19:09.330183+0000","last_became_peered":"2024-09-12T03:19:09.330183+0000","last_unstale":"2024-09-12T03:20:49.247769+0000","last_undegraded":"2024-09-12T03:20:49.247769+0000","last_fullsized":"2024-09-12T03:20:49.247769+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:39.334116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252190+0000","last_change":"2024-09-12T03:19:09.338601+0000","last_active":"2024-09-12T03:20:49.252190+0000","last_peered":"2024-09-12T03:20:49.252190+0000","last_clean":"2024-09-12T03:20:49.252190+0000","last_became_active":"2024-09-12T03:19:09.337903+0000","last_became_peered":"2024-09-12T03:19:09.337903+0000","last_unstale":"2024-09-12T03:20:49.252190+0000","last_undegraded":"2024-09-12T03:20:49.252190+0000","last_fullsized":"2024-09-12T03:20:49.252190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:17.810139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250990+0000","last_change":"2024-09-12T03:19:09.326629+0000","last_active":"2024-09-12T03:20:49.250990+0000","last_peered":"2024-09-12T03:20:49.250990+0000","last_clean":"2024-09-12T03:20:49.250990+0000","last_became_active":"2024-09-12T03:19:09.326333+0000","last_became_peered":"2024-09-12T03:19:09.326333+0000","last_unstale":"2024-09-12T03:20:49.250990+0000","last_undegraded":"2024-09-12T03:20:49.250990+0000","last_fullsized":"2024-09-12T03:20:49.250990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:20.807593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256412+0000","last_change":"2024-09-12T03:19:09.339919+0000","last_active":"2024-09-12T03:20:49.256412+0000","last_peered":"2024-09-12T03:20:49.256412+0000","last_clean":"2024-09-12T03:20:49.256412+0000","last_became_active":"2024-09-12T03:19:09.339618+0000","last_became_peered":"2024-09-12T03:19:09.339618+0000","last_unstale":"2024-09-12T03:20:49.256412+0000","last_undegraded":"2024-09-12T03:20:49.256412+0000","last_fullsized":"2024-09-12T03:20:49.256412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:11:03.740079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.246446+0000","last_change":"2024-09-12T03:19:09.336148+0000","last_active":"2024-09-12T03:20:49.246446+0000","last_peered":"2024-09-12T03:20:49.246446+0000","last_clean":"2024-09-12T03:20:49.246446+0000","last_became_active":"2024-09-12T03:19:09.335779+0000","last_became_peered":"2024-09-12T03:19:09.335779+0000","last_unstale":"2024-09-12T03:20:49.246446+0000","last_undegraded":"2024-09-12T03:20:49.246446+0000","last_fullsized":"2024-09-12T03:20:49.246446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:18.234320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.247971+0000","last_change":"2024-09-12T03:19:20.382536+0000","last_active":"2024-09-12T03:20:49.247971+0000","last_peered":"2024-09-12T03:20:49.247971+0000","last_clean":"2024-09-12T03:20:49.247971+0000","last_became_active":"2024-09-12T03:19:20.382118+0000","last_became_peered":"2024-09-12T03:19:20.382118+0000","last_unstale":"2024-09-12T03:20:49.247971+0000","last_undegraded":"2024-09-12T03:20:49.247971+0000","last_fullsized":"2024-09-12T03:20:49.247971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:55.836646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252455+0000","last_change":"2024-09-12T03:19:09.345064+0000","last_active":"2024-09-12T03:20:49.252455+0000","last_peered":"2024-09-12T03:20:49.252455+0000","last_clean":"2024-09-12T03:20:49.252455+0000","last_became_active":"2024-09-12T03:19:09.344628+0000","last_became_peered":"2024-09-12T03:19:09.344628+0000","last_unstale":"2024-09-12T03:20:49.252455+0000","last_undegraded":"2024-09-12T03:20:49.252455+0000","last_fullsized":"2024-09-12T03:20:49.252455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:50.011751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.244081+0000","last_change":"2024-09-12T03:19:09.338052+0000","last_active":"2024-09-12T03:20:49.244081+0000","last_peered":"2024-09-12T03:20:49.244081+0000","last_clean":"2024-09-12T03:20:49.244081+0000","last_became_active":"2024-09-12T03:19:09.337710+0000","last_became_peered":"2024-09-12T03:19:09.337710+0000","last_unstale":"2024-09-12T03:20:49.244081+0000","last_undegraded":"2024-09-12T03:20:49.244081+0000","last_fullsized":"2024-09-12T03:20:49.244081+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:36.521269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'9","reported_seq":134,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:54.518339+0000","last_change":"2024-09-12T03:19:07.208341+0000","last_active":"2024-09-12T03:20:54.518339+0000","last_peered":"2024-09-12T03:20:54.518339+0000","last_clean":"2024-09-12T03:20:54.518339+0000","last_became_active":"2024-09-12T03:19:07.207385+0000","last_became_peered":"2024-09-12T03:19:07.207385+0000","last_unstale":"2024-09-12T03:20:54.518339+0000","last_undegraded":"2024-09-12T03:20:54.518339+0000","last_fullsized":"2024-09-12T03:20:54.518339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:18.476795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1857,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254115+0000","last_change":"2024-09-12T03:19:09.339997+0000","last_active":"2024-09-12T03:20:49.254115+0000","last_peered":"2024-09-12T03:20:49.254115+0000","last_clean":"2024-09-12T03:20:49.254115+0000","last_became_active":"2024-09-12T03:19:09.339261+0000","last_became_peered":"2024-09-12T03:19:09.339261+0000","last_unstale":"2024-09-12T03:20:49.254115+0000","last_undegraded":"2024-09-12T03:20:49.254115+0000","last_fullsized":"2024-09-12T03:20:49.254115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:07.670345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374030+0000","last_change":"2024-09-12T03:19:07.200222+0000","last_active":"2024-09-12T03:19:20.374030+0000","last_peered":"2024-09-12T03:19:20.374030+0000","last_clean":"2024-09-12T03:19:20.374030+0000","last_became_active":"2024-09-12T03:19:07.199819+0000","last_became_peered":"2024-09-12T03:19:07.199819+0000","last_unstale":"2024-09-12T03:19:20.374030+0000","last_undegraded":"2024-09-12T03:19:20.374030+0000","last_fullsized":"2024-09-12T03:19:20.374030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:39:28.933609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375686+0000","last_change":"2024-09-12T03:19:07.207005+0000","last_active":"2024-09-12T03:19:20.375686+0000","last_peered":"2024-09-12T03:19:20.375686+0000","last_clean":"2024-09-12T03:19:20.375686+0000","last_became_active":"2024-09-12T03:19:07.206685+0000","last_became_peered":"2024-09-12T03:19:07.206685+0000","last_unstale":"2024-09-12T03:19:20.375686+0000","last_undegraded":"2024-09-12T03:19:20.375686+0000","last_fullsized":"2024-09-12T03:19:20.375686+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:31.111706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252939+0000","last_change":"2024-09-12T03:19:09.335871+0000","last_active":"2024-09-12T03:20:49.252939+0000","last_peered":"2024-09-12T03:20:49.252939+0000","last_clean":"2024-09-12T03:20:49.252939+0000","last_became_active":"2024-09-12T03:19:09.335568+0000","last_became_peered":"2024-09-12T03:19:09.335568+0000","last_unstale":"2024-09-12T03:20:49.252939+0000","last_undegraded":"2024-09-12T03:20:49.252939+0000","last_fullsized":"2024-09-12T03:20:49.252939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:52:34.471512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386049+0000","last_change":"2024-09-12T03:19:07.208320+0000","last_active":"2024-09-12T03:19:20.386049+0000","last_peered":"2024-09-12T03:19:20.386049+0000","last_clean":"2024-09-12T03:19:20.386049+0000","last_became_active":"2024-09-12T03:19:07.207873+0000","last_became_peered":"2024-09-12T03:19:07.207873+0000","last_unstale":"2024-09-12T03:19:20.386049+0000","last_undegraded":"2024-09-12T03:19:20.386049+0000","last_fullsized":"2024-09-12T03:19:20.386049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:00.128595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250364+0000","last_change":"2024-09-12T03:19:09.331930+0000","last_active":"2024-09-12T03:20:49.250364+0000","last_peered":"2024-09-12T03:20:49.250364+0000","last_clean":"2024-09-12T03:20:49.250364+0000","last_became_active":"2024-09-12T03:19:09.331558+0000","last_became_peered":"2024-09-12T03:19:09.331558+0000","last_unstale":"2024-09-12T03:20:49.250364+0000","last_undegraded":"2024-09-12T03:20:49.250364+0000","last_fullsized":"2024-09-12T03:20:49.250364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:00:27.935717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":120,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:51.511645+0000","last_change":"2024-09-12T03:19:07.205473+0000","last_active":"2024-09-12T03:20:51.511645+0000","last_peered":"2024-09-12T03:20:51.511645+0000","last_clean":"2024-09-12T03:20:51.511645+0000","last_became_active":"2024-09-12T03:19:07.205134+0000","last_became_peered":"2024-09-12T03:19:07.205134+0000","last_unstale":"2024-09-12T03:20:51.511645+0000","last_undegraded":"2024-09-12T03:20:51.511645+0000","last_fullsized":"2024-09-12T03:20:51.511645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:23.443957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265317+0000","last_change":"2024-09-12T03:19:09.333868+0000","last_active":"2024-09-12T03:20:49.265317+0000","last_peered":"2024-09-12T03:20:49.265317+0000","last_clean":"2024-09-12T03:20:49.265317+0000","last_became_active":"2024-09-12T03:19:09.333484+0000","last_became_peered":"2024-09-12T03:19:09.333484+0000","last_unstale":"2024-09-12T03:20:49.265317+0000","last_undegraded":"2024-09-12T03:20:49.265317+0000","last_fullsized":"2024-09-12T03:20:49.265317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:21:10.226821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386303+0000","last_change":"2024-09-12T03:19:20.386303+0000","last_active":"2024-09-12T03:19:20.386303+0000","last_peered":"2024-09-12T03:19:20.386303+0000","last_clean":"2024-09-12T03:19:20.386303+0000","last_became_active":"2024-09-12T03:19:20.383198+0000","last_became_peered":"2024-09-12T03:19:20.383198+0000","last_unstale":"2024-09-12T03:19:20.386303+0000","last_undegraded":"2024-09-12T03:19:20.386303+0000","last_fullsized":"2024-09-12T03:19:20.386303+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:42.224849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.257056+0000","last_change":"2024-09-12T03:19:09.340849+0000","last_active":"2024-09-12T03:20:49.257056+0000","last_peered":"2024-09-12T03:20:49.257056+0000","last_clean":"2024-09-12T03:20:49.257056+0000","last_became_active":"2024-09-12T03:19:09.340517+0000","last_became_peered":"2024-09-12T03:19:09.340517+0000","last_unstale":"2024-09-12T03:20:49.257056+0000","last_undegraded":"2024-09-12T03:20:49.257056+0000","last_fullsized":"2024-09-12T03:20:49.257056+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:42.018269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386240+0000","last_change":"2024-09-12T03:19:07.389147+0000","last_active":"2024-09-12T03:19:20.386240+0000","last_peered":"2024-09-12T03:19:20.386240+0000","last_clean":"2024-09-12T03:19:20.386240+0000","last_became_active":"2024-09-12T03:19:07.388744+0000","last_became_peered":"2024-09-12T03:19:07.388744+0000","last_unstale":"2024-09-12T03:19:20.386240+0000","last_undegraded":"2024-09-12T03:19:20.386240+0000","last_fullsized":"2024-09-12T03:19:20.386240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:16.700644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.264622+0000","last_change":"2024-09-12T03:19:09.329243+0000","last_active":"2024-09-12T03:20:49.264622+0000","last_peered":"2024-09-12T03:20:49.264622+0000","last_clean":"2024-09-12T03:20:49.264622+0000","last_became_active":"2024-09-12T03:19:09.328610+0000","last_became_peered":"2024-09-12T03:19:09.328610+0000","last_unstale":"2024-09-12T03:20:49.264622+0000","last_undegraded":"2024-09-12T03:20:49.264622+0000","last_fullsized":"2024-09-12T03:20:49.264622+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:49.452942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'32","reported_seq":251,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:52.513814+0000","last_change":"2024-09-12T03:19:07.207804+0000","last_active":"2024-09-12T03:20:52.513814+0000","last_peered":"2024-09-12T03:20:52.513814+0000","last_clean":"2024-09-12T03:20:52.513814+0000","last_became_active":"2024-09-12T03:19:07.207409+0000","last_became_peered":"2024-09-12T03:19:07.207409+0000","last_unstale":"2024-09-12T03:20:52.513814+0000","last_undegraded":"2024-09-12T03:20:52.513814+0000","last_fullsized":"2024-09-12T03:20:52.513814+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:10.732423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60321,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":197,"num_read_kb":157,"num_write":33,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.255978+0000","last_change":"2024-09-12T03:19:09.333102+0000","last_active":"2024-09-12T03:20:49.255978+0000","last_peered":"2024-09-12T03:20:49.255978+0000","last_clean":"2024-09-12T03:20:49.255978+0000","last_became_active":"2024-09-12T03:19:09.332712+0000","last_became_peered":"2024-09-12T03:19:09.332712+0000","last_unstale":"2024-09-12T03:20:49.255978+0000","last_undegraded":"2024-09-12T03:20:49.255978+0000","last_fullsized":"2024-09-12T03:20:49.255978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:22.912213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382902+0000","last_change":"2024-09-12T03:19:20.382348+0000","last_active":"2024-09-12T03:19:20.382902+0000","last_peered":"2024-09-12T03:19:20.382902+0000","last_clean":"2024-09-12T03:19:20.382902+0000","last_became_active":"2024-09-12T03:19:20.375142+0000","last_became_peered":"2024-09-12T03:19:20.375142+0000","last_unstale":"2024-09-12T03:19:20.382902+0000","last_undegraded":"2024-09-12T03:19:20.382902+0000","last_fullsized":"2024-09-12T03:19:20.382902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:04.573599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.246879+0000","last_change":"2024-09-12T03:19:09.333194+0000","last_active":"2024-09-12T03:20:49.246879+0000","last_peered":"2024-09-12T03:20:49.246879+0000","last_clean":"2024-09-12T03:20:49.246879+0000","last_became_active":"2024-09-12T03:19:09.332865+0000","last_became_peered":"2024-09-12T03:19:09.332865+0000","last_unstale":"2024-09-12T03:20:49.246879+0000","last_undegraded":"2024-09-12T03:20:49.246879+0000","last_fullsized":"2024-09-12T03:20:49.246879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:57.096138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386176+0000","last_change":"2024-09-12T03:19:07.209289+0000","last_active":"2024-09-12T03:19:20.386176+0000","last_peered":"2024-09-12T03:19:20.386176+0000","last_clean":"2024-09-12T03:19:20.386176+0000","last_became_active":"2024-09-12T03:19:07.208863+0000","last_became_peered":"2024-09-12T03:19:07.208863+0000","last_unstale":"2024-09-12T03:19:20.386176+0000","last_undegraded":"2024-09-12T03:19:20.386176+0000","last_fullsized":"2024-09-12T03:19:20.386176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:26.168148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256973+0000","last_change":"2024-09-12T03:19:09.343620+0000","last_active":"2024-09-12T03:20:49.256973+0000","last_peered":"2024-09-12T03:20:49.256973+0000","last_clean":"2024-09-12T03:20:49.256973+0000","last_became_active":"2024-09-12T03:19:09.343299+0000","last_became_peered":"2024-09-12T03:19:09.343299+0000","last_unstale":"2024-09-12T03:20:49.256973+0000","last_undegraded":"2024-09-12T03:20:49.256973+0000","last_fullsized":"2024-09-12T03:20:49.256973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:34.268886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":126,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:55.551460+0000","last_change":"2024-09-12T03:19:07.206160+0000","last_active":"2024-09-12T03:20:55.551460+0000","last_peered":"2024-09-12T03:20:55.551460+0000","last_clean":"2024-09-12T03:20:55.551460+0000","last_became_active":"2024-09-12T03:19:07.205509+0000","last_became_peered":"2024-09-12T03:19:07.205509+0000","last_unstale":"2024-09-12T03:20:55.551460+0000","last_undegraded":"2024-09-12T03:20:55.551460+0000","last_fullsized":"2024-09-12T03:20:55.551460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:35.362831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250896+0000","last_change":"2024-09-12T03:19:09.337595+0000","last_active":"2024-09-12T03:20:49.250896+0000","last_peered":"2024-09-12T03:20:49.250896+0000","last_clean":"2024-09-12T03:20:49.250896+0000","last_became_active":"2024-09-12T03:19:09.337281+0000","last_became_peered":"2024-09-12T03:19:09.337281+0000","last_unstale":"2024-09-12T03:20:49.250896+0000","last_undegraded":"2024-09-12T03:20:49.250896+0000","last_fullsized":"2024-09-12T03:20:49.250896+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:35.761679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373995+0000","last_change":"2024-09-12T03:19:07.389336+0000","last_active":"2024-09-12T03:19:20.373995+0000","last_peered":"2024-09-12T03:19:20.373995+0000","last_clean":"2024-09-12T03:19:20.373995+0000","last_became_active":"2024-09-12T03:19:07.388649+0000","last_became_peered":"2024-09-12T03:19:07.388649+0000","last_unstale":"2024-09-12T03:19:20.373995+0000","last_undegraded":"2024-09-12T03:19:20.373995+0000","last_fullsized":"2024-09-12T03:19:20.373995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:33.312700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.253821+0000","last_change":"2024-09-12T03:19:09.340674+0000","last_active":"2024-09-12T03:20:49.253821+0000","last_peered":"2024-09-12T03:20:49.253821+0000","last_clean":"2024-09-12T03:20:49.253821+0000","last_became_active":"2024-09-12T03:19:09.340144+0000","last_became_peered":"2024-09-12T03:19:09.340144+0000","last_unstale":"2024-09-12T03:20:49.253821+0000","last_undegraded":"2024-09-12T03:20:49.253821+0000","last_fullsized":"2024-09-12T03:20:49.253821+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:21:15.605086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373837+0000","last_change":"2024-09-12T03:19:07.206513+0000","last_active":"2024-09-12T03:19:20.373837+0000","last_peered":"2024-09-12T03:19:20.373837+0000","last_clean":"2024-09-12T03:19:20.373837+0000","last_became_active":"2024-09-12T03:19:07.205974+0000","last_became_peered":"2024-09-12T03:19:07.205974+0000","last_unstale":"2024-09-12T03:19:20.373837+0000","last_undegraded":"2024-09-12T03:19:20.373837+0000","last_fullsized":"2024-09-12T03:19:20.373837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:43:10.002123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252033+0000","last_change":"2024-09-12T03:19:09.342200+0000","last_active":"2024-09-12T03:20:49.252033+0000","last_peered":"2024-09-12T03:20:49.252033+0000","last_clean":"2024-09-12T03:20:49.252033+0000","last_became_active":"2024-09-12T03:19:09.341901+0000","last_became_peered":"2024-09-12T03:19:09.341901+0000","last_unstale":"2024-09-12T03:20:49.252033+0000","last_undegraded":"2024-09-12T03:20:49.252033+0000","last_fullsized":"2024-09-12T03:20:49.252033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:27.486214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375373+0000","last_change":"2024-09-12T03:19:07.207514+0000","last_active":"2024-09-12T03:19:20.375373+0000","last_peered":"2024-09-12T03:19:20.375373+0000","last_clean":"2024-09-12T03:19:20.375373+0000","last_became_active":"2024-09-12T03:19:07.205925+0000","last_became_peered":"2024-09-12T03:19:07.205925+0000","last_unstale":"2024-09-12T03:19:20.375373+0000","last_undegraded":"2024-09-12T03:19:20.375373+0000","last_fullsized":"2024-09-12T03:19:20.375373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:20.244703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256005+0000","last_change":"2024-09-12T03:19:09.337837+0000","last_active":"2024-09-12T03:20:49.256005+0000","last_peered":"2024-09-12T03:20:49.256005+0000","last_clean":"2024-09-12T03:20:49.256005+0000","last_became_active":"2024-09-12T03:19:09.337502+0000","last_became_peered":"2024-09-12T03:19:09.337502+0000","last_unstale":"2024-09-12T03:20:49.256005+0000","last_undegraded":"2024-09-12T03:20:49.256005+0000","last_fullsized":"2024-09-12T03:20:49.256005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:30.167139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386356+0000","last_change":"2024-09-12T03:19:07.209130+0000","last_active":"2024-09-12T03:19:20.386356+0000","last_peered":"2024-09-12T03:19:20.386356+0000","last_clean":"2024-09-12T03:19:20.386356+0000","last_became_active":"2024-09-12T03:19:07.208753+0000","last_became_peered":"2024-09-12T03:19:07.208753+0000","last_unstale":"2024-09-12T03:19:20.386356+0000","last_undegraded":"2024-09-12T03:19:20.386356+0000","last_fullsized":"2024-09-12T03:19:20.386356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:34.719292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250623+0000","last_change":"2024-09-12T03:19:09.336382+0000","last_active":"2024-09-12T03:20:49.250623+0000","last_peered":"2024-09-12T03:20:49.250623+0000","last_clean":"2024-09-12T03:20:49.250623+0000","last_became_active":"2024-09-12T03:19:09.336015+0000","last_became_peered":"2024-09-12T03:19:09.336015+0000","last_unstale":"2024-09-12T03:20:49.250623+0000","last_undegraded":"2024-09-12T03:20:49.250623+0000","last_fullsized":"2024-09-12T03:20:49.250623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:19.430849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375044+0000","last_change":"2024-09-12T03:19:07.528567+0000","last_active":"2024-09-12T03:19:20.375044+0000","last_peered":"2024-09-12T03:19:20.375044+0000","last_clean":"2024-09-12T03:19:20.375044+0000","last_became_active":"2024-09-12T03:19:07.528123+0000","last_became_peered":"2024-09-12T03:19:07.528123+0000","last_unstale":"2024-09-12T03:19:20.375044+0000","last_undegraded":"2024-09-12T03:19:20.375044+0000","last_fullsized":"2024-09-12T03:19:20.375044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:34.416714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256025+0000","last_change":"2024-09-12T03:19:09.338836+0000","last_active":"2024-09-12T03:20:49.256025+0000","last_peered":"2024-09-12T03:20:49.256025+0000","last_clean":"2024-09-12T03:20:49.256025+0000","last_became_active":"2024-09-12T03:19:09.337571+0000","last_became_peered":"2024-09-12T03:19:09.337571+0000","last_unstale":"2024-09-12T03:20:49.256025+0000","last_undegraded":"2024-09-12T03:20:49.256025+0000","last_fullsized":"2024-09-12T03:20:49.256025+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:39.039878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":330,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372792+0000","last_change":"2024-09-12T03:18:01.051311+0000","last_active":"2024-09-12T03:19:20.372792+0000","last_peered":"2024-09-12T03:19:20.372792+0000","last_clean":"2024-09-12T03:19:20.372792+0000","last_became_active":"2024-09-12T03:18:01.050849+0000","last_became_peered":"2024-09-12T03:18:01.050849+0000","last_unstale":"2024-09-12T03:19:20.372792+0000","last_undegraded":"2024-09-12T03:19:20.372792+0000","last_fullsized":"2024-09-12T03:19:20.372792+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:14:38.096222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:14:38.096222+0000","last_clean_scrub_stamp":"2024-09-12T03:14:38.096222+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:33.067603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382035+0000","last_change":"2024-09-12T03:19:07.206045+0000","last_active":"2024-09-12T03:19:20.382035+0000","last_peered":"2024-09-12T03:19:20.382035+0000","last_clean":"2024-09-12T03:19:20.382035+0000","last_became_active":"2024-09-12T03:19:07.205594+0000","last_became_peered":"2024-09-12T03:19:07.205594+0000","last_unstale":"2024-09-12T03:19:20.382035+0000","last_undegraded":"2024-09-12T03:19:20.382035+0000","last_fullsized":"2024-09-12T03:19:20.382035+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:12.811906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.244733+0000","last_change":"2024-09-12T03:19:20.387274+0000","last_active":"2024-09-12T03:20:49.244733+0000","last_peered":"2024-09-12T03:20:49.244733+0000","last_clean":"2024-09-12T03:20:49.244733+0000","last_became_active":"2024-09-12T03:19:20.386878+0000","last_became_peered":"2024-09-12T03:19:20.386878+0000","last_unstale":"2024-09-12T03:20:49.244733+0000","last_undegraded":"2024-09-12T03:20:49.244733+0000","last_fullsized":"2024-09-12T03:20:49.244733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:26.674032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372302+0000","last_change":"2024-09-12T03:19:07.206212+0000","last_active":"2024-09-12T03:19:20.372302+0000","last_peered":"2024-09-12T03:19:20.372302+0000","last_clean":"2024-09-12T03:19:20.372302+0000","last_became_active":"2024-09-12T03:19:07.205899+0000","last_became_peered":"2024-09-12T03:19:07.205899+0000","last_unstale":"2024-09-12T03:19:20.372302+0000","last_undegraded":"2024-09-12T03:19:20.372302+0000","last_fullsized":"2024-09-12T03:19:20.372302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:02.169436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250795+0000","last_change":"2024-09-12T03:19:20.377494+0000","last_active":"2024-09-12T03:20:49.250795+0000","last_peered":"2024-09-12T03:20:49.250795+0000","last_clean":"2024-09-12T03:20:49.250795+0000","last_became_active":"2024-09-12T03:19:20.377091+0000","last_became_peered":"2024-09-12T03:19:20.377091+0000","last_unstale":"2024-09-12T03:20:49.250795+0000","last_undegraded":"2024-09-12T03:20:49.250795+0000","last_fullsized":"2024-09-12T03:20:49.250795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:30.926853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":119,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:51.542783+0000","last_change":"2024-09-12T03:19:07.388486+0000","last_active":"2024-09-12T03:20:51.542783+0000","last_peered":"2024-09-12T03:20:51.542783+0000","last_clean":"2024-09-12T03:20:51.542783+0000","last_became_active":"2024-09-12T03:19:07.388125+0000","last_became_peered":"2024-09-12T03:19:07.388125+0000","last_unstale":"2024-09-12T03:20:51.542783+0000","last_undegraded":"2024-09-12T03:20:51.542783+0000","last_fullsized":"2024-09-12T03:20:51.542783+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:46.218601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254069+0000","last_change":"2024-09-12T03:19:09.344308+0000","last_active":"2024-09-12T03:20:49.254069+0000","last_peered":"2024-09-12T03:20:49.254069+0000","last_clean":"2024-09-12T03:20:49.254069+0000","last_became_active":"2024-09-12T03:19:09.343987+0000","last_became_peered":"2024-09-12T03:19:09.343987+0000","last_unstale":"2024-09-12T03:20:49.254069+0000","last_undegraded":"2024-09-12T03:20:49.254069+0000","last_fullsized":"2024-09-12T03:20:49.254069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:16:11.616555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372560+0000","last_change":"2024-09-12T03:19:07.208619+0000","last_active":"2024-09-12T03:19:20.372560+0000","last_peered":"2024-09-12T03:19:20.372560+0000","last_clean":"2024-09-12T03:19:20.372560+0000","last_became_active":"2024-09-12T03:19:07.208254+0000","last_became_peered":"2024-09-12T03:19:07.208254+0000","last_unstale":"2024-09-12T03:19:20.372560+0000","last_undegraded":"2024-09-12T03:19:20.372560+0000","last_fullsized":"2024-09-12T03:19:20.372560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:56.177948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.238148+0000","last_change":"2024-09-12T03:19:09.341837+0000","last_active":"2024-09-12T03:20:49.238148+0000","last_peered":"2024-09-12T03:20:49.238148+0000","last_clean":"2024-09-12T03:20:49.238148+0000","last_became_active":"2024-09-12T03:19:09.341069+0000","last_became_peered":"2024-09-12T03:19:09.341069+0000","last_unstale":"2024-09-12T03:20:49.238148+0000","last_undegraded":"2024-09-12T03:20:49.238148+0000","last_fullsized":"2024-09-12T03:20:49.238148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:04.238986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372460+0000","last_change":"2024-09-12T03:19:07.388870+0000","last_active":"2024-09-12T03:19:20.372460+0000","last_peered":"2024-09-12T03:19:20.372460+0000","last_clean":"2024-09-12T03:19:20.372460+0000","last_became_active":"2024-09-12T03:19:07.388555+0000","last_became_peered":"2024-09-12T03:19:07.388555+0000","last_unstale":"2024-09-12T03:19:20.372460+0000","last_undegraded":"2024-09-12T03:19:20.372460+0000","last_fullsized":"2024-09-12T03:19:20.372460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:29.789454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.247542+0000","last_change":"2024-09-12T03:19:09.334714+0000","last_active":"2024-09-12T03:20:49.247542+0000","last_peered":"2024-09-12T03:20:49.247542+0000","last_clean":"2024-09-12T03:20:49.247542+0000","last_became_active":"2024-09-12T03:19:09.334413+0000","last_became_peered":"2024-09-12T03:19:09.334413+0000","last_unstale":"2024-09-12T03:20:49.247542+0000","last_undegraded":"2024-09-12T03:20:49.247542+0000","last_fullsized":"2024-09-12T03:20:49.247542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:52.222340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373797+0000","last_change":"2024-09-12T03:19:07.389055+0000","last_active":"2024-09-12T03:19:20.373797+0000","last_peered":"2024-09-12T03:19:20.373797+0000","last_clean":"2024-09-12T03:19:20.373797+0000","last_became_active":"2024-09-12T03:19:07.388673+0000","last_became_peered":"2024-09-12T03:19:07.388673+0000","last_unstale":"2024-09-12T03:19:20.373797+0000","last_undegraded":"2024-09-12T03:19:20.373797+0000","last_fullsized":"2024-09-12T03:19:20.373797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:20.974533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248932+0000","last_change":"2024-09-12T03:19:09.340105+0000","last_active":"2024-09-12T03:20:49.248932+0000","last_peered":"2024-09-12T03:20:49.248932+0000","last_clean":"2024-09-12T03:20:49.248932+0000","last_became_active":"2024-09-12T03:19:09.339642+0000","last_became_peered":"2024-09-12T03:19:09.339642+0000","last_unstale":"2024-09-12T03:20:49.248932+0000","last_undegraded":"2024-09-12T03:20:49.248932+0000","last_fullsized":"2024-09-12T03:20:49.248932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:01.624236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'15","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:52.343269+0000","last_change":"2024-09-12T03:19:07.207232+0000","last_active":"2024-09-12T03:20:52.343269+0000","last_peered":"2024-09-12T03:20:52.343269+0000","last_clean":"2024-09-12T03:20:52.343269+0000","last_became_active":"2024-09-12T03:19:07.206921+0000","last_became_peered":"2024-09-12T03:19:07.206921+0000","last_unstale":"2024-09-12T03:20:52.343269+0000","last_undegraded":"2024-09-12T03:20:52.343269+0000","last_fullsized":"2024-09-12T03:20:52.343269+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:25.457853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252490+0000","last_change":"2024-09-12T03:19:09.330442+0000","last_active":"2024-09-12T03:20:49.252490+0000","last_peered":"2024-09-12T03:20:49.252490+0000","last_clean":"2024-09-12T03:20:49.252490+0000","last_became_active":"2024-09-12T03:19:09.330081+0000","last_became_peered":"2024-09-12T03:19:09.330081+0000","last_unstale":"2024-09-12T03:20:49.252490+0000","last_undegraded":"2024-09-12T03:20:49.252490+0000","last_fullsized":"2024-09-12T03:20:49.252490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:56.152562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382392+0000","last_change":"2024-09-12T03:19:07.528646+0000","last_active":"2024-09-12T03:19:20.382392+0000","last_peered":"2024-09-12T03:19:20.382392+0000","last_clean":"2024-09-12T03:19:20.382392+0000","last_became_active":"2024-09-12T03:19:07.528334+0000","last_became_peered":"2024-09-12T03:19:07.528334+0000","last_unstale":"2024-09-12T03:19:20.382392+0000","last_undegraded":"2024-09-12T03:19:20.382392+0000","last_fullsized":"2024-09-12T03:19:20.382392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:40.425579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256685+0000","last_change":"2024-09-12T03:19:09.331993+0000","last_active":"2024-09-12T03:20:49.256685+0000","last_peered":"2024-09-12T03:20:49.256685+0000","last_clean":"2024-09-12T03:20:49.256685+0000","last_became_active":"2024-09-12T03:19:09.331698+0000","last_became_peered":"2024-09-12T03:19:09.331698+0000","last_unstale":"2024-09-12T03:20:49.256685+0000","last_undegraded":"2024-09-12T03:20:49.256685+0000","last_fullsized":"2024-09-12T03:20:49.256685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:03:54.182833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370179+0000","last_change":"2024-09-12T03:19:07.388297+0000","last_active":"2024-09-12T03:19:20.370179+0000","last_peered":"2024-09-12T03:19:20.370179+0000","last_clean":"2024-09-12T03:19:20.370179+0000","last_became_active":"2024-09-12T03:19:07.388038+0000","last_became_peered":"2024-09-12T03:19:07.388038+0000","last_unstale":"2024-09-12T03:19:20.370179+0000","last_undegraded":"2024-09-12T03:19:20.370179+0000","last_fullsized":"2024-09-12T03:19:20.370179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:45.874195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.212080+0000","last_change":"2024-09-12T03:19:09.331525+0000","last_active":"2024-09-12T03:20:49.212080+0000","last_peered":"2024-09-12T03:20:49.212080+0000","last_clean":"2024-09-12T03:20:49.212080+0000","last_became_active":"2024-09-12T03:19:09.330903+0000","last_became_peered":"2024-09-12T03:19:09.330903+0000","last_unstale":"2024-09-12T03:20:49.212080+0000","last_undegraded":"2024-09-12T03:20:49.212080+0000","last_fullsized":"2024-09-12T03:20:49.212080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:02.455132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372235+0000","last_change":"2024-09-12T03:19:07.205181+0000","last_active":"2024-09-12T03:19:20.372235+0000","last_peered":"2024-09-12T03:19:20.372235+0000","last_clean":"2024-09-12T03:19:20.372235+0000","last_became_active":"2024-09-12T03:19:07.204726+0000","last_became_peered":"2024-09-12T03:19:07.204726+0000","last_unstale":"2024-09-12T03:19:20.372235+0000","last_undegraded":"2024-09-12T03:19:20.372235+0000","last_fullsized":"2024-09-12T03:19:20.372235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:26.112800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250768+0000","last_change":"2024-09-12T03:19:09.345338+0000","last_active":"2024-09-12T03:20:49.250768+0000","last_peered":"2024-09-12T03:20:49.250768+0000","last_clean":"2024-09-12T03:20:49.250768+0000","last_became_active":"2024-09-12T03:19:09.345023+0000","last_became_peered":"2024-09-12T03:19:09.345023+0000","last_unstale":"2024-09-12T03:20:49.250768+0000","last_undegraded":"2024-09-12T03:20:49.250768+0000","last_fullsized":"2024-09-12T03:20:49.250768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:39.184065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374938+0000","last_change":"2024-09-12T03:19:07.211954+0000","last_active":"2024-09-12T03:19:20.374938+0000","last_peered":"2024-09-12T03:19:20.374938+0000","last_clean":"2024-09-12T03:19:20.374938+0000","last_became_active":"2024-09-12T03:19:07.211509+0000","last_became_peered":"2024-09-12T03:19:07.211509+0000","last_unstale":"2024-09-12T03:19:20.374938+0000","last_undegraded":"2024-09-12T03:19:20.374938+0000","last_fullsized":"2024-09-12T03:19:20.374938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:27.461924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.244962+0000","last_change":"2024-09-12T03:19:09.331168+0000","last_active":"2024-09-12T03:20:49.244962+0000","last_peered":"2024-09-12T03:20:49.244962+0000","last_clean":"2024-09-12T03:20:49.244962+0000","last_became_active":"2024-09-12T03:19:09.329592+0000","last_became_peered":"2024-09-12T03:19:09.329592+0000","last_unstale":"2024-09-12T03:20:49.244962+0000","last_undegraded":"2024-09-12T03:20:49.244962+0000","last_fullsized":"2024-09-12T03:20:49.244962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:46.127861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373572+0000","last_change":"2024-09-12T03:19:07.207802+0000","last_active":"2024-09-12T03:19:20.373572+0000","last_peered":"2024-09-12T03:19:20.373572+0000","last_clean":"2024-09-12T03:19:20.373572+0000","last_became_active":"2024-09-12T03:19:07.207428+0000","last_became_peered":"2024-09-12T03:19:07.207428+0000","last_unstale":"2024-09-12T03:19:20.373572+0000","last_undegraded":"2024-09-12T03:19:20.373572+0000","last_fullsized":"2024-09-12T03:19:20.373572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:45.135019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.255394+0000","last_change":"2024-09-12T03:19:09.336094+0000","last_active":"2024-09-12T03:20:49.255394+0000","last_peered":"2024-09-12T03:20:49.255394+0000","last_clean":"2024-09-12T03:20:49.255394+0000","last_became_active":"2024-09-12T03:19:09.335779+0000","last_became_peered":"2024-09-12T03:19:09.335779+0000","last_unstale":"2024-09-12T03:20:49.255394+0000","last_undegraded":"2024-09-12T03:20:49.255394+0000","last_fullsized":"2024-09-12T03:20:49.255394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:37.019082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254523+0000","last_change":"2024-09-12T03:19:09.344537+0000","last_active":"2024-09-12T03:20:49.254523+0000","last_peered":"2024-09-12T03:20:49.254523+0000","last_clean":"2024-09-12T03:20:49.254523+0000","last_became_active":"2024-09-12T03:19:09.343879+0000","last_became_peered":"2024-09-12T03:19:09.343879+0000","last_unstale":"2024-09-12T03:20:49.254523+0000","last_undegraded":"2024-09-12T03:20:49.254523+0000","last_fullsized":"2024-09-12T03:20:49.254523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:40.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373973+0000","last_change":"2024-09-12T03:19:07.208040+0000","last_active":"2024-09-12T03:19:20.373973+0000","last_peered":"2024-09-12T03:19:20.373973+0000","last_clean":"2024-09-12T03:19:20.373973+0000","last_became_active":"2024-09-12T03:19:07.207658+0000","last_became_peered":"2024-09-12T03:19:07.207658+0000","last_unstale":"2024-09-12T03:19:20.373973+0000","last_undegraded":"2024-09-12T03:19:20.373973+0000","last_fullsized":"2024-09-12T03:19:20.373973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:48.620707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265759+0000","last_change":"2024-09-12T03:19:09.333837+0000","last_active":"2024-09-12T03:20:49.265759+0000","last_peered":"2024-09-12T03:20:49.265759+0000","last_clean":"2024-09-12T03:20:49.265759+0000","last_became_active":"2024-09-12T03:19:09.333194+0000","last_became_peered":"2024-09-12T03:19:09.333194+0000","last_unstale":"2024-09-12T03:20:49.265759+0000","last_undegraded":"2024-09-12T03:20:49.265759+0000","last_fullsized":"2024-09-12T03:20:49.265759+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:18.766181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.385455+0000","last_change":"2024-09-12T03:19:20.384978+0000","last_active":"2024-09-12T03:19:20.385455+0000","last_peered":"2024-09-12T03:19:20.385455+0000","last_clean":"2024-09-12T03:19:20.385455+0000","last_became_active":"2024-09-12T03:19:20.376883+0000","last_became_peered":"2024-09-12T03:19:20.376883+0000","last_unstale":"2024-09-12T03:19:20.385455+0000","last_undegraded":"2024-09-12T03:19:20.385455+0000","last_fullsized":"2024-09-12T03:19:20.385455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:32.542334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265022+0000","last_change":"2024-09-12T03:19:09.334684+0000","last_active":"2024-09-12T03:20:49.265022+0000","last_peered":"2024-09-12T03:20:49.265022+0000","last_clean":"2024-09-12T03:20:49.265022+0000","last_became_active":"2024-09-12T03:19:09.334281+0000","last_became_peered":"2024-09-12T03:19:09.334281+0000","last_unstale":"2024-09-12T03:20:49.265022+0000","last_undegraded":"2024-09-12T03:20:49.265022+0000","last_fullsized":"2024-09-12T03:20:49.265022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:44.837067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372829+0000","last_change":"2024-09-12T03:19:07.388555+0000","last_active":"2024-09-12T03:19:20.372829+0000","last_peered":"2024-09-12T03:19:20.372829+0000","last_clean":"2024-09-12T03:19:20.372829+0000","last_became_active":"2024-09-12T03:19:07.388206+0000","last_became_peered":"2024-09-12T03:19:07.388206+0000","last_unstale":"2024-09-12T03:19:20.372829+0000","last_undegraded":"2024-09-12T03:19:20.372829+0000","last_fullsized":"2024-09-12T03:19:20.372829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:33.628294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248105+0000","last_change":"2024-09-12T03:19:09.339108+0000","last_active":"2024-09-12T03:20:49.248105+0000","last_peered":"2024-09-12T03:20:49.248105+0000","last_clean":"2024-09-12T03:20:49.248105+0000","last_became_active":"2024-09-12T03:19:09.338819+0000","last_became_peered":"2024-09-12T03:19:09.338819+0000","last_unstale":"2024-09-12T03:20:49.248105+0000","last_undegraded":"2024-09-12T03:20:49.248105+0000","last_fullsized":"2024-09-12T03:20:49.248105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:32.864900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372798+0000","last_change":"2024-09-12T03:19:07.207197+0000","last_active":"2024-09-12T03:19:20.372798+0000","last_peered":"2024-09-12T03:19:20.372798+0000","last_clean":"2024-09-12T03:19:20.372798+0000","last_became_active":"2024-09-12T03:19:07.206879+0000","last_became_peered":"2024-09-12T03:19:07.206879+0000","last_unstale":"2024-09-12T03:19:20.372798+0000","last_undegraded":"2024-09-12T03:19:20.372798+0000","last_fullsized":"2024-09-12T03:19:20.372798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:00:46.269636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65559,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":601,"num_read_kb":367,"num_write":179,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":209550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579938,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":433387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36892,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708260,"statfs":{"total":95995035648,"available":95957258240,"internally_reserved":0,"allocated":1585152,"data_stored":1085155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972395,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":433589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201329,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":433509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430262,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1499136,"data_stored":1025890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593787,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":432978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822720,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1511424,"data_stored":1025967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":897024,"data_stored":433808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":433289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":501409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40440,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704712,"statfs":{"total":95995035648,"available":95953625088,"internally_reserved":0,"allocated":958464,"data_stored":498858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":434249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":66087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:20:58.019 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:20:58.311 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:58.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.792+0000 7f952c9ce640 1 -- 172.21.15.6:0/1156963304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9528102790 msgr2=0x7f95281067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:58.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.792+0000 7f952c9ce640 1 --2- 172.21.15.6:0/1156963304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9528102790 0x7f95281067e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f951c009900 tx=0x7f951c02a410 comp rx=0 tx=0).stop 2024-09-12T03:20:58.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.792+0000 7f952c9ce640 1 -- 172.21.15.6:0/1156963304 shutdown_connections 2024-09-12T03:20:58.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.792+0000 7f952c9ce640 1 --2- 172.21.15.6:0/1156963304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9528106f10 0x7f952810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:58.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.792+0000 7f952c9ce640 1 --2- 172.21.15.6:0/1156963304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9528102790 0x7f95281067e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:58.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.792+0000 7f952c9ce640 1 --2- 172.21.15.6:0/1156963304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9528101de0 0x7f95281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:58.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.792+0000 7f952c9ce640 1 -- 172.21.15.6:0/1156963304 >> 172.21.15.6:0/1156963304 conn(0x7f95280fd8c0 msgr2=0x7f95280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:58.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.792+0000 7f952c9ce640 1 -- 172.21.15.6:0/1156963304 shutdown_connections 2024-09-12T03:20:58.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.792+0000 7f952c9ce640 1 -- 172.21.15.6:0/1156963304 wait complete. 2024-09-12T03:20:58.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.793+0000 7f952c9ce640 1 Processor -- start 2024-09-12T03:20:58.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.793+0000 7f952c9ce640 1 -- start start 2024-09-12T03:20:58.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.793+0000 7f952c9ce640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9528101de0 0x7f95280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:58.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.793+0000 7f952c9ce640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9528102790 0x7f9528074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:58.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.793+0000 7f952c9ce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9528106f10 0x7f9528074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:58.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.793+0000 7f952c9ce640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9528111f70 con 0x7f9528102790 2024-09-12T03:20:58.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.793+0000 7f952c9ce640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f9528111df0 con 0x7f9528101de0 2024-09-12T03:20:58.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.793+0000 7f952c9ce640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f95281120f0 con 0x7f9528106f10 2024-09-12T03:20:58.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.793+0000 7f9525d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9528102790 0x7f9528074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:58.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.793+0000 7f9525d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9528102790 0x7f9528074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36508/0 (socket says 172.21.15.6:36508) 2024-09-12T03:20:58.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.793+0000 7f9525d9b640 1 -- 172.21.15.6:0/1235961736 learned_addr learned my addr 172.21.15.6:0/1235961736 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:20:58.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.793+0000 7f952659c640 1 --2- 172.21.15.6:0/1235961736 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9528101de0 0x7f95280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:58.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.793+0000 7f9526d9d640 1 --2- 172.21.15.6:0/1235961736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9528106f10 0x7f9528074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:58.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.794+0000 7f9525d9b640 1 -- 172.21.15.6:0/1235961736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9528106f10 msgr2=0x7f9528074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:58.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.794+0000 7f9525d9b640 1 --2- 172.21.15.6:0/1235961736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9528106f10 0x7f9528074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:58.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.794+0000 7f9525d9b640 1 -- 172.21.15.6:0/1235961736 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9528101de0 msgr2=0x7f95280744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:58.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.794+0000 7f9525d9b640 1 --2- 172.21.15.6:0/1235961736 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9528101de0 0x7f95280744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:58.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.794+0000 7f9525d9b640 1 -- 172.21.15.6:0/1235961736 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95281a6900 con 0x7f9528102790 2024-09-12T03:20:58.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.794+0000 7f9525d9b640 1 --2- 172.21.15.6:0/1235961736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9528102790 0x7f9528074a30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f951c006010 tx=0x7f951c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:58.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.794+0000 7f95177fe640 1 -- 172.21.15.6:0/1235961736 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c03f030 con 0x7f9528102790 2024-09-12T03:20:58.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.794+0000 7f952c9ce640 1 -- 172.21.15.6:0/1235961736 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95281a6b90 con 0x7f9528102790 2024-09-12T03:20:58.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.794+0000 7f95177fe640 1 -- 172.21.15.6:0/1235961736 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f951c003ce0 con 0x7f9528102790 2024-09-12T03:20:58.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.794+0000 7f95177fe640 1 -- 172.21.15.6:0/1235961736 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c035070 con 0x7f9528102790 2024-09-12T03:20:58.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.794+0000 7f952c9ce640 1 -- 172.21.15.6:0/1235961736 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95281a7070 con 0x7f9528102790 2024-09-12T03:20:58.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.795+0000 7f952c9ce640 1 -- 172.21.15.6:0/1235961736 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95281021c0 con 0x7f9528102790 2024-09-12T03:20:58.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.797+0000 7f95177fe640 1 -- 172.21.15.6:0/1235961736 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f951c042030 con 0x7f9528102790 2024-09-12T03:20:58.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.797+0000 7f95177fe640 1 --2- 172.21.15.6:0/1235961736 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f95100a2c10 0x7f95100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:58.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.797+0000 7f952659c640 1 --2- 172.21.15.6:0/1235961736 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f95100a2c10 0x7f95100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:58.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.797+0000 7f95177fe640 1 -- 172.21.15.6:0/1235961736 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f951c0ef950 con 0x7f9528102790 2024-09-12T03:20:58.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.798+0000 7f952659c640 1 --2- 172.21.15.6:0/1235961736 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f95100a2c10 0x7f95100a50d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9518005d00 tx=0x7f951800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:58.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:58.800+0000 7f95177fe640 1 -- 172.21.15.6:0/1235961736 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f951c0bc1f0 con 0x7f9528102790 2024-09-12T03:20:59.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:59.013+0000 7f952c9ce640 1 -- 172.21.15.6:0/1235961736 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9528075a20 con 0x7f9528102790 2024-09-12T03:20:59.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:59.014+0000 7f95177fe640 1 -- 172.21.15.6:0/1235961736 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11908 (secure 0 0 0) 0x7f9528075a20 con 0x7f9528102790 2024-09-12T03:20:59.014 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:20:59.014 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"btime":"2024-09-12T03:20:52:974598+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:20:52.261388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24463]},"id":1}]} 2024-09-12T03:20:59.015 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 25 2024-09-12T03:20:59.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:59.016+0000 7f952c9ce640 1 -- 172.21.15.6:0/1235961736 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f95100a2c10 msgr2=0x7f95100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:59.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:59.016+0000 7f952c9ce640 1 --2- 172.21.15.6:0/1235961736 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f95100a2c10 0x7f95100a50d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9518005d00 tx=0x7f951800b410 comp rx=0 tx=0).stop 2024-09-12T03:20:59.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:59.016+0000 7f952c9ce640 1 -- 172.21.15.6:0/1235961736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9528102790 msgr2=0x7f9528074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:59.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:59.016+0000 7f952c9ce640 1 --2- 172.21.15.6:0/1235961736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9528102790 0x7f9528074a30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f951c006010 tx=0x7f951c0042f0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:59.016+0000 7f952c9ce640 1 -- 172.21.15.6:0/1235961736 shutdown_connections 2024-09-12T03:20:59.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:59.016+0000 7f952c9ce640 1 --2- 172.21.15.6:0/1235961736 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f95100a2c10 0x7f95100a50d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:59.016+0000 7f952c9ce640 1 --2- 172.21.15.6:0/1235961736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9528106f10 0x7f9528074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:59.016+0000 7f952c9ce640 1 --2- 172.21.15.6:0/1235961736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9528102790 0x7f9528074a30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:59.016+0000 7f952c9ce640 1 --2- 172.21.15.6:0/1235961736 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9528101de0 0x7f95280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:59.016+0000 7f952c9ce640 1 -- 172.21.15.6:0/1235961736 >> 172.21.15.6:0/1235961736 conn(0x7f95280fd8c0 msgr2=0x7f95280fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:59.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:59.016+0000 7f952c9ce640 1 -- 172.21.15.6:0/1235961736 shutdown_connections 2024-09-12T03:20:59.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:20:59.016+0000 7f952c9ce640 1 -- 172.21.15.6:0/1235961736 wait complete. 2024-09-12T03:20:59.314 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:59 smithi006 ceph-mon[30991]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T03:20:59.315 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:59 smithi006 ceph-mon[30991]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:20:59.315 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:59 smithi006 ceph-mon[30991]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:20:59.315 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:20:59 smithi006 ceph-mon[30991]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:20:59.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:59 smithi161 ceph-mon[35485]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T03:20:59.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:59 smithi161 ceph-mon[35485]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:20:59.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:59 smithi161 ceph-mon[35485]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:20:59.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:20:59 smithi161 ceph-mon[35485]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:20:59.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:59 smithi118 ceph-mon[35048]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T03:20:59.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:59 smithi118 ceph-mon[35048]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:20:59.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:59 smithi118 ceph-mon[35048]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:20:59.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:20:59 smithi118 ceph-mon[35048]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:20:59.702 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:20:59.957 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:00.326 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:00 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1235961736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:00.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:00 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1235961736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.408+0000 7fde687c7640 1 -- 172.21.15.6:0/45331712 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde60101420 msgr2=0x7fde6010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.408+0000 7fde687c7640 1 --2- 172.21.15.6:0/45331712 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde60101420 0x7fde6010d950 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fde58009cd0 tx=0x7fde5802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.408+0000 7fde687c7640 1 -- 172.21.15.6:0/45331712 shutdown_connections 2024-09-12T03:21:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.408+0000 7fde687c7640 1 --2- 172.21.15.6:0/45331712 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde60101420 0x7fde6010d950 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.408+0000 7fde687c7640 1 --2- 172.21.15.6:0/45331712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde60100a80 0x7fde60100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.408+0000 7fde687c7640 1 --2- 172.21.15.6:0/45331712 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde60106a80 0x7fde60106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.408+0000 7fde687c7640 1 -- 172.21.15.6:0/45331712 >> 172.21.15.6:0/45331712 conn(0x7fde600fc7a0 msgr2=0x7fde600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.408+0000 7fde687c7640 1 -- 172.21.15.6:0/45331712 shutdown_connections 2024-09-12T03:21:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.409+0000 7fde687c7640 1 -- 172.21.15.6:0/45331712 wait complete. 2024-09-12T03:21:00.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.409+0000 7fde687c7640 1 Processor -- start 2024-09-12T03:21:00.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.409+0000 7fde687c7640 1 -- start start 2024-09-12T03:21:00.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.409+0000 7fde687c7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde60100a80 0x7fde60198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.409+0000 7fde687c7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde60101420 0x7fde60198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.409+0000 7fde687c7640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde60106a80 0x7fde6019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.409+0000 7fde687c7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fde60110e80 con 0x7fde60100a80 2024-09-12T03:21:00.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.409+0000 7fde687c7640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fde60110d00 con 0x7fde60106a80 2024-09-12T03:21:00.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.409+0000 7fde687c7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fde60111000 con 0x7fde60101420 2024-09-12T03:21:00.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.409+0000 7fde66563640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde60100a80 0x7fde60198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.409+0000 7fde66563640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde60100a80 0x7fde60198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44804/0 (socket says 172.21.15.6:44804) 2024-09-12T03:21:00.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.410+0000 7fde66563640 1 -- 172.21.15.6:0/2256929201 learned_addr learned my addr 172.21.15.6:0/2256929201 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:00.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.410+0000 7fde66d64640 1 --2- 172.21.15.6:0/2256929201 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde60106a80 0x7fde6019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.410+0000 7fde65d62640 1 --2- 172.21.15.6:0/2256929201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde60101420 0x7fde60198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.410+0000 7fde66563640 1 -- 172.21.15.6:0/2256929201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde60101420 msgr2=0x7fde60198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.410+0000 7fde66563640 1 --2- 172.21.15.6:0/2256929201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde60101420 0x7fde60198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.410+0000 7fde66563640 1 -- 172.21.15.6:0/2256929201 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde60106a80 msgr2=0x7fde6019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.410+0000 7fde66563640 1 --2- 172.21.15.6:0/2256929201 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde60106a80 0x7fde6019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.410+0000 7fde66563640 1 -- 172.21.15.6:0/2256929201 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde6019d3c0 con 0x7fde60100a80 2024-09-12T03:21:00.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.410+0000 7fde66563640 1 --2- 172.21.15.6:0/2256929201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde60100a80 0x7fde60198410 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00d950 tx=0x7fde5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:00.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.410+0000 7fde577fe640 1 -- 172.21.15.6:0/2256929201 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c017030 con 0x7fde60100a80 2024-09-12T03:21:00.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.410+0000 7fde577fe640 1 -- 172.21.15.6:0/2256929201 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde5c004020 con 0x7fde60100a80 2024-09-12T03:21:00.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.410+0000 7fde687c7640 1 -- 172.21.15.6:0/2256929201 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde6019d6b0 con 0x7fde60100a80 2024-09-12T03:21:00.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.410+0000 7fde577fe640 1 -- 172.21.15.6:0/2256929201 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c0047a0 con 0x7fde60100a80 2024-09-12T03:21:00.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.410+0000 7fde687c7640 1 -- 172.21.15.6:0/2256929201 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde601a4f90 con 0x7fde60100a80 2024-09-12T03:21:00.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.411+0000 7fde687c7640 1 -- 172.21.15.6:0/2256929201 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde60100ee0 con 0x7fde60100a80 2024-09-12T03:21:00.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.412+0000 7fde577fe640 1 -- 172.21.15.6:0/2256929201 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fde5c0041d0 con 0x7fde60100a80 2024-09-12T03:21:00.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.413+0000 7fde577fe640 1 --2- 172.21.15.6:0/2256929201 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fde4c0a2c90 0x7fde4c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.413+0000 7fde65d62640 1 --2- 172.21.15.6:0/2256929201 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fde4c0a2c90 0x7fde4c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.413+0000 7fde577fe640 1 -- 172.21.15.6:0/2256929201 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fde5c0cf090 con 0x7fde60100a80 2024-09-12T03:21:00.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.414+0000 7fde65d62640 1 --2- 172.21.15.6:0/2256929201 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fde4c0a2c90 0x7fde4c0a5150 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fde600feca0 tx=0x7fde5000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:00.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.416+0000 7fde577fe640 1 -- 172.21.15.6:0/2256929201 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde5c09bee0 con 0x7fde60100a80 2024-09-12T03:21:00.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:00 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1235961736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:00.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.593+0000 7fde687c7640 1 -- 172.21.15.6:0/2256929201 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde60199770 con 0x7fde60100a80 2024-09-12T03:21:00.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.594+0000 7fde577fe640 1 -- 172.21.15.6:0/2256929201 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22607 (secure 0 0 0) 0x7fde5c09b7e0 con 0x7fde60100a80 2024-09-12T03:21:00.595 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:21:00.595 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":82,"fsid":"89d99938-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:11:29.530545+0000","modified":"2024-09-12T03:19:20.366710+0000","last_up_change":"2024-09-12T03:18:21.292572+0000","last_in_change":"2024-09-12T03:17:58.013800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:14:37.662010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:06.143398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:08.305279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f55d68b-95a0-4ee2-a46f-bc422b38471e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6803","nonce":726729286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6805","nonce":726729286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6809","nonce":726729286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6807","nonce":726729286}]},"public_addr":"172.21.15.6:6803/726729286","cluster_addr":"172.21.15.6:6805/726729286","heartbeat_back_addr":"172.21.15.6:6809/726729286","heartbeat_front_addr":"172.21.15.6:6807/726729286","state":["exists","up"]},{"osd":1,"uuid":"83afdb01-c731-4de1-96c6-3f798549b1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6801","nonce":3377555415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6803","nonce":3377555415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6807","nonce":3377555415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6805","nonce":3377555415}]},"public_addr":"172.21.15.118:6801/3377555415","cluster_addr":"172.21.15.118:6803/3377555415","heartbeat_back_addr":"172.21.15.118:6807/3377555415","heartbeat_front_addr":"172.21.15.118:6805/3377555415","state":["exists","up"]},{"osd":2,"uuid":"aed718b4-b849-403a-b81c-3a140b5c380c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6801","nonce":392541009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6803","nonce":392541009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6807","nonce":392541009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6805","nonce":392541009}]},"public_addr":"172.21.15.161:6801/392541009","cluster_addr":"172.21.15.161:6803/392541009","heartbeat_back_addr":"172.21.15.161:6807/392541009","heartbeat_front_addr":"172.21.15.161:6805/392541009","state":["exists","up"]},{"osd":3,"uuid":"25ced4c1-bf31-402d-8397-d42fd57f615f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6811","nonce":1892039151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6813","nonce":1892039151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6817","nonce":1892039151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6815","nonce":1892039151}]},"public_addr":"172.21.15.6:6811/1892039151","cluster_addr":"172.21.15.6:6813/1892039151","heartbeat_back_addr":"172.21.15.6:6817/1892039151","heartbeat_front_addr":"172.21.15.6:6815/1892039151","state":["exists","up"]},{"osd":4,"uuid":"1d884604-920b-4c35-b41e-d629ad9f9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6809","nonce":150177024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6811","nonce":150177024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6815","nonce":150177024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6813","nonce":150177024}]},"public_addr":"172.21.15.118:6809/150177024","cluster_addr":"172.21.15.118:6811/150177024","heartbeat_back_addr":"172.21.15.118:6815/150177024","heartbeat_front_addr":"172.21.15.118:6813/150177024","state":["exists","up"]},{"osd":5,"uuid":"1d40c0ef-99f6-426f-90c7-27579edc66af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6809","nonce":3758167854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6811","nonce":3758167854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6815","nonce":3758167854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6813","nonce":3758167854}]},"public_addr":"172.21.15.161:6809/3758167854","cluster_addr":"172.21.15.161:6811/3758167854","heartbeat_back_addr":"172.21.15.161:6815/3758167854","heartbeat_front_addr":"172.21.15.161:6813/3758167854","state":["exists","up"]},{"osd":6,"uuid":"9d9f9510-b107-40c2-9a98-9012715944b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6819","nonce":2129224874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6821","nonce":2129224874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6825","nonce":2129224874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6823","nonce":2129224874}]},"public_addr":"172.21.15.6:6819/2129224874","cluster_addr":"172.21.15.6:6821/2129224874","heartbeat_back_addr":"172.21.15.6:6825/2129224874","heartbeat_front_addr":"172.21.15.6:6823/2129224874","state":["exists","up"]},{"osd":7,"uuid":"84635ae1-3240-4954-af14-76bb3aa162e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6817","nonce":269980353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6819","nonce":269980353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6823","nonce":269980353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6821","nonce":269980353}]},"public_addr":"172.21.15.118:6817/269980353","cluster_addr":"172.21.15.118:6819/269980353","heartbeat_back_addr":"172.21.15.118:6823/269980353","heartbeat_front_addr":"172.21.15.118:6821/269980353","state":["exists","up"]},{"osd":8,"uuid":"6fd58e55-1492-4595-8647-823ea3e3c5d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6817","nonce":1783416134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6819","nonce":1783416134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6823","nonce":1783416134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6821","nonce":1783416134}]},"public_addr":"172.21.15.161:6817/1783416134","cluster_addr":"172.21.15.161:6819/1783416134","heartbeat_back_addr":"172.21.15.161:6823/1783416134","heartbeat_front_addr":"172.21.15.161:6821/1783416134","state":["exists","up"]},{"osd":9,"uuid":"90508a6d-a49c-43c6-a600-5bbccd8702ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6827","nonce":2987332365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6829","nonce":2987332365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6833","nonce":2987332365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6831","nonce":2987332365}]},"public_addr":"172.21.15.6:6827/2987332365","cluster_addr":"172.21.15.6:6829/2987332365","heartbeat_back_addr":"172.21.15.6:6833/2987332365","heartbeat_front_addr":"172.21.15.6:6831/2987332365","state":["exists","up"]},{"osd":10,"uuid":"911e86d1-669a-4935-82c0-7da51d141df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6825","nonce":2167146341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6827","nonce":2167146341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6831","nonce":2167146341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6829","nonce":2167146341}]},"public_addr":"172.21.15.118:6825/2167146341","cluster_addr":"172.21.15.118:6827/2167146341","heartbeat_back_addr":"172.21.15.118:6831/2167146341","heartbeat_front_addr":"172.21.15.118:6829/2167146341","state":["exists","up"]},{"osd":11,"uuid":"5a15c6a7-ff43-473c-bb48-0c129c316df5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6825","nonce":2889795796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6827","nonce":2889795796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6831","nonce":2889795796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6829","nonce":2889795796}]},"public_addr":"172.21.15.161:6825/2889795796","cluster_addr":"172.21.15.161:6827/2889795796","heartbeat_back_addr":"172.21.15.161:6831/2889795796","heartbeat_front_addr":"172.21.15.161:6829/2889795796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:42.912205+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:08.410669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:34.299134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:59.093181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:24.116635+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:49.566433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:15.719320+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:40.541017+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:05.178178+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:31.163877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:55.405110+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:19.417156+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6800/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/2620491916":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/4037656392":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/1377116573":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/3062800125":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:6800/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/4145338968":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/825701128":"2024-09-13T03:11:55.244257+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:21:00.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.597+0000 7fde687c7640 1 -- 172.21.15.6:0/2256929201 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fde4c0a2c90 msgr2=0x7fde4c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.597+0000 7fde687c7640 1 --2- 172.21.15.6:0/2256929201 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fde4c0a2c90 0x7fde4c0a5150 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fde600feca0 tx=0x7fde5000b410 comp rx=0 tx=0).stop 2024-09-12T03:21:00.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.597+0000 7fde687c7640 1 -- 172.21.15.6:0/2256929201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde60100a80 msgr2=0x7fde60198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.597+0000 7fde687c7640 1 --2- 172.21.15.6:0/2256929201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde60100a80 0x7fde60198410 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00d950 tx=0x7fde5c00de10 comp rx=0 tx=0).stop 2024-09-12T03:21:00.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.597+0000 7fde687c7640 1 -- 172.21.15.6:0/2256929201 shutdown_connections 2024-09-12T03:21:00.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.597+0000 7fde687c7640 1 --2- 172.21.15.6:0/2256929201 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fde4c0a2c90 0x7fde4c0a5150 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.597+0000 7fde687c7640 1 --2- 172.21.15.6:0/2256929201 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde60106a80 0x7fde6019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.597+0000 7fde687c7640 1 --2- 172.21.15.6:0/2256929201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde60101420 0x7fde60198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.597+0000 7fde687c7640 1 --2- 172.21.15.6:0/2256929201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde60100a80 0x7fde60198410 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.597+0000 7fde687c7640 1 -- 172.21.15.6:0/2256929201 >> 172.21.15.6:0/2256929201 conn(0x7fde600fc7a0 msgr2=0x7fde6006a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:00.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.597+0000 7fde687c7640 1 -- 172.21.15.6:0/2256929201 shutdown_connections 2024-09-12T03:21:00.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:00.597+0000 7fde687c7640 1 -- 172.21.15.6:0/2256929201 wait complete. 2024-09-12T03:21:01.272 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:01 smithi006 ceph-mon[30991]: pgmap v332: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-12T03:21:01.272 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:01 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2256929201' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:21:01.301 DEBUG:tasks.fwd_scrub:Yielding 2024-09-12T03:21:01.301 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-12T03:21:01.314 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T03:21:01.315 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:21:01.315 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-12T03:21:01.315 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:21:01.315 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:21:01.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:01 smithi161 ceph-mon[35485]: pgmap v332: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-12T03:21:01.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:01 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2256929201' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:21:01.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:01 smithi118 ceph-mon[35048]: pgmap v332: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-12T03:21:01.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:01 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2256929201' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:21:01.591 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:01.592 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:02.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.123+0000 7f6e5af07640 1 -- 172.21.15.6:0/2072076812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e54106f10 msgr2=0x7f6e5410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.123+0000 7f6e5af07640 1 --2- 172.21.15.6:0/2072076812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e54106f10 0x7f6e5410ea90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009ca0 tx=0x7f6e4c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.123+0000 7f6e5af07640 1 -- 172.21.15.6:0/2072076812 shutdown_connections 2024-09-12T03:21:02.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.123+0000 7f6e5af07640 1 --2- 172.21.15.6:0/2072076812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e54106f10 0x7f6e5410ea90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.123+0000 7f6e5af07640 1 --2- 172.21.15.6:0/2072076812 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e54102790 0x7f6e541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.123+0000 7f6e5af07640 1 --2- 172.21.15.6:0/2072076812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e54101de0 0x7f6e541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.123+0000 7f6e5af07640 1 -- 172.21.15.6:0/2072076812 >> 172.21.15.6:0/2072076812 conn(0x7f6e540fd8c0 msgr2=0x7f6e540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:02.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.123+0000 7f6e5af07640 1 -- 172.21.15.6:0/2072076812 shutdown_connections 2024-09-12T03:21:02.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.123+0000 7f6e5af07640 1 -- 172.21.15.6:0/2072076812 wait complete. 2024-09-12T03:21:02.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.124+0000 7f6e5af07640 1 Processor -- start 2024-09-12T03:21:02.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.124+0000 7f6e5af07640 1 -- start start 2024-09-12T03:21:02.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.124+0000 7f6e5af07640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e54101de0 0x7f6e541983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.124+0000 7f6e5af07640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e54102790 0x7f6e541988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.124+0000 7f6e5af07640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e54106f10 0x7f6e5419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.124+0000 7f6e5af07640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6e54111f70 con 0x7f6e54102790 2024-09-12T03:21:02.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.124+0000 7f6e5af07640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f6e54111df0 con 0x7f6e54101de0 2024-09-12T03:21:02.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.124+0000 7f6e5af07640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6e541120f0 con 0x7f6e54106f10 2024-09-12T03:21:02.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.124+0000 7f6e4bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e54102790 0x7f6e541988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.124+0000 7f6e4bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e54102790 0x7f6e541988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44824/0 (socket says 172.21.15.6:44824) 2024-09-12T03:21:02.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.124+0000 7f6e4bfff640 1 -- 172.21.15.6:0/1952235823 learned_addr learned my addr 172.21.15.6:0/1952235823 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:02.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.124+0000 7f6e58ca3640 1 --2- 172.21.15.6:0/1952235823 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e54101de0 0x7f6e541983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.124+0000 7f6e594a4640 1 --2- 172.21.15.6:0/1952235823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e54106f10 0x7f6e5419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.125+0000 7f6e4bfff640 1 -- 172.21.15.6:0/1952235823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e54106f10 msgr2=0x7f6e5419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.125+0000 7f6e4bfff640 1 --2- 172.21.15.6:0/1952235823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e54106f10 0x7f6e5419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.125+0000 7f6e4bfff640 1 -- 172.21.15.6:0/1952235823 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e54101de0 msgr2=0x7f6e541983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.125+0000 7f6e4bfff640 1 --2- 172.21.15.6:0/1952235823 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e54101de0 0x7f6e541983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.125+0000 7f6e4bfff640 1 -- 172.21.15.6:0/1952235823 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e5419d360 con 0x7f6e54102790 2024-09-12T03:21:02.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.125+0000 7f6e4bfff640 1 --2- 172.21.15.6:0/1952235823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e54102790 0x7f6e541988f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400c890 tx=0x7f6e4400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:02.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.125+0000 7f6e49ffb640 1 -- 172.21.15.6:0/1952235823 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44004220 con 0x7f6e54102790 2024-09-12T03:21:02.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.125+0000 7f6e49ffb640 1 -- 172.21.15.6:0/1952235823 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e440043c0 con 0x7f6e54102790 2024-09-12T03:21:02.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.125+0000 7f6e5af07640 1 -- 172.21.15.6:0/1952235823 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e5419d650 con 0x7f6e54102790 2024-09-12T03:21:02.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.125+0000 7f6e49ffb640 1 -- 172.21.15.6:0/1952235823 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e440049e0 con 0x7f6e54102790 2024-09-12T03:21:02.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.125+0000 7f6e5af07640 1 -- 172.21.15.6:0/1952235823 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e541a4f30 con 0x7f6e54102790 2024-09-12T03:21:02.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.126+0000 7f6e5af07640 1 -- 172.21.15.6:0/1952235823 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e541021c0 con 0x7f6e54102790 2024-09-12T03:21:02.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.127+0000 7f6e49ffb640 1 -- 172.21.15.6:0/1952235823 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f6e440079b0 con 0x7f6e54102790 2024-09-12T03:21:02.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.128+0000 7f6e49ffb640 1 --2- 172.21.15.6:0/1952235823 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f6e3c0a2c10 0x7f6e3c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.128+0000 7f6e49ffb640 1 -- 172.21.15.6:0/1952235823 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f6e440cf1a0 con 0x7f6e54102790 2024-09-12T03:21:02.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.128+0000 7f6e58ca3640 1 --2- 172.21.15.6:0/1952235823 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f6e3c0a2c10 0x7f6e3c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.128+0000 7f6e58ca3640 1 --2- 172.21.15.6:0/1952235823 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f6e3c0a2c10 0x7f6e3c0a50d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000bbe0 tx=0x7f6e5000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:02.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.132+0000 7f6e49ffb640 1 -- 172.21.15.6:0/1952235823 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6e44014030 con 0x7f6e54102790 2024-09-12T03:21:02.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.312+0000 7f6e5af07640 1 -- 172.21.15.6:0/1952235823 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6e54062630 con 0x7f6e3c0a2c10 2024-09-12T03:21:02.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.320+0000 7f6e49ffb640 1 -- 172.21.15.6:0/1952235823 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258835 (secure 0 0 0) 0x7f6e54062630 con 0x7f6e3c0a2c10 2024-09-12T03:21:02.322 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:21:02.323 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-09-12T03:21:02.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.323+0000 7f6e5af07640 1 -- 172.21.15.6:0/1952235823 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f6e3c0a2c10 msgr2=0x7f6e3c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.323+0000 7f6e5af07640 1 --2- 172.21.15.6:0/1952235823 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f6e3c0a2c10 0x7f6e3c0a50d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000bbe0 tx=0x7f6e5000b390 comp rx=0 tx=0).stop 2024-09-12T03:21:02.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.323+0000 7f6e5af07640 1 -- 172.21.15.6:0/1952235823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e54102790 msgr2=0x7f6e541988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.323+0000 7f6e5af07640 1 --2- 172.21.15.6:0/1952235823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e54102790 0x7f6e541988f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400c890 tx=0x7f6e4400cd50 comp rx=0 tx=0).stop 2024-09-12T03:21:02.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.323+0000 7f6e5af07640 1 -- 172.21.15.6:0/1952235823 shutdown_connections 2024-09-12T03:21:02.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.323+0000 7f6e5af07640 1 --2- 172.21.15.6:0/1952235823 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f6e3c0a2c10 0x7f6e3c0a50d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.323+0000 7f6e5af07640 1 --2- 172.21.15.6:0/1952235823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e54106f10 0x7f6e5419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.323+0000 7f6e5af07640 1 --2- 172.21.15.6:0/1952235823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e54102790 0x7f6e541988f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.323+0000 7f6e5af07640 1 --2- 172.21.15.6:0/1952235823 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e54101de0 0x7f6e541983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.323+0000 7f6e5af07640 1 -- 172.21.15.6:0/1952235823 >> 172.21.15.6:0/1952235823 conn(0x7f6e540fd8c0 msgr2=0x7f6e540fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:02.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.323+0000 7f6e5af07640 1 -- 172.21.15.6:0/1952235823 shutdown_connections 2024-09-12T03:21:02.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.323+0000 7f6e5af07640 1 -- 172.21.15.6:0/1952235823 wait complete. 2024-09-12T03:21:02.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.774+0000 7f3380bcd640 1 -- 172.21.15.6:0/2958921530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f337c102570 msgr2=0x7f337c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.774+0000 7f3380bcd640 1 --2- 172.21.15.6:0/2958921530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f337c102570 0x7f337c1029b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3364009900 tx=0x7f336402a480 comp rx=0 tx=0).stop 2024-09-12T03:21:02.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.774+0000 7f3380bcd640 1 -- 172.21.15.6:0/2958921530 shutdown_connections 2024-09-12T03:21:02.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.774+0000 7f3380bcd640 1 --2- 172.21.15.6:0/2958921530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f337c102ef0 0x7f337c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.774+0000 7f3380bcd640 1 --2- 172.21.15.6:0/2958921530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f337c102570 0x7f337c1029b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.774+0000 7f3380bcd640 1 --2- 172.21.15.6:0/2958921530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f337c100e20 0x7f337c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.774+0000 7f3380bcd640 1 -- 172.21.15.6:0/2958921530 >> 172.21.15.6:0/2958921530 conn(0x7f337c0fc7d0 msgr2=0x7f337c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:02.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.774+0000 7f3380bcd640 1 -- 172.21.15.6:0/2958921530 shutdown_connections 2024-09-12T03:21:02.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.774+0000 7f3380bcd640 1 -- 172.21.15.6:0/2958921530 wait complete. 2024-09-12T03:21:02.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.775+0000 7f3380bcd640 1 Processor -- start 2024-09-12T03:21:02.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.775+0000 7f3380bcd640 1 -- start start 2024-09-12T03:21:02.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.775+0000 7f3380bcd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f337c100e20 0x7f337c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.775+0000 7f3380bcd640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f337c102570 0x7f337c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.775+0000 7f3380bcd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f337c102ef0 0x7f337c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.775+0000 7f3380bcd640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f337c110f80 con 0x7f337c102ef0 2024-09-12T03:21:02.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.775+0000 7f3380bcd640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f337c110e00 con 0x7f337c102570 2024-09-12T03:21:02.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.775+0000 7f3380bcd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f337c111100 con 0x7f337c100e20 2024-09-12T03:21:02.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.775+0000 7f337ad9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f337c102ef0 0x7f337c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.776+0000 7f337ad9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f337c102ef0 0x7f337c19cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44842/0 (socket says 172.21.15.6:44842) 2024-09-12T03:21:02.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.776+0000 7f337ad9d640 1 -- 172.21.15.6:0/1581049232 learned_addr learned my addr 172.21.15.6:0/1581049232 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:02.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.776+0000 7f337a59c640 1 --2- 172.21.15.6:0/1581049232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f337c100e20 0x7f337c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.776+0000 7f3373fff640 1 --2- 172.21.15.6:0/1581049232 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f337c102570 0x7f337c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.776+0000 7f337ad9d640 1 -- 172.21.15.6:0/1581049232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f337c100e20 msgr2=0x7f337c198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.776+0000 7f337ad9d640 1 --2- 172.21.15.6:0/1581049232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f337c100e20 0x7f337c198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.776+0000 7f337ad9d640 1 -- 172.21.15.6:0/1581049232 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f337c102570 msgr2=0x7f337c198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.776+0000 7f337ad9d640 1 --2- 172.21.15.6:0/1581049232 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f337c102570 0x7f337c198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.776+0000 7f337ad9d640 1 -- 172.21.15.6:0/1581049232 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337c19d4d0 con 0x7f337c102ef0 2024-09-12T03:21:02.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.776+0000 7f337ad9d640 1 --2- 172.21.15.6:0/1581049232 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f337c102ef0 0x7f337c19cdf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f337400cad0 tx=0x7f337400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:02.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.776+0000 7f33737fe640 1 -- 172.21.15.6:0/1581049232 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337400ae20 con 0x7f337c102ef0 2024-09-12T03:21:02.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.776+0000 7f33737fe640 1 -- 172.21.15.6:0/1581049232 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3374005020 con 0x7f337c102ef0 2024-09-12T03:21:02.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.776+0000 7f3380bcd640 1 -- 172.21.15.6:0/1581049232 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f337c19d7c0 con 0x7f337c102ef0 2024-09-12T03:21:02.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.776+0000 7f33737fe640 1 -- 172.21.15.6:0/1581049232 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337400f6f0 con 0x7f337c102ef0 2024-09-12T03:21:02.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.776+0000 7f3380bcd640 1 -- 172.21.15.6:0/1581049232 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f337c0fcbb0 con 0x7f337c102ef0 2024-09-12T03:21:02.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.779+0000 7f33737fe640 1 -- 172.21.15.6:0/1581049232 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f33740051d0 con 0x7f337c102ef0 2024-09-12T03:21:02.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.779+0000 7f33737fe640 1 --2- 172.21.15.6:0/1581049232 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f33680a2c90 0x7f33680a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.780+0000 7f33737fe640 1 -- 172.21.15.6:0/1581049232 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f33740cf210 con 0x7f337c102ef0 2024-09-12T03:21:02.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.780+0000 7f337a59c640 1 --2- 172.21.15.6:0/1581049232 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f33680a2c90 0x7f33680a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.780+0000 7f3382fa6740 1 Processor -- start 2024-09-12T03:21:02.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.780+0000 7f3382fa6740 1 -- start start 2024-09-12T03:21:02.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.780+0000 7f337a59c640 1 --2- 172.21.15.6:0/1581049232 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f33680a2c90 0x7f33680a5150 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f336c005fd0 tx=0x7f336c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:02.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.780+0000 7f3382fa6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a67f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.780+0000 7f3382fa6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f548a68480 0x55f548a72800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.780+0000 7f3382fa6740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a72d40 0x55f548a75130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.780+0000 7f3382fa6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55f5489cd280 con 0x55f548a67b60 2024-09-12T03:21:02.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.780+0000 7f3382fa6740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55f548974b10 con 0x55f548a72d40 2024-09-12T03:21:02.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.780+0000 7f3382fa6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f5489c9240 con 0x55f548a68480 2024-09-12T03:21:02.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.780+0000 7f337a59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a67f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.780+0000 7f337a59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a67f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44858/0 (socket says 172.21.15.6:44858) 2024-09-12T03:21:02.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.780+0000 7f337a59c640 1 -- 172.21.15.6:0/1276729803 learned_addr learned my addr 172.21.15.6:0/1276729803 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:02.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.781+0000 7f3373fff640 1 --2- 172.21.15.6:0/1276729803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f548a68480 0x55f548a72800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.781+0000 7f337ad9d640 1 --2- 172.21.15.6:0/1276729803 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a72d40 0x55f548a75130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.781+0000 7f337a59c640 1 -- 172.21.15.6:0/1276729803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f548a68480 msgr2=0x55f548a72800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:02.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.781+0000 7f337a59c640 1 --2- 172.21.15.6:0/1276729803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f548a68480 0x55f548a72800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.781+0000 7f337a59c640 1 -- 172.21.15.6:0/1276729803 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a72d40 msgr2=0x55f548a75130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:02.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.781+0000 7f337a59c640 1 --2- 172.21.15.6:0/1276729803 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a72d40 0x55f548a75130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.781+0000 7f337a59c640 1 -- 172.21.15.6:0/1276729803 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5489811f0 con 0x55f548a67b60 2024-09-12T03:21:02.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.781+0000 7f337ad9d640 1 --2- 172.21.15.6:0/1276729803 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a72d40 0x55f548a75130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:02.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.781+0000 7f337a59c640 1 --2- 172.21.15.6:0/1276729803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a67f40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f336c00b620 tx=0x7f336c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de15609d22c59f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:02.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.781+0000 7f3370ff9640 1 -- 172.21.15.6:0/1276729803 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c003e90 con 0x55f548a67b60 2024-09-12T03:21:02.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.781+0000 7f3370ff9640 1 -- 172.21.15.6:0/1276729803 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f336c005500 con 0x55f548a67b60 2024-09-12T03:21:02.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.781+0000 7f3370ff9640 1 -- 172.21.15.6:0/1276729803 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c01e660 con 0x55f548a67b60 2024-09-12T03:21:02.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.782+0000 7f3382fa6740 1 -- 172.21.15.6:0/1276729803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 msgr2=0x55f548a67f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.782+0000 7f3382fa6740 1 --2- 172.21.15.6:0/1276729803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a67f40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f336c00b620 tx=0x7f336c013000 comp rx=0 tx=0).stop 2024-09-12T03:21:02.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.782+0000 7f3382fa6740 1 -- 172.21.15.6:0/1276729803 shutdown_connections 2024-09-12T03:21:02.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.782+0000 7f3382fa6740 1 --2- 172.21.15.6:0/1276729803 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a72d40 0x55f548a75130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.782+0000 7f3382fa6740 1 --2- 172.21.15.6:0/1276729803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f548a68480 0x55f548a72800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.782+0000 7f3382fa6740 1 --2- 172.21.15.6:0/1276729803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a67f40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.782+0000 7f3382fa6740 1 -- 172.21.15.6:0/1276729803 >> 172.21.15.6:0/1276729803 conn(0x55f548a640f0 msgr2=0x55f548a644d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:02.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.782+0000 7f3382fa6740 1 -- 172.21.15.6:0/1276729803 shutdown_connections 2024-09-12T03:21:02.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.782+0000 7f3382fa6740 1 -- 172.21.15.6:0/1276729803 wait complete. 2024-09-12T03:21:02.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.782+0000 7f3382fa6740 1 -- start start 2024-09-12T03:21:02.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.782+0000 7f3382fa6740 1 Processor -- start 2024-09-12T03:21:02.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.783+0000 7f3382fa6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a6fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.783+0000 7f3382fa6740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a68480 0x55f548a70160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.783+0000 7f3382fa6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f548a72d40 0x55f548a706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.783+0000 7f3382fa6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55f548a75910 con 0x55f548a67b60 2024-09-12T03:21:02.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.783+0000 7f3382fa6740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55f548987f10 con 0x55f548a68480 2024-09-12T03:21:02.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.783+0000 7f3382fa6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f548a75a90 con 0x55f548a72d40 2024-09-12T03:21:02.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.783+0000 7f3382fa6740 10 client.0 _ll_register_callbacks cb 0x55f548950a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:02.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.783+0000 7f337a59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a6fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.783+0000 7f337a59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a6fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44872/0 (socket says 172.21.15.6:44872) 2024-09-12T03:21:02.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.783+0000 7f337a59c640 1 -- 172.21.15.6:0/1145308443 learned_addr learned my addr 172.21.15.6:0/1145308443 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:02.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.783+0000 7f337ad9d640 1 --2- 172.21.15.6:0/1145308443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f548a72d40 0x55f548a706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.783+0000 7f3373fff640 1 --2- 172.21.15.6:0/1145308443 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a68480 0x55f548a70160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.783+0000 7f337a59c640 1 -- 172.21.15.6:0/1145308443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f548a72d40 msgr2=0x55f548a706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:02.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.783+0000 7f337a59c640 1 --2- 172.21.15.6:0/1145308443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f548a72d40 0x55f548a706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.783+0000 7f337a59c640 1 -- 172.21.15.6:0/1145308443 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a68480 msgr2=0x55f548a70160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:02.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.783+0000 7f337a59c640 1 --2- 172.21.15.6:0/1145308443 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a68480 0x55f548a70160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.783+0000 7f337a59c640 1 -- 172.21.15.6:0/1145308443 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f54873e6a0 con 0x55f548a67b60 2024-09-12T03:21:02.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.783+0000 7f337a59c640 1 --2- 172.21.15.6:0/1145308443 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a6fc20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f336c004650 tx=0x7f336c003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de26388fc614780 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:02.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.783+0000 7f3361ffb640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:21:02.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.784+0000 7f3361ffb640 1 -- 172.21.15.6:0/1145308443 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c036060 con 0x55f548a67b60 2024-09-12T03:21:02.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.784+0000 7f3361ffb640 1 -- 172.21.15.6:0/1145308443 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f336c0046c0 con 0x55f548a67b60 2024-09-12T03:21:02.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.784+0000 7f3382fa6740 1 -- 172.21.15.6:0/1145308443 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f336c005500 con 0x55f548a67b60 2024-09-12T03:21:02.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.784+0000 7f3361ffb640 1 -- 172.21.15.6:0/1145308443 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f336c0048c0 con 0x55f548a67b60 2024-09-12T03:21:02.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.784+0000 7f3382fa6740 10 client.15207 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:02.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.784+0000 7f3382fa6740 1 -- 172.21.15.6:0/1145308443 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f548a81570 con 0x55f548a67b60 2024-09-12T03:21:02.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.784+0000 7f3361ffb640 1 -- 172.21.15.6:0/1145308443 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f336c005500 con 0x55f548a67b60 2024-09-12T03:21:02.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.784+0000 7f3361ffb640 1 -- 172.21.15.6:0/1145308443 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f336c026400 con 0x55f548a67b60 2024-09-12T03:21:02.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.784+0000 7f3382fa6740 10 client.15207 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:02.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.784+0000 7f3382fa6740 10 client.15207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:21:02.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.784+0000 7f3382fa6740 20 client.15207 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:02.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.784+0000 7f3382fa6740 1 --2- 172.21.15.6:0/1145308443 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55f548a82980 0x55f548a84d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.784+0000 7f3382fa6740 4 client.15207 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:02.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.784+0000 7f3382fa6740 1 -- 172.21.15.6:0/1145308443 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5489da7f0 con 0x55f548a82980 2024-09-12T03:21:02.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.785+0000 7f3373fff640 1 --2- 172.21.15.6:0/1145308443 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55f548a82980 0x55f548a84d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.785+0000 7f3373fff640 1 --2- 172.21.15.6:0/1145308443 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55f548a82980 0x55f548a84d70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=416a19f3b3f46495 server_cookie=30a4f2e4f7fa1a64 in_seq=0 out_seq=0 2024-09-12T03:21:02.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.785+0000 7f3361ffb640 10 client.15207 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:02.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.786+0000 7f3361ffb640 1 -- 172.21.15.6:0/1145308443 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5489da7f0 con 0x55f548a82980 2024-09-12T03:21:02.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.786+0000 7f3361ffb640 10 client.15207 handle_command_reply: tid=0 2024-09-12T03:21:02.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.786+0000 7f3382fa6740 1 client.15207 shutdown 2024-09-12T03:21:02.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.786+0000 7f3382fa6740 1 -- 172.21.15.6:0/1145308443 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 msgr2=0x55f548a6fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.786+0000 7f3382fa6740 1 --2- 172.21.15.6:0/1145308443 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a6fc20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f336c004650 tx=0x7f336c003a90 comp rx=0 tx=0).stop 2024-09-12T03:21:02.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.786+0000 7f3382fa6740 1 -- 172.21.15.6:0/1145308443 shutdown_connections 2024-09-12T03:21:02.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.786+0000 7f3382fa6740 1 --2- 172.21.15.6:0/1145308443 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55f548a82980 0x55f548a84d70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.786+0000 7f3361ffb640 0 client.15207 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:02.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.786+0000 7f3382fa6740 1 --2- 172.21.15.6:0/1145308443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f548a72d40 0x55f548a706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.786+0000 7f3382fa6740 1 --2- 172.21.15.6:0/1145308443 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a68480 0x55f548a70160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.786+0000 7f3382fa6740 1 --2- 172.21.15.6:0/1145308443 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a6fc20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.786+0000 7f3382fa6740 1 -- 172.21.15.6:0/1145308443 >> 172.21.15.6:0/1145308443 conn(0x55f548a640f0 msgr2=0x55f548a69dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:02.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.786+0000 7f3382fa6740 1 -- 172.21.15.6:0/1145308443 shutdown_connections 2024-09-12T03:21:02.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.786+0000 7f3382fa6740 1 -- 172.21.15.6:0/1145308443 wait complete. 2024-09-12T03:21:02.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.786+0000 7f3382fa6740 20 client.15207 trim_cache size 0 max 16384 2024-09-12T03:21:02.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.805+0000 7f3382fa6740 1 Processor -- start 2024-09-12T03:21:02.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.805+0000 7f3382fa6740 1 -- start start 2024-09-12T03:21:02.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.805+0000 7f3382fa6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a6ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.805+0000 7f3382fa6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f548a68480 0x55f548a8a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.805+0000 7f3382fa6740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a72d40 0x55f548a8aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.805+0000 7f3382fa6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1b2030 con 0x55f548a67b60 2024-09-12T03:21:02.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.805+0000 7f3382fa6740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1af900 con 0x55f548a72d40 2024-09-12T03:21:02.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.805+0000 7f3382fa6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1ad770 con 0x55f548a68480 2024-09-12T03:21:02.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.805+0000 7f337a59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a6ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.805+0000 7f337a59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a6ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44880/0 (socket says 172.21.15.6:44880) 2024-09-12T03:21:02.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.805+0000 7f337a59c640 1 -- 172.21.15.6:0/1185571861 learned_addr learned my addr 172.21.15.6:0/1185571861 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:02.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.806+0000 7f337ad9d640 1 --2- 172.21.15.6:0/1185571861 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a72d40 0x55f548a8aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.806+0000 7f337a59c640 1 -- 172.21.15.6:0/1185571861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f548a68480 msgr2=0x55f548a8a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:02.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.806+0000 7f337a59c640 1 --2- 172.21.15.6:0/1185571861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f548a68480 0x55f548a8a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.806+0000 7f337a59c640 1 -- 172.21.15.6:0/1185571861 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a72d40 msgr2=0x55f548a8aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:02.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.806+0000 7f337a59c640 1 --2- 172.21.15.6:0/1185571861 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a72d40 0x55f548a8aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.806+0000 7f337a59c640 1 -- 172.21.15.6:0/1185571861 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f548a63f20 con 0x55f548a67b60 2024-09-12T03:21:02.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.806+0000 7f337a59c640 1 --2- 172.21.15.6:0/1185571861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a6ead0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f336c003c10 tx=0x7f336c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff72af6b86eed162 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:02.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.806+0000 7f3363fff640 1 -- 172.21.15.6:0/1185571861 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c040080 con 0x55f548a67b60 2024-09-12T03:21:02.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.806+0000 7f3363fff640 1 -- 172.21.15.6:0/1185571861 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f336c026b70 con 0x55f548a67b60 2024-09-12T03:21:02.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.806+0000 7f3363fff640 1 -- 172.21.15.6:0/1185571861 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c045470 con 0x55f548a67b60 2024-09-12T03:21:02.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.807+0000 7f3382fa6740 1 -- 172.21.15.6:0/1185571861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 msgr2=0x55f548a6ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.807+0000 7f3382fa6740 1 --2- 172.21.15.6:0/1185571861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a6ead0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f336c003c10 tx=0x7f336c00bf40 comp rx=0 tx=0).stop 2024-09-12T03:21:02.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.807+0000 7f3382fa6740 1 -- 172.21.15.6:0/1185571861 shutdown_connections 2024-09-12T03:21:02.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.807+0000 7f3382fa6740 1 --2- 172.21.15.6:0/1185571861 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a72d40 0x55f548a8aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.807+0000 7f3382fa6740 1 --2- 172.21.15.6:0/1185571861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f548a68480 0x55f548a8a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.807+0000 7f3382fa6740 1 --2- 172.21.15.6:0/1185571861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a6ead0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.807+0000 7f3382fa6740 1 -- 172.21.15.6:0/1185571861 >> 172.21.15.6:0/1185571861 conn(0x55f548a640f0 msgr2=0x55f548a6e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:02.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.807+0000 7f3382fa6740 1 -- 172.21.15.6:0/1185571861 shutdown_connections 2024-09-12T03:21:02.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.807+0000 7f3382fa6740 1 -- 172.21.15.6:0/1185571861 wait complete. 2024-09-12T03:21:02.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.807+0000 7f3382fa6740 1 -- start start 2024-09-12T03:21:02.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.808+0000 7f3382fa6740 1 Processor -- start 2024-09-12T03:21:02.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.808+0000 7f3382fa6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a7b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.808+0000 7f3382fa6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f548a68480 0x55f548a76930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.808+0000 7f3382fa6740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a72d40 0x55f548a770c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.808+0000 7f3382fa6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1addc0 con 0x55f548a67b60 2024-09-12T03:21:02.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.808+0000 7f3382fa6740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1af230 con 0x55f548a72d40 2024-09-12T03:21:02.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.808+0000 7f3382fa6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1ae490 con 0x55f548a68480 2024-09-12T03:21:02.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.808+0000 7f337a59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a7b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.808+0000 7f3382fa6740 10 client.0 _ll_register_callbacks cb 0x7f3364009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:02.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.808+0000 7f337a59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a7b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44884/0 (socket says 172.21.15.6:44884) 2024-09-12T03:21:02.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.808+0000 7f337a59c640 1 -- 172.21.15.6:0/3135791982 learned_addr learned my addr 172.21.15.6:0/3135791982 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:02.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.808+0000 7f3373fff640 1 --2- 172.21.15.6:0/3135791982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f548a68480 0x55f548a76930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.808+0000 7f337ad9d640 1 --2- 172.21.15.6:0/3135791982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a72d40 0x55f548a770c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.808+0000 7f337a59c640 1 -- 172.21.15.6:0/3135791982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f548a68480 msgr2=0x55f548a76930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:02.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.808+0000 7f337a59c640 1 --2- 172.21.15.6:0/3135791982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f548a68480 0x55f548a76930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.808+0000 7f337a59c640 1 -- 172.21.15.6:0/3135791982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a72d40 msgr2=0x55f548a770c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:02.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.808+0000 7f337a59c640 1 --2- 172.21.15.6:0/3135791982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a72d40 0x55f548a770c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.808+0000 7f337a59c640 1 -- 172.21.15.6:0/3135791982 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f548aec4b0 con 0x55f548a67b60 2024-09-12T03:21:02.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.808+0000 7f337ad9d640 1 --2- 172.21.15.6:0/3135791982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a72d40 0x55f548a770c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:02.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.809+0000 7f337a59c640 1 --2- 172.21.15.6:0/3135791982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a7b5b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f336c011d20 tx=0x7f336c026af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae6b51b955e6dd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:02.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.809+0000 7f3361ffb640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:21:02.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.809+0000 7f3361ffb640 1 -- 172.21.15.6:0/3135791982 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c040040 con 0x55f548a67b60 2024-09-12T03:21:02.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.809+0000 7f3361ffb640 1 -- 172.21.15.6:0/3135791982 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f336c045470 con 0x55f548a67b60 2024-09-12T03:21:02.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.809+0000 7f3382fa6740 1 -- 172.21.15.6:0/3135791982 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f336c026b70 con 0x55f548a67b60 2024-09-12T03:21:02.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.809+0000 7f3361ffb640 1 -- 172.21.15.6:0/3135791982 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f336c054070 con 0x55f548a67b60 2024-09-12T03:21:02.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.809+0000 7f3382fa6740 10 client.15213 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:02.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.809+0000 7f3382fa6740 1 -- 172.21.15.6:0/3135791982 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f548aed680 con 0x55f548a67b60 2024-09-12T03:21:02.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.809+0000 7f3361ffb640 1 -- 172.21.15.6:0/3135791982 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f336c026b70 con 0x55f548a67b60 2024-09-12T03:21:02.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.809+0000 7f3361ffb640 1 -- 172.21.15.6:0/3135791982 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f336c045b50 con 0x55f548a67b60 2024-09-12T03:21:02.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.809+0000 7f3382fa6740 10 client.15213 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:02.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.809+0000 7f3382fa6740 10 client.15213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:21:02.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.809+0000 7f3382fa6740 20 client.15213 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:02.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.809+0000 7f3382fa6740 1 --2- 172.21.15.6:0/3135791982 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55f548a82980 0x55f548aefd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.809+0000 7f3382fa6740 4 client.15213 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:21:02.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.809+0000 7f3382fa6740 1 -- 172.21.15.6:0/3135791982 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f5489d4d00 con 0x55f548a82980 2024-09-12T03:21:02.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.810+0000 7f3373fff640 1 --2- 172.21.15.6:0/3135791982 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55f548a82980 0x55f548aefd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.810+0000 7f3361ffb640 10 client.15213 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:02.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.810+0000 7f3373fff640 1 --2- 172.21.15.6:0/3135791982 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55f548a82980 0x55f548aefd80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9130d3c3f4a9405 server_cookie=962ff0a1720098c8 in_seq=0 out_seq=0 2024-09-12T03:21:02.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.811+0000 7f3361ffb640 1 -- 172.21.15.6:0/3135791982 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f5489d4d00 con 0x55f548a82980 2024-09-12T03:21:02.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.811+0000 7f3361ffb640 10 client.15213 handle_command_reply: tid=0 2024-09-12T03:21:02.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.811+0000 7f3382fa6740 1 client.15213 shutdown 2024-09-12T03:21:02.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.811+0000 7f3382fa6740 1 -- 172.21.15.6:0/3135791982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 msgr2=0x55f548a7b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.811+0000 7f3382fa6740 1 --2- 172.21.15.6:0/3135791982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a7b5b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f336c011d20 tx=0x7f336c026af0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.811+0000 7f3382fa6740 1 -- 172.21.15.6:0/3135791982 shutdown_connections 2024-09-12T03:21:02.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.811+0000 7f3382fa6740 1 --2- 172.21.15.6:0/3135791982 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55f548a82980 0x55f548aefd80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.812+0000 7f3382fa6740 1 --2- 172.21.15.6:0/3135791982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f548a72d40 0x55f548a770c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.812+0000 7f3382fa6740 1 --2- 172.21.15.6:0/3135791982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f548a68480 0x55f548a76930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.812+0000 7f3382fa6740 1 --2- 172.21.15.6:0/3135791982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f548a67b60 0x55f548a7b5b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.812+0000 7f3361ffb640 0 client.15213 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:02.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.812+0000 7f3382fa6740 1 -- 172.21.15.6:0/3135791982 >> 172.21.15.6:0/3135791982 conn(0x55f548a640f0 msgr2=0x55f548a63270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:02.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.812+0000 7f3382fa6740 1 -- 172.21.15.6:0/3135791982 shutdown_connections 2024-09-12T03:21:02.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.812+0000 7f3382fa6740 1 -- 172.21.15.6:0/3135791982 wait complete. 2024-09-12T03:21:02.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.812+0000 7f3382fa6740 20 client.15213 trim_cache size 0 max 16384 2024-09-12T03:21:02.819 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:21:02.819 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2024-09-12T03:21:02.819 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "b0b04be6-d1b5-4239-8706-c51f499d089b", 2024-09-12T03:21:02.819 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2024-09-12T03:21:02.819 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:21:02.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.812+0000 7f3380bcd640 1 -- 172.21.15.6:0/1581049232 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f33680a2c90 msgr2=0x7f33680a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.812+0000 7f3380bcd640 1 --2- 172.21.15.6:0/1581049232 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f33680a2c90 0x7f33680a5150 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f336c005fd0 tx=0x7f336c00b390 comp rx=0 tx=0).stop 2024-09-12T03:21:02.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.812+0000 7f3380bcd640 1 -- 172.21.15.6:0/1581049232 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f337c102ef0 msgr2=0x7f337c19cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.812+0000 7f3380bcd640 1 --2- 172.21.15.6:0/1581049232 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f337c102ef0 0x7f337c19cdf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f337400cad0 tx=0x7f337400cf90 comp rx=0 tx=0).stop 2024-09-12T03:21:02.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.812+0000 7f3380bcd640 1 -- 172.21.15.6:0/1581049232 shutdown_connections 2024-09-12T03:21:02.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.813+0000 7f3380bcd640 1 --2- 172.21.15.6:0/1581049232 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f33680a2c90 0x7f33680a5150 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.813+0000 7f3380bcd640 1 --2- 172.21.15.6:0/1581049232 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f337c102ef0 0x7f337c19cdf0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.813+0000 7f3380bcd640 1 --2- 172.21.15.6:0/1581049232 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f337c102570 0x7f337c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.813+0000 7f3380bcd640 1 --2- 172.21.15.6:0/1581049232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f337c100e20 0x7f337c198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.813+0000 7f3380bcd640 1 -- 172.21.15.6:0/1581049232 >> 172.21.15.6:0/1581049232 conn(0x7f337c0fc7d0 msgr2=0x7f337c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:02.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.813+0000 7f3380bcd640 1 -- 172.21.15.6:0/1581049232 shutdown_connections 2024-09-12T03:21:02.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:02.813+0000 7f3380bcd640 1 -- 172.21.15.6:0/1581049232 wait complete. 2024-09-12T03:21:03.264 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:03 smithi118 ceph-mon[35048]: pgmap v333: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-12T03:21:03.311 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:03 smithi006 ceph-mon[30991]: pgmap v333: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-12T03:21:03.339 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-12T03:21:01.265062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664447,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":754,"num_read_kb":500,"num_write":365,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382608,"kb_used_data":12560,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559216,"statfs":{"total":1151940427776,"available":1151548637184,"internally_reserved":0,"allocated":12861440,"data_stored":7184073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10078,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":45,"num_write":18,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002841"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265565+0000","last_change":"2024-09-12T03:19:09.343829+0000","last_active":"2024-09-12T03:20:49.265565+0000","last_peered":"2024-09-12T03:20:49.265565+0000","last_clean":"2024-09-12T03:20:49.265565+0000","last_became_active":"2024-09-12T03:19:09.343228+0000","last_became_peered":"2024-09-12T03:19:09.343228+0000","last_unstale":"2024-09-12T03:20:49.265565+0000","last_undegraded":"2024-09-12T03:20:49.265565+0000","last_fullsized":"2024-09-12T03:20:49.265565+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:08.720934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.237141+0000","last_change":"2024-09-12T03:19:09.331829+0000","last_active":"2024-09-12T03:20:49.237141+0000","last_peered":"2024-09-12T03:20:49.237141+0000","last_clean":"2024-09-12T03:20:49.237141+0000","last_became_active":"2024-09-12T03:19:09.331276+0000","last_became_peered":"2024-09-12T03:19:09.331276+0000","last_unstale":"2024-09-12T03:20:49.237141+0000","last_undegraded":"2024-09-12T03:20:49.237141+0000","last_fullsized":"2024-09-12T03:20:49.237141+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:48.672085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.263951+0000","last_change":"2024-09-12T03:19:09.338120+0000","last_active":"2024-09-12T03:20:49.263951+0000","last_peered":"2024-09-12T03:20:49.263951+0000","last_clean":"2024-09-12T03:20:49.263951+0000","last_became_active":"2024-09-12T03:19:09.337773+0000","last_became_peered":"2024-09-12T03:19:09.337773+0000","last_unstale":"2024-09-12T03:20:49.263951+0000","last_undegraded":"2024-09-12T03:20:49.263951+0000","last_fullsized":"2024-09-12T03:20:49.263951+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:13.716047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254592+0000","last_change":"2024-09-12T03:19:09.339404+0000","last_active":"2024-09-12T03:20:49.254592+0000","last_peered":"2024-09-12T03:20:49.254592+0000","last_clean":"2024-09-12T03:20:49.254592+0000","last_became_active":"2024-09-12T03:19:09.339105+0000","last_became_peered":"2024-09-12T03:19:09.339105+0000","last_unstale":"2024-09-12T03:20:49.254592+0000","last_undegraded":"2024-09-12T03:20:49.254592+0000","last_fullsized":"2024-09-12T03:20:49.254592+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:37.930963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.242261+0000","last_change":"2024-09-12T03:19:09.326285+0000","last_active":"2024-09-12T03:20:49.242261+0000","last_peered":"2024-09-12T03:20:49.242261+0000","last_clean":"2024-09-12T03:20:49.242261+0000","last_became_active":"2024-09-12T03:19:09.325852+0000","last_became_peered":"2024-09-12T03:19:09.325852+0000","last_unstale":"2024-09-12T03:20:49.242261+0000","last_undegraded":"2024-09-12T03:20:49.242261+0000","last_fullsized":"2024-09-12T03:20:49.242261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:44:25.387014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.249918+0000","last_change":"2024-09-12T03:19:20.376197+0000","last_active":"2024-09-12T03:20:49.249918+0000","last_peered":"2024-09-12T03:20:49.249918+0000","last_clean":"2024-09-12T03:20:49.249918+0000","last_became_active":"2024-09-12T03:19:20.375744+0000","last_became_peered":"2024-09-12T03:19:20.375744+0000","last_unstale":"2024-09-12T03:20:49.249918+0000","last_undegraded":"2024-09-12T03:20:49.249918+0000","last_fullsized":"2024-09-12T03:20:49.249918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:08.663121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.245988+0000","last_change":"2024-09-12T03:19:09.335284+0000","last_active":"2024-09-12T03:20:49.245988+0000","last_peered":"2024-09-12T03:20:49.245988+0000","last_clean":"2024-09-12T03:20:49.245988+0000","last_became_active":"2024-09-12T03:19:09.334948+0000","last_became_peered":"2024-09-12T03:19:09.334948+0000","last_unstale":"2024-09-12T03:20:49.245988+0000","last_undegraded":"2024-09-12T03:20:49.245988+0000","last_fullsized":"2024-09-12T03:20:49.245988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:32.778433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250337+0000","last_change":"2024-09-12T03:19:09.338398+0000","last_active":"2024-09-12T03:20:49.250337+0000","last_peered":"2024-09-12T03:20:49.250337+0000","last_clean":"2024-09-12T03:20:49.250337+0000","last_became_active":"2024-09-12T03:19:09.337750+0000","last_became_peered":"2024-09-12T03:19:09.337750+0000","last_unstale":"2024-09-12T03:20:49.250337+0000","last_undegraded":"2024-09-12T03:20:49.250337+0000","last_fullsized":"2024-09-12T03:20:49.250337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:07.542322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265597+0000","last_change":"2024-09-12T03:19:09.340703+0000","last_active":"2024-09-12T03:20:49.265597+0000","last_peered":"2024-09-12T03:20:49.265597+0000","last_clean":"2024-09-12T03:20:49.265597+0000","last_became_active":"2024-09-12T03:19:09.340386+0000","last_became_peered":"2024-09-12T03:19:09.340386+0000","last_unstale":"2024-09-12T03:20:49.265597+0000","last_undegraded":"2024-09-12T03:20:49.265597+0000","last_fullsized":"2024-09-12T03:20:49.265597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:19.990894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248198+0000","last_change":"2024-09-12T03:19:09.341489+0000","last_active":"2024-09-12T03:20:49.248198+0000","last_peered":"2024-09-12T03:20:49.248198+0000","last_clean":"2024-09-12T03:20:49.248198+0000","last_became_active":"2024-09-12T03:19:09.341183+0000","last_became_peered":"2024-09-12T03:19:09.341183+0000","last_unstale":"2024-09-12T03:20:49.248198+0000","last_undegraded":"2024-09-12T03:20:49.248198+0000","last_fullsized":"2024-09-12T03:20:49.248198+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:05.688402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.255447+0000","last_change":"2024-09-12T03:19:09.339289+0000","last_active":"2024-09-12T03:20:49.255447+0000","last_peered":"2024-09-12T03:20:49.255447+0000","last_clean":"2024-09-12T03:20:49.255447+0000","last_became_active":"2024-09-12T03:19:09.338466+0000","last_became_peered":"2024-09-12T03:19:09.338466+0000","last_unstale":"2024-09-12T03:20:49.255447+0000","last_undegraded":"2024-09-12T03:20:49.255447+0000","last_fullsized":"2024-09-12T03:20:49.255447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:47:13.070057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254364+0000","last_change":"2024-09-12T03:19:09.337754+0000","last_active":"2024-09-12T03:20:49.254364+0000","last_peered":"2024-09-12T03:20:49.254364+0000","last_clean":"2024-09-12T03:20:49.254364+0000","last_became_active":"2024-09-12T03:19:09.332789+0000","last_became_peered":"2024-09-12T03:19:09.332789+0000","last_unstale":"2024-09-12T03:20:49.254364+0000","last_undegraded":"2024-09-12T03:20:49.254364+0000","last_fullsized":"2024-09-12T03:20:49.254364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:34.622136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250957+0000","last_change":"2024-09-12T03:19:09.333577+0000","last_active":"2024-09-12T03:20:49.250957+0000","last_peered":"2024-09-12T03:20:49.250957+0000","last_clean":"2024-09-12T03:20:49.250957+0000","last_became_active":"2024-09-12T03:19:09.333234+0000","last_became_peered":"2024-09-12T03:19:09.333234+0000","last_unstale":"2024-09-12T03:20:49.250957+0000","last_undegraded":"2024-09-12T03:20:49.250957+0000","last_fullsized":"2024-09-12T03:20:49.250957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:22.604640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.237367+0000","last_change":"2024-09-12T03:19:09.331607+0000","last_active":"2024-09-12T03:20:49.237367+0000","last_peered":"2024-09-12T03:20:49.237367+0000","last_clean":"2024-09-12T03:20:49.237367+0000","last_became_active":"2024-09-12T03:19:09.331253+0000","last_became_peered":"2024-09-12T03:19:09.331253+0000","last_unstale":"2024-09-12T03:20:49.237367+0000","last_undegraded":"2024-09-12T03:20:49.237367+0000","last_fullsized":"2024-09-12T03:20:49.237367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:11.969265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256379+0000","last_change":"2024-09-12T03:19:09.340768+0000","last_active":"2024-09-12T03:20:49.256379+0000","last_peered":"2024-09-12T03:20:49.256379+0000","last_clean":"2024-09-12T03:20:49.256379+0000","last_became_active":"2024-09-12T03:19:09.340437+0000","last_became_peered":"2024-09-12T03:19:09.340437+0000","last_unstale":"2024-09-12T03:20:49.256379+0000","last_undegraded":"2024-09-12T03:20:49.256379+0000","last_fullsized":"2024-09-12T03:20:49.256379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:32.645691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.263546+0000","last_change":"2024-09-12T03:19:09.333829+0000","last_active":"2024-09-12T03:20:49.263546+0000","last_peered":"2024-09-12T03:20:49.263546+0000","last_clean":"2024-09-12T03:20:49.263546+0000","last_became_active":"2024-09-12T03:19:09.333462+0000","last_became_peered":"2024-09-12T03:19:09.333462+0000","last_unstale":"2024-09-12T03:20:49.263546+0000","last_undegraded":"2024-09-12T03:20:49.263546+0000","last_fullsized":"2024-09-12T03:20:49.263546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:18:57.586836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265659+0000","last_change":"2024-09-12T03:19:09.344068+0000","last_active":"2024-09-12T03:20:49.265659+0000","last_peered":"2024-09-12T03:20:49.265659+0000","last_clean":"2024-09-12T03:20:49.265659+0000","last_became_active":"2024-09-12T03:19:09.343751+0000","last_became_peered":"2024-09-12T03:19:09.343751+0000","last_unstale":"2024-09-12T03:20:49.265659+0000","last_undegraded":"2024-09-12T03:20:49.265659+0000","last_fullsized":"2024-09-12T03:20:49.265659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:52.425547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.249413+0000","last_change":"2024-09-12T03:19:09.330828+0000","last_active":"2024-09-12T03:20:49.249413+0000","last_peered":"2024-09-12T03:20:49.249413+0000","last_clean":"2024-09-12T03:20:49.249413+0000","last_became_active":"2024-09-12T03:19:09.330497+0000","last_became_peered":"2024-09-12T03:19:09.330497+0000","last_unstale":"2024-09-12T03:20:49.249413+0000","last_undegraded":"2024-09-12T03:20:49.249413+0000","last_fullsized":"2024-09-12T03:20:49.249413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:21:20.064177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.245548+0000","last_change":"2024-09-12T03:19:09.344403+0000","last_active":"2024-09-12T03:20:49.245548+0000","last_peered":"2024-09-12T03:20:49.245548+0000","last_clean":"2024-09-12T03:20:49.245548+0000","last_became_active":"2024-09-12T03:19:09.343946+0000","last_became_peered":"2024-09-12T03:19:09.343946+0000","last_unstale":"2024-09-12T03:20:49.245548+0000","last_undegraded":"2024-09-12T03:20:49.245548+0000","last_fullsized":"2024-09-12T03:20:49.245548+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:48.386117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248484+0000","last_change":"2024-09-12T03:19:09.341481+0000","last_active":"2024-09-12T03:20:49.248484+0000","last_peered":"2024-09-12T03:20:49.248484+0000","last_clean":"2024-09-12T03:20:49.248484+0000","last_became_active":"2024-09-12T03:19:09.341142+0000","last_became_peered":"2024-09-12T03:19:09.341142+0000","last_unstale":"2024-09-12T03:20:49.248484+0000","last_undegraded":"2024-09-12T03:20:49.248484+0000","last_fullsized":"2024-09-12T03:20:49.248484+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:42.371221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.249677+0000","last_change":"2024-09-12T03:19:09.330621+0000","last_active":"2024-09-12T03:20:49.249677+0000","last_peered":"2024-09-12T03:20:49.249677+0000","last_clean":"2024-09-12T03:20:49.249677+0000","last_became_active":"2024-09-12T03:19:09.330169+0000","last_became_peered":"2024-09-12T03:19:09.330169+0000","last_unstale":"2024-09-12T03:20:49.249677+0000","last_undegraded":"2024-09-12T03:20:49.249677+0000","last_fullsized":"2024-09-12T03:20:49.249677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:16.634141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.264220+0000","last_change":"2024-09-12T03:19:09.332817+0000","last_active":"2024-09-12T03:20:49.264220+0000","last_peered":"2024-09-12T03:20:49.264220+0000","last_clean":"2024-09-12T03:20:49.264220+0000","last_became_active":"2024-09-12T03:19:09.332236+0000","last_became_peered":"2024-09-12T03:19:09.332236+0000","last_unstale":"2024-09-12T03:20:49.264220+0000","last_undegraded":"2024-09-12T03:20:49.264220+0000","last_fullsized":"2024-09-12T03:20:49.264220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:28.024294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252477+0000","last_change":"2024-09-12T03:19:09.337910+0000","last_active":"2024-09-12T03:20:49.252477+0000","last_peered":"2024-09-12T03:20:49.252477+0000","last_clean":"2024-09-12T03:20:49.252477+0000","last_became_active":"2024-09-12T03:19:09.337176+0000","last_became_peered":"2024-09-12T03:19:09.337176+0000","last_unstale":"2024-09-12T03:20:49.252477+0000","last_undegraded":"2024-09-12T03:20:49.252477+0000","last_fullsized":"2024-09-12T03:20:49.252477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:53.795898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.239957+0000","last_change":"2024-09-12T03:19:09.337902+0000","last_active":"2024-09-12T03:20:49.239957+0000","last_peered":"2024-09-12T03:20:49.239957+0000","last_clean":"2024-09-12T03:20:49.239957+0000","last_became_active":"2024-09-12T03:19:09.337567+0000","last_became_peered":"2024-09-12T03:19:09.337567+0000","last_unstale":"2024-09-12T03:20:49.239957+0000","last_undegraded":"2024-09-12T03:20:49.239957+0000","last_fullsized":"2024-09-12T03:20:49.239957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:08.471313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248557+0000","last_change":"2024-09-12T03:19:09.339850+0000","last_active":"2024-09-12T03:20:49.248557+0000","last_peered":"2024-09-12T03:20:49.248557+0000","last_clean":"2024-09-12T03:20:49.248557+0000","last_became_active":"2024-09-12T03:19:09.339563+0000","last_became_peered":"2024-09-12T03:19:09.339563+0000","last_unstale":"2024-09-12T03:20:49.248557+0000","last_undegraded":"2024-09-12T03:20:49.248557+0000","last_fullsized":"2024-09-12T03:20:49.248557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:27.473351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.247769+0000","last_change":"2024-09-12T03:19:09.330555+0000","last_active":"2024-09-12T03:20:49.247769+0000","last_peered":"2024-09-12T03:20:49.247769+0000","last_clean":"2024-09-12T03:20:49.247769+0000","last_became_active":"2024-09-12T03:19:09.330183+0000","last_became_peered":"2024-09-12T03:19:09.330183+0000","last_unstale":"2024-09-12T03:20:49.247769+0000","last_undegraded":"2024-09-12T03:20:49.247769+0000","last_fullsized":"2024-09-12T03:20:49.247769+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:39.334116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252190+0000","last_change":"2024-09-12T03:19:09.338601+0000","last_active":"2024-09-12T03:20:49.252190+0000","last_peered":"2024-09-12T03:20:49.252190+0000","last_clean":"2024-09-12T03:20:49.252190+0000","last_became_active":"2024-09-12T03:19:09.337903+0000","last_became_peered":"2024-09-12T03:19:09.337903+0000","last_unstale":"2024-09-12T03:20:49.252190+0000","last_undegraded":"2024-09-12T03:20:49.252190+0000","last_fullsized":"2024-09-12T03:20:49.252190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:17.810139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250990+0000","last_change":"2024-09-12T03:19:09.326629+0000","last_active":"2024-09-12T03:20:49.250990+0000","last_peered":"2024-09-12T03:20:49.250990+0000","last_clean":"2024-09-12T03:20:49.250990+0000","last_became_active":"2024-09-12T03:19:09.326333+0000","last_became_peered":"2024-09-12T03:19:09.326333+0000","last_unstale":"2024-09-12T03:20:49.250990+0000","last_undegraded":"2024-09-12T03:20:49.250990+0000","last_fullsized":"2024-09-12T03:20:49.250990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:20.807593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256412+0000","last_change":"2024-09-12T03:19:09.339919+0000","last_active":"2024-09-12T03:20:49.256412+0000","last_peered":"2024-09-12T03:20:49.256412+0000","last_clean":"2024-09-12T03:20:49.256412+0000","last_became_active":"2024-09-12T03:19:09.339618+0000","last_became_peered":"2024-09-12T03:19:09.339618+0000","last_unstale":"2024-09-12T03:20:49.256412+0000","last_undegraded":"2024-09-12T03:20:49.256412+0000","last_fullsized":"2024-09-12T03:20:49.256412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:11:03.740079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.246446+0000","last_change":"2024-09-12T03:19:09.336148+0000","last_active":"2024-09-12T03:20:49.246446+0000","last_peered":"2024-09-12T03:20:49.246446+0000","last_clean":"2024-09-12T03:20:49.246446+0000","last_became_active":"2024-09-12T03:19:09.335779+0000","last_became_peered":"2024-09-12T03:19:09.335779+0000","last_unstale":"2024-09-12T03:20:49.246446+0000","last_undegraded":"2024-09-12T03:20:49.246446+0000","last_fullsized":"2024-09-12T03:20:49.246446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:18.234320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.247971+0000","last_change":"2024-09-12T03:19:20.382536+0000","last_active":"2024-09-12T03:20:49.247971+0000","last_peered":"2024-09-12T03:20:49.247971+0000","last_clean":"2024-09-12T03:20:49.247971+0000","last_became_active":"2024-09-12T03:19:20.382118+0000","last_became_peered":"2024-09-12T03:19:20.382118+0000","last_unstale":"2024-09-12T03:20:49.247971+0000","last_undegraded":"2024-09-12T03:20:49.247971+0000","last_fullsized":"2024-09-12T03:20:49.247971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:55.836646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252455+0000","last_change":"2024-09-12T03:19:09.345064+0000","last_active":"2024-09-12T03:20:49.252455+0000","last_peered":"2024-09-12T03:20:49.252455+0000","last_clean":"2024-09-12T03:20:49.252455+0000","last_became_active":"2024-09-12T03:19:09.344628+0000","last_became_peered":"2024-09-12T03:19:09.344628+0000","last_unstale":"2024-09-12T03:20:49.252455+0000","last_undegraded":"2024-09-12T03:20:49.252455+0000","last_fullsized":"2024-09-12T03:20:49.252455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:50.011751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.244081+0000","last_change":"2024-09-12T03:19:09.338052+0000","last_active":"2024-09-12T03:20:49.244081+0000","last_peered":"2024-09-12T03:20:49.244081+0000","last_clean":"2024-09-12T03:20:49.244081+0000","last_became_active":"2024-09-12T03:19:09.337710+0000","last_became_peered":"2024-09-12T03:19:09.337710+0000","last_unstale":"2024-09-12T03:20:49.244081+0000","last_undegraded":"2024-09-12T03:20:49.244081+0000","last_fullsized":"2024-09-12T03:20:49.244081+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:36.521269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'9","reported_seq":139,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:59.526609+0000","last_change":"2024-09-12T03:19:07.208341+0000","last_active":"2024-09-12T03:20:59.526609+0000","last_peered":"2024-09-12T03:20:59.526609+0000","last_clean":"2024-09-12T03:20:59.526609+0000","last_became_active":"2024-09-12T03:19:07.207385+0000","last_became_peered":"2024-09-12T03:19:07.207385+0000","last_unstale":"2024-09-12T03:20:59.526609+0000","last_undegraded":"2024-09-12T03:20:59.526609+0000","last_fullsized":"2024-09-12T03:20:59.526609+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:18.476795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1857,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254115+0000","last_change":"2024-09-12T03:19:09.339997+0000","last_active":"2024-09-12T03:20:49.254115+0000","last_peered":"2024-09-12T03:20:49.254115+0000","last_clean":"2024-09-12T03:20:49.254115+0000","last_became_active":"2024-09-12T03:19:09.339261+0000","last_became_peered":"2024-09-12T03:19:09.339261+0000","last_unstale":"2024-09-12T03:20:49.254115+0000","last_undegraded":"2024-09-12T03:20:49.254115+0000","last_fullsized":"2024-09-12T03:20:49.254115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:07.670345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374030+0000","last_change":"2024-09-12T03:19:07.200222+0000","last_active":"2024-09-12T03:19:20.374030+0000","last_peered":"2024-09-12T03:19:20.374030+0000","last_clean":"2024-09-12T03:19:20.374030+0000","last_became_active":"2024-09-12T03:19:07.199819+0000","last_became_peered":"2024-09-12T03:19:07.199819+0000","last_unstale":"2024-09-12T03:19:20.374030+0000","last_undegraded":"2024-09-12T03:19:20.374030+0000","last_fullsized":"2024-09-12T03:19:20.374030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:39:28.933609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375686+0000","last_change":"2024-09-12T03:19:07.207005+0000","last_active":"2024-09-12T03:19:20.375686+0000","last_peered":"2024-09-12T03:19:20.375686+0000","last_clean":"2024-09-12T03:19:20.375686+0000","last_became_active":"2024-09-12T03:19:07.206685+0000","last_became_peered":"2024-09-12T03:19:07.206685+0000","last_unstale":"2024-09-12T03:19:20.375686+0000","last_undegraded":"2024-09-12T03:19:20.375686+0000","last_fullsized":"2024-09-12T03:19:20.375686+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:31.111706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252939+0000","last_change":"2024-09-12T03:19:09.335871+0000","last_active":"2024-09-12T03:20:49.252939+0000","last_peered":"2024-09-12T03:20:49.252939+0000","last_clean":"2024-09-12T03:20:49.252939+0000","last_became_active":"2024-09-12T03:19:09.335568+0000","last_became_peered":"2024-09-12T03:19:09.335568+0000","last_unstale":"2024-09-12T03:20:49.252939+0000","last_undegraded":"2024-09-12T03:20:49.252939+0000","last_fullsized":"2024-09-12T03:20:49.252939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:52:34.471512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386049+0000","last_change":"2024-09-12T03:19:07.208320+0000","last_active":"2024-09-12T03:19:20.386049+0000","last_peered":"2024-09-12T03:19:20.386049+0000","last_clean":"2024-09-12T03:19:20.386049+0000","last_became_active":"2024-09-12T03:19:07.207873+0000","last_became_peered":"2024-09-12T03:19:07.207873+0000","last_unstale":"2024-09-12T03:19:20.386049+0000","last_undegraded":"2024-09-12T03:19:20.386049+0000","last_fullsized":"2024-09-12T03:19:20.386049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:00.128595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250364+0000","last_change":"2024-09-12T03:19:09.331930+0000","last_active":"2024-09-12T03:20:49.250364+0000","last_peered":"2024-09-12T03:20:49.250364+0000","last_clean":"2024-09-12T03:20:49.250364+0000","last_became_active":"2024-09-12T03:19:09.331558+0000","last_became_peered":"2024-09-12T03:19:09.331558+0000","last_unstale":"2024-09-12T03:20:49.250364+0000","last_undegraded":"2024-09-12T03:20:49.250364+0000","last_fullsized":"2024-09-12T03:20:49.250364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:00:27.935717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":125,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:56.520992+0000","last_change":"2024-09-12T03:19:07.205473+0000","last_active":"2024-09-12T03:20:56.520992+0000","last_peered":"2024-09-12T03:20:56.520992+0000","last_clean":"2024-09-12T03:20:56.520992+0000","last_became_active":"2024-09-12T03:19:07.205134+0000","last_became_peered":"2024-09-12T03:19:07.205134+0000","last_unstale":"2024-09-12T03:20:56.520992+0000","last_undegraded":"2024-09-12T03:20:56.520992+0000","last_fullsized":"2024-09-12T03:20:56.520992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:23.443957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265317+0000","last_change":"2024-09-12T03:19:09.333868+0000","last_active":"2024-09-12T03:20:49.265317+0000","last_peered":"2024-09-12T03:20:49.265317+0000","last_clean":"2024-09-12T03:20:49.265317+0000","last_became_active":"2024-09-12T03:19:09.333484+0000","last_became_peered":"2024-09-12T03:19:09.333484+0000","last_unstale":"2024-09-12T03:20:49.265317+0000","last_undegraded":"2024-09-12T03:20:49.265317+0000","last_fullsized":"2024-09-12T03:20:49.265317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:21:10.226821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386303+0000","last_change":"2024-09-12T03:19:20.386303+0000","last_active":"2024-09-12T03:19:20.386303+0000","last_peered":"2024-09-12T03:19:20.386303+0000","last_clean":"2024-09-12T03:19:20.386303+0000","last_became_active":"2024-09-12T03:19:20.383198+0000","last_became_peered":"2024-09-12T03:19:20.383198+0000","last_unstale":"2024-09-12T03:19:20.386303+0000","last_undegraded":"2024-09-12T03:19:20.386303+0000","last_fullsized":"2024-09-12T03:19:20.386303+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:42.224849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.257056+0000","last_change":"2024-09-12T03:19:09.340849+0000","last_active":"2024-09-12T03:20:49.257056+0000","last_peered":"2024-09-12T03:20:49.257056+0000","last_clean":"2024-09-12T03:20:49.257056+0000","last_became_active":"2024-09-12T03:19:09.340517+0000","last_became_peered":"2024-09-12T03:19:09.340517+0000","last_unstale":"2024-09-12T03:20:49.257056+0000","last_undegraded":"2024-09-12T03:20:49.257056+0000","last_fullsized":"2024-09-12T03:20:49.257056+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:42.018269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386240+0000","last_change":"2024-09-12T03:19:07.389147+0000","last_active":"2024-09-12T03:19:20.386240+0000","last_peered":"2024-09-12T03:19:20.386240+0000","last_clean":"2024-09-12T03:19:20.386240+0000","last_became_active":"2024-09-12T03:19:07.388744+0000","last_became_peered":"2024-09-12T03:19:07.388744+0000","last_unstale":"2024-09-12T03:19:20.386240+0000","last_undegraded":"2024-09-12T03:19:20.386240+0000","last_fullsized":"2024-09-12T03:19:20.386240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:16.700644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.264622+0000","last_change":"2024-09-12T03:19:09.329243+0000","last_active":"2024-09-12T03:20:49.264622+0000","last_peered":"2024-09-12T03:20:49.264622+0000","last_clean":"2024-09-12T03:20:49.264622+0000","last_became_active":"2024-09-12T03:19:09.328610+0000","last_became_peered":"2024-09-12T03:19:09.328610+0000","last_unstale":"2024-09-12T03:20:49.264622+0000","last_undegraded":"2024-09-12T03:20:49.264622+0000","last_fullsized":"2024-09-12T03:20:49.264622+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:49.452942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'34","reported_seq":265,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:57.523509+0000","last_change":"2024-09-12T03:19:07.207804+0000","last_active":"2024-09-12T03:20:57.523509+0000","last_peered":"2024-09-12T03:20:57.523509+0000","last_clean":"2024-09-12T03:20:57.523509+0000","last_became_active":"2024-09-12T03:19:07.207409+0000","last_became_peered":"2024-09-12T03:19:07.207409+0000","last_unstale":"2024-09-12T03:20:57.523509+0000","last_undegraded":"2024-09-12T03:20:57.523509+0000","last_fullsized":"2024-09-12T03:20:57.523509+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:10.732423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68607,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":209,"num_read_kb":171,"num_write":35,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.255978+0000","last_change":"2024-09-12T03:19:09.333102+0000","last_active":"2024-09-12T03:20:49.255978+0000","last_peered":"2024-09-12T03:20:49.255978+0000","last_clean":"2024-09-12T03:20:49.255978+0000","last_became_active":"2024-09-12T03:19:09.332712+0000","last_became_peered":"2024-09-12T03:19:09.332712+0000","last_unstale":"2024-09-12T03:20:49.255978+0000","last_undegraded":"2024-09-12T03:20:49.255978+0000","last_fullsized":"2024-09-12T03:20:49.255978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:22.912213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382902+0000","last_change":"2024-09-12T03:19:20.382348+0000","last_active":"2024-09-12T03:19:20.382902+0000","last_peered":"2024-09-12T03:19:20.382902+0000","last_clean":"2024-09-12T03:19:20.382902+0000","last_became_active":"2024-09-12T03:19:20.375142+0000","last_became_peered":"2024-09-12T03:19:20.375142+0000","last_unstale":"2024-09-12T03:19:20.382902+0000","last_undegraded":"2024-09-12T03:19:20.382902+0000","last_fullsized":"2024-09-12T03:19:20.382902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:04.573599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.246879+0000","last_change":"2024-09-12T03:19:09.333194+0000","last_active":"2024-09-12T03:20:49.246879+0000","last_peered":"2024-09-12T03:20:49.246879+0000","last_clean":"2024-09-12T03:20:49.246879+0000","last_became_active":"2024-09-12T03:19:09.332865+0000","last_became_peered":"2024-09-12T03:19:09.332865+0000","last_unstale":"2024-09-12T03:20:49.246879+0000","last_undegraded":"2024-09-12T03:20:49.246879+0000","last_fullsized":"2024-09-12T03:20:49.246879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:57.096138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386176+0000","last_change":"2024-09-12T03:19:07.209289+0000","last_active":"2024-09-12T03:19:20.386176+0000","last_peered":"2024-09-12T03:19:20.386176+0000","last_clean":"2024-09-12T03:19:20.386176+0000","last_became_active":"2024-09-12T03:19:07.208863+0000","last_became_peered":"2024-09-12T03:19:07.208863+0000","last_unstale":"2024-09-12T03:19:20.386176+0000","last_undegraded":"2024-09-12T03:19:20.386176+0000","last_fullsized":"2024-09-12T03:19:20.386176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:26.168148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256973+0000","last_change":"2024-09-12T03:19:09.343620+0000","last_active":"2024-09-12T03:20:49.256973+0000","last_peered":"2024-09-12T03:20:49.256973+0000","last_clean":"2024-09-12T03:20:49.256973+0000","last_became_active":"2024-09-12T03:19:09.343299+0000","last_became_peered":"2024-09-12T03:19:09.343299+0000","last_unstale":"2024-09-12T03:20:49.256973+0000","last_undegraded":"2024-09-12T03:20:49.256973+0000","last_fullsized":"2024-09-12T03:20:49.256973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:34.268886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":126,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:55.551460+0000","last_change":"2024-09-12T03:19:07.206160+0000","last_active":"2024-09-12T03:20:55.551460+0000","last_peered":"2024-09-12T03:20:55.551460+0000","last_clean":"2024-09-12T03:20:55.551460+0000","last_became_active":"2024-09-12T03:19:07.205509+0000","last_became_peered":"2024-09-12T03:19:07.205509+0000","last_unstale":"2024-09-12T03:20:55.551460+0000","last_undegraded":"2024-09-12T03:20:55.551460+0000","last_fullsized":"2024-09-12T03:20:55.551460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:35.362831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250896+0000","last_change":"2024-09-12T03:19:09.337595+0000","last_active":"2024-09-12T03:20:49.250896+0000","last_peered":"2024-09-12T03:20:49.250896+0000","last_clean":"2024-09-12T03:20:49.250896+0000","last_became_active":"2024-09-12T03:19:09.337281+0000","last_became_peered":"2024-09-12T03:19:09.337281+0000","last_unstale":"2024-09-12T03:20:49.250896+0000","last_undegraded":"2024-09-12T03:20:49.250896+0000","last_fullsized":"2024-09-12T03:20:49.250896+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:35.761679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373995+0000","last_change":"2024-09-12T03:19:07.389336+0000","last_active":"2024-09-12T03:19:20.373995+0000","last_peered":"2024-09-12T03:19:20.373995+0000","last_clean":"2024-09-12T03:19:20.373995+0000","last_became_active":"2024-09-12T03:19:07.388649+0000","last_became_peered":"2024-09-12T03:19:07.388649+0000","last_unstale":"2024-09-12T03:19:20.373995+0000","last_undegraded":"2024-09-12T03:19:20.373995+0000","last_fullsized":"2024-09-12T03:19:20.373995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:33.312700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.253821+0000","last_change":"2024-09-12T03:19:09.340674+0000","last_active":"2024-09-12T03:20:49.253821+0000","last_peered":"2024-09-12T03:20:49.253821+0000","last_clean":"2024-09-12T03:20:49.253821+0000","last_became_active":"2024-09-12T03:19:09.340144+0000","last_became_peered":"2024-09-12T03:19:09.340144+0000","last_unstale":"2024-09-12T03:20:49.253821+0000","last_undegraded":"2024-09-12T03:20:49.253821+0000","last_fullsized":"2024-09-12T03:20:49.253821+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:21:15.605086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373837+0000","last_change":"2024-09-12T03:19:07.206513+0000","last_active":"2024-09-12T03:19:20.373837+0000","last_peered":"2024-09-12T03:19:20.373837+0000","last_clean":"2024-09-12T03:19:20.373837+0000","last_became_active":"2024-09-12T03:19:07.205974+0000","last_became_peered":"2024-09-12T03:19:07.205974+0000","last_unstale":"2024-09-12T03:19:20.373837+0000","last_undegraded":"2024-09-12T03:19:20.373837+0000","last_fullsized":"2024-09-12T03:19:20.373837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:43:10.002123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252033+0000","last_change":"2024-09-12T03:19:09.342200+0000","last_active":"2024-09-12T03:20:49.252033+0000","last_peered":"2024-09-12T03:20:49.252033+0000","last_clean":"2024-09-12T03:20:49.252033+0000","last_became_active":"2024-09-12T03:19:09.341901+0000","last_became_peered":"2024-09-12T03:19:09.341901+0000","last_unstale":"2024-09-12T03:20:49.252033+0000","last_undegraded":"2024-09-12T03:20:49.252033+0000","last_fullsized":"2024-09-12T03:20:49.252033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:27.486214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375373+0000","last_change":"2024-09-12T03:19:07.207514+0000","last_active":"2024-09-12T03:19:20.375373+0000","last_peered":"2024-09-12T03:19:20.375373+0000","last_clean":"2024-09-12T03:19:20.375373+0000","last_became_active":"2024-09-12T03:19:07.205925+0000","last_became_peered":"2024-09-12T03:19:07.205925+0000","last_unstale":"2024-09-12T03:19:20.375373+0000","last_undegraded":"2024-09-12T03:19:20.375373+0000","last_fullsized":"2024-09-12T03:19:20.375373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:20.244703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256005+0000","last_change":"2024-09-12T03:19:09.337837+0000","last_active":"2024-09-12T03:20:49.256005+0000","last_peered":"2024-09-12T03:20:49.256005+0000","last_clean":"2024-09-12T03:20:49.256005+0000","last_became_active":"2024-09-12T03:19:09.337502+0000","last_became_peered":"2024-09-12T03:19:09.337502+0000","last_unstale":"2024-09-12T03:20:49.256005+0000","last_undegraded":"2024-09-12T03:20:49.256005+0000","last_fullsized":"2024-09-12T03:20:49.256005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:30.167139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386356+0000","last_change":"2024-09-12T03:19:07.209130+0000","last_active":"2024-09-12T03:19:20.386356+0000","last_peered":"2024-09-12T03:19:20.386356+0000","last_clean":"2024-09-12T03:19:20.386356+0000","last_became_active":"2024-09-12T03:19:07.208753+0000","last_became_peered":"2024-09-12T03:19:07.208753+0000","last_unstale":"2024-09-12T03:19:20.386356+0000","last_undegraded":"2024-09-12T03:19:20.386356+0000","last_fullsized":"2024-09-12T03:19:20.386356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:34.719292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250623+0000","last_change":"2024-09-12T03:19:09.336382+0000","last_active":"2024-09-12T03:20:49.250623+0000","last_peered":"2024-09-12T03:20:49.250623+0000","last_clean":"2024-09-12T03:20:49.250623+0000","last_became_active":"2024-09-12T03:19:09.336015+0000","last_became_peered":"2024-09-12T03:19:09.336015+0000","last_unstale":"2024-09-12T03:20:49.250623+0000","last_undegraded":"2024-09-12T03:20:49.250623+0000","last_fullsized":"2024-09-12T03:20:49.250623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:19.430849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375044+0000","last_change":"2024-09-12T03:19:07.528567+0000","last_active":"2024-09-12T03:19:20.375044+0000","last_peered":"2024-09-12T03:19:20.375044+0000","last_clean":"2024-09-12T03:19:20.375044+0000","last_became_active":"2024-09-12T03:19:07.528123+0000","last_became_peered":"2024-09-12T03:19:07.528123+0000","last_unstale":"2024-09-12T03:19:20.375044+0000","last_undegraded":"2024-09-12T03:19:20.375044+0000","last_fullsized":"2024-09-12T03:19:20.375044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:34.416714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256025+0000","last_change":"2024-09-12T03:19:09.338836+0000","last_active":"2024-09-12T03:20:49.256025+0000","last_peered":"2024-09-12T03:20:49.256025+0000","last_clean":"2024-09-12T03:20:49.256025+0000","last_became_active":"2024-09-12T03:19:09.337571+0000","last_became_peered":"2024-09-12T03:19:09.337571+0000","last_unstale":"2024-09-12T03:20:49.256025+0000","last_undegraded":"2024-09-12T03:20:49.256025+0000","last_fullsized":"2024-09-12T03:20:49.256025+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:39.039878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":330,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372792+0000","last_change":"2024-09-12T03:18:01.051311+0000","last_active":"2024-09-12T03:19:20.372792+0000","last_peered":"2024-09-12T03:19:20.372792+0000","last_clean":"2024-09-12T03:19:20.372792+0000","last_became_active":"2024-09-12T03:18:01.050849+0000","last_became_peered":"2024-09-12T03:18:01.050849+0000","last_unstale":"2024-09-12T03:19:20.372792+0000","last_undegraded":"2024-09-12T03:19:20.372792+0000","last_fullsized":"2024-09-12T03:19:20.372792+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:14:38.096222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:14:38.096222+0000","last_clean_scrub_stamp":"2024-09-12T03:14:38.096222+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:33.067603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382035+0000","last_change":"2024-09-12T03:19:07.206045+0000","last_active":"2024-09-12T03:19:20.382035+0000","last_peered":"2024-09-12T03:19:20.382035+0000","last_clean":"2024-09-12T03:19:20.382035+0000","last_became_active":"2024-09-12T03:19:07.205594+0000","last_became_peered":"2024-09-12T03:19:07.205594+0000","last_unstale":"2024-09-12T03:19:20.382035+0000","last_undegraded":"2024-09-12T03:19:20.382035+0000","last_fullsized":"2024-09-12T03:19:20.382035+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:12.811906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.244733+0000","last_change":"2024-09-12T03:19:20.387274+0000","last_active":"2024-09-12T03:20:49.244733+0000","last_peered":"2024-09-12T03:20:49.244733+0000","last_clean":"2024-09-12T03:20:49.244733+0000","last_became_active":"2024-09-12T03:19:20.386878+0000","last_became_peered":"2024-09-12T03:19:20.386878+0000","last_unstale":"2024-09-12T03:20:49.244733+0000","last_undegraded":"2024-09-12T03:20:49.244733+0000","last_fullsized":"2024-09-12T03:20:49.244733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:26.674032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372302+0000","last_change":"2024-09-12T03:19:07.206212+0000","last_active":"2024-09-12T03:19:20.372302+0000","last_peered":"2024-09-12T03:19:20.372302+0000","last_clean":"2024-09-12T03:19:20.372302+0000","last_became_active":"2024-09-12T03:19:07.205899+0000","last_became_peered":"2024-09-12T03:19:07.205899+0000","last_unstale":"2024-09-12T03:19:20.372302+0000","last_undegraded":"2024-09-12T03:19:20.372302+0000","last_fullsized":"2024-09-12T03:19:20.372302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:02.169436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250795+0000","last_change":"2024-09-12T03:19:20.377494+0000","last_active":"2024-09-12T03:20:49.250795+0000","last_peered":"2024-09-12T03:20:49.250795+0000","last_clean":"2024-09-12T03:20:49.250795+0000","last_became_active":"2024-09-12T03:19:20.377091+0000","last_became_peered":"2024-09-12T03:19:20.377091+0000","last_unstale":"2024-09-12T03:20:49.250795+0000","last_undegraded":"2024-09-12T03:20:49.250795+0000","last_fullsized":"2024-09-12T03:20:49.250795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:30.926853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":125,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:56.552378+0000","last_change":"2024-09-12T03:19:07.388486+0000","last_active":"2024-09-12T03:20:56.552378+0000","last_peered":"2024-09-12T03:20:56.552378+0000","last_clean":"2024-09-12T03:20:56.552378+0000","last_became_active":"2024-09-12T03:19:07.388125+0000","last_became_peered":"2024-09-12T03:19:07.388125+0000","last_unstale":"2024-09-12T03:20:56.552378+0000","last_undegraded":"2024-09-12T03:20:56.552378+0000","last_fullsized":"2024-09-12T03:20:56.552378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:46.218601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254069+0000","last_change":"2024-09-12T03:19:09.344308+0000","last_active":"2024-09-12T03:20:49.254069+0000","last_peered":"2024-09-12T03:20:49.254069+0000","last_clean":"2024-09-12T03:20:49.254069+0000","last_became_active":"2024-09-12T03:19:09.343987+0000","last_became_peered":"2024-09-12T03:19:09.343987+0000","last_unstale":"2024-09-12T03:20:49.254069+0000","last_undegraded":"2024-09-12T03:20:49.254069+0000","last_fullsized":"2024-09-12T03:20:49.254069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:16:11.616555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372560+0000","last_change":"2024-09-12T03:19:07.208619+0000","last_active":"2024-09-12T03:19:20.372560+0000","last_peered":"2024-09-12T03:19:20.372560+0000","last_clean":"2024-09-12T03:19:20.372560+0000","last_became_active":"2024-09-12T03:19:07.208254+0000","last_became_peered":"2024-09-12T03:19:07.208254+0000","last_unstale":"2024-09-12T03:19:20.372560+0000","last_undegraded":"2024-09-12T03:19:20.372560+0000","last_fullsized":"2024-09-12T03:19:20.372560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:56.177948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.238148+0000","last_change":"2024-09-12T03:19:09.341837+0000","last_active":"2024-09-12T03:20:49.238148+0000","last_peered":"2024-09-12T03:20:49.238148+0000","last_clean":"2024-09-12T03:20:49.238148+0000","last_became_active":"2024-09-12T03:19:09.341069+0000","last_became_peered":"2024-09-12T03:19:09.341069+0000","last_unstale":"2024-09-12T03:20:49.238148+0000","last_undegraded":"2024-09-12T03:20:49.238148+0000","last_fullsized":"2024-09-12T03:20:49.238148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:04.238986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372460+0000","last_change":"2024-09-12T03:19:07.388870+0000","last_active":"2024-09-12T03:19:20.372460+0000","last_peered":"2024-09-12T03:19:20.372460+0000","last_clean":"2024-09-12T03:19:20.372460+0000","last_became_active":"2024-09-12T03:19:07.388555+0000","last_became_peered":"2024-09-12T03:19:07.388555+0000","last_unstale":"2024-09-12T03:19:20.372460+0000","last_undegraded":"2024-09-12T03:19:20.372460+0000","last_fullsized":"2024-09-12T03:19:20.372460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:29.789454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.247542+0000","last_change":"2024-09-12T03:19:09.334714+0000","last_active":"2024-09-12T03:20:49.247542+0000","last_peered":"2024-09-12T03:20:49.247542+0000","last_clean":"2024-09-12T03:20:49.247542+0000","last_became_active":"2024-09-12T03:19:09.334413+0000","last_became_peered":"2024-09-12T03:19:09.334413+0000","last_unstale":"2024-09-12T03:20:49.247542+0000","last_undegraded":"2024-09-12T03:20:49.247542+0000","last_fullsized":"2024-09-12T03:20:49.247542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:52.222340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373797+0000","last_change":"2024-09-12T03:19:07.389055+0000","last_active":"2024-09-12T03:19:20.373797+0000","last_peered":"2024-09-12T03:19:20.373797+0000","last_clean":"2024-09-12T03:19:20.373797+0000","last_became_active":"2024-09-12T03:19:07.388673+0000","last_became_peered":"2024-09-12T03:19:07.388673+0000","last_unstale":"2024-09-12T03:19:20.373797+0000","last_undegraded":"2024-09-12T03:19:20.373797+0000","last_fullsized":"2024-09-12T03:19:20.373797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:20.974533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248932+0000","last_change":"2024-09-12T03:19:09.340105+0000","last_active":"2024-09-12T03:20:49.248932+0000","last_peered":"2024-09-12T03:20:49.248932+0000","last_clean":"2024-09-12T03:20:49.248932+0000","last_became_active":"2024-09-12T03:19:09.339642+0000","last_became_peered":"2024-09-12T03:19:09.339642+0000","last_unstale":"2024-09-12T03:20:49.248932+0000","last_undegraded":"2024-09-12T03:20:49.248932+0000","last_fullsized":"2024-09-12T03:20:49.248932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:01.624236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'15","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:52.343269+0000","last_change":"2024-09-12T03:19:07.207232+0000","last_active":"2024-09-12T03:20:52.343269+0000","last_peered":"2024-09-12T03:20:52.343269+0000","last_clean":"2024-09-12T03:20:52.343269+0000","last_became_active":"2024-09-12T03:19:07.206921+0000","last_became_peered":"2024-09-12T03:19:07.206921+0000","last_unstale":"2024-09-12T03:20:52.343269+0000","last_undegraded":"2024-09-12T03:20:52.343269+0000","last_fullsized":"2024-09-12T03:20:52.343269+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:25.457853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252490+0000","last_change":"2024-09-12T03:19:09.330442+0000","last_active":"2024-09-12T03:20:49.252490+0000","last_peered":"2024-09-12T03:20:49.252490+0000","last_clean":"2024-09-12T03:20:49.252490+0000","last_became_active":"2024-09-12T03:19:09.330081+0000","last_became_peered":"2024-09-12T03:19:09.330081+0000","last_unstale":"2024-09-12T03:20:49.252490+0000","last_undegraded":"2024-09-12T03:20:49.252490+0000","last_fullsized":"2024-09-12T03:20:49.252490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:56.152562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382392+0000","last_change":"2024-09-12T03:19:07.528646+0000","last_active":"2024-09-12T03:19:20.382392+0000","last_peered":"2024-09-12T03:19:20.382392+0000","last_clean":"2024-09-12T03:19:20.382392+0000","last_became_active":"2024-09-12T03:19:07.528334+0000","last_became_peered":"2024-09-12T03:19:07.528334+0000","last_unstale":"2024-09-12T03:19:20.382392+0000","last_undegraded":"2024-09-12T03:19:20.382392+0000","last_fullsized":"2024-09-12T03:19:20.382392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:40.425579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256685+0000","last_change":"2024-09-12T03:19:09.331993+0000","last_active":"2024-09-12T03:20:49.256685+0000","last_peered":"2024-09-12T03:20:49.256685+0000","last_clean":"2024-09-12T03:20:49.256685+0000","last_became_active":"2024-09-12T03:19:09.331698+0000","last_became_peered":"2024-09-12T03:19:09.331698+0000","last_unstale":"2024-09-12T03:20:49.256685+0000","last_undegraded":"2024-09-12T03:20:49.256685+0000","last_fullsized":"2024-09-12T03:20:49.256685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:03:54.182833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370179+0000","last_change":"2024-09-12T03:19:07.388297+0000","last_active":"2024-09-12T03:19:20.370179+0000","last_peered":"2024-09-12T03:19:20.370179+0000","last_clean":"2024-09-12T03:19:20.370179+0000","last_became_active":"2024-09-12T03:19:07.388038+0000","last_became_peered":"2024-09-12T03:19:07.388038+0000","last_unstale":"2024-09-12T03:19:20.370179+0000","last_undegraded":"2024-09-12T03:19:20.370179+0000","last_fullsized":"2024-09-12T03:19:20.370179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:45.874195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.212080+0000","last_change":"2024-09-12T03:19:09.331525+0000","last_active":"2024-09-12T03:20:49.212080+0000","last_peered":"2024-09-12T03:20:49.212080+0000","last_clean":"2024-09-12T03:20:49.212080+0000","last_became_active":"2024-09-12T03:19:09.330903+0000","last_became_peered":"2024-09-12T03:19:09.330903+0000","last_unstale":"2024-09-12T03:20:49.212080+0000","last_undegraded":"2024-09-12T03:20:49.212080+0000","last_fullsized":"2024-09-12T03:20:49.212080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:02.455132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372235+0000","last_change":"2024-09-12T03:19:07.205181+0000","last_active":"2024-09-12T03:19:20.372235+0000","last_peered":"2024-09-12T03:19:20.372235+0000","last_clean":"2024-09-12T03:19:20.372235+0000","last_became_active":"2024-09-12T03:19:07.204726+0000","last_became_peered":"2024-09-12T03:19:07.204726+0000","last_unstale":"2024-09-12T03:19:20.372235+0000","last_undegraded":"2024-09-12T03:19:20.372235+0000","last_fullsized":"2024-09-12T03:19:20.372235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:26.112800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250768+0000","last_change":"2024-09-12T03:19:09.345338+0000","last_active":"2024-09-12T03:20:49.250768+0000","last_peered":"2024-09-12T03:20:49.250768+0000","last_clean":"2024-09-12T03:20:49.250768+0000","last_became_active":"2024-09-12T03:19:09.345023+0000","last_became_peered":"2024-09-12T03:19:09.345023+0000","last_unstale":"2024-09-12T03:20:49.250768+0000","last_undegraded":"2024-09-12T03:20:49.250768+0000","last_fullsized":"2024-09-12T03:20:49.250768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:39.184065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374938+0000","last_change":"2024-09-12T03:19:07.211954+0000","last_active":"2024-09-12T03:19:20.374938+0000","last_peered":"2024-09-12T03:19:20.374938+0000","last_clean":"2024-09-12T03:19:20.374938+0000","last_became_active":"2024-09-12T03:19:07.211509+0000","last_became_peered":"2024-09-12T03:19:07.211509+0000","last_unstale":"2024-09-12T03:19:20.374938+0000","last_undegraded":"2024-09-12T03:19:20.374938+0000","last_fullsized":"2024-09-12T03:19:20.374938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:27.461924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.244962+0000","last_change":"2024-09-12T03:19:09.331168+0000","last_active":"2024-09-12T03:20:49.244962+0000","last_peered":"2024-09-12T03:20:49.244962+0000","last_clean":"2024-09-12T03:20:49.244962+0000","last_became_active":"2024-09-12T03:19:09.329592+0000","last_became_peered":"2024-09-12T03:19:09.329592+0000","last_unstale":"2024-09-12T03:20:49.244962+0000","last_undegraded":"2024-09-12T03:20:49.244962+0000","last_fullsized":"2024-09-12T03:20:49.244962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:46.127861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373572+0000","last_change":"2024-09-12T03:19:07.207802+0000","last_active":"2024-09-12T03:19:20.373572+0000","last_peered":"2024-09-12T03:19:20.373572+0000","last_clean":"2024-09-12T03:19:20.373572+0000","last_became_active":"2024-09-12T03:19:07.207428+0000","last_became_peered":"2024-09-12T03:19:07.207428+0000","last_unstale":"2024-09-12T03:19:20.373572+0000","last_undegraded":"2024-09-12T03:19:20.373572+0000","last_fullsized":"2024-09-12T03:19:20.373572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:45.135019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.255394+0000","last_change":"2024-09-12T03:19:09.336094+0000","last_active":"2024-09-12T03:20:49.255394+0000","last_peered":"2024-09-12T03:20:49.255394+0000","last_clean":"2024-09-12T03:20:49.255394+0000","last_became_active":"2024-09-12T03:19:09.335779+0000","last_became_peered":"2024-09-12T03:19:09.335779+0000","last_unstale":"2024-09-12T03:20:49.255394+0000","last_undegraded":"2024-09-12T03:20:49.255394+0000","last_fullsized":"2024-09-12T03:20:49.255394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:37.019082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254523+0000","last_change":"2024-09-12T03:19:09.344537+0000","last_active":"2024-09-12T03:20:49.254523+0000","last_peered":"2024-09-12T03:20:49.254523+0000","last_clean":"2024-09-12T03:20:49.254523+0000","last_became_active":"2024-09-12T03:19:09.343879+0000","last_became_peered":"2024-09-12T03:19:09.343879+0000","last_unstale":"2024-09-12T03:20:49.254523+0000","last_undegraded":"2024-09-12T03:20:49.254523+0000","last_fullsized":"2024-09-12T03:20:49.254523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:40.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373973+0000","last_change":"2024-09-12T03:19:07.208040+0000","last_active":"2024-09-12T03:19:20.373973+0000","last_peered":"2024-09-12T03:19:20.373973+0000","last_clean":"2024-09-12T03:19:20.373973+0000","last_became_active":"2024-09-12T03:19:07.207658+0000","last_became_peered":"2024-09-12T03:19:07.207658+0000","last_unstale":"2024-09-12T03:19:20.373973+0000","last_undegraded":"2024-09-12T03:19:20.373973+0000","last_fullsized":"2024-09-12T03:19:20.373973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:48.620707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265759+0000","last_change":"2024-09-12T03:19:09.333837+0000","last_active":"2024-09-12T03:20:49.265759+0000","last_peered":"2024-09-12T03:20:49.265759+0000","last_clean":"2024-09-12T03:20:49.265759+0000","last_became_active":"2024-09-12T03:19:09.333194+0000","last_became_peered":"2024-09-12T03:19:09.333194+0000","last_unstale":"2024-09-12T03:20:49.265759+0000","last_undegraded":"2024-09-12T03:20:49.265759+0000","last_fullsized":"2024-09-12T03:20:49.265759+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:18.766181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.385455+0000","last_change":"2024-09-12T03:19:20.384978+0000","last_active":"2024-09-12T03:19:20.385455+0000","last_peered":"2024-09-12T03:19:20.385455+0000","last_clean":"2024-09-12T03:19:20.385455+0000","last_became_active":"2024-09-12T03:19:20.376883+0000","last_became_peered":"2024-09-12T03:19:20.376883+0000","last_unstale":"2024-09-12T03:19:20.385455+0000","last_undegraded":"2024-09-12T03:19:20.385455+0000","last_fullsized":"2024-09-12T03:19:20.385455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:32.542334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265022+0000","last_change":"2024-09-12T03:19:09.334684+0000","last_active":"2024-09-12T03:20:49.265022+0000","last_peered":"2024-09-12T03:20:49.265022+0000","last_clean":"2024-09-12T03:20:49.265022+0000","last_became_active":"2024-09-12T03:19:09.334281+0000","last_became_peered":"2024-09-12T03:19:09.334281+0000","last_unstale":"2024-09-12T03:20:49.265022+0000","last_undegraded":"2024-09-12T03:20:49.265022+0000","last_fullsized":"2024-09-12T03:20:49.265022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:44.837067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372829+0000","last_change":"2024-09-12T03:19:07.388555+0000","last_active":"2024-09-12T03:19:20.372829+0000","last_peered":"2024-09-12T03:19:20.372829+0000","last_clean":"2024-09-12T03:19:20.372829+0000","last_became_active":"2024-09-12T03:19:07.388206+0000","last_became_peered":"2024-09-12T03:19:07.388206+0000","last_unstale":"2024-09-12T03:19:20.372829+0000","last_undegraded":"2024-09-12T03:19:20.372829+0000","last_fullsized":"2024-09-12T03:19:20.372829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:33.628294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248105+0000","last_change":"2024-09-12T03:19:09.339108+0000","last_active":"2024-09-12T03:20:49.248105+0000","last_peered":"2024-09-12T03:20:49.248105+0000","last_clean":"2024-09-12T03:20:49.248105+0000","last_became_active":"2024-09-12T03:19:09.338819+0000","last_became_peered":"2024-09-12T03:19:09.338819+0000","last_unstale":"2024-09-12T03:20:49.248105+0000","last_undegraded":"2024-09-12T03:20:49.248105+0000","last_fullsized":"2024-09-12T03:20:49.248105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:32.864900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372798+0000","last_change":"2024-09-12T03:19:07.207197+0000","last_active":"2024-09-12T03:19:20.372798+0000","last_peered":"2024-09-12T03:19:20.372798+0000","last_clean":"2024-09-12T03:19:20.372798+0000","last_became_active":"2024-09-12T03:19:07.206879+0000","last_became_peered":"2024-09-12T03:19:07.206879+0000","last_unstale":"2024-09-12T03:19:20.372798+0000","last_undegraded":"2024-09-12T03:19:20.372798+0000","last_fullsized":"2024-09-12T03:19:20.372798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:00:46.269636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73845,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":628,"num_read_kb":391,"num_write":182,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":221535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579939,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":433387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36900,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708252,"statfs":{"total":95995035648,"available":95957250048,"internally_reserved":0,"allocated":1593344,"data_stored":1093834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972396,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":433589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201330,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":433509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430263,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1499136,"data_stored":1026676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593788,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":432978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822721,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1511424,"data_stored":1025967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":897024,"data_stored":433808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":433289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":501409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40444,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704708,"statfs":{"total":95995035648,"available":95953620992,"internally_reserved":0,"allocated":962560,"data_stored":501378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":434249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:21:03.343 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:21:03.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:03 smithi161 ceph-mon[35485]: pgmap v333: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-12T03:21:04.009 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:21:04.057 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:04.318 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:04 smithi006 ceph-mon[30991]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:21:04.318 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:04 smithi006 ceph-mon[30991]: scrub queued for path: / 2024-09-12T03:21:04.318 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:04 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:04.318 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:04 smithi006 ceph-mon[30991]: scrub summary: active paths [/] 2024-09-12T03:21:04.318 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:04 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:04.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:04 smithi161 ceph-mon[35485]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:21:04.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:04 smithi161 ceph-mon[35485]: scrub queued for path: / 2024-09-12T03:21:04.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:04 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:04.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:04 smithi161 ceph-mon[35485]: scrub summary: active paths [/] 2024-09-12T03:21:04.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:04 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:04.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:04 smithi118 ceph-mon[35048]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:21:04.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:04 smithi118 ceph-mon[35048]: scrub queued for path: / 2024-09-12T03:21:04.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:04 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:04.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:04 smithi118 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-12T03:21:04.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:04 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:04.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.567+0000 7f49aecfb640 1 -- 172.21.15.6:0/1350455429 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49a8102790 msgr2=0x7f49a81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:04.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.567+0000 7f49aecfb640 1 --2- 172.21.15.6:0/1350455429 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49a8102790 0x7f49a81067e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4994009900 tx=0x7f499402a410 comp rx=0 tx=0).stop 2024-09-12T03:21:04.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.567+0000 7f49aecfb640 1 -- 172.21.15.6:0/1350455429 shutdown_connections 2024-09-12T03:21:04.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.567+0000 7f49aecfb640 1 --2- 172.21.15.6:0/1350455429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49a8106f10 0x7f49a810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.567+0000 7f49aecfb640 1 --2- 172.21.15.6:0/1350455429 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49a8102790 0x7f49a81067e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.567+0000 7f49aecfb640 1 --2- 172.21.15.6:0/1350455429 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49a8101de0 0x7f49a81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.567+0000 7f49aecfb640 1 -- 172.21.15.6:0/1350455429 >> 172.21.15.6:0/1350455429 conn(0x7f49a80fd8c0 msgr2=0x7f49a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:04.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.567+0000 7f49aecfb640 1 -- 172.21.15.6:0/1350455429 shutdown_connections 2024-09-12T03:21:04.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.567+0000 7f49aecfb640 1 -- 172.21.15.6:0/1350455429 wait complete. 2024-09-12T03:21:04.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.567+0000 7f49aecfb640 1 Processor -- start 2024-09-12T03:21:04.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.568+0000 7f49aecfb640 1 -- start start 2024-09-12T03:21:04.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.568+0000 7f49aecfb640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49a8101de0 0x7f49a8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.568+0000 7f49aecfb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49a8102790 0x7f49a81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.568+0000 7f49aecfb640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49a8106f10 0x7f49a819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.568+0000 7f49aecfb640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f49a8111f70 con 0x7f49a8106f10 2024-09-12T03:21:04.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.568+0000 7f49aecfb640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f49a8111df0 con 0x7f49a8101de0 2024-09-12T03:21:04.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.568+0000 7f49aecfb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81120f0 con 0x7f49a8102790 2024-09-12T03:21:04.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.568+0000 7f49ad298640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49a8106f10 0x7f49a819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.568+0000 7f49ad298640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49a8106f10 0x7f49a819cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44916/0 (socket says 172.21.15.6:44916) 2024-09-12T03:21:04.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.568+0000 7f49ad298640 1 -- 172.21.15.6:0/1339661713 learned_addr learned my addr 172.21.15.6:0/1339661713 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:04.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.568+0000 7f49a7fff640 1 --2- 172.21.15.6:0/1339661713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49a8102790 0x7f49a81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.568+0000 7f49aca97640 1 --2- 172.21.15.6:0/1339661713 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49a8101de0 0x7f49a8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.568+0000 7f49ad298640 1 -- 172.21.15.6:0/1339661713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49a8102790 msgr2=0x7f49a81989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:04.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.568+0000 7f49ad298640 1 --2- 172.21.15.6:0/1339661713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49a8102790 0x7f49a81989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.568+0000 7f49ad298640 1 -- 172.21.15.6:0/1339661713 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49a8101de0 msgr2=0x7f49a8198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:04.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.568+0000 7f49ad298640 1 --2- 172.21.15.6:0/1339661713 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49a8101de0 0x7f49a8198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.569+0000 7f49ad298640 1 -- 172.21.15.6:0/1339661713 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a819d440 con 0x7f49a8106f10 2024-09-12T03:21:04.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.569+0000 7f49ad298640 1 --2- 172.21.15.6:0/1339661713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49a8106f10 0x7f49a819cd60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f49a000eaa0 tx=0x7f49a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:04.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.569+0000 7f49a5ffb640 1 -- 172.21.15.6:0/1339661713 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a000cdb0 con 0x7f49a8106f10 2024-09-12T03:21:04.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.569+0000 7f49a5ffb640 1 -- 172.21.15.6:0/1339661713 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49a0005020 con 0x7f49a8106f10 2024-09-12T03:21:04.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.569+0000 7f49aecfb640 1 -- 172.21.15.6:0/1339661713 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a819d730 con 0x7f49a8106f10 2024-09-12T03:21:04.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.569+0000 7f49a5ffb640 1 -- 172.21.15.6:0/1339661713 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a00116b0 con 0x7f49a8106f10 2024-09-12T03:21:04.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.570+0000 7f49aecfb640 1 -- 172.21.15.6:0/1339661713 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a81a5010 con 0x7f49a8106f10 2024-09-12T03:21:04.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.571+0000 7f49aecfb640 1 -- 172.21.15.6:0/1339661713 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a81021c0 con 0x7f49a8106f10 2024-09-12T03:21:04.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.572+0000 7f49a5ffb640 1 -- 172.21.15.6:0/1339661713 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f49a0005560 con 0x7f49a8106f10 2024-09-12T03:21:04.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.572+0000 7f49a5ffb640 1 --2- 172.21.15.6:0/1339661713 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f49980a2c10 0x7f49980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.573+0000 7f49a5ffb640 1 -- 172.21.15.6:0/1339661713 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f49a00cf2d0 con 0x7f49a8106f10 2024-09-12T03:21:04.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.573+0000 7f49aca97640 1 --2- 172.21.15.6:0/1339661713 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f49980a2c10 0x7f49980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.573+0000 7f49aca97640 1 --2- 172.21.15.6:0/1339661713 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f49980a2c10 0x7f49980a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f499c00bbe0 tx=0x7f499c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:04.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.577+0000 7f49a5ffb640 1 -- 172.21.15.6:0/1339661713 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f49a0016030 con 0x7f49a8106f10 2024-09-12T03:21:04.604 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:04.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.767+0000 7f49aecfb640 1 -- 172.21.15.6:0/1339661713 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f49a810a160 con 0x7f49980a2c10 2024-09-12T03:21:04.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.773+0000 7f49a5ffb640 1 -- 172.21.15.6:0/1339661713 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258838 (secure 0 0 0) 0x7f499c010c30 con 0x7f49980a2c10 2024-09-12T03:21:04.775 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:21:04.776 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-09-12T03:21:04.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.776+0000 7f49aecfb640 1 -- 172.21.15.6:0/1339661713 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f49980a2c10 msgr2=0x7f49980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:04.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.776+0000 7f49aecfb640 1 --2- 172.21.15.6:0/1339661713 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f49980a2c10 0x7f49980a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f499c00bbe0 tx=0x7f499c00b390 comp rx=0 tx=0).stop 2024-09-12T03:21:04.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.776+0000 7f49aecfb640 1 -- 172.21.15.6:0/1339661713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49a8106f10 msgr2=0x7f49a819cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:04.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.776+0000 7f49aecfb640 1 --2- 172.21.15.6:0/1339661713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49a8106f10 0x7f49a819cd60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f49a000eaa0 tx=0x7f49a000ef60 comp rx=0 tx=0).stop 2024-09-12T03:21:04.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.776+0000 7f49aecfb640 1 -- 172.21.15.6:0/1339661713 shutdown_connections 2024-09-12T03:21:04.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.776+0000 7f49aecfb640 1 --2- 172.21.15.6:0/1339661713 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f49980a2c10 0x7f49980a50d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.776+0000 7f49aecfb640 1 --2- 172.21.15.6:0/1339661713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49a8106f10 0x7f49a819cd60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.776+0000 7f49aecfb640 1 --2- 172.21.15.6:0/1339661713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49a8102790 0x7f49a81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.776+0000 7f49aecfb640 1 --2- 172.21.15.6:0/1339661713 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49a8101de0 0x7f49a8198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.776+0000 7f49aecfb640 1 -- 172.21.15.6:0/1339661713 >> 172.21.15.6:0/1339661713 conn(0x7f49a80fd8c0 msgr2=0x7f49a80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:04.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.776+0000 7f49aecfb640 1 -- 172.21.15.6:0/1339661713 shutdown_connections 2024-09-12T03:21:04.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:04.776+0000 7f49aecfb640 1 -- 172.21.15.6:0/1339661713 wait complete. 2024-09-12T03:21:05.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:05 smithi006 ceph-mon[30991]: pgmap v334: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-12T03:21:05.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.273+0000 7f3b3a9ba640 1 -- 172.21.15.6:0/566656888 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b341036c0 msgr2=0x7f3b3410bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:05.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.273+0000 7f3b3a9ba640 1 --2- 172.21.15.6:0/566656888 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b341036c0 0x7f3b3410bbb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009900 tx=0x7f3b1c02a480 comp rx=0 tx=0).stop 2024-09-12T03:21:05.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.273+0000 7f3b3a9ba640 1 -- 172.21.15.6:0/566656888 shutdown_connections 2024-09-12T03:21:05.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.273+0000 7f3b3a9ba640 1 --2- 172.21.15.6:0/566656888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b3410c180 0x7f3b3410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.273+0000 7f3b3a9ba640 1 --2- 172.21.15.6:0/566656888 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b341036c0 0x7f3b3410bbb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.273+0000 7f3b3a9ba640 1 --2- 172.21.15.6:0/566656888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b34102da0 0x7f3b34103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.273+0000 7f3b3a9ba640 1 -- 172.21.15.6:0/566656888 >> 172.21.15.6:0/566656888 conn(0x7f3b340fc7a0 msgr2=0x7f3b340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:05.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.273+0000 7f3b3a9ba640 1 -- 172.21.15.6:0/566656888 shutdown_connections 2024-09-12T03:21:05.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.273+0000 7f3b3a9ba640 1 -- 172.21.15.6:0/566656888 wait complete. 2024-09-12T03:21:05.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.274+0000 7f3b3a9ba640 1 Processor -- start 2024-09-12T03:21:05.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.274+0000 7f3b3a9ba640 1 -- start start 2024-09-12T03:21:05.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.274+0000 7f3b3a9ba640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b34102da0 0x7f3b341983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.274+0000 7f3b3a9ba640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b341036c0 0x7f3b34198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.274+0000 7f3b3a9ba640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b3410c180 0x7f3b3419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.274+0000 7f3b3a9ba640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3b34111f60 con 0x7f3b341036c0 2024-09-12T03:21:05.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.274+0000 7f3b3a9ba640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f3b34111de0 con 0x7f3b3410c180 2024-09-12T03:21:05.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.274+0000 7f3b3a9ba640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3b341120e0 con 0x7f3b34102da0 2024-09-12T03:21:05.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.274+0000 7f3b2bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b341036c0 0x7f3b34198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.274+0000 7f3b2bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b341036c0 0x7f3b34198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44932/0 (socket says 172.21.15.6:44932) 2024-09-12T03:21:05.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.274+0000 7f3b2bfff640 1 -- 172.21.15.6:0/3598472936 learned_addr learned my addr 172.21.15.6:0/3598472936 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:05.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.274+0000 7f3b33fff640 1 --2- 172.21.15.6:0/3598472936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b34102da0 0x7f3b341983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.274+0000 7f3b38f57640 1 --2- 172.21.15.6:0/3598472936 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b3410c180 0x7f3b3419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.274+0000 7f3b2bfff640 1 -- 172.21.15.6:0/3598472936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b34102da0 msgr2=0x7f3b341983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:05.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.274+0000 7f3b2bfff640 1 --2- 172.21.15.6:0/3598472936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b34102da0 0x7f3b341983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.274+0000 7f3b2bfff640 1 -- 172.21.15.6:0/3598472936 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b3410c180 msgr2=0x7f3b3419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:05.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.274+0000 7f3b2bfff640 1 --2- 172.21.15.6:0/3598472936 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b3410c180 0x7f3b3419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.274+0000 7f3b2bfff640 1 -- 172.21.15.6:0/3598472936 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3419d390 con 0x7f3b341036c0 2024-09-12T03:21:05.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.274+0000 7f3b38f57640 1 --2- 172.21.15.6:0/3598472936 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b3410c180 0x7f3b3419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:05.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.275+0000 7f3b2bfff640 1 --2- 172.21.15.6:0/3598472936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b341036c0 0x7f3b34198920 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c006010 tx=0x7f3b1c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:05.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.275+0000 7f3b31ffb640 1 -- 172.21.15.6:0/3598472936 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c03f030 con 0x7f3b341036c0 2024-09-12T03:21:05.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.275+0000 7f3b3a9ba640 1 -- 172.21.15.6:0/3598472936 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b3419d620 con 0x7f3b341036c0 2024-09-12T03:21:05.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.275+0000 7f3b31ffb640 1 -- 172.21.15.6:0/3598472936 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b1c003ce0 con 0x7f3b341036c0 2024-09-12T03:21:05.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.275+0000 7f3b31ffb640 1 -- 172.21.15.6:0/3598472936 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c035070 con 0x7f3b341036c0 2024-09-12T03:21:05.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.275+0000 7f3b3a9ba640 1 -- 172.21.15.6:0/3598472936 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b341a4f60 con 0x7f3b341036c0 2024-09-12T03:21:05.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.277+0000 7f3b31ffb640 1 -- 172.21.15.6:0/3598472936 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f3b1c042030 con 0x7f3b341036c0 2024-09-12T03:21:05.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.277+0000 7f3b3cd93740 1 Processor -- start 2024-09-12T03:21:05.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.277+0000 7f3b3cd93740 1 -- start start 2024-09-12T03:21:05.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.278+0000 7f3b3cd93740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3b5360 0x56094f3b5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.278+0000 7f3b3cd93740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56094f3b5c80 0x56094f3c4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.278+0000 7f3b3cd93740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3c4a90 0x56094f3c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.278+0000 7f3b3cd93740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56094f31b2e0 con 0x56094f3b5360 2024-09-12T03:21:05.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.278+0000 7f3b3cd93740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x56094f2c2ce0 con 0x56094f3c4a90 2024-09-12T03:21:05.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.278+0000 7f3b3cd93740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56094f3172a0 con 0x56094f3b5c80 2024-09-12T03:21:05.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.278+0000 7f3b38f57640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3b5360 0x56094f3b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.278+0000 7f3b38f57640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3b5360 0x56094f3b5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44948/0 (socket says 172.21.15.6:44948) 2024-09-12T03:21:05.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.278+0000 7f3b38f57640 1 -- 172.21.15.6:0/200918698 learned_addr learned my addr 172.21.15.6:0/200918698 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:05.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.278+0000 7f3b33fff640 1 --2- 172.21.15.6:0/200918698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56094f3b5c80 0x56094f3c4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.278+0000 7f3b31ffb640 1 --2- 172.21.15.6:0/3598472936 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3b200a2c10 0x7f3b200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.278+0000 7f3b2bfff640 1 --2- 172.21.15.6:0/200918698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3c4a90 0x56094f3c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.278+0000 7f3b38f57640 1 -- 172.21.15.6:0/200918698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56094f3b5c80 msgr2=0x56094f3c4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:05.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.278+0000 7f3b38f57640 1 --2- 172.21.15.6:0/200918698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56094f3b5c80 0x56094f3c4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.278+0000 7f3b38f57640 1 -- 172.21.15.6:0/200918698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3c4a90 msgr2=0x56094f3c6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:05.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.278+0000 7f3b38f57640 1 --2- 172.21.15.6:0/200918698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3c4a90 0x56094f3c6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.278+0000 7f3b38f57640 1 -- 172.21.15.6:0/200918698 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56094f2a7440 con 0x56094f3b5360 2024-09-12T03:21:05.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.278+0000 7f3b2bfff640 1 --2- 172.21.15.6:0/200918698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3c4a90 0x56094f3c6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:05.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.278+0000 7f3b38f57640 1 --2- 172.21.15.6:0/3598472936 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3b200a2c10 0x7f3b200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.278+0000 7f3b31ffb640 1 -- 172.21.15.6:0/3598472936 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f3b1c0ef7d0 con 0x7f3b341036c0 2024-09-12T03:21:05.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.279+0000 7f3b38f57640 1 --2- 172.21.15.6:0/200918698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3b5360 0x56094f3b5740 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c005c80 tx=0x7f3b2c00e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d71cb2bfddd010b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:05.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.279+0000 7f3b2a7fc640 1 -- 172.21.15.6:0/200918698 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c022070 con 0x56094f3b5360 2024-09-12T03:21:05.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.279+0000 7f3b2a7fc640 1 -- 172.21.15.6:0/200918698 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b2c011c30 con 0x56094f3b5360 2024-09-12T03:21:05.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.279+0000 7f3b2a7fc640 1 -- 172.21.15.6:0/200918698 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c01a400 con 0x56094f3b5360 2024-09-12T03:21:05.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.279+0000 7f3b38f57640 1 --2- 172.21.15.6:0/3598472936 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3b200a2c10 0x7f3b200a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c01f000 tx=0x7f3b2c020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:05.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.279+0000 7f3b3cd93740 1 -- 172.21.15.6:0/200918698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3b5360 msgr2=0x56094f3b5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:05.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.279+0000 7f3b3cd93740 1 --2- 172.21.15.6:0/200918698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3b5360 0x56094f3b5740 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c005c80 tx=0x7f3b2c00e7e0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.280+0000 7f3b3cd93740 1 -- 172.21.15.6:0/200918698 shutdown_connections 2024-09-12T03:21:05.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.280+0000 7f3b3cd93740 1 --2- 172.21.15.6:0/200918698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3c4a90 0x56094f3c6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.280+0000 7f3b3cd93740 1 --2- 172.21.15.6:0/200918698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56094f3b5c80 0x56094f3c4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.280+0000 7f3b3cd93740 1 --2- 172.21.15.6:0/200918698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3b5360 0x56094f3b5740 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.280+0000 7f3b3cd93740 1 -- 172.21.15.6:0/200918698 >> 172.21.15.6:0/200918698 conn(0x56094f3b1a30 msgr2=0x56094f3b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:05.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.280+0000 7f3b3cd93740 1 -- 172.21.15.6:0/200918698 shutdown_connections 2024-09-12T03:21:05.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.280+0000 7f3b3cd93740 1 -- 172.21.15.6:0/200918698 wait complete. 2024-09-12T03:21:05.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.280+0000 7f3b3cd93740 1 -- start start 2024-09-12T03:21:05.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.280+0000 7f3b3cd93740 1 Processor -- start 2024-09-12T03:21:05.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.280+0000 7f3b3cd93740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56094f3b5360 0x56094f3c4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.280+0000 7f3b3cd93740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3b5c80 0x56094f3c16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.280+0000 7f3b3cd93740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3c4a90 0x56094f3c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.280+0000 7f3b3cd93740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56094f3c7660 con 0x56094f3b5c80 2024-09-12T03:21:05.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.280+0000 7f3b3cd93740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x56094f2d6050 con 0x56094f3c4a90 2024-09-12T03:21:05.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.280+0000 7f3b3cd93740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56094f3c77e0 con 0x56094f3b5360 2024-09-12T03:21:05.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.280+0000 7f3b3cd93740 10 client.0 _ll_register_callbacks cb 0x56094f316160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:05.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.281+0000 7f3b2bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3b5c80 0x56094f3c16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.281+0000 7f3b2bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3b5c80 0x56094f3c16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44958/0 (socket says 172.21.15.6:44958) 2024-09-12T03:21:05.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.281+0000 7f3b2bfff640 1 -- 172.21.15.6:0/1582370498 learned_addr learned my addr 172.21.15.6:0/1582370498 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:05.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.281+0000 7f3b38f57640 1 --2- 172.21.15.6:0/1582370498 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3c4a90 0x56094f3c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.281+0000 7f3b33fff640 1 --2- 172.21.15.6:0/1582370498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56094f3b5360 0x56094f3c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.281+0000 7f3b2bfff640 1 -- 172.21.15.6:0/1582370498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56094f3b5360 msgr2=0x56094f3c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:05.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.281+0000 7f3b2bfff640 1 --2- 172.21.15.6:0/1582370498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56094f3b5360 0x56094f3c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.281+0000 7f3b2bfff640 1 -- 172.21.15.6:0/1582370498 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3c4a90 msgr2=0x56094f3c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:05.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.281+0000 7f3b2bfff640 1 --2- 172.21.15.6:0/1582370498 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3c4a90 0x56094f3c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.281+0000 7f3b2bfff640 1 -- 172.21.15.6:0/1582370498 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56094f08c3c0 con 0x56094f3b5c80 2024-09-12T03:21:05.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.281+0000 7f3b2bfff640 1 --2- 172.21.15.6:0/1582370498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3b5c80 0x56094f3c16b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0c4700 tx=0x7f3b1c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f7360d581ef413 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:05.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.281+0000 7f3b137fe640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:21:05.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.281+0000 7f3b137fe640 1 -- 172.21.15.6:0/1582370498 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c049070 con 0x56094f3b5c80 2024-09-12T03:21:05.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.281+0000 7f3b137fe640 1 -- 172.21.15.6:0/1582370498 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f3b1c0c47a0 con 0x56094f3b5c80 2024-09-12T03:21:05.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.281+0000 7f3b3cd93740 1 -- 172.21.15.6:0/1582370498 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b2c011c30 con 0x56094f3b5c80 2024-09-12T03:21:05.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.281+0000 7f3b137fe640 1 -- 172.21.15.6:0/1582370498 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3b1c0bb920 con 0x56094f3b5c80 2024-09-12T03:21:05.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.281+0000 7f3b3cd93740 10 client.15231 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:05.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.281+0000 7f3b3cd93740 1 -- 172.21.15.6:0/1582370498 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56094f3d2fe0 con 0x56094f3b5c80 2024-09-12T03:21:05.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.282+0000 7f3b137fe640 1 -- 172.21.15.6:0/1582370498 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b2c011c30 con 0x56094f3b5c80 2024-09-12T03:21:05.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.282+0000 7f3b137fe640 1 -- 172.21.15.6:0/1582370498 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3b1c0be020 con 0x56094f3b5c80 2024-09-12T03:21:05.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.282+0000 7f3b3cd93740 10 client.15231 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:05.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.282+0000 7f3b3cd93740 10 client.15231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:21:05.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.282+0000 7f3b3cd93740 20 client.15231 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:05.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.282+0000 7f3b3cd93740 1 --2- 172.21.15.6:0/1582370498 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56094f3d43d0 0x56094f3d67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.282+0000 7f3b3cd93740 4 client.15231 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:05.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.282+0000 7f3b3cd93740 1 -- 172.21.15.6:0/1582370498 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56094f328850 con 0x56094f3d43d0 2024-09-12T03:21:05.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.282+0000 7f3b33fff640 1 --2- 172.21.15.6:0/1582370498 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56094f3d43d0 0x56094f3d67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.283+0000 7f3b33fff640 1 --2- 172.21.15.6:0/1582370498 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56094f3d43d0 0x56094f3d67c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fcc077b18d0cba8 server_cookie=5bc04fafee5f7d96 in_seq=0 out_seq=0 2024-09-12T03:21:05.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.283+0000 7f3b137fe640 10 client.15231 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:05.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.283+0000 7f3b137fe640 1 -- 172.21.15.6:0/1582370498 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56094f328850 con 0x56094f3d43d0 2024-09-12T03:21:05.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.283+0000 7f3b137fe640 10 client.15231 handle_command_reply: tid=0 2024-09-12T03:21:05.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.283+0000 7f3b3cd93740 1 client.15231 shutdown 2024-09-12T03:21:05.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.284+0000 7f3b3cd93740 1 -- 172.21.15.6:0/1582370498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3b5c80 msgr2=0x56094f3c16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:05.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.284+0000 7f3b3cd93740 1 --2- 172.21.15.6:0/1582370498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3b5c80 0x56094f3c16b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0c4700 tx=0x7f3b1c009560 comp rx=0 tx=0).stop 2024-09-12T03:21:05.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.284+0000 7f3b3cd93740 1 -- 172.21.15.6:0/1582370498 shutdown_connections 2024-09-12T03:21:05.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.284+0000 7f3b3cd93740 1 --2- 172.21.15.6:0/1582370498 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56094f3d43d0 0x56094f3d67c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.284+0000 7f3b3cd93740 1 --2- 172.21.15.6:0/1582370498 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3c4a90 0x56094f3c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.284+0000 7f3b3cd93740 1 --2- 172.21.15.6:0/1582370498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3b5c80 0x56094f3c16b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.284+0000 7f3b137fe640 0 client.15231 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:05.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.284+0000 7f3b3cd93740 1 --2- 172.21.15.6:0/1582370498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56094f3b5360 0x56094f3c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.284+0000 7f3b3cd93740 1 -- 172.21.15.6:0/1582370498 >> 172.21.15.6:0/1582370498 conn(0x56094f3b1a30 msgr2=0x56094f3b7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:05.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.284+0000 7f3b3cd93740 1 -- 172.21.15.6:0/1582370498 shutdown_connections 2024-09-12T03:21:05.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.284+0000 7f3b3cd93740 1 -- 172.21.15.6:0/1582370498 wait complete. 2024-09-12T03:21:05.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.284+0000 7f3b3cd93740 20 client.15231 trim_cache size 0 max 16384 2024-09-12T03:21:05.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.302+0000 7f3b3cd93740 1 Processor -- start 2024-09-12T03:21:05.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.302+0000 7f3b3cd93740 1 -- start start 2024-09-12T03:21:05.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.302+0000 7f3b3cd93740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56094f3b5360 0x56094f3d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.302+0000 7f3b3cd93740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3b5c80 0x56094f3dc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.302+0000 7f3b3cd93740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3c4a90 0x56094f3dc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.302+0000 7f3b3cd93740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3b341ada20 con 0x56094f3c4a90 2024-09-12T03:21:05.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.302+0000 7f3b3cd93740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f3b341ab2f0 con 0x56094f3b5c80 2024-09-12T03:21:05.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.302+0000 7f3b3cd93740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3b341a9140 con 0x56094f3b5360 2024-09-12T03:21:05.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.302+0000 7f3b38f57640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3c4a90 0x56094f3dc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.302+0000 7f3b38f57640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3c4a90 0x56094f3dc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44966/0 (socket says 172.21.15.6:44966) 2024-09-12T03:21:05.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.302+0000 7f3b38f57640 1 -- 172.21.15.6:0/3568144245 learned_addr learned my addr 172.21.15.6:0/3568144245 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:05.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.302+0000 7f3b33fff640 1 --2- 172.21.15.6:0/3568144245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56094f3b5360 0x56094f3d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.302+0000 7f3b2bfff640 1 --2- 172.21.15.6:0/3568144245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3b5c80 0x56094f3dc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.302+0000 7f3b38f57640 1 -- 172.21.15.6:0/3568144245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56094f3b5360 msgr2=0x56094f3d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:05.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.302+0000 7f3b38f57640 1 --2- 172.21.15.6:0/3568144245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56094f3b5360 0x56094f3d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.302+0000 7f3b38f57640 1 -- 172.21.15.6:0/3568144245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3b5c80 msgr2=0x56094f3dc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:05.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.302+0000 7f3b38f57640 1 --2- 172.21.15.6:0/3568144245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3b5c80 0x56094f3dc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.302+0000 7f3b38f57640 1 -- 172.21.15.6:0/3568144245 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56094f3c1f80 con 0x56094f3c4a90 2024-09-12T03:21:05.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.302+0000 7f3b2bfff640 1 --2- 172.21.15.6:0/3568144245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3b5c80 0x56094f3dc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:05.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.303+0000 7f3b38f57640 1 --2- 172.21.15.6:0/3568144245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3c4a90 0x56094f3dc5d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c007650 tx=0x7f3b2c007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246d3bf46d41d931 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:05.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.303+0000 7f3b29ffb640 1 -- 172.21.15.6:0/3568144245 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c036070 con 0x56094f3c4a90 2024-09-12T03:21:05.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.303+0000 7f3b29ffb640 1 -- 172.21.15.6:0/3568144245 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b2c01a720 con 0x56094f3c4a90 2024-09-12T03:21:05.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.303+0000 7f3b29ffb640 1 -- 172.21.15.6:0/3568144245 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c022040 con 0x56094f3c4a90 2024-09-12T03:21:05.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.303+0000 7f3b3cd93740 1 -- 172.21.15.6:0/3568144245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3c4a90 msgr2=0x56094f3dc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:05.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.303+0000 7f3b3cd93740 1 --2- 172.21.15.6:0/3568144245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3c4a90 0x56094f3dc5d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c007650 tx=0x7f3b2c007f30 comp rx=0 tx=0).stop 2024-09-12T03:21:05.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.303+0000 7f3b3cd93740 1 -- 172.21.15.6:0/3568144245 shutdown_connections 2024-09-12T03:21:05.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.303+0000 7f3b3cd93740 1 --2- 172.21.15.6:0/3568144245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3c4a90 0x56094f3dc5d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.303+0000 7f3b3cd93740 1 --2- 172.21.15.6:0/3568144245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3b5c80 0x56094f3dc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.304+0000 7f3b3cd93740 1 --2- 172.21.15.6:0/3568144245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56094f3b5360 0x56094f3d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.304+0000 7f3b3cd93740 1 -- 172.21.15.6:0/3568144245 >> 172.21.15.6:0/3568144245 conn(0x56094f3b1a30 msgr2=0x56094f3bcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:05.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.304+0000 7f3b3cd93740 1 -- 172.21.15.6:0/3568144245 shutdown_connections 2024-09-12T03:21:05.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.304+0000 7f3b3cd93740 1 -- 172.21.15.6:0/3568144245 wait complete. 2024-09-12T03:21:05.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.304+0000 7f3b3cd93740 1 -- start start 2024-09-12T03:21:05.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.304+0000 7f3b3cd93740 1 Processor -- start 2024-09-12T03:21:05.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.304+0000 7f3b3cd93740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3b5360 0x56094f3cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.304+0000 7f3b3cd93740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3b5c80 0x56094f3ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.304+0000 7f3b3cd93740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56094f3c4a90 0x56094f3c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.304+0000 7f3b3cd93740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3b341a97b0 con 0x56094f3b5360 2024-09-12T03:21:05.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.304+0000 7f3b3cd93740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f3b341aac20 con 0x56094f3b5c80 2024-09-12T03:21:05.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.304+0000 7f3b3cd93740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3b341a9e80 con 0x56094f3c4a90 2024-09-12T03:21:05.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.305+0000 7f3b3cd93740 10 client.0 _ll_register_callbacks cb 0x7f3b1c005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:05.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.305+0000 7f3b33fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3b5360 0x56094f3cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.305+0000 7f3b33fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3b5360 0x56094f3cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44974/0 (socket says 172.21.15.6:44974) 2024-09-12T03:21:05.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.305+0000 7f3b33fff640 1 -- 172.21.15.6:0/2798394683 learned_addr learned my addr 172.21.15.6:0/2798394683 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:05.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.305+0000 7f3b2bfff640 1 --2- 172.21.15.6:0/2798394683 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3b5c80 0x56094f3ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.305+0000 7f3b38f57640 1 --2- 172.21.15.6:0/2798394683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56094f3c4a90 0x56094f3c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.305+0000 7f3b33fff640 1 -- 172.21.15.6:0/2798394683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56094f3c4a90 msgr2=0x56094f3c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:05.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.305+0000 7f3b33fff640 1 --2- 172.21.15.6:0/2798394683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56094f3c4a90 0x56094f3c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.305+0000 7f3b33fff640 1 -- 172.21.15.6:0/2798394683 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3b5c80 msgr2=0x56094f3ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:05.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.305+0000 7f3b33fff640 1 --2- 172.21.15.6:0/2798394683 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3b5c80 0x56094f3ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.305+0000 7f3b33fff640 1 -- 172.21.15.6:0/2798394683 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56094f3c9f80 con 0x56094f3b5360 2024-09-12T03:21:05.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.305+0000 7f3b38f57640 1 --2- 172.21.15.6:0/2798394683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56094f3c4a90 0x56094f3c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:05.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.305+0000 7f3b2bfff640 1 --2- 172.21.15.6:0/2798394683 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3b5c80 0x56094f3ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:05.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.305+0000 7f3b33fff640 1 --2- 172.21.15.6:0/2798394683 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3b5360 0x56094f3cca70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400b800 tx=0x7f3b240035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c8eb84e89c3b1d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:05.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.305+0000 7f3b137fe640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:21:05.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.305+0000 7f3b137fe640 1 -- 172.21.15.6:0/2798394683 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24005220 con 0x56094f3b5360 2024-09-12T03:21:05.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.305+0000 7f3b137fe640 1 -- 172.21.15.6:0/2798394683 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f3b240053c0 con 0x56094f3b5360 2024-09-12T03:21:05.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.305+0000 7f3b3cd93740 1 -- 172.21.15.6:0/2798394683 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b2c01a720 con 0x56094f3b5360 2024-09-12T03:21:05.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.305+0000 7f3b137fe640 1 -- 172.21.15.6:0/2798394683 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3b240055c0 con 0x56094f3b5360 2024-09-12T03:21:05.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.306+0000 7f3b3cd93740 10 client.15237 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:05.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.306+0000 7f3b3cd93740 1 -- 172.21.15.6:0/2798394683 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56094f43f6a0 con 0x56094f3b5360 2024-09-12T03:21:05.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.306+0000 7f3b137fe640 1 -- 172.21.15.6:0/2798394683 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b2c01a720 con 0x56094f3b5360 2024-09-12T03:21:05.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.306+0000 7f3b137fe640 1 -- 172.21.15.6:0/2798394683 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3b24024030 con 0x56094f3b5360 2024-09-12T03:21:05.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.306+0000 7f3b3cd93740 10 client.15237 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:05.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.306+0000 7f3b3cd93740 10 client.15237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:21:05.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.306+0000 7f3b3cd93740 20 client.15237 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:05.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.306+0000 7f3b3cd93740 1 --2- 172.21.15.6:0/2798394683 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56094f43fe20 0x56094f442280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.306+0000 7f3b3cd93740 4 client.15237 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:21:05.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.306+0000 7f3b3cd93740 1 -- 172.21.15.6:0/2798394683 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56094f322d60 con 0x56094f43fe20 2024-09-12T03:21:05.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.306+0000 7f3b2bfff640 1 --2- 172.21.15.6:0/2798394683 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56094f43fe20 0x56094f442280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.307+0000 7f3b2bfff640 1 --2- 172.21.15.6:0/2798394683 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56094f43fe20 0x56094f442280 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d46b22b83d4beb server_cookie=ca9483dd0e6b952b in_seq=0 out_seq=0 2024-09-12T03:21:05.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.307+0000 7f3b137fe640 10 client.15237 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:05.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.307+0000 7f3b137fe640 1 -- 172.21.15.6:0/2798394683 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56094f322d60 con 0x56094f43fe20 2024-09-12T03:21:05.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.307+0000 7f3b137fe640 10 client.15237 handle_command_reply: tid=0 2024-09-12T03:21:05.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.307+0000 7f3b3cd93740 1 client.15237 shutdown 2024-09-12T03:21:05.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.307+0000 7f3b3cd93740 1 -- 172.21.15.6:0/2798394683 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3b5360 msgr2=0x56094f3cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:05.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.307+0000 7f3b3cd93740 1 --2- 172.21.15.6:0/2798394683 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3b5360 0x56094f3cca70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400b800 tx=0x7f3b240035f0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.307+0000 7f3b3cd93740 1 -- 172.21.15.6:0/2798394683 shutdown_connections 2024-09-12T03:21:05.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.307+0000 7f3b3cd93740 1 --2- 172.21.15.6:0/2798394683 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56094f43fe20 0x56094f442280 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.307+0000 7f3b3cd93740 1 --2- 172.21.15.6:0/2798394683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56094f3c4a90 0x56094f3c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.307+0000 7f3b3cd93740 1 --2- 172.21.15.6:0/2798394683 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56094f3b5c80 0x56094f3ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.307+0000 7f3b3cd93740 1 --2- 172.21.15.6:0/2798394683 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56094f3b5360 0x56094f3cca70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.307+0000 7f3b3cd93740 1 -- 172.21.15.6:0/2798394683 >> 172.21.15.6:0/2798394683 conn(0x56094f3b1a30 msgr2=0x56094f3c2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:05.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.307+0000 7f3b137fe640 0 client.15237 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:05.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.307+0000 7f3b3cd93740 1 -- 172.21.15.6:0/2798394683 shutdown_connections 2024-09-12T03:21:05.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.307+0000 7f3b3cd93740 1 -- 172.21.15.6:0/2798394683 wait complete. 2024-09-12T03:21:05.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.307+0000 7f3b3cd93740 20 client.15237 trim_cache size 0 max 16384 2024-09-12T03:21:05.313 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:21:05.313 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:21:05.313 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2024-09-12T03:21:05.313 INFO:teuthology.orchestra.run.smithi006.stdout: "b0b04be6-d1b5-4239-8706-c51f499d089b": { 2024-09-12T03:21:05.314 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-09-12T03:21:05.314 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "b0b04be6-d1b5-4239-8706-c51f499d089b", 2024-09-12T03:21:05.314 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2024-09-12T03:21:05.314 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:21:05.314 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:21:05.314 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:21:05.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.308+0000 7f3b3a9ba640 1 -- 172.21.15.6:0/3598472936 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3b200a2c10 msgr2=0x7f3b200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:05.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.308+0000 7f3b3a9ba640 1 --2- 172.21.15.6:0/3598472936 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3b200a2c10 0x7f3b200a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c01f000 tx=0x7f3b2c020000 comp rx=0 tx=0).stop 2024-09-12T03:21:05.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.308+0000 7f3b3a9ba640 1 -- 172.21.15.6:0/3598472936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b341036c0 msgr2=0x7f3b34198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:05.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.308+0000 7f3b3a9ba640 1 --2- 172.21.15.6:0/3598472936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b341036c0 0x7f3b34198920 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c006010 tx=0x7f3b1c0042f0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.308+0000 7f3b3a9ba640 1 -- 172.21.15.6:0/3598472936 shutdown_connections 2024-09-12T03:21:05.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.308+0000 7f3b3a9ba640 1 --2- 172.21.15.6:0/3598472936 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3b200a2c10 0x7f3b200a50d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.308+0000 7f3b3a9ba640 1 --2- 172.21.15.6:0/3598472936 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b3410c180 0x7f3b3419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.308+0000 7f3b3a9ba640 1 --2- 172.21.15.6:0/3598472936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b341036c0 0x7f3b34198920 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.308+0000 7f3b3a9ba640 1 --2- 172.21.15.6:0/3598472936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b34102da0 0x7f3b341983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.308+0000 7f3b3a9ba640 1 -- 172.21.15.6:0/3598472936 >> 172.21.15.6:0/3598472936 conn(0x7f3b340fc7a0 msgr2=0x7f3b340fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:05.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.308+0000 7f3b3a9ba640 1 -- 172.21.15.6:0/3598472936 shutdown_connections 2024-09-12T03:21:05.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:05.308+0000 7f3b3a9ba640 1 -- 172.21.15.6:0/3598472936 wait complete. 2024-09-12T03:21:05.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:05 smithi161 ceph-mon[35485]: pgmap v334: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-12T03:21:05.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:05 smithi118 ceph-mon[35048]: pgmap v334: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-12T03:21:05.897 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-09-12T03:21:03.265684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664447,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":780,"num_read_kb":516,"num_write":365,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382612,"kb_used_data":12564,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559212,"statfs":{"total":1151940427776,"available":1151548633088,"internally_reserved":0,"allocated":12865536,"data_stored":7186593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9937,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":93,"num_read_kb":60,"num_write":17,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002974"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265565+0000","last_change":"2024-09-12T03:19:09.343829+0000","last_active":"2024-09-12T03:20:49.265565+0000","last_peered":"2024-09-12T03:20:49.265565+0000","last_clean":"2024-09-12T03:20:49.265565+0000","last_became_active":"2024-09-12T03:19:09.343228+0000","last_became_peered":"2024-09-12T03:19:09.343228+0000","last_unstale":"2024-09-12T03:20:49.265565+0000","last_undegraded":"2024-09-12T03:20:49.265565+0000","last_fullsized":"2024-09-12T03:20:49.265565+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:08.720934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.237141+0000","last_change":"2024-09-12T03:19:09.331829+0000","last_active":"2024-09-12T03:20:49.237141+0000","last_peered":"2024-09-12T03:20:49.237141+0000","last_clean":"2024-09-12T03:20:49.237141+0000","last_became_active":"2024-09-12T03:19:09.331276+0000","last_became_peered":"2024-09-12T03:19:09.331276+0000","last_unstale":"2024-09-12T03:20:49.237141+0000","last_undegraded":"2024-09-12T03:20:49.237141+0000","last_fullsized":"2024-09-12T03:20:49.237141+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:48.672085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.263951+0000","last_change":"2024-09-12T03:19:09.338120+0000","last_active":"2024-09-12T03:20:49.263951+0000","last_peered":"2024-09-12T03:20:49.263951+0000","last_clean":"2024-09-12T03:20:49.263951+0000","last_became_active":"2024-09-12T03:19:09.337773+0000","last_became_peered":"2024-09-12T03:19:09.337773+0000","last_unstale":"2024-09-12T03:20:49.263951+0000","last_undegraded":"2024-09-12T03:20:49.263951+0000","last_fullsized":"2024-09-12T03:20:49.263951+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:13.716047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254592+0000","last_change":"2024-09-12T03:19:09.339404+0000","last_active":"2024-09-12T03:20:49.254592+0000","last_peered":"2024-09-12T03:20:49.254592+0000","last_clean":"2024-09-12T03:20:49.254592+0000","last_became_active":"2024-09-12T03:19:09.339105+0000","last_became_peered":"2024-09-12T03:19:09.339105+0000","last_unstale":"2024-09-12T03:20:49.254592+0000","last_undegraded":"2024-09-12T03:20:49.254592+0000","last_fullsized":"2024-09-12T03:20:49.254592+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:37.930963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.242261+0000","last_change":"2024-09-12T03:19:09.326285+0000","last_active":"2024-09-12T03:20:49.242261+0000","last_peered":"2024-09-12T03:20:49.242261+0000","last_clean":"2024-09-12T03:20:49.242261+0000","last_became_active":"2024-09-12T03:19:09.325852+0000","last_became_peered":"2024-09-12T03:19:09.325852+0000","last_unstale":"2024-09-12T03:20:49.242261+0000","last_undegraded":"2024-09-12T03:20:49.242261+0000","last_fullsized":"2024-09-12T03:20:49.242261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:44:25.387014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.249918+0000","last_change":"2024-09-12T03:19:20.376197+0000","last_active":"2024-09-12T03:20:49.249918+0000","last_peered":"2024-09-12T03:20:49.249918+0000","last_clean":"2024-09-12T03:20:49.249918+0000","last_became_active":"2024-09-12T03:19:20.375744+0000","last_became_peered":"2024-09-12T03:19:20.375744+0000","last_unstale":"2024-09-12T03:20:49.249918+0000","last_undegraded":"2024-09-12T03:20:49.249918+0000","last_fullsized":"2024-09-12T03:20:49.249918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:08.663121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.245988+0000","last_change":"2024-09-12T03:19:09.335284+0000","last_active":"2024-09-12T03:20:49.245988+0000","last_peered":"2024-09-12T03:20:49.245988+0000","last_clean":"2024-09-12T03:20:49.245988+0000","last_became_active":"2024-09-12T03:19:09.334948+0000","last_became_peered":"2024-09-12T03:19:09.334948+0000","last_unstale":"2024-09-12T03:20:49.245988+0000","last_undegraded":"2024-09-12T03:20:49.245988+0000","last_fullsized":"2024-09-12T03:20:49.245988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:32.778433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250337+0000","last_change":"2024-09-12T03:19:09.338398+0000","last_active":"2024-09-12T03:20:49.250337+0000","last_peered":"2024-09-12T03:20:49.250337+0000","last_clean":"2024-09-12T03:20:49.250337+0000","last_became_active":"2024-09-12T03:19:09.337750+0000","last_became_peered":"2024-09-12T03:19:09.337750+0000","last_unstale":"2024-09-12T03:20:49.250337+0000","last_undegraded":"2024-09-12T03:20:49.250337+0000","last_fullsized":"2024-09-12T03:20:49.250337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:07.542322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265597+0000","last_change":"2024-09-12T03:19:09.340703+0000","last_active":"2024-09-12T03:20:49.265597+0000","last_peered":"2024-09-12T03:20:49.265597+0000","last_clean":"2024-09-12T03:20:49.265597+0000","last_became_active":"2024-09-12T03:19:09.340386+0000","last_became_peered":"2024-09-12T03:19:09.340386+0000","last_unstale":"2024-09-12T03:20:49.265597+0000","last_undegraded":"2024-09-12T03:20:49.265597+0000","last_fullsized":"2024-09-12T03:20:49.265597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:19.990894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248198+0000","last_change":"2024-09-12T03:19:09.341489+0000","last_active":"2024-09-12T03:20:49.248198+0000","last_peered":"2024-09-12T03:20:49.248198+0000","last_clean":"2024-09-12T03:20:49.248198+0000","last_became_active":"2024-09-12T03:19:09.341183+0000","last_became_peered":"2024-09-12T03:19:09.341183+0000","last_unstale":"2024-09-12T03:20:49.248198+0000","last_undegraded":"2024-09-12T03:20:49.248198+0000","last_fullsized":"2024-09-12T03:20:49.248198+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:05.688402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.255447+0000","last_change":"2024-09-12T03:19:09.339289+0000","last_active":"2024-09-12T03:20:49.255447+0000","last_peered":"2024-09-12T03:20:49.255447+0000","last_clean":"2024-09-12T03:20:49.255447+0000","last_became_active":"2024-09-12T03:19:09.338466+0000","last_became_peered":"2024-09-12T03:19:09.338466+0000","last_unstale":"2024-09-12T03:20:49.255447+0000","last_undegraded":"2024-09-12T03:20:49.255447+0000","last_fullsized":"2024-09-12T03:20:49.255447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:47:13.070057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254364+0000","last_change":"2024-09-12T03:19:09.337754+0000","last_active":"2024-09-12T03:20:49.254364+0000","last_peered":"2024-09-12T03:20:49.254364+0000","last_clean":"2024-09-12T03:20:49.254364+0000","last_became_active":"2024-09-12T03:19:09.332789+0000","last_became_peered":"2024-09-12T03:19:09.332789+0000","last_unstale":"2024-09-12T03:20:49.254364+0000","last_undegraded":"2024-09-12T03:20:49.254364+0000","last_fullsized":"2024-09-12T03:20:49.254364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:34.622136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250957+0000","last_change":"2024-09-12T03:19:09.333577+0000","last_active":"2024-09-12T03:20:49.250957+0000","last_peered":"2024-09-12T03:20:49.250957+0000","last_clean":"2024-09-12T03:20:49.250957+0000","last_became_active":"2024-09-12T03:19:09.333234+0000","last_became_peered":"2024-09-12T03:19:09.333234+0000","last_unstale":"2024-09-12T03:20:49.250957+0000","last_undegraded":"2024-09-12T03:20:49.250957+0000","last_fullsized":"2024-09-12T03:20:49.250957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:22.604640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.237367+0000","last_change":"2024-09-12T03:19:09.331607+0000","last_active":"2024-09-12T03:20:49.237367+0000","last_peered":"2024-09-12T03:20:49.237367+0000","last_clean":"2024-09-12T03:20:49.237367+0000","last_became_active":"2024-09-12T03:19:09.331253+0000","last_became_peered":"2024-09-12T03:19:09.331253+0000","last_unstale":"2024-09-12T03:20:49.237367+0000","last_undegraded":"2024-09-12T03:20:49.237367+0000","last_fullsized":"2024-09-12T03:20:49.237367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:11.969265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256379+0000","last_change":"2024-09-12T03:19:09.340768+0000","last_active":"2024-09-12T03:20:49.256379+0000","last_peered":"2024-09-12T03:20:49.256379+0000","last_clean":"2024-09-12T03:20:49.256379+0000","last_became_active":"2024-09-12T03:19:09.340437+0000","last_became_peered":"2024-09-12T03:19:09.340437+0000","last_unstale":"2024-09-12T03:20:49.256379+0000","last_undegraded":"2024-09-12T03:20:49.256379+0000","last_fullsized":"2024-09-12T03:20:49.256379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:32.645691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.263546+0000","last_change":"2024-09-12T03:19:09.333829+0000","last_active":"2024-09-12T03:20:49.263546+0000","last_peered":"2024-09-12T03:20:49.263546+0000","last_clean":"2024-09-12T03:20:49.263546+0000","last_became_active":"2024-09-12T03:19:09.333462+0000","last_became_peered":"2024-09-12T03:19:09.333462+0000","last_unstale":"2024-09-12T03:20:49.263546+0000","last_undegraded":"2024-09-12T03:20:49.263546+0000","last_fullsized":"2024-09-12T03:20:49.263546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:18:57.586836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265659+0000","last_change":"2024-09-12T03:19:09.344068+0000","last_active":"2024-09-12T03:20:49.265659+0000","last_peered":"2024-09-12T03:20:49.265659+0000","last_clean":"2024-09-12T03:20:49.265659+0000","last_became_active":"2024-09-12T03:19:09.343751+0000","last_became_peered":"2024-09-12T03:19:09.343751+0000","last_unstale":"2024-09-12T03:20:49.265659+0000","last_undegraded":"2024-09-12T03:20:49.265659+0000","last_fullsized":"2024-09-12T03:20:49.265659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:52.425547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.249413+0000","last_change":"2024-09-12T03:19:09.330828+0000","last_active":"2024-09-12T03:20:49.249413+0000","last_peered":"2024-09-12T03:20:49.249413+0000","last_clean":"2024-09-12T03:20:49.249413+0000","last_became_active":"2024-09-12T03:19:09.330497+0000","last_became_peered":"2024-09-12T03:19:09.330497+0000","last_unstale":"2024-09-12T03:20:49.249413+0000","last_undegraded":"2024-09-12T03:20:49.249413+0000","last_fullsized":"2024-09-12T03:20:49.249413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:21:20.064177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.245548+0000","last_change":"2024-09-12T03:19:09.344403+0000","last_active":"2024-09-12T03:20:49.245548+0000","last_peered":"2024-09-12T03:20:49.245548+0000","last_clean":"2024-09-12T03:20:49.245548+0000","last_became_active":"2024-09-12T03:19:09.343946+0000","last_became_peered":"2024-09-12T03:19:09.343946+0000","last_unstale":"2024-09-12T03:20:49.245548+0000","last_undegraded":"2024-09-12T03:20:49.245548+0000","last_fullsized":"2024-09-12T03:20:49.245548+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:48.386117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248484+0000","last_change":"2024-09-12T03:19:09.341481+0000","last_active":"2024-09-12T03:20:49.248484+0000","last_peered":"2024-09-12T03:20:49.248484+0000","last_clean":"2024-09-12T03:20:49.248484+0000","last_became_active":"2024-09-12T03:19:09.341142+0000","last_became_peered":"2024-09-12T03:19:09.341142+0000","last_unstale":"2024-09-12T03:20:49.248484+0000","last_undegraded":"2024-09-12T03:20:49.248484+0000","last_fullsized":"2024-09-12T03:20:49.248484+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:42.371221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.249677+0000","last_change":"2024-09-12T03:19:09.330621+0000","last_active":"2024-09-12T03:20:49.249677+0000","last_peered":"2024-09-12T03:20:49.249677+0000","last_clean":"2024-09-12T03:20:49.249677+0000","last_became_active":"2024-09-12T03:19:09.330169+0000","last_became_peered":"2024-09-12T03:19:09.330169+0000","last_unstale":"2024-09-12T03:20:49.249677+0000","last_undegraded":"2024-09-12T03:20:49.249677+0000","last_fullsized":"2024-09-12T03:20:49.249677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:16.634141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.264220+0000","last_change":"2024-09-12T03:19:09.332817+0000","last_active":"2024-09-12T03:20:49.264220+0000","last_peered":"2024-09-12T03:20:49.264220+0000","last_clean":"2024-09-12T03:20:49.264220+0000","last_became_active":"2024-09-12T03:19:09.332236+0000","last_became_peered":"2024-09-12T03:19:09.332236+0000","last_unstale":"2024-09-12T03:20:49.264220+0000","last_undegraded":"2024-09-12T03:20:49.264220+0000","last_fullsized":"2024-09-12T03:20:49.264220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:28.024294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252477+0000","last_change":"2024-09-12T03:19:09.337910+0000","last_active":"2024-09-12T03:20:49.252477+0000","last_peered":"2024-09-12T03:20:49.252477+0000","last_clean":"2024-09-12T03:20:49.252477+0000","last_became_active":"2024-09-12T03:19:09.337176+0000","last_became_peered":"2024-09-12T03:19:09.337176+0000","last_unstale":"2024-09-12T03:20:49.252477+0000","last_undegraded":"2024-09-12T03:20:49.252477+0000","last_fullsized":"2024-09-12T03:20:49.252477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:53.795898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.239957+0000","last_change":"2024-09-12T03:19:09.337902+0000","last_active":"2024-09-12T03:20:49.239957+0000","last_peered":"2024-09-12T03:20:49.239957+0000","last_clean":"2024-09-12T03:20:49.239957+0000","last_became_active":"2024-09-12T03:19:09.337567+0000","last_became_peered":"2024-09-12T03:19:09.337567+0000","last_unstale":"2024-09-12T03:20:49.239957+0000","last_undegraded":"2024-09-12T03:20:49.239957+0000","last_fullsized":"2024-09-12T03:20:49.239957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:08.471313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248557+0000","last_change":"2024-09-12T03:19:09.339850+0000","last_active":"2024-09-12T03:20:49.248557+0000","last_peered":"2024-09-12T03:20:49.248557+0000","last_clean":"2024-09-12T03:20:49.248557+0000","last_became_active":"2024-09-12T03:19:09.339563+0000","last_became_peered":"2024-09-12T03:19:09.339563+0000","last_unstale":"2024-09-12T03:20:49.248557+0000","last_undegraded":"2024-09-12T03:20:49.248557+0000","last_fullsized":"2024-09-12T03:20:49.248557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:27.473351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.247769+0000","last_change":"2024-09-12T03:19:09.330555+0000","last_active":"2024-09-12T03:20:49.247769+0000","last_peered":"2024-09-12T03:20:49.247769+0000","last_clean":"2024-09-12T03:20:49.247769+0000","last_became_active":"2024-09-12T03:19:09.330183+0000","last_became_peered":"2024-09-12T03:19:09.330183+0000","last_unstale":"2024-09-12T03:20:49.247769+0000","last_undegraded":"2024-09-12T03:20:49.247769+0000","last_fullsized":"2024-09-12T03:20:49.247769+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:39.334116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252190+0000","last_change":"2024-09-12T03:19:09.338601+0000","last_active":"2024-09-12T03:20:49.252190+0000","last_peered":"2024-09-12T03:20:49.252190+0000","last_clean":"2024-09-12T03:20:49.252190+0000","last_became_active":"2024-09-12T03:19:09.337903+0000","last_became_peered":"2024-09-12T03:19:09.337903+0000","last_unstale":"2024-09-12T03:20:49.252190+0000","last_undegraded":"2024-09-12T03:20:49.252190+0000","last_fullsized":"2024-09-12T03:20:49.252190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:17.810139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250990+0000","last_change":"2024-09-12T03:19:09.326629+0000","last_active":"2024-09-12T03:20:49.250990+0000","last_peered":"2024-09-12T03:20:49.250990+0000","last_clean":"2024-09-12T03:20:49.250990+0000","last_became_active":"2024-09-12T03:19:09.326333+0000","last_became_peered":"2024-09-12T03:19:09.326333+0000","last_unstale":"2024-09-12T03:20:49.250990+0000","last_undegraded":"2024-09-12T03:20:49.250990+0000","last_fullsized":"2024-09-12T03:20:49.250990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:20.807593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256412+0000","last_change":"2024-09-12T03:19:09.339919+0000","last_active":"2024-09-12T03:20:49.256412+0000","last_peered":"2024-09-12T03:20:49.256412+0000","last_clean":"2024-09-12T03:20:49.256412+0000","last_became_active":"2024-09-12T03:19:09.339618+0000","last_became_peered":"2024-09-12T03:19:09.339618+0000","last_unstale":"2024-09-12T03:20:49.256412+0000","last_undegraded":"2024-09-12T03:20:49.256412+0000","last_fullsized":"2024-09-12T03:20:49.256412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:11:03.740079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.246446+0000","last_change":"2024-09-12T03:19:09.336148+0000","last_active":"2024-09-12T03:20:49.246446+0000","last_peered":"2024-09-12T03:20:49.246446+0000","last_clean":"2024-09-12T03:20:49.246446+0000","last_became_active":"2024-09-12T03:19:09.335779+0000","last_became_peered":"2024-09-12T03:19:09.335779+0000","last_unstale":"2024-09-12T03:20:49.246446+0000","last_undegraded":"2024-09-12T03:20:49.246446+0000","last_fullsized":"2024-09-12T03:20:49.246446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:18.234320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.247971+0000","last_change":"2024-09-12T03:19:20.382536+0000","last_active":"2024-09-12T03:20:49.247971+0000","last_peered":"2024-09-12T03:20:49.247971+0000","last_clean":"2024-09-12T03:20:49.247971+0000","last_became_active":"2024-09-12T03:19:20.382118+0000","last_became_peered":"2024-09-12T03:19:20.382118+0000","last_unstale":"2024-09-12T03:20:49.247971+0000","last_undegraded":"2024-09-12T03:20:49.247971+0000","last_fullsized":"2024-09-12T03:20:49.247971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:55.836646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252455+0000","last_change":"2024-09-12T03:19:09.345064+0000","last_active":"2024-09-12T03:20:49.252455+0000","last_peered":"2024-09-12T03:20:49.252455+0000","last_clean":"2024-09-12T03:20:49.252455+0000","last_became_active":"2024-09-12T03:19:09.344628+0000","last_became_peered":"2024-09-12T03:19:09.344628+0000","last_unstale":"2024-09-12T03:20:49.252455+0000","last_undegraded":"2024-09-12T03:20:49.252455+0000","last_fullsized":"2024-09-12T03:20:49.252455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:50.011751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.244081+0000","last_change":"2024-09-12T03:19:09.338052+0000","last_active":"2024-09-12T03:20:49.244081+0000","last_peered":"2024-09-12T03:20:49.244081+0000","last_clean":"2024-09-12T03:20:49.244081+0000","last_became_active":"2024-09-12T03:19:09.337710+0000","last_became_peered":"2024-09-12T03:19:09.337710+0000","last_unstale":"2024-09-12T03:20:49.244081+0000","last_undegraded":"2024-09-12T03:20:49.244081+0000","last_fullsized":"2024-09-12T03:20:49.244081+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:36.521269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'9","reported_seq":139,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:59.526609+0000","last_change":"2024-09-12T03:19:07.208341+0000","last_active":"2024-09-12T03:20:59.526609+0000","last_peered":"2024-09-12T03:20:59.526609+0000","last_clean":"2024-09-12T03:20:59.526609+0000","last_became_active":"2024-09-12T03:19:07.207385+0000","last_became_peered":"2024-09-12T03:19:07.207385+0000","last_unstale":"2024-09-12T03:20:59.526609+0000","last_undegraded":"2024-09-12T03:20:59.526609+0000","last_fullsized":"2024-09-12T03:20:59.526609+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:18.476795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1857,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254115+0000","last_change":"2024-09-12T03:19:09.339997+0000","last_active":"2024-09-12T03:20:49.254115+0000","last_peered":"2024-09-12T03:20:49.254115+0000","last_clean":"2024-09-12T03:20:49.254115+0000","last_became_active":"2024-09-12T03:19:09.339261+0000","last_became_peered":"2024-09-12T03:19:09.339261+0000","last_unstale":"2024-09-12T03:20:49.254115+0000","last_undegraded":"2024-09-12T03:20:49.254115+0000","last_fullsized":"2024-09-12T03:20:49.254115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:07.670345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374030+0000","last_change":"2024-09-12T03:19:07.200222+0000","last_active":"2024-09-12T03:19:20.374030+0000","last_peered":"2024-09-12T03:19:20.374030+0000","last_clean":"2024-09-12T03:19:20.374030+0000","last_became_active":"2024-09-12T03:19:07.199819+0000","last_became_peered":"2024-09-12T03:19:07.199819+0000","last_unstale":"2024-09-12T03:19:20.374030+0000","last_undegraded":"2024-09-12T03:19:20.374030+0000","last_fullsized":"2024-09-12T03:19:20.374030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:39:28.933609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375686+0000","last_change":"2024-09-12T03:19:07.207005+0000","last_active":"2024-09-12T03:19:20.375686+0000","last_peered":"2024-09-12T03:19:20.375686+0000","last_clean":"2024-09-12T03:19:20.375686+0000","last_became_active":"2024-09-12T03:19:07.206685+0000","last_became_peered":"2024-09-12T03:19:07.206685+0000","last_unstale":"2024-09-12T03:19:20.375686+0000","last_undegraded":"2024-09-12T03:19:20.375686+0000","last_fullsized":"2024-09-12T03:19:20.375686+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:31.111706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252939+0000","last_change":"2024-09-12T03:19:09.335871+0000","last_active":"2024-09-12T03:20:49.252939+0000","last_peered":"2024-09-12T03:20:49.252939+0000","last_clean":"2024-09-12T03:20:49.252939+0000","last_became_active":"2024-09-12T03:19:09.335568+0000","last_became_peered":"2024-09-12T03:19:09.335568+0000","last_unstale":"2024-09-12T03:20:49.252939+0000","last_undegraded":"2024-09-12T03:20:49.252939+0000","last_fullsized":"2024-09-12T03:20:49.252939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:52:34.471512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386049+0000","last_change":"2024-09-12T03:19:07.208320+0000","last_active":"2024-09-12T03:19:20.386049+0000","last_peered":"2024-09-12T03:19:20.386049+0000","last_clean":"2024-09-12T03:19:20.386049+0000","last_became_active":"2024-09-12T03:19:07.207873+0000","last_became_peered":"2024-09-12T03:19:07.207873+0000","last_unstale":"2024-09-12T03:19:20.386049+0000","last_undegraded":"2024-09-12T03:19:20.386049+0000","last_fullsized":"2024-09-12T03:19:20.386049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:00.128595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250364+0000","last_change":"2024-09-12T03:19:09.331930+0000","last_active":"2024-09-12T03:20:49.250364+0000","last_peered":"2024-09-12T03:20:49.250364+0000","last_clean":"2024-09-12T03:20:49.250364+0000","last_became_active":"2024-09-12T03:19:09.331558+0000","last_became_peered":"2024-09-12T03:19:09.331558+0000","last_unstale":"2024-09-12T03:20:49.250364+0000","last_undegraded":"2024-09-12T03:20:49.250364+0000","last_fullsized":"2024-09-12T03:20:49.250364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:00:27.935717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":130,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.529083+0000","last_change":"2024-09-12T03:19:07.205473+0000","last_active":"2024-09-12T03:21:01.529083+0000","last_peered":"2024-09-12T03:21:01.529083+0000","last_clean":"2024-09-12T03:21:01.529083+0000","last_became_active":"2024-09-12T03:19:07.205134+0000","last_became_peered":"2024-09-12T03:19:07.205134+0000","last_unstale":"2024-09-12T03:21:01.529083+0000","last_undegraded":"2024-09-12T03:21:01.529083+0000","last_fullsized":"2024-09-12T03:21:01.529083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:23.443957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265317+0000","last_change":"2024-09-12T03:19:09.333868+0000","last_active":"2024-09-12T03:20:49.265317+0000","last_peered":"2024-09-12T03:20:49.265317+0000","last_clean":"2024-09-12T03:20:49.265317+0000","last_became_active":"2024-09-12T03:19:09.333484+0000","last_became_peered":"2024-09-12T03:19:09.333484+0000","last_unstale":"2024-09-12T03:20:49.265317+0000","last_undegraded":"2024-09-12T03:20:49.265317+0000","last_fullsized":"2024-09-12T03:20:49.265317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:21:10.226821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386303+0000","last_change":"2024-09-12T03:19:20.386303+0000","last_active":"2024-09-12T03:19:20.386303+0000","last_peered":"2024-09-12T03:19:20.386303+0000","last_clean":"2024-09-12T03:19:20.386303+0000","last_became_active":"2024-09-12T03:19:20.383198+0000","last_became_peered":"2024-09-12T03:19:20.383198+0000","last_unstale":"2024-09-12T03:19:20.386303+0000","last_undegraded":"2024-09-12T03:19:20.386303+0000","last_fullsized":"2024-09-12T03:19:20.386303+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:42.224849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.257056+0000","last_change":"2024-09-12T03:19:09.340849+0000","last_active":"2024-09-12T03:20:49.257056+0000","last_peered":"2024-09-12T03:20:49.257056+0000","last_clean":"2024-09-12T03:20:49.257056+0000","last_became_active":"2024-09-12T03:19:09.340517+0000","last_became_peered":"2024-09-12T03:19:09.340517+0000","last_unstale":"2024-09-12T03:20:49.257056+0000","last_undegraded":"2024-09-12T03:20:49.257056+0000","last_fullsized":"2024-09-12T03:20:49.257056+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:42.018269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386240+0000","last_change":"2024-09-12T03:19:07.389147+0000","last_active":"2024-09-12T03:19:20.386240+0000","last_peered":"2024-09-12T03:19:20.386240+0000","last_clean":"2024-09-12T03:19:20.386240+0000","last_became_active":"2024-09-12T03:19:07.388744+0000","last_became_peered":"2024-09-12T03:19:07.388744+0000","last_unstale":"2024-09-12T03:19:20.386240+0000","last_undegraded":"2024-09-12T03:19:20.386240+0000","last_fullsized":"2024-09-12T03:19:20.386240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:16.700644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.264622+0000","last_change":"2024-09-12T03:19:09.329243+0000","last_active":"2024-09-12T03:20:49.264622+0000","last_peered":"2024-09-12T03:20:49.264622+0000","last_clean":"2024-09-12T03:20:49.264622+0000","last_became_active":"2024-09-12T03:19:09.328610+0000","last_became_peered":"2024-09-12T03:19:09.328610+0000","last_unstale":"2024-09-12T03:20:49.264622+0000","last_undegraded":"2024-09-12T03:20:49.264622+0000","last_fullsized":"2024-09-12T03:20:49.264622+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:49.452942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'34","reported_seq":276,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:21:02.531095+0000","last_change":"2024-09-12T03:19:07.207804+0000","last_active":"2024-09-12T03:21:02.531095+0000","last_peered":"2024-09-12T03:21:02.531095+0000","last_clean":"2024-09-12T03:21:02.531095+0000","last_became_active":"2024-09-12T03:19:07.207409+0000","last_became_peered":"2024-09-12T03:19:07.207409+0000","last_unstale":"2024-09-12T03:21:02.531095+0000","last_undegraded":"2024-09-12T03:21:02.531095+0000","last_fullsized":"2024-09-12T03:21:02.531095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:10.732423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68607,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":220,"num_read_kb":177,"num_write":35,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.255978+0000","last_change":"2024-09-12T03:19:09.333102+0000","last_active":"2024-09-12T03:20:49.255978+0000","last_peered":"2024-09-12T03:20:49.255978+0000","last_clean":"2024-09-12T03:20:49.255978+0000","last_became_active":"2024-09-12T03:19:09.332712+0000","last_became_peered":"2024-09-12T03:19:09.332712+0000","last_unstale":"2024-09-12T03:20:49.255978+0000","last_undegraded":"2024-09-12T03:20:49.255978+0000","last_fullsized":"2024-09-12T03:20:49.255978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:22.912213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382902+0000","last_change":"2024-09-12T03:19:20.382348+0000","last_active":"2024-09-12T03:19:20.382902+0000","last_peered":"2024-09-12T03:19:20.382902+0000","last_clean":"2024-09-12T03:19:20.382902+0000","last_became_active":"2024-09-12T03:19:20.375142+0000","last_became_peered":"2024-09-12T03:19:20.375142+0000","last_unstale":"2024-09-12T03:19:20.382902+0000","last_undegraded":"2024-09-12T03:19:20.382902+0000","last_fullsized":"2024-09-12T03:19:20.382902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:04.573599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.246879+0000","last_change":"2024-09-12T03:19:09.333194+0000","last_active":"2024-09-12T03:20:49.246879+0000","last_peered":"2024-09-12T03:20:49.246879+0000","last_clean":"2024-09-12T03:20:49.246879+0000","last_became_active":"2024-09-12T03:19:09.332865+0000","last_became_peered":"2024-09-12T03:19:09.332865+0000","last_unstale":"2024-09-12T03:20:49.246879+0000","last_undegraded":"2024-09-12T03:20:49.246879+0000","last_fullsized":"2024-09-12T03:20:49.246879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:57.096138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386176+0000","last_change":"2024-09-12T03:19:07.209289+0000","last_active":"2024-09-12T03:19:20.386176+0000","last_peered":"2024-09-12T03:19:20.386176+0000","last_clean":"2024-09-12T03:19:20.386176+0000","last_became_active":"2024-09-12T03:19:07.208863+0000","last_became_peered":"2024-09-12T03:19:07.208863+0000","last_unstale":"2024-09-12T03:19:20.386176+0000","last_undegraded":"2024-09-12T03:19:20.386176+0000","last_fullsized":"2024-09-12T03:19:20.386176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:26.168148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256973+0000","last_change":"2024-09-12T03:19:09.343620+0000","last_active":"2024-09-12T03:20:49.256973+0000","last_peered":"2024-09-12T03:20:49.256973+0000","last_clean":"2024-09-12T03:20:49.256973+0000","last_became_active":"2024-09-12T03:19:09.343299+0000","last_became_peered":"2024-09-12T03:19:09.343299+0000","last_unstale":"2024-09-12T03:20:49.256973+0000","last_undegraded":"2024-09-12T03:20:49.256973+0000","last_fullsized":"2024-09-12T03:20:49.256973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:34.268886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":131,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.559527+0000","last_change":"2024-09-12T03:19:07.206160+0000","last_active":"2024-09-12T03:21:00.559527+0000","last_peered":"2024-09-12T03:21:00.559527+0000","last_clean":"2024-09-12T03:21:00.559527+0000","last_became_active":"2024-09-12T03:19:07.205509+0000","last_became_peered":"2024-09-12T03:19:07.205509+0000","last_unstale":"2024-09-12T03:21:00.559527+0000","last_undegraded":"2024-09-12T03:21:00.559527+0000","last_fullsized":"2024-09-12T03:21:00.559527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:35.362831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250896+0000","last_change":"2024-09-12T03:19:09.337595+0000","last_active":"2024-09-12T03:20:49.250896+0000","last_peered":"2024-09-12T03:20:49.250896+0000","last_clean":"2024-09-12T03:20:49.250896+0000","last_became_active":"2024-09-12T03:19:09.337281+0000","last_became_peered":"2024-09-12T03:19:09.337281+0000","last_unstale":"2024-09-12T03:20:49.250896+0000","last_undegraded":"2024-09-12T03:20:49.250896+0000","last_fullsized":"2024-09-12T03:20:49.250896+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:35.761679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373995+0000","last_change":"2024-09-12T03:19:07.389336+0000","last_active":"2024-09-12T03:19:20.373995+0000","last_peered":"2024-09-12T03:19:20.373995+0000","last_clean":"2024-09-12T03:19:20.373995+0000","last_became_active":"2024-09-12T03:19:07.388649+0000","last_became_peered":"2024-09-12T03:19:07.388649+0000","last_unstale":"2024-09-12T03:19:20.373995+0000","last_undegraded":"2024-09-12T03:19:20.373995+0000","last_fullsized":"2024-09-12T03:19:20.373995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:33.312700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.253821+0000","last_change":"2024-09-12T03:19:09.340674+0000","last_active":"2024-09-12T03:20:49.253821+0000","last_peered":"2024-09-12T03:20:49.253821+0000","last_clean":"2024-09-12T03:20:49.253821+0000","last_became_active":"2024-09-12T03:19:09.340144+0000","last_became_peered":"2024-09-12T03:19:09.340144+0000","last_unstale":"2024-09-12T03:20:49.253821+0000","last_undegraded":"2024-09-12T03:20:49.253821+0000","last_fullsized":"2024-09-12T03:20:49.253821+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:21:15.605086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373837+0000","last_change":"2024-09-12T03:19:07.206513+0000","last_active":"2024-09-12T03:19:20.373837+0000","last_peered":"2024-09-12T03:19:20.373837+0000","last_clean":"2024-09-12T03:19:20.373837+0000","last_became_active":"2024-09-12T03:19:07.205974+0000","last_became_peered":"2024-09-12T03:19:07.205974+0000","last_unstale":"2024-09-12T03:19:20.373837+0000","last_undegraded":"2024-09-12T03:19:20.373837+0000","last_fullsized":"2024-09-12T03:19:20.373837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:43:10.002123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252033+0000","last_change":"2024-09-12T03:19:09.342200+0000","last_active":"2024-09-12T03:20:49.252033+0000","last_peered":"2024-09-12T03:20:49.252033+0000","last_clean":"2024-09-12T03:20:49.252033+0000","last_became_active":"2024-09-12T03:19:09.341901+0000","last_became_peered":"2024-09-12T03:19:09.341901+0000","last_unstale":"2024-09-12T03:20:49.252033+0000","last_undegraded":"2024-09-12T03:20:49.252033+0000","last_fullsized":"2024-09-12T03:20:49.252033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:27.486214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375373+0000","last_change":"2024-09-12T03:19:07.207514+0000","last_active":"2024-09-12T03:19:20.375373+0000","last_peered":"2024-09-12T03:19:20.375373+0000","last_clean":"2024-09-12T03:19:20.375373+0000","last_became_active":"2024-09-12T03:19:07.205925+0000","last_became_peered":"2024-09-12T03:19:07.205925+0000","last_unstale":"2024-09-12T03:19:20.375373+0000","last_undegraded":"2024-09-12T03:19:20.375373+0000","last_fullsized":"2024-09-12T03:19:20.375373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:20.244703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256005+0000","last_change":"2024-09-12T03:19:09.337837+0000","last_active":"2024-09-12T03:20:49.256005+0000","last_peered":"2024-09-12T03:20:49.256005+0000","last_clean":"2024-09-12T03:20:49.256005+0000","last_became_active":"2024-09-12T03:19:09.337502+0000","last_became_peered":"2024-09-12T03:19:09.337502+0000","last_unstale":"2024-09-12T03:20:49.256005+0000","last_undegraded":"2024-09-12T03:20:49.256005+0000","last_fullsized":"2024-09-12T03:20:49.256005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:30.167139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386356+0000","last_change":"2024-09-12T03:19:07.209130+0000","last_active":"2024-09-12T03:19:20.386356+0000","last_peered":"2024-09-12T03:19:20.386356+0000","last_clean":"2024-09-12T03:19:20.386356+0000","last_became_active":"2024-09-12T03:19:07.208753+0000","last_became_peered":"2024-09-12T03:19:07.208753+0000","last_unstale":"2024-09-12T03:19:20.386356+0000","last_undegraded":"2024-09-12T03:19:20.386356+0000","last_fullsized":"2024-09-12T03:19:20.386356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:34.719292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250623+0000","last_change":"2024-09-12T03:19:09.336382+0000","last_active":"2024-09-12T03:20:49.250623+0000","last_peered":"2024-09-12T03:20:49.250623+0000","last_clean":"2024-09-12T03:20:49.250623+0000","last_became_active":"2024-09-12T03:19:09.336015+0000","last_became_peered":"2024-09-12T03:19:09.336015+0000","last_unstale":"2024-09-12T03:20:49.250623+0000","last_undegraded":"2024-09-12T03:20:49.250623+0000","last_fullsized":"2024-09-12T03:20:49.250623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:19.430849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375044+0000","last_change":"2024-09-12T03:19:07.528567+0000","last_active":"2024-09-12T03:19:20.375044+0000","last_peered":"2024-09-12T03:19:20.375044+0000","last_clean":"2024-09-12T03:19:20.375044+0000","last_became_active":"2024-09-12T03:19:07.528123+0000","last_became_peered":"2024-09-12T03:19:07.528123+0000","last_unstale":"2024-09-12T03:19:20.375044+0000","last_undegraded":"2024-09-12T03:19:20.375044+0000","last_fullsized":"2024-09-12T03:19:20.375044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:34.416714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256025+0000","last_change":"2024-09-12T03:19:09.338836+0000","last_active":"2024-09-12T03:20:49.256025+0000","last_peered":"2024-09-12T03:20:49.256025+0000","last_clean":"2024-09-12T03:20:49.256025+0000","last_became_active":"2024-09-12T03:19:09.337571+0000","last_became_peered":"2024-09-12T03:19:09.337571+0000","last_unstale":"2024-09-12T03:20:49.256025+0000","last_undegraded":"2024-09-12T03:20:49.256025+0000","last_fullsized":"2024-09-12T03:20:49.256025+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:39.039878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":330,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372792+0000","last_change":"2024-09-12T03:18:01.051311+0000","last_active":"2024-09-12T03:19:20.372792+0000","last_peered":"2024-09-12T03:19:20.372792+0000","last_clean":"2024-09-12T03:19:20.372792+0000","last_became_active":"2024-09-12T03:18:01.050849+0000","last_became_peered":"2024-09-12T03:18:01.050849+0000","last_unstale":"2024-09-12T03:19:20.372792+0000","last_undegraded":"2024-09-12T03:19:20.372792+0000","last_fullsized":"2024-09-12T03:19:20.372792+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:14:38.096222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:14:38.096222+0000","last_clean_scrub_stamp":"2024-09-12T03:14:38.096222+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:33.067603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382035+0000","last_change":"2024-09-12T03:19:07.206045+0000","last_active":"2024-09-12T03:19:20.382035+0000","last_peered":"2024-09-12T03:19:20.382035+0000","last_clean":"2024-09-12T03:19:20.382035+0000","last_became_active":"2024-09-12T03:19:07.205594+0000","last_became_peered":"2024-09-12T03:19:07.205594+0000","last_unstale":"2024-09-12T03:19:20.382035+0000","last_undegraded":"2024-09-12T03:19:20.382035+0000","last_fullsized":"2024-09-12T03:19:20.382035+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:12.811906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.244733+0000","last_change":"2024-09-12T03:19:20.387274+0000","last_active":"2024-09-12T03:20:49.244733+0000","last_peered":"2024-09-12T03:20:49.244733+0000","last_clean":"2024-09-12T03:20:49.244733+0000","last_became_active":"2024-09-12T03:19:20.386878+0000","last_became_peered":"2024-09-12T03:19:20.386878+0000","last_unstale":"2024-09-12T03:20:49.244733+0000","last_undegraded":"2024-09-12T03:20:49.244733+0000","last_fullsized":"2024-09-12T03:20:49.244733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:26.674032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372302+0000","last_change":"2024-09-12T03:19:07.206212+0000","last_active":"2024-09-12T03:19:20.372302+0000","last_peered":"2024-09-12T03:19:20.372302+0000","last_clean":"2024-09-12T03:19:20.372302+0000","last_became_active":"2024-09-12T03:19:07.205899+0000","last_became_peered":"2024-09-12T03:19:07.205899+0000","last_unstale":"2024-09-12T03:19:20.372302+0000","last_undegraded":"2024-09-12T03:19:20.372302+0000","last_fullsized":"2024-09-12T03:19:20.372302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:02.169436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250795+0000","last_change":"2024-09-12T03:19:20.377494+0000","last_active":"2024-09-12T03:20:49.250795+0000","last_peered":"2024-09-12T03:20:49.250795+0000","last_clean":"2024-09-12T03:20:49.250795+0000","last_became_active":"2024-09-12T03:19:20.377091+0000","last_became_peered":"2024-09-12T03:19:20.377091+0000","last_unstale":"2024-09-12T03:20:49.250795+0000","last_undegraded":"2024-09-12T03:20:49.250795+0000","last_fullsized":"2024-09-12T03:20:49.250795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:30.926853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":130,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.560665+0000","last_change":"2024-09-12T03:19:07.388486+0000","last_active":"2024-09-12T03:21:01.560665+0000","last_peered":"2024-09-12T03:21:01.560665+0000","last_clean":"2024-09-12T03:21:01.560665+0000","last_became_active":"2024-09-12T03:19:07.388125+0000","last_became_peered":"2024-09-12T03:19:07.388125+0000","last_unstale":"2024-09-12T03:21:01.560665+0000","last_undegraded":"2024-09-12T03:21:01.560665+0000","last_fullsized":"2024-09-12T03:21:01.560665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:46.218601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254069+0000","last_change":"2024-09-12T03:19:09.344308+0000","last_active":"2024-09-12T03:20:49.254069+0000","last_peered":"2024-09-12T03:20:49.254069+0000","last_clean":"2024-09-12T03:20:49.254069+0000","last_became_active":"2024-09-12T03:19:09.343987+0000","last_became_peered":"2024-09-12T03:19:09.343987+0000","last_unstale":"2024-09-12T03:20:49.254069+0000","last_undegraded":"2024-09-12T03:20:49.254069+0000","last_fullsized":"2024-09-12T03:20:49.254069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:16:11.616555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372560+0000","last_change":"2024-09-12T03:19:07.208619+0000","last_active":"2024-09-12T03:19:20.372560+0000","last_peered":"2024-09-12T03:19:20.372560+0000","last_clean":"2024-09-12T03:19:20.372560+0000","last_became_active":"2024-09-12T03:19:07.208254+0000","last_became_peered":"2024-09-12T03:19:07.208254+0000","last_unstale":"2024-09-12T03:19:20.372560+0000","last_undegraded":"2024-09-12T03:19:20.372560+0000","last_fullsized":"2024-09-12T03:19:20.372560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:56.177948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.238148+0000","last_change":"2024-09-12T03:19:09.341837+0000","last_active":"2024-09-12T03:20:49.238148+0000","last_peered":"2024-09-12T03:20:49.238148+0000","last_clean":"2024-09-12T03:20:49.238148+0000","last_became_active":"2024-09-12T03:19:09.341069+0000","last_became_peered":"2024-09-12T03:19:09.341069+0000","last_unstale":"2024-09-12T03:20:49.238148+0000","last_undegraded":"2024-09-12T03:20:49.238148+0000","last_fullsized":"2024-09-12T03:20:49.238148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:04.238986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372460+0000","last_change":"2024-09-12T03:19:07.388870+0000","last_active":"2024-09-12T03:19:20.372460+0000","last_peered":"2024-09-12T03:19:20.372460+0000","last_clean":"2024-09-12T03:19:20.372460+0000","last_became_active":"2024-09-12T03:19:07.388555+0000","last_became_peered":"2024-09-12T03:19:07.388555+0000","last_unstale":"2024-09-12T03:19:20.372460+0000","last_undegraded":"2024-09-12T03:19:20.372460+0000","last_fullsized":"2024-09-12T03:19:20.372460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:29.789454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.247542+0000","last_change":"2024-09-12T03:19:09.334714+0000","last_active":"2024-09-12T03:20:49.247542+0000","last_peered":"2024-09-12T03:20:49.247542+0000","last_clean":"2024-09-12T03:20:49.247542+0000","last_became_active":"2024-09-12T03:19:09.334413+0000","last_became_peered":"2024-09-12T03:19:09.334413+0000","last_unstale":"2024-09-12T03:20:49.247542+0000","last_undegraded":"2024-09-12T03:20:49.247542+0000","last_fullsized":"2024-09-12T03:20:49.247542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:52.222340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373797+0000","last_change":"2024-09-12T03:19:07.389055+0000","last_active":"2024-09-12T03:19:20.373797+0000","last_peered":"2024-09-12T03:19:20.373797+0000","last_clean":"2024-09-12T03:19:20.373797+0000","last_became_active":"2024-09-12T03:19:07.388673+0000","last_became_peered":"2024-09-12T03:19:07.388673+0000","last_unstale":"2024-09-12T03:19:20.373797+0000","last_undegraded":"2024-09-12T03:19:20.373797+0000","last_fullsized":"2024-09-12T03:19:20.373797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:20.974533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248932+0000","last_change":"2024-09-12T03:19:09.340105+0000","last_active":"2024-09-12T03:20:49.248932+0000","last_peered":"2024-09-12T03:20:49.248932+0000","last_clean":"2024-09-12T03:20:49.248932+0000","last_became_active":"2024-09-12T03:19:09.339642+0000","last_became_peered":"2024-09-12T03:19:09.339642+0000","last_unstale":"2024-09-12T03:20:49.248932+0000","last_undegraded":"2024-09-12T03:20:49.248932+0000","last_fullsized":"2024-09-12T03:20:49.248932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:01.624236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'15","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:52.343269+0000","last_change":"2024-09-12T03:19:07.207232+0000","last_active":"2024-09-12T03:20:52.343269+0000","last_peered":"2024-09-12T03:20:52.343269+0000","last_clean":"2024-09-12T03:20:52.343269+0000","last_became_active":"2024-09-12T03:19:07.206921+0000","last_became_peered":"2024-09-12T03:19:07.206921+0000","last_unstale":"2024-09-12T03:20:52.343269+0000","last_undegraded":"2024-09-12T03:20:52.343269+0000","last_fullsized":"2024-09-12T03:20:52.343269+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:25.457853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252490+0000","last_change":"2024-09-12T03:19:09.330442+0000","last_active":"2024-09-12T03:20:49.252490+0000","last_peered":"2024-09-12T03:20:49.252490+0000","last_clean":"2024-09-12T03:20:49.252490+0000","last_became_active":"2024-09-12T03:19:09.330081+0000","last_became_peered":"2024-09-12T03:19:09.330081+0000","last_unstale":"2024-09-12T03:20:49.252490+0000","last_undegraded":"2024-09-12T03:20:49.252490+0000","last_fullsized":"2024-09-12T03:20:49.252490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:56.152562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382392+0000","last_change":"2024-09-12T03:19:07.528646+0000","last_active":"2024-09-12T03:19:20.382392+0000","last_peered":"2024-09-12T03:19:20.382392+0000","last_clean":"2024-09-12T03:19:20.382392+0000","last_became_active":"2024-09-12T03:19:07.528334+0000","last_became_peered":"2024-09-12T03:19:07.528334+0000","last_unstale":"2024-09-12T03:19:20.382392+0000","last_undegraded":"2024-09-12T03:19:20.382392+0000","last_fullsized":"2024-09-12T03:19:20.382392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:40.425579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256685+0000","last_change":"2024-09-12T03:19:09.331993+0000","last_active":"2024-09-12T03:20:49.256685+0000","last_peered":"2024-09-12T03:20:49.256685+0000","last_clean":"2024-09-12T03:20:49.256685+0000","last_became_active":"2024-09-12T03:19:09.331698+0000","last_became_peered":"2024-09-12T03:19:09.331698+0000","last_unstale":"2024-09-12T03:20:49.256685+0000","last_undegraded":"2024-09-12T03:20:49.256685+0000","last_fullsized":"2024-09-12T03:20:49.256685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:03:54.182833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370179+0000","last_change":"2024-09-12T03:19:07.388297+0000","last_active":"2024-09-12T03:19:20.370179+0000","last_peered":"2024-09-12T03:19:20.370179+0000","last_clean":"2024-09-12T03:19:20.370179+0000","last_became_active":"2024-09-12T03:19:07.388038+0000","last_became_peered":"2024-09-12T03:19:07.388038+0000","last_unstale":"2024-09-12T03:19:20.370179+0000","last_undegraded":"2024-09-12T03:19:20.370179+0000","last_fullsized":"2024-09-12T03:19:20.370179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:45.874195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.212080+0000","last_change":"2024-09-12T03:19:09.331525+0000","last_active":"2024-09-12T03:20:49.212080+0000","last_peered":"2024-09-12T03:20:49.212080+0000","last_clean":"2024-09-12T03:20:49.212080+0000","last_became_active":"2024-09-12T03:19:09.330903+0000","last_became_peered":"2024-09-12T03:19:09.330903+0000","last_unstale":"2024-09-12T03:20:49.212080+0000","last_undegraded":"2024-09-12T03:20:49.212080+0000","last_fullsized":"2024-09-12T03:20:49.212080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:02.455132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372235+0000","last_change":"2024-09-12T03:19:07.205181+0000","last_active":"2024-09-12T03:19:20.372235+0000","last_peered":"2024-09-12T03:19:20.372235+0000","last_clean":"2024-09-12T03:19:20.372235+0000","last_became_active":"2024-09-12T03:19:07.204726+0000","last_became_peered":"2024-09-12T03:19:07.204726+0000","last_unstale":"2024-09-12T03:19:20.372235+0000","last_undegraded":"2024-09-12T03:19:20.372235+0000","last_fullsized":"2024-09-12T03:19:20.372235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:26.112800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250768+0000","last_change":"2024-09-12T03:19:09.345338+0000","last_active":"2024-09-12T03:20:49.250768+0000","last_peered":"2024-09-12T03:20:49.250768+0000","last_clean":"2024-09-12T03:20:49.250768+0000","last_became_active":"2024-09-12T03:19:09.345023+0000","last_became_peered":"2024-09-12T03:19:09.345023+0000","last_unstale":"2024-09-12T03:20:49.250768+0000","last_undegraded":"2024-09-12T03:20:49.250768+0000","last_fullsized":"2024-09-12T03:20:49.250768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:39.184065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374938+0000","last_change":"2024-09-12T03:19:07.211954+0000","last_active":"2024-09-12T03:19:20.374938+0000","last_peered":"2024-09-12T03:19:20.374938+0000","last_clean":"2024-09-12T03:19:20.374938+0000","last_became_active":"2024-09-12T03:19:07.211509+0000","last_became_peered":"2024-09-12T03:19:07.211509+0000","last_unstale":"2024-09-12T03:19:20.374938+0000","last_undegraded":"2024-09-12T03:19:20.374938+0000","last_fullsized":"2024-09-12T03:19:20.374938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:27.461924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.244962+0000","last_change":"2024-09-12T03:19:09.331168+0000","last_active":"2024-09-12T03:20:49.244962+0000","last_peered":"2024-09-12T03:20:49.244962+0000","last_clean":"2024-09-12T03:20:49.244962+0000","last_became_active":"2024-09-12T03:19:09.329592+0000","last_became_peered":"2024-09-12T03:19:09.329592+0000","last_unstale":"2024-09-12T03:20:49.244962+0000","last_undegraded":"2024-09-12T03:20:49.244962+0000","last_fullsized":"2024-09-12T03:20:49.244962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:46.127861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373572+0000","last_change":"2024-09-12T03:19:07.207802+0000","last_active":"2024-09-12T03:19:20.373572+0000","last_peered":"2024-09-12T03:19:20.373572+0000","last_clean":"2024-09-12T03:19:20.373572+0000","last_became_active":"2024-09-12T03:19:07.207428+0000","last_became_peered":"2024-09-12T03:19:07.207428+0000","last_unstale":"2024-09-12T03:19:20.373572+0000","last_undegraded":"2024-09-12T03:19:20.373572+0000","last_fullsized":"2024-09-12T03:19:20.373572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:45.135019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.255394+0000","last_change":"2024-09-12T03:19:09.336094+0000","last_active":"2024-09-12T03:20:49.255394+0000","last_peered":"2024-09-12T03:20:49.255394+0000","last_clean":"2024-09-12T03:20:49.255394+0000","last_became_active":"2024-09-12T03:19:09.335779+0000","last_became_peered":"2024-09-12T03:19:09.335779+0000","last_unstale":"2024-09-12T03:20:49.255394+0000","last_undegraded":"2024-09-12T03:20:49.255394+0000","last_fullsized":"2024-09-12T03:20:49.255394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:37.019082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254523+0000","last_change":"2024-09-12T03:19:09.344537+0000","last_active":"2024-09-12T03:20:49.254523+0000","last_peered":"2024-09-12T03:20:49.254523+0000","last_clean":"2024-09-12T03:20:49.254523+0000","last_became_active":"2024-09-12T03:19:09.343879+0000","last_became_peered":"2024-09-12T03:19:09.343879+0000","last_unstale":"2024-09-12T03:20:49.254523+0000","last_undegraded":"2024-09-12T03:20:49.254523+0000","last_fullsized":"2024-09-12T03:20:49.254523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:40.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373973+0000","last_change":"2024-09-12T03:19:07.208040+0000","last_active":"2024-09-12T03:19:20.373973+0000","last_peered":"2024-09-12T03:19:20.373973+0000","last_clean":"2024-09-12T03:19:20.373973+0000","last_became_active":"2024-09-12T03:19:07.207658+0000","last_became_peered":"2024-09-12T03:19:07.207658+0000","last_unstale":"2024-09-12T03:19:20.373973+0000","last_undegraded":"2024-09-12T03:19:20.373973+0000","last_fullsized":"2024-09-12T03:19:20.373973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:48.620707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265759+0000","last_change":"2024-09-12T03:19:09.333837+0000","last_active":"2024-09-12T03:20:49.265759+0000","last_peered":"2024-09-12T03:20:49.265759+0000","last_clean":"2024-09-12T03:20:49.265759+0000","last_became_active":"2024-09-12T03:19:09.333194+0000","last_became_peered":"2024-09-12T03:19:09.333194+0000","last_unstale":"2024-09-12T03:20:49.265759+0000","last_undegraded":"2024-09-12T03:20:49.265759+0000","last_fullsized":"2024-09-12T03:20:49.265759+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:18.766181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.385455+0000","last_change":"2024-09-12T03:19:20.384978+0000","last_active":"2024-09-12T03:19:20.385455+0000","last_peered":"2024-09-12T03:19:20.385455+0000","last_clean":"2024-09-12T03:19:20.385455+0000","last_became_active":"2024-09-12T03:19:20.376883+0000","last_became_peered":"2024-09-12T03:19:20.376883+0000","last_unstale":"2024-09-12T03:19:20.385455+0000","last_undegraded":"2024-09-12T03:19:20.385455+0000","last_fullsized":"2024-09-12T03:19:20.385455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:32.542334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265022+0000","last_change":"2024-09-12T03:19:09.334684+0000","last_active":"2024-09-12T03:20:49.265022+0000","last_peered":"2024-09-12T03:20:49.265022+0000","last_clean":"2024-09-12T03:20:49.265022+0000","last_became_active":"2024-09-12T03:19:09.334281+0000","last_became_peered":"2024-09-12T03:19:09.334281+0000","last_unstale":"2024-09-12T03:20:49.265022+0000","last_undegraded":"2024-09-12T03:20:49.265022+0000","last_fullsized":"2024-09-12T03:20:49.265022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:44.837067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372829+0000","last_change":"2024-09-12T03:19:07.388555+0000","last_active":"2024-09-12T03:19:20.372829+0000","last_peered":"2024-09-12T03:19:20.372829+0000","last_clean":"2024-09-12T03:19:20.372829+0000","last_became_active":"2024-09-12T03:19:07.388206+0000","last_became_peered":"2024-09-12T03:19:07.388206+0000","last_unstale":"2024-09-12T03:19:20.372829+0000","last_undegraded":"2024-09-12T03:19:20.372829+0000","last_fullsized":"2024-09-12T03:19:20.372829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:33.628294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248105+0000","last_change":"2024-09-12T03:19:09.339108+0000","last_active":"2024-09-12T03:20:49.248105+0000","last_peered":"2024-09-12T03:20:49.248105+0000","last_clean":"2024-09-12T03:20:49.248105+0000","last_became_active":"2024-09-12T03:19:09.338819+0000","last_became_peered":"2024-09-12T03:19:09.338819+0000","last_unstale":"2024-09-12T03:20:49.248105+0000","last_undegraded":"2024-09-12T03:20:49.248105+0000","last_fullsized":"2024-09-12T03:20:49.248105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:32.864900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372798+0000","last_change":"2024-09-12T03:19:07.207197+0000","last_active":"2024-09-12T03:19:20.372798+0000","last_peered":"2024-09-12T03:19:20.372798+0000","last_clean":"2024-09-12T03:19:20.372798+0000","last_became_active":"2024-09-12T03:19:07.206879+0000","last_became_peered":"2024-09-12T03:19:07.206879+0000","last_unstale":"2024-09-12T03:19:20.372798+0000","last_undegraded":"2024-09-12T03:19:20.372798+0000","last_fullsized":"2024-09-12T03:19:20.372798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:00:46.269636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73845,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":654,"num_read_kb":407,"num_write":182,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":221535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579940,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":433387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36900,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708252,"statfs":{"total":95995035648,"available":95957250048,"internally_reserved":0,"allocated":1593344,"data_stored":1093834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972396,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":433589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201331,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":433509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430264,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1499136,"data_stored":1026676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593789,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":432978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822721,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1511424,"data_stored":1025967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953543,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":897024,"data_stored":433808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":433289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":503929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40444,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704708,"statfs":{"total":95995035648,"available":95953620992,"internally_reserved":0,"allocated":962560,"data_stored":501378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":434249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:21:05.901 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T03:21:05.901 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:21:06.266 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:06 smithi006 ceph-mon[30991]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:21:06.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:06 smithi161 ceph-mon[35485]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:21:06.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:06 smithi118 ceph-mon[35048]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:21:06.582 INFO:tasks.cephfs.filesystem:scrub status for tag:b0b04be6-d1b5-4239-8706-c51f499d089b - {'path': '/', 'tag': 'b0b04be6-d1b5-4239-8706-c51f499d089b', 'options': 'recursive,force'} 2024-09-12T03:21:06.623 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:07.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:07 smithi006 ceph-mon[30991]: pgmap v335: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-12T03:21:07.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.133+0000 7fb9253c6640 1 -- 172.21.15.6:0/2988167108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb920101de0 msgr2=0x7fb9201021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:07.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.133+0000 7fb9253c6640 1 --2- 172.21.15.6:0/2988167108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb920101de0 0x7fb9201021c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb910009900 tx=0x7fb91002a410 comp rx=0 tx=0).stop 2024-09-12T03:21:07.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.133+0000 7fb9253c6640 1 -- 172.21.15.6:0/2988167108 shutdown_connections 2024-09-12T03:21:07.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.133+0000 7fb9253c6640 1 --2- 172.21.15.6:0/2988167108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb920106f10 0x7fb92010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:07.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.133+0000 7fb9253c6640 1 --2- 172.21.15.6:0/2988167108 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb920102790 0x7fb9201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:07.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.133+0000 7fb9253c6640 1 --2- 172.21.15.6:0/2988167108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb920101de0 0x7fb9201021c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:07.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.133+0000 7fb9253c6640 1 -- 172.21.15.6:0/2988167108 >> 172.21.15.6:0/2988167108 conn(0x7fb9200fd8c0 msgr2=0x7fb9200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:07.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.133+0000 7fb9253c6640 1 -- 172.21.15.6:0/2988167108 shutdown_connections 2024-09-12T03:21:07.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.133+0000 7fb9253c6640 1 -- 172.21.15.6:0/2988167108 wait complete. 2024-09-12T03:21:07.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.134+0000 7fb9253c6640 1 Processor -- start 2024-09-12T03:21:07.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.134+0000 7fb9253c6640 1 -- start start 2024-09-12T03:21:07.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.134+0000 7fb9253c6640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb920101de0 0x7fb9201a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:07.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.134+0000 7fb9253c6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb920102790 0x7fb9201a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:07.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.134+0000 7fb9253c6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb920106f10 0x7fb92010dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:07.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.134+0000 7fb9253c6640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb920111f70 con 0x7fb920102790 2024-09-12T03:21:07.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.134+0000 7fb9253c6640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fb920111df0 con 0x7fb920101de0 2024-09-12T03:21:07.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.134+0000 7fb9253c6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb9201120f0 con 0x7fb920106f10 2024-09-12T03:21:07.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.134+0000 7fb917fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb920102790 0x7fb9201a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:07.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.134+0000 7fb917fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb920102790 0x7fb9201a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44988/0 (socket says 172.21.15.6:44988) 2024-09-12T03:21:07.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.134+0000 7fb917fff640 1 -- 172.21.15.6:0/261963571 learned_addr learned my addr 172.21.15.6:0/261963571 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:07.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.134+0000 7fb91effd640 1 --2- 172.21.15.6:0/261963571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb920101de0 0x7fb9201a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:07.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.134+0000 7fb91f7fe640 1 --2- 172.21.15.6:0/261963571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb920106f10 0x7fb92010dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:07.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.135+0000 7fb917fff640 1 -- 172.21.15.6:0/261963571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb920106f10 msgr2=0x7fb92010dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:07.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.135+0000 7fb917fff640 1 --2- 172.21.15.6:0/261963571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb920106f10 0x7fb92010dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:07.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.135+0000 7fb917fff640 1 -- 172.21.15.6:0/261963571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb920101de0 msgr2=0x7fb9201a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:07.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.135+0000 7fb917fff640 1 --2- 172.21.15.6:0/261963571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb920101de0 0x7fb9201a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:07.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.135+0000 7fb917fff640 1 -- 172.21.15.6:0/261963571 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92010e3d0 con 0x7fb920102790 2024-09-12T03:21:07.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.135+0000 7fb91effd640 1 --2- 172.21.15.6:0/261963571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb920101de0 0x7fb9201a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:07.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.135+0000 7fb917fff640 1 --2- 172.21.15.6:0/261963571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb920102790 0x7fb9201a4c10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb90800c890 tx=0x7fb90800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:07.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.135+0000 7fb91cff9640 1 -- 172.21.15.6:0/261963571 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb908004220 con 0x7fb920102790 2024-09-12T03:21:07.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.135+0000 7fb91cff9640 1 -- 172.21.15.6:0/261963571 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9080043c0 con 0x7fb920102790 2024-09-12T03:21:07.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.135+0000 7fb9253c6640 1 -- 172.21.15.6:0/261963571 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92010e6c0 con 0x7fb920102790 2024-09-12T03:21:07.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.135+0000 7fb91cff9640 1 -- 172.21.15.6:0/261963571 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9080049e0 con 0x7fb920102790 2024-09-12T03:21:07.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.135+0000 7fb9253c6640 1 -- 172.21.15.6:0/261963571 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9201ab5a0 con 0x7fb920102790 2024-09-12T03:21:07.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.136+0000 7fb9253c6640 1 -- 172.21.15.6:0/261963571 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9201021c0 con 0x7fb920102790 2024-09-12T03:21:07.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.137+0000 7fb91cff9640 1 -- 172.21.15.6:0/261963571 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fb9080079b0 con 0x7fb920102790 2024-09-12T03:21:07.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.138+0000 7fb91cff9640 1 --2- 172.21.15.6:0/261963571 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb90c0a2c10 0x7fb90c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:07.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.138+0000 7fb91cff9640 1 -- 172.21.15.6:0/261963571 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fb9080cf310 con 0x7fb920102790 2024-09-12T03:21:07.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.138+0000 7fb91effd640 1 --2- 172.21.15.6:0/261963571 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb90c0a2c10 0x7fb90c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:07.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.138+0000 7fb91effd640 1 --2- 172.21.15.6:0/261963571 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb90c0a2c10 0x7fb90c0a50d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb910000c00 tx=0x7fb910032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:07.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.142+0000 7fb91cff9640 1 -- 172.21.15.6:0/261963571 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb908014030 con 0x7fb920102790 2024-09-12T03:21:07.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.314+0000 7fb9253c6640 1 -- 172.21.15.6:0/261963571 --> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb920062630 con 0x7fb90c0a2c10 2024-09-12T03:21:07.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.320+0000 7fb91cff9640 1 -- 172.21.15.6:0/261963571 <== mgr.14150 v2:172.21.15.6:6800/5477767 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258837 (secure 0 0 0) 0x7fb920062630 con 0x7fb90c0a2c10 2024-09-12T03:21:07.322 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:21:07.323 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-09-12T03:21:07.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.323+0000 7fb9253c6640 1 -- 172.21.15.6:0/261963571 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb90c0a2c10 msgr2=0x7fb90c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:07.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.323+0000 7fb9253c6640 1 --2- 172.21.15.6:0/261963571 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb90c0a2c10 0x7fb90c0a50d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb910000c00 tx=0x7fb910032c70 comp rx=0 tx=0).stop 2024-09-12T03:21:07.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.323+0000 7fb9253c6640 1 -- 172.21.15.6:0/261963571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb920102790 msgr2=0x7fb9201a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:07.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.323+0000 7fb9253c6640 1 --2- 172.21.15.6:0/261963571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb920102790 0x7fb9201a4c10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb90800c890 tx=0x7fb90800cd50 comp rx=0 tx=0).stop 2024-09-12T03:21:07.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.323+0000 7fb9253c6640 1 -- 172.21.15.6:0/261963571 shutdown_connections 2024-09-12T03:21:07.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.323+0000 7fb9253c6640 1 --2- 172.21.15.6:0/261963571 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb90c0a2c10 0x7fb90c0a50d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:07.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.323+0000 7fb9253c6640 1 --2- 172.21.15.6:0/261963571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb920106f10 0x7fb92010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:07.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.323+0000 7fb9253c6640 1 --2- 172.21.15.6:0/261963571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb920102790 0x7fb9201a4c10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:07.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.323+0000 7fb9253c6640 1 --2- 172.21.15.6:0/261963571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb920101de0 0x7fb9201a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:07.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.323+0000 7fb9253c6640 1 -- 172.21.15.6:0/261963571 >> 172.21.15.6:0/261963571 conn(0x7fb9200fd8c0 msgr2=0x7fb9200fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:07.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.323+0000 7fb9253c6640 1 -- 172.21.15.6:0/261963571 shutdown_connections 2024-09-12T03:21:07.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:07.323+0000 7fb9253c6640 1 -- 172.21.15.6:0/261963571 wait complete. 2024-09-12T03:21:07.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:07 smithi161 ceph-mon[35485]: pgmap v335: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-12T03:21:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:07 smithi118 ceph-mon[35048]: pgmap v335: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-12T03:21:08.135 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-12T03:21:07.266541+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664447,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":791,"num_read_kb":517,"num_write":365,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382612,"kb_used_data":12564,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559212,"statfs":{"total":1151940427776,"available":1151548633088,"internally_reserved":0,"allocated":12865536,"data_stored":7186593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9380,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":44,"num_write":6,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002890"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265565+0000","last_change":"2024-09-12T03:19:09.343829+0000","last_active":"2024-09-12T03:20:49.265565+0000","last_peered":"2024-09-12T03:20:49.265565+0000","last_clean":"2024-09-12T03:20:49.265565+0000","last_became_active":"2024-09-12T03:19:09.343228+0000","last_became_peered":"2024-09-12T03:19:09.343228+0000","last_unstale":"2024-09-12T03:20:49.265565+0000","last_undegraded":"2024-09-12T03:20:49.265565+0000","last_fullsized":"2024-09-12T03:20:49.265565+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:08.720934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.237141+0000","last_change":"2024-09-12T03:19:09.331829+0000","last_active":"2024-09-12T03:20:49.237141+0000","last_peered":"2024-09-12T03:20:49.237141+0000","last_clean":"2024-09-12T03:20:49.237141+0000","last_became_active":"2024-09-12T03:19:09.331276+0000","last_became_peered":"2024-09-12T03:19:09.331276+0000","last_unstale":"2024-09-12T03:20:49.237141+0000","last_undegraded":"2024-09-12T03:20:49.237141+0000","last_fullsized":"2024-09-12T03:20:49.237141+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:48.672085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.263951+0000","last_change":"2024-09-12T03:19:09.338120+0000","last_active":"2024-09-12T03:20:49.263951+0000","last_peered":"2024-09-12T03:20:49.263951+0000","last_clean":"2024-09-12T03:20:49.263951+0000","last_became_active":"2024-09-12T03:19:09.337773+0000","last_became_peered":"2024-09-12T03:19:09.337773+0000","last_unstale":"2024-09-12T03:20:49.263951+0000","last_undegraded":"2024-09-12T03:20:49.263951+0000","last_fullsized":"2024-09-12T03:20:49.263951+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:13.716047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254592+0000","last_change":"2024-09-12T03:19:09.339404+0000","last_active":"2024-09-12T03:20:49.254592+0000","last_peered":"2024-09-12T03:20:49.254592+0000","last_clean":"2024-09-12T03:20:49.254592+0000","last_became_active":"2024-09-12T03:19:09.339105+0000","last_became_peered":"2024-09-12T03:19:09.339105+0000","last_unstale":"2024-09-12T03:20:49.254592+0000","last_undegraded":"2024-09-12T03:20:49.254592+0000","last_fullsized":"2024-09-12T03:20:49.254592+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:37.930963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.242261+0000","last_change":"2024-09-12T03:19:09.326285+0000","last_active":"2024-09-12T03:20:49.242261+0000","last_peered":"2024-09-12T03:20:49.242261+0000","last_clean":"2024-09-12T03:20:49.242261+0000","last_became_active":"2024-09-12T03:19:09.325852+0000","last_became_peered":"2024-09-12T03:19:09.325852+0000","last_unstale":"2024-09-12T03:20:49.242261+0000","last_undegraded":"2024-09-12T03:20:49.242261+0000","last_fullsized":"2024-09-12T03:20:49.242261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:44:25.387014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.249918+0000","last_change":"2024-09-12T03:19:20.376197+0000","last_active":"2024-09-12T03:20:49.249918+0000","last_peered":"2024-09-12T03:20:49.249918+0000","last_clean":"2024-09-12T03:20:49.249918+0000","last_became_active":"2024-09-12T03:19:20.375744+0000","last_became_peered":"2024-09-12T03:19:20.375744+0000","last_unstale":"2024-09-12T03:20:49.249918+0000","last_undegraded":"2024-09-12T03:20:49.249918+0000","last_fullsized":"2024-09-12T03:20:49.249918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:08.663121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.245988+0000","last_change":"2024-09-12T03:19:09.335284+0000","last_active":"2024-09-12T03:20:49.245988+0000","last_peered":"2024-09-12T03:20:49.245988+0000","last_clean":"2024-09-12T03:20:49.245988+0000","last_became_active":"2024-09-12T03:19:09.334948+0000","last_became_peered":"2024-09-12T03:19:09.334948+0000","last_unstale":"2024-09-12T03:20:49.245988+0000","last_undegraded":"2024-09-12T03:20:49.245988+0000","last_fullsized":"2024-09-12T03:20:49.245988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:32.778433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250337+0000","last_change":"2024-09-12T03:19:09.338398+0000","last_active":"2024-09-12T03:20:49.250337+0000","last_peered":"2024-09-12T03:20:49.250337+0000","last_clean":"2024-09-12T03:20:49.250337+0000","last_became_active":"2024-09-12T03:19:09.337750+0000","last_became_peered":"2024-09-12T03:19:09.337750+0000","last_unstale":"2024-09-12T03:20:49.250337+0000","last_undegraded":"2024-09-12T03:20:49.250337+0000","last_fullsized":"2024-09-12T03:20:49.250337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:07.542322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265597+0000","last_change":"2024-09-12T03:19:09.340703+0000","last_active":"2024-09-12T03:20:49.265597+0000","last_peered":"2024-09-12T03:20:49.265597+0000","last_clean":"2024-09-12T03:20:49.265597+0000","last_became_active":"2024-09-12T03:19:09.340386+0000","last_became_peered":"2024-09-12T03:19:09.340386+0000","last_unstale":"2024-09-12T03:20:49.265597+0000","last_undegraded":"2024-09-12T03:20:49.265597+0000","last_fullsized":"2024-09-12T03:20:49.265597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:19.990894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248198+0000","last_change":"2024-09-12T03:19:09.341489+0000","last_active":"2024-09-12T03:20:49.248198+0000","last_peered":"2024-09-12T03:20:49.248198+0000","last_clean":"2024-09-12T03:20:49.248198+0000","last_became_active":"2024-09-12T03:19:09.341183+0000","last_became_peered":"2024-09-12T03:19:09.341183+0000","last_unstale":"2024-09-12T03:20:49.248198+0000","last_undegraded":"2024-09-12T03:20:49.248198+0000","last_fullsized":"2024-09-12T03:20:49.248198+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:05.688402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.255447+0000","last_change":"2024-09-12T03:19:09.339289+0000","last_active":"2024-09-12T03:20:49.255447+0000","last_peered":"2024-09-12T03:20:49.255447+0000","last_clean":"2024-09-12T03:20:49.255447+0000","last_became_active":"2024-09-12T03:19:09.338466+0000","last_became_peered":"2024-09-12T03:19:09.338466+0000","last_unstale":"2024-09-12T03:20:49.255447+0000","last_undegraded":"2024-09-12T03:20:49.255447+0000","last_fullsized":"2024-09-12T03:20:49.255447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:47:13.070057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254364+0000","last_change":"2024-09-12T03:19:09.337754+0000","last_active":"2024-09-12T03:20:49.254364+0000","last_peered":"2024-09-12T03:20:49.254364+0000","last_clean":"2024-09-12T03:20:49.254364+0000","last_became_active":"2024-09-12T03:19:09.332789+0000","last_became_peered":"2024-09-12T03:19:09.332789+0000","last_unstale":"2024-09-12T03:20:49.254364+0000","last_undegraded":"2024-09-12T03:20:49.254364+0000","last_fullsized":"2024-09-12T03:20:49.254364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:34.622136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250957+0000","last_change":"2024-09-12T03:19:09.333577+0000","last_active":"2024-09-12T03:20:49.250957+0000","last_peered":"2024-09-12T03:20:49.250957+0000","last_clean":"2024-09-12T03:20:49.250957+0000","last_became_active":"2024-09-12T03:19:09.333234+0000","last_became_peered":"2024-09-12T03:19:09.333234+0000","last_unstale":"2024-09-12T03:20:49.250957+0000","last_undegraded":"2024-09-12T03:20:49.250957+0000","last_fullsized":"2024-09-12T03:20:49.250957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:22.604640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.237367+0000","last_change":"2024-09-12T03:19:09.331607+0000","last_active":"2024-09-12T03:20:49.237367+0000","last_peered":"2024-09-12T03:20:49.237367+0000","last_clean":"2024-09-12T03:20:49.237367+0000","last_became_active":"2024-09-12T03:19:09.331253+0000","last_became_peered":"2024-09-12T03:19:09.331253+0000","last_unstale":"2024-09-12T03:20:49.237367+0000","last_undegraded":"2024-09-12T03:20:49.237367+0000","last_fullsized":"2024-09-12T03:20:49.237367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:11.969265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256379+0000","last_change":"2024-09-12T03:19:09.340768+0000","last_active":"2024-09-12T03:20:49.256379+0000","last_peered":"2024-09-12T03:20:49.256379+0000","last_clean":"2024-09-12T03:20:49.256379+0000","last_became_active":"2024-09-12T03:19:09.340437+0000","last_became_peered":"2024-09-12T03:19:09.340437+0000","last_unstale":"2024-09-12T03:20:49.256379+0000","last_undegraded":"2024-09-12T03:20:49.256379+0000","last_fullsized":"2024-09-12T03:20:49.256379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:32.645691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.263546+0000","last_change":"2024-09-12T03:19:09.333829+0000","last_active":"2024-09-12T03:20:49.263546+0000","last_peered":"2024-09-12T03:20:49.263546+0000","last_clean":"2024-09-12T03:20:49.263546+0000","last_became_active":"2024-09-12T03:19:09.333462+0000","last_became_peered":"2024-09-12T03:19:09.333462+0000","last_unstale":"2024-09-12T03:20:49.263546+0000","last_undegraded":"2024-09-12T03:20:49.263546+0000","last_fullsized":"2024-09-12T03:20:49.263546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:18:57.586836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265659+0000","last_change":"2024-09-12T03:19:09.344068+0000","last_active":"2024-09-12T03:20:49.265659+0000","last_peered":"2024-09-12T03:20:49.265659+0000","last_clean":"2024-09-12T03:20:49.265659+0000","last_became_active":"2024-09-12T03:19:09.343751+0000","last_became_peered":"2024-09-12T03:19:09.343751+0000","last_unstale":"2024-09-12T03:20:49.265659+0000","last_undegraded":"2024-09-12T03:20:49.265659+0000","last_fullsized":"2024-09-12T03:20:49.265659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:52.425547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.249413+0000","last_change":"2024-09-12T03:19:09.330828+0000","last_active":"2024-09-12T03:20:49.249413+0000","last_peered":"2024-09-12T03:20:49.249413+0000","last_clean":"2024-09-12T03:20:49.249413+0000","last_became_active":"2024-09-12T03:19:09.330497+0000","last_became_peered":"2024-09-12T03:19:09.330497+0000","last_unstale":"2024-09-12T03:20:49.249413+0000","last_undegraded":"2024-09-12T03:20:49.249413+0000","last_fullsized":"2024-09-12T03:20:49.249413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:21:20.064177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.245548+0000","last_change":"2024-09-12T03:19:09.344403+0000","last_active":"2024-09-12T03:20:49.245548+0000","last_peered":"2024-09-12T03:20:49.245548+0000","last_clean":"2024-09-12T03:20:49.245548+0000","last_became_active":"2024-09-12T03:19:09.343946+0000","last_became_peered":"2024-09-12T03:19:09.343946+0000","last_unstale":"2024-09-12T03:20:49.245548+0000","last_undegraded":"2024-09-12T03:20:49.245548+0000","last_fullsized":"2024-09-12T03:20:49.245548+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:48.386117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248484+0000","last_change":"2024-09-12T03:19:09.341481+0000","last_active":"2024-09-12T03:20:49.248484+0000","last_peered":"2024-09-12T03:20:49.248484+0000","last_clean":"2024-09-12T03:20:49.248484+0000","last_became_active":"2024-09-12T03:19:09.341142+0000","last_became_peered":"2024-09-12T03:19:09.341142+0000","last_unstale":"2024-09-12T03:20:49.248484+0000","last_undegraded":"2024-09-12T03:20:49.248484+0000","last_fullsized":"2024-09-12T03:20:49.248484+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:42.371221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.249677+0000","last_change":"2024-09-12T03:19:09.330621+0000","last_active":"2024-09-12T03:20:49.249677+0000","last_peered":"2024-09-12T03:20:49.249677+0000","last_clean":"2024-09-12T03:20:49.249677+0000","last_became_active":"2024-09-12T03:19:09.330169+0000","last_became_peered":"2024-09-12T03:19:09.330169+0000","last_unstale":"2024-09-12T03:20:49.249677+0000","last_undegraded":"2024-09-12T03:20:49.249677+0000","last_fullsized":"2024-09-12T03:20:49.249677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:16.634141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.264220+0000","last_change":"2024-09-12T03:19:09.332817+0000","last_active":"2024-09-12T03:20:49.264220+0000","last_peered":"2024-09-12T03:20:49.264220+0000","last_clean":"2024-09-12T03:20:49.264220+0000","last_became_active":"2024-09-12T03:19:09.332236+0000","last_became_peered":"2024-09-12T03:19:09.332236+0000","last_unstale":"2024-09-12T03:20:49.264220+0000","last_undegraded":"2024-09-12T03:20:49.264220+0000","last_fullsized":"2024-09-12T03:20:49.264220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:28.024294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252477+0000","last_change":"2024-09-12T03:19:09.337910+0000","last_active":"2024-09-12T03:20:49.252477+0000","last_peered":"2024-09-12T03:20:49.252477+0000","last_clean":"2024-09-12T03:20:49.252477+0000","last_became_active":"2024-09-12T03:19:09.337176+0000","last_became_peered":"2024-09-12T03:19:09.337176+0000","last_unstale":"2024-09-12T03:20:49.252477+0000","last_undegraded":"2024-09-12T03:20:49.252477+0000","last_fullsized":"2024-09-12T03:20:49.252477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:53.795898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.239957+0000","last_change":"2024-09-12T03:19:09.337902+0000","last_active":"2024-09-12T03:20:49.239957+0000","last_peered":"2024-09-12T03:20:49.239957+0000","last_clean":"2024-09-12T03:20:49.239957+0000","last_became_active":"2024-09-12T03:19:09.337567+0000","last_became_peered":"2024-09-12T03:19:09.337567+0000","last_unstale":"2024-09-12T03:20:49.239957+0000","last_undegraded":"2024-09-12T03:20:49.239957+0000","last_fullsized":"2024-09-12T03:20:49.239957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:08.471313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248557+0000","last_change":"2024-09-12T03:19:09.339850+0000","last_active":"2024-09-12T03:20:49.248557+0000","last_peered":"2024-09-12T03:20:49.248557+0000","last_clean":"2024-09-12T03:20:49.248557+0000","last_became_active":"2024-09-12T03:19:09.339563+0000","last_became_peered":"2024-09-12T03:19:09.339563+0000","last_unstale":"2024-09-12T03:20:49.248557+0000","last_undegraded":"2024-09-12T03:20:49.248557+0000","last_fullsized":"2024-09-12T03:20:49.248557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:27.473351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.247769+0000","last_change":"2024-09-12T03:19:09.330555+0000","last_active":"2024-09-12T03:20:49.247769+0000","last_peered":"2024-09-12T03:20:49.247769+0000","last_clean":"2024-09-12T03:20:49.247769+0000","last_became_active":"2024-09-12T03:19:09.330183+0000","last_became_peered":"2024-09-12T03:19:09.330183+0000","last_unstale":"2024-09-12T03:20:49.247769+0000","last_undegraded":"2024-09-12T03:20:49.247769+0000","last_fullsized":"2024-09-12T03:20:49.247769+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:39.334116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252190+0000","last_change":"2024-09-12T03:19:09.338601+0000","last_active":"2024-09-12T03:20:49.252190+0000","last_peered":"2024-09-12T03:20:49.252190+0000","last_clean":"2024-09-12T03:20:49.252190+0000","last_became_active":"2024-09-12T03:19:09.337903+0000","last_became_peered":"2024-09-12T03:19:09.337903+0000","last_unstale":"2024-09-12T03:20:49.252190+0000","last_undegraded":"2024-09-12T03:20:49.252190+0000","last_fullsized":"2024-09-12T03:20:49.252190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:17.810139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250990+0000","last_change":"2024-09-12T03:19:09.326629+0000","last_active":"2024-09-12T03:20:49.250990+0000","last_peered":"2024-09-12T03:20:49.250990+0000","last_clean":"2024-09-12T03:20:49.250990+0000","last_became_active":"2024-09-12T03:19:09.326333+0000","last_became_peered":"2024-09-12T03:19:09.326333+0000","last_unstale":"2024-09-12T03:20:49.250990+0000","last_undegraded":"2024-09-12T03:20:49.250990+0000","last_fullsized":"2024-09-12T03:20:49.250990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:20.807593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256412+0000","last_change":"2024-09-12T03:19:09.339919+0000","last_active":"2024-09-12T03:20:49.256412+0000","last_peered":"2024-09-12T03:20:49.256412+0000","last_clean":"2024-09-12T03:20:49.256412+0000","last_became_active":"2024-09-12T03:19:09.339618+0000","last_became_peered":"2024-09-12T03:19:09.339618+0000","last_unstale":"2024-09-12T03:20:49.256412+0000","last_undegraded":"2024-09-12T03:20:49.256412+0000","last_fullsized":"2024-09-12T03:20:49.256412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:11:03.740079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.246446+0000","last_change":"2024-09-12T03:19:09.336148+0000","last_active":"2024-09-12T03:20:49.246446+0000","last_peered":"2024-09-12T03:20:49.246446+0000","last_clean":"2024-09-12T03:20:49.246446+0000","last_became_active":"2024-09-12T03:19:09.335779+0000","last_became_peered":"2024-09-12T03:19:09.335779+0000","last_unstale":"2024-09-12T03:20:49.246446+0000","last_undegraded":"2024-09-12T03:20:49.246446+0000","last_fullsized":"2024-09-12T03:20:49.246446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:18.234320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.247971+0000","last_change":"2024-09-12T03:19:20.382536+0000","last_active":"2024-09-12T03:20:49.247971+0000","last_peered":"2024-09-12T03:20:49.247971+0000","last_clean":"2024-09-12T03:20:49.247971+0000","last_became_active":"2024-09-12T03:19:20.382118+0000","last_became_peered":"2024-09-12T03:19:20.382118+0000","last_unstale":"2024-09-12T03:20:49.247971+0000","last_undegraded":"2024-09-12T03:20:49.247971+0000","last_fullsized":"2024-09-12T03:20:49.247971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:55.836646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252455+0000","last_change":"2024-09-12T03:19:09.345064+0000","last_active":"2024-09-12T03:20:49.252455+0000","last_peered":"2024-09-12T03:20:49.252455+0000","last_clean":"2024-09-12T03:20:49.252455+0000","last_became_active":"2024-09-12T03:19:09.344628+0000","last_became_peered":"2024-09-12T03:19:09.344628+0000","last_unstale":"2024-09-12T03:20:49.252455+0000","last_undegraded":"2024-09-12T03:20:49.252455+0000","last_fullsized":"2024-09-12T03:20:49.252455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:50.011751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.244081+0000","last_change":"2024-09-12T03:19:09.338052+0000","last_active":"2024-09-12T03:20:49.244081+0000","last_peered":"2024-09-12T03:20:49.244081+0000","last_clean":"2024-09-12T03:20:49.244081+0000","last_became_active":"2024-09-12T03:19:09.337710+0000","last_became_peered":"2024-09-12T03:19:09.337710+0000","last_unstale":"2024-09-12T03:20:49.244081+0000","last_undegraded":"2024-09-12T03:20:49.244081+0000","last_fullsized":"2024-09-12T03:20:49.244081+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:36.521269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'9","reported_seq":144,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:21:04.534436+0000","last_change":"2024-09-12T03:19:07.208341+0000","last_active":"2024-09-12T03:21:04.534436+0000","last_peered":"2024-09-12T03:21:04.534436+0000","last_clean":"2024-09-12T03:21:04.534436+0000","last_became_active":"2024-09-12T03:19:07.207385+0000","last_became_peered":"2024-09-12T03:19:07.207385+0000","last_unstale":"2024-09-12T03:21:04.534436+0000","last_undegraded":"2024-09-12T03:21:04.534436+0000","last_fullsized":"2024-09-12T03:21:04.534436+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:18.476795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1857,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254115+0000","last_change":"2024-09-12T03:19:09.339997+0000","last_active":"2024-09-12T03:20:49.254115+0000","last_peered":"2024-09-12T03:20:49.254115+0000","last_clean":"2024-09-12T03:20:49.254115+0000","last_became_active":"2024-09-12T03:19:09.339261+0000","last_became_peered":"2024-09-12T03:19:09.339261+0000","last_unstale":"2024-09-12T03:20:49.254115+0000","last_undegraded":"2024-09-12T03:20:49.254115+0000","last_fullsized":"2024-09-12T03:20:49.254115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:07.670345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374030+0000","last_change":"2024-09-12T03:19:07.200222+0000","last_active":"2024-09-12T03:19:20.374030+0000","last_peered":"2024-09-12T03:19:20.374030+0000","last_clean":"2024-09-12T03:19:20.374030+0000","last_became_active":"2024-09-12T03:19:07.199819+0000","last_became_peered":"2024-09-12T03:19:07.199819+0000","last_unstale":"2024-09-12T03:19:20.374030+0000","last_undegraded":"2024-09-12T03:19:20.374030+0000","last_fullsized":"2024-09-12T03:19:20.374030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:39:28.933609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375686+0000","last_change":"2024-09-12T03:19:07.207005+0000","last_active":"2024-09-12T03:19:20.375686+0000","last_peered":"2024-09-12T03:19:20.375686+0000","last_clean":"2024-09-12T03:19:20.375686+0000","last_became_active":"2024-09-12T03:19:07.206685+0000","last_became_peered":"2024-09-12T03:19:07.206685+0000","last_unstale":"2024-09-12T03:19:20.375686+0000","last_undegraded":"2024-09-12T03:19:20.375686+0000","last_fullsized":"2024-09-12T03:19:20.375686+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:31.111706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252939+0000","last_change":"2024-09-12T03:19:09.335871+0000","last_active":"2024-09-12T03:20:49.252939+0000","last_peered":"2024-09-12T03:20:49.252939+0000","last_clean":"2024-09-12T03:20:49.252939+0000","last_became_active":"2024-09-12T03:19:09.335568+0000","last_became_peered":"2024-09-12T03:19:09.335568+0000","last_unstale":"2024-09-12T03:20:49.252939+0000","last_undegraded":"2024-09-12T03:20:49.252939+0000","last_fullsized":"2024-09-12T03:20:49.252939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:52:34.471512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386049+0000","last_change":"2024-09-12T03:19:07.208320+0000","last_active":"2024-09-12T03:19:20.386049+0000","last_peered":"2024-09-12T03:19:20.386049+0000","last_clean":"2024-09-12T03:19:20.386049+0000","last_became_active":"2024-09-12T03:19:07.207873+0000","last_became_peered":"2024-09-12T03:19:07.207873+0000","last_unstale":"2024-09-12T03:19:20.386049+0000","last_undegraded":"2024-09-12T03:19:20.386049+0000","last_fullsized":"2024-09-12T03:19:20.386049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:00.128595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250364+0000","last_change":"2024-09-12T03:19:09.331930+0000","last_active":"2024-09-12T03:20:49.250364+0000","last_peered":"2024-09-12T03:20:49.250364+0000","last_clean":"2024-09-12T03:20:49.250364+0000","last_became_active":"2024-09-12T03:19:09.331558+0000","last_became_peered":"2024-09-12T03:19:09.331558+0000","last_unstale":"2024-09-12T03:20:49.250364+0000","last_undegraded":"2024-09-12T03:20:49.250364+0000","last_fullsized":"2024-09-12T03:20:49.250364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:00:27.935717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":130,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.529083+0000","last_change":"2024-09-12T03:19:07.205473+0000","last_active":"2024-09-12T03:21:01.529083+0000","last_peered":"2024-09-12T03:21:01.529083+0000","last_clean":"2024-09-12T03:21:01.529083+0000","last_became_active":"2024-09-12T03:19:07.205134+0000","last_became_peered":"2024-09-12T03:19:07.205134+0000","last_unstale":"2024-09-12T03:21:01.529083+0000","last_undegraded":"2024-09-12T03:21:01.529083+0000","last_fullsized":"2024-09-12T03:21:01.529083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:23.443957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265317+0000","last_change":"2024-09-12T03:19:09.333868+0000","last_active":"2024-09-12T03:20:49.265317+0000","last_peered":"2024-09-12T03:20:49.265317+0000","last_clean":"2024-09-12T03:20:49.265317+0000","last_became_active":"2024-09-12T03:19:09.333484+0000","last_became_peered":"2024-09-12T03:19:09.333484+0000","last_unstale":"2024-09-12T03:20:49.265317+0000","last_undegraded":"2024-09-12T03:20:49.265317+0000","last_fullsized":"2024-09-12T03:20:49.265317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:21:10.226821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386303+0000","last_change":"2024-09-12T03:19:20.386303+0000","last_active":"2024-09-12T03:19:20.386303+0000","last_peered":"2024-09-12T03:19:20.386303+0000","last_clean":"2024-09-12T03:19:20.386303+0000","last_became_active":"2024-09-12T03:19:20.383198+0000","last_became_peered":"2024-09-12T03:19:20.383198+0000","last_unstale":"2024-09-12T03:19:20.386303+0000","last_undegraded":"2024-09-12T03:19:20.386303+0000","last_fullsized":"2024-09-12T03:19:20.386303+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:42.224849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.257056+0000","last_change":"2024-09-12T03:19:09.340849+0000","last_active":"2024-09-12T03:20:49.257056+0000","last_peered":"2024-09-12T03:20:49.257056+0000","last_clean":"2024-09-12T03:20:49.257056+0000","last_became_active":"2024-09-12T03:19:09.340517+0000","last_became_peered":"2024-09-12T03:19:09.340517+0000","last_unstale":"2024-09-12T03:20:49.257056+0000","last_undegraded":"2024-09-12T03:20:49.257056+0000","last_fullsized":"2024-09-12T03:20:49.257056+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:42.018269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386240+0000","last_change":"2024-09-12T03:19:07.389147+0000","last_active":"2024-09-12T03:19:20.386240+0000","last_peered":"2024-09-12T03:19:20.386240+0000","last_clean":"2024-09-12T03:19:20.386240+0000","last_became_active":"2024-09-12T03:19:07.388744+0000","last_became_peered":"2024-09-12T03:19:07.388744+0000","last_unstale":"2024-09-12T03:19:20.386240+0000","last_undegraded":"2024-09-12T03:19:20.386240+0000","last_fullsized":"2024-09-12T03:19:20.386240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:16.700644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.264622+0000","last_change":"2024-09-12T03:19:09.329243+0000","last_active":"2024-09-12T03:20:49.264622+0000","last_peered":"2024-09-12T03:20:49.264622+0000","last_clean":"2024-09-12T03:20:49.264622+0000","last_became_active":"2024-09-12T03:19:09.328610+0000","last_became_peered":"2024-09-12T03:19:09.328610+0000","last_unstale":"2024-09-12T03:20:49.264622+0000","last_undegraded":"2024-09-12T03:20:49.264622+0000","last_fullsized":"2024-09-12T03:20:49.264622+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:49.452942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'34","reported_seq":276,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:21:02.531095+0000","last_change":"2024-09-12T03:19:07.207804+0000","last_active":"2024-09-12T03:21:02.531095+0000","last_peered":"2024-09-12T03:21:02.531095+0000","last_clean":"2024-09-12T03:21:02.531095+0000","last_became_active":"2024-09-12T03:19:07.207409+0000","last_became_peered":"2024-09-12T03:19:07.207409+0000","last_unstale":"2024-09-12T03:21:02.531095+0000","last_undegraded":"2024-09-12T03:21:02.531095+0000","last_fullsized":"2024-09-12T03:21:02.531095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:10.732423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68607,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":220,"num_read_kb":177,"num_write":35,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.255978+0000","last_change":"2024-09-12T03:19:09.333102+0000","last_active":"2024-09-12T03:20:49.255978+0000","last_peered":"2024-09-12T03:20:49.255978+0000","last_clean":"2024-09-12T03:20:49.255978+0000","last_became_active":"2024-09-12T03:19:09.332712+0000","last_became_peered":"2024-09-12T03:19:09.332712+0000","last_unstale":"2024-09-12T03:20:49.255978+0000","last_undegraded":"2024-09-12T03:20:49.255978+0000","last_fullsized":"2024-09-12T03:20:49.255978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:22.912213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382902+0000","last_change":"2024-09-12T03:19:20.382348+0000","last_active":"2024-09-12T03:19:20.382902+0000","last_peered":"2024-09-12T03:19:20.382902+0000","last_clean":"2024-09-12T03:19:20.382902+0000","last_became_active":"2024-09-12T03:19:20.375142+0000","last_became_peered":"2024-09-12T03:19:20.375142+0000","last_unstale":"2024-09-12T03:19:20.382902+0000","last_undegraded":"2024-09-12T03:19:20.382902+0000","last_fullsized":"2024-09-12T03:19:20.382902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:04.573599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.246879+0000","last_change":"2024-09-12T03:19:09.333194+0000","last_active":"2024-09-12T03:20:49.246879+0000","last_peered":"2024-09-12T03:20:49.246879+0000","last_clean":"2024-09-12T03:20:49.246879+0000","last_became_active":"2024-09-12T03:19:09.332865+0000","last_became_peered":"2024-09-12T03:19:09.332865+0000","last_unstale":"2024-09-12T03:20:49.246879+0000","last_undegraded":"2024-09-12T03:20:49.246879+0000","last_fullsized":"2024-09-12T03:20:49.246879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:57.096138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386176+0000","last_change":"2024-09-12T03:19:07.209289+0000","last_active":"2024-09-12T03:19:20.386176+0000","last_peered":"2024-09-12T03:19:20.386176+0000","last_clean":"2024-09-12T03:19:20.386176+0000","last_became_active":"2024-09-12T03:19:07.208863+0000","last_became_peered":"2024-09-12T03:19:07.208863+0000","last_unstale":"2024-09-12T03:19:20.386176+0000","last_undegraded":"2024-09-12T03:19:20.386176+0000","last_fullsized":"2024-09-12T03:19:20.386176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:26.168148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256973+0000","last_change":"2024-09-12T03:19:09.343620+0000","last_active":"2024-09-12T03:20:49.256973+0000","last_peered":"2024-09-12T03:20:49.256973+0000","last_clean":"2024-09-12T03:20:49.256973+0000","last_became_active":"2024-09-12T03:19:09.343299+0000","last_became_peered":"2024-09-12T03:19:09.343299+0000","last_unstale":"2024-09-12T03:20:49.256973+0000","last_undegraded":"2024-09-12T03:20:49.256973+0000","last_fullsized":"2024-09-12T03:20:49.256973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:34.268886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":136,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:21:05.568470+0000","last_change":"2024-09-12T03:19:07.206160+0000","last_active":"2024-09-12T03:21:05.568470+0000","last_peered":"2024-09-12T03:21:05.568470+0000","last_clean":"2024-09-12T03:21:05.568470+0000","last_became_active":"2024-09-12T03:19:07.205509+0000","last_became_peered":"2024-09-12T03:19:07.205509+0000","last_unstale":"2024-09-12T03:21:05.568470+0000","last_undegraded":"2024-09-12T03:21:05.568470+0000","last_fullsized":"2024-09-12T03:21:05.568470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:35.362831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250896+0000","last_change":"2024-09-12T03:19:09.337595+0000","last_active":"2024-09-12T03:20:49.250896+0000","last_peered":"2024-09-12T03:20:49.250896+0000","last_clean":"2024-09-12T03:20:49.250896+0000","last_became_active":"2024-09-12T03:19:09.337281+0000","last_became_peered":"2024-09-12T03:19:09.337281+0000","last_unstale":"2024-09-12T03:20:49.250896+0000","last_undegraded":"2024-09-12T03:20:49.250896+0000","last_fullsized":"2024-09-12T03:20:49.250896+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:35.761679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373995+0000","last_change":"2024-09-12T03:19:07.389336+0000","last_active":"2024-09-12T03:19:20.373995+0000","last_peered":"2024-09-12T03:19:20.373995+0000","last_clean":"2024-09-12T03:19:20.373995+0000","last_became_active":"2024-09-12T03:19:07.388649+0000","last_became_peered":"2024-09-12T03:19:07.388649+0000","last_unstale":"2024-09-12T03:19:20.373995+0000","last_undegraded":"2024-09-12T03:19:20.373995+0000","last_fullsized":"2024-09-12T03:19:20.373995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:33.312700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.253821+0000","last_change":"2024-09-12T03:19:09.340674+0000","last_active":"2024-09-12T03:20:49.253821+0000","last_peered":"2024-09-12T03:20:49.253821+0000","last_clean":"2024-09-12T03:20:49.253821+0000","last_became_active":"2024-09-12T03:19:09.340144+0000","last_became_peered":"2024-09-12T03:19:09.340144+0000","last_unstale":"2024-09-12T03:20:49.253821+0000","last_undegraded":"2024-09-12T03:20:49.253821+0000","last_fullsized":"2024-09-12T03:20:49.253821+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:21:15.605086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373837+0000","last_change":"2024-09-12T03:19:07.206513+0000","last_active":"2024-09-12T03:19:20.373837+0000","last_peered":"2024-09-12T03:19:20.373837+0000","last_clean":"2024-09-12T03:19:20.373837+0000","last_became_active":"2024-09-12T03:19:07.205974+0000","last_became_peered":"2024-09-12T03:19:07.205974+0000","last_unstale":"2024-09-12T03:19:20.373837+0000","last_undegraded":"2024-09-12T03:19:20.373837+0000","last_fullsized":"2024-09-12T03:19:20.373837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:43:10.002123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252033+0000","last_change":"2024-09-12T03:19:09.342200+0000","last_active":"2024-09-12T03:20:49.252033+0000","last_peered":"2024-09-12T03:20:49.252033+0000","last_clean":"2024-09-12T03:20:49.252033+0000","last_became_active":"2024-09-12T03:19:09.341901+0000","last_became_peered":"2024-09-12T03:19:09.341901+0000","last_unstale":"2024-09-12T03:20:49.252033+0000","last_undegraded":"2024-09-12T03:20:49.252033+0000","last_fullsized":"2024-09-12T03:20:49.252033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:27.486214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375373+0000","last_change":"2024-09-12T03:19:07.207514+0000","last_active":"2024-09-12T03:19:20.375373+0000","last_peered":"2024-09-12T03:19:20.375373+0000","last_clean":"2024-09-12T03:19:20.375373+0000","last_became_active":"2024-09-12T03:19:07.205925+0000","last_became_peered":"2024-09-12T03:19:07.205925+0000","last_unstale":"2024-09-12T03:19:20.375373+0000","last_undegraded":"2024-09-12T03:19:20.375373+0000","last_fullsized":"2024-09-12T03:19:20.375373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:20.244703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256005+0000","last_change":"2024-09-12T03:19:09.337837+0000","last_active":"2024-09-12T03:20:49.256005+0000","last_peered":"2024-09-12T03:20:49.256005+0000","last_clean":"2024-09-12T03:20:49.256005+0000","last_became_active":"2024-09-12T03:19:09.337502+0000","last_became_peered":"2024-09-12T03:19:09.337502+0000","last_unstale":"2024-09-12T03:20:49.256005+0000","last_undegraded":"2024-09-12T03:20:49.256005+0000","last_fullsized":"2024-09-12T03:20:49.256005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:30.167139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.386356+0000","last_change":"2024-09-12T03:19:07.209130+0000","last_active":"2024-09-12T03:19:20.386356+0000","last_peered":"2024-09-12T03:19:20.386356+0000","last_clean":"2024-09-12T03:19:20.386356+0000","last_became_active":"2024-09-12T03:19:07.208753+0000","last_became_peered":"2024-09-12T03:19:07.208753+0000","last_unstale":"2024-09-12T03:19:20.386356+0000","last_undegraded":"2024-09-12T03:19:20.386356+0000","last_fullsized":"2024-09-12T03:19:20.386356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:34.719292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250623+0000","last_change":"2024-09-12T03:19:09.336382+0000","last_active":"2024-09-12T03:20:49.250623+0000","last_peered":"2024-09-12T03:20:49.250623+0000","last_clean":"2024-09-12T03:20:49.250623+0000","last_became_active":"2024-09-12T03:19:09.336015+0000","last_became_peered":"2024-09-12T03:19:09.336015+0000","last_unstale":"2024-09-12T03:20:49.250623+0000","last_undegraded":"2024-09-12T03:20:49.250623+0000","last_fullsized":"2024-09-12T03:20:49.250623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:19.430849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.375044+0000","last_change":"2024-09-12T03:19:07.528567+0000","last_active":"2024-09-12T03:19:20.375044+0000","last_peered":"2024-09-12T03:19:20.375044+0000","last_clean":"2024-09-12T03:19:20.375044+0000","last_became_active":"2024-09-12T03:19:07.528123+0000","last_became_peered":"2024-09-12T03:19:07.528123+0000","last_unstale":"2024-09-12T03:19:20.375044+0000","last_undegraded":"2024-09-12T03:19:20.375044+0000","last_fullsized":"2024-09-12T03:19:20.375044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:34.416714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256025+0000","last_change":"2024-09-12T03:19:09.338836+0000","last_active":"2024-09-12T03:20:49.256025+0000","last_peered":"2024-09-12T03:20:49.256025+0000","last_clean":"2024-09-12T03:20:49.256025+0000","last_became_active":"2024-09-12T03:19:09.337571+0000","last_became_peered":"2024-09-12T03:19:09.337571+0000","last_unstale":"2024-09-12T03:20:49.256025+0000","last_undegraded":"2024-09-12T03:20:49.256025+0000","last_fullsized":"2024-09-12T03:20:49.256025+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:39.039878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":330,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372792+0000","last_change":"2024-09-12T03:18:01.051311+0000","last_active":"2024-09-12T03:19:20.372792+0000","last_peered":"2024-09-12T03:19:20.372792+0000","last_clean":"2024-09-12T03:19:20.372792+0000","last_became_active":"2024-09-12T03:18:01.050849+0000","last_became_peered":"2024-09-12T03:18:01.050849+0000","last_unstale":"2024-09-12T03:19:20.372792+0000","last_undegraded":"2024-09-12T03:19:20.372792+0000","last_fullsized":"2024-09-12T03:19:20.372792+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:14:38.096222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:14:38.096222+0000","last_clean_scrub_stamp":"2024-09-12T03:14:38.096222+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:33.067603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382035+0000","last_change":"2024-09-12T03:19:07.206045+0000","last_active":"2024-09-12T03:19:20.382035+0000","last_peered":"2024-09-12T03:19:20.382035+0000","last_clean":"2024-09-12T03:19:20.382035+0000","last_became_active":"2024-09-12T03:19:07.205594+0000","last_became_peered":"2024-09-12T03:19:07.205594+0000","last_unstale":"2024-09-12T03:19:20.382035+0000","last_undegraded":"2024-09-12T03:19:20.382035+0000","last_fullsized":"2024-09-12T03:19:20.382035+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:12.811906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.244733+0000","last_change":"2024-09-12T03:19:20.387274+0000","last_active":"2024-09-12T03:20:49.244733+0000","last_peered":"2024-09-12T03:20:49.244733+0000","last_clean":"2024-09-12T03:20:49.244733+0000","last_became_active":"2024-09-12T03:19:20.386878+0000","last_became_peered":"2024-09-12T03:19:20.386878+0000","last_unstale":"2024-09-12T03:20:49.244733+0000","last_undegraded":"2024-09-12T03:20:49.244733+0000","last_fullsized":"2024-09-12T03:20:49.244733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:26.674032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372302+0000","last_change":"2024-09-12T03:19:07.206212+0000","last_active":"2024-09-12T03:19:20.372302+0000","last_peered":"2024-09-12T03:19:20.372302+0000","last_clean":"2024-09-12T03:19:20.372302+0000","last_became_active":"2024-09-12T03:19:07.205899+0000","last_became_peered":"2024-09-12T03:19:07.205899+0000","last_unstale":"2024-09-12T03:19:20.372302+0000","last_undegraded":"2024-09-12T03:19:20.372302+0000","last_fullsized":"2024-09-12T03:19:20.372302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:02.169436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250795+0000","last_change":"2024-09-12T03:19:20.377494+0000","last_active":"2024-09-12T03:20:49.250795+0000","last_peered":"2024-09-12T03:20:49.250795+0000","last_clean":"2024-09-12T03:20:49.250795+0000","last_became_active":"2024-09-12T03:19:20.377091+0000","last_became_peered":"2024-09-12T03:19:20.377091+0000","last_unstale":"2024-09-12T03:20:49.250795+0000","last_undegraded":"2024-09-12T03:20:49.250795+0000","last_fullsized":"2024-09-12T03:20:49.250795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:30.926853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":130,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.560665+0000","last_change":"2024-09-12T03:19:07.388486+0000","last_active":"2024-09-12T03:21:01.560665+0000","last_peered":"2024-09-12T03:21:01.560665+0000","last_clean":"2024-09-12T03:21:01.560665+0000","last_became_active":"2024-09-12T03:19:07.388125+0000","last_became_peered":"2024-09-12T03:19:07.388125+0000","last_unstale":"2024-09-12T03:21:01.560665+0000","last_undegraded":"2024-09-12T03:21:01.560665+0000","last_fullsized":"2024-09-12T03:21:01.560665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:46.218601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254069+0000","last_change":"2024-09-12T03:19:09.344308+0000","last_active":"2024-09-12T03:20:49.254069+0000","last_peered":"2024-09-12T03:20:49.254069+0000","last_clean":"2024-09-12T03:20:49.254069+0000","last_became_active":"2024-09-12T03:19:09.343987+0000","last_became_peered":"2024-09-12T03:19:09.343987+0000","last_unstale":"2024-09-12T03:20:49.254069+0000","last_undegraded":"2024-09-12T03:20:49.254069+0000","last_fullsized":"2024-09-12T03:20:49.254069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:16:11.616555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372560+0000","last_change":"2024-09-12T03:19:07.208619+0000","last_active":"2024-09-12T03:19:20.372560+0000","last_peered":"2024-09-12T03:19:20.372560+0000","last_clean":"2024-09-12T03:19:20.372560+0000","last_became_active":"2024-09-12T03:19:07.208254+0000","last_became_peered":"2024-09-12T03:19:07.208254+0000","last_unstale":"2024-09-12T03:19:20.372560+0000","last_undegraded":"2024-09-12T03:19:20.372560+0000","last_fullsized":"2024-09-12T03:19:20.372560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:56.177948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.238148+0000","last_change":"2024-09-12T03:19:09.341837+0000","last_active":"2024-09-12T03:20:49.238148+0000","last_peered":"2024-09-12T03:20:49.238148+0000","last_clean":"2024-09-12T03:20:49.238148+0000","last_became_active":"2024-09-12T03:19:09.341069+0000","last_became_peered":"2024-09-12T03:19:09.341069+0000","last_unstale":"2024-09-12T03:20:49.238148+0000","last_undegraded":"2024-09-12T03:20:49.238148+0000","last_fullsized":"2024-09-12T03:20:49.238148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:04.238986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372460+0000","last_change":"2024-09-12T03:19:07.388870+0000","last_active":"2024-09-12T03:19:20.372460+0000","last_peered":"2024-09-12T03:19:20.372460+0000","last_clean":"2024-09-12T03:19:20.372460+0000","last_became_active":"2024-09-12T03:19:07.388555+0000","last_became_peered":"2024-09-12T03:19:07.388555+0000","last_unstale":"2024-09-12T03:19:20.372460+0000","last_undegraded":"2024-09-12T03:19:20.372460+0000","last_fullsized":"2024-09-12T03:19:20.372460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:29.789454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.247542+0000","last_change":"2024-09-12T03:19:09.334714+0000","last_active":"2024-09-12T03:20:49.247542+0000","last_peered":"2024-09-12T03:20:49.247542+0000","last_clean":"2024-09-12T03:20:49.247542+0000","last_became_active":"2024-09-12T03:19:09.334413+0000","last_became_peered":"2024-09-12T03:19:09.334413+0000","last_unstale":"2024-09-12T03:20:49.247542+0000","last_undegraded":"2024-09-12T03:20:49.247542+0000","last_fullsized":"2024-09-12T03:20:49.247542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:52.222340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373797+0000","last_change":"2024-09-12T03:19:07.389055+0000","last_active":"2024-09-12T03:19:20.373797+0000","last_peered":"2024-09-12T03:19:20.373797+0000","last_clean":"2024-09-12T03:19:20.373797+0000","last_became_active":"2024-09-12T03:19:07.388673+0000","last_became_peered":"2024-09-12T03:19:07.388673+0000","last_unstale":"2024-09-12T03:19:20.373797+0000","last_undegraded":"2024-09-12T03:19:20.373797+0000","last_fullsized":"2024-09-12T03:19:20.373797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:20.974533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248932+0000","last_change":"2024-09-12T03:19:09.340105+0000","last_active":"2024-09-12T03:20:49.248932+0000","last_peered":"2024-09-12T03:20:49.248932+0000","last_clean":"2024-09-12T03:20:49.248932+0000","last_became_active":"2024-09-12T03:19:09.339642+0000","last_became_peered":"2024-09-12T03:19:09.339642+0000","last_unstale":"2024-09-12T03:20:49.248932+0000","last_undegraded":"2024-09-12T03:20:49.248932+0000","last_fullsized":"2024-09-12T03:20:49.248932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:01.624236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'15","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:52.343269+0000","last_change":"2024-09-12T03:19:07.207232+0000","last_active":"2024-09-12T03:20:52.343269+0000","last_peered":"2024-09-12T03:20:52.343269+0000","last_clean":"2024-09-12T03:20:52.343269+0000","last_became_active":"2024-09-12T03:19:07.206921+0000","last_became_peered":"2024-09-12T03:19:07.206921+0000","last_unstale":"2024-09-12T03:20:52.343269+0000","last_undegraded":"2024-09-12T03:20:52.343269+0000","last_fullsized":"2024-09-12T03:20:52.343269+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:25.457853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.252490+0000","last_change":"2024-09-12T03:19:09.330442+0000","last_active":"2024-09-12T03:20:49.252490+0000","last_peered":"2024-09-12T03:20:49.252490+0000","last_clean":"2024-09-12T03:20:49.252490+0000","last_became_active":"2024-09-12T03:19:09.330081+0000","last_became_peered":"2024-09-12T03:19:09.330081+0000","last_unstale":"2024-09-12T03:20:49.252490+0000","last_undegraded":"2024-09-12T03:20:49.252490+0000","last_fullsized":"2024-09-12T03:20:49.252490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:56.152562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.382392+0000","last_change":"2024-09-12T03:19:07.528646+0000","last_active":"2024-09-12T03:19:20.382392+0000","last_peered":"2024-09-12T03:19:20.382392+0000","last_clean":"2024-09-12T03:19:20.382392+0000","last_became_active":"2024-09-12T03:19:07.528334+0000","last_became_peered":"2024-09-12T03:19:07.528334+0000","last_unstale":"2024-09-12T03:19:20.382392+0000","last_undegraded":"2024-09-12T03:19:20.382392+0000","last_fullsized":"2024-09-12T03:19:20.382392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:40.425579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.256685+0000","last_change":"2024-09-12T03:19:09.331993+0000","last_active":"2024-09-12T03:20:49.256685+0000","last_peered":"2024-09-12T03:20:49.256685+0000","last_clean":"2024-09-12T03:20:49.256685+0000","last_became_active":"2024-09-12T03:19:09.331698+0000","last_became_peered":"2024-09-12T03:19:09.331698+0000","last_unstale":"2024-09-12T03:20:49.256685+0000","last_undegraded":"2024-09-12T03:20:49.256685+0000","last_fullsized":"2024-09-12T03:20:49.256685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:03:54.182833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.370179+0000","last_change":"2024-09-12T03:19:07.388297+0000","last_active":"2024-09-12T03:19:20.370179+0000","last_peered":"2024-09-12T03:19:20.370179+0000","last_clean":"2024-09-12T03:19:20.370179+0000","last_became_active":"2024-09-12T03:19:07.388038+0000","last_became_peered":"2024-09-12T03:19:07.388038+0000","last_unstale":"2024-09-12T03:19:20.370179+0000","last_undegraded":"2024-09-12T03:19:20.370179+0000","last_fullsized":"2024-09-12T03:19:20.370179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:45.874195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.212080+0000","last_change":"2024-09-12T03:19:09.331525+0000","last_active":"2024-09-12T03:20:49.212080+0000","last_peered":"2024-09-12T03:20:49.212080+0000","last_clean":"2024-09-12T03:20:49.212080+0000","last_became_active":"2024-09-12T03:19:09.330903+0000","last_became_peered":"2024-09-12T03:19:09.330903+0000","last_unstale":"2024-09-12T03:20:49.212080+0000","last_undegraded":"2024-09-12T03:20:49.212080+0000","last_fullsized":"2024-09-12T03:20:49.212080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:02.455132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372235+0000","last_change":"2024-09-12T03:19:07.205181+0000","last_active":"2024-09-12T03:19:20.372235+0000","last_peered":"2024-09-12T03:19:20.372235+0000","last_clean":"2024-09-12T03:19:20.372235+0000","last_became_active":"2024-09-12T03:19:07.204726+0000","last_became_peered":"2024-09-12T03:19:07.204726+0000","last_unstale":"2024-09-12T03:19:20.372235+0000","last_undegraded":"2024-09-12T03:19:20.372235+0000","last_fullsized":"2024-09-12T03:19:20.372235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:26.112800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.250768+0000","last_change":"2024-09-12T03:19:09.345338+0000","last_active":"2024-09-12T03:20:49.250768+0000","last_peered":"2024-09-12T03:20:49.250768+0000","last_clean":"2024-09-12T03:20:49.250768+0000","last_became_active":"2024-09-12T03:19:09.345023+0000","last_became_peered":"2024-09-12T03:19:09.345023+0000","last_unstale":"2024-09-12T03:20:49.250768+0000","last_undegraded":"2024-09-12T03:20:49.250768+0000","last_fullsized":"2024-09-12T03:20:49.250768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:39.184065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.374938+0000","last_change":"2024-09-12T03:19:07.211954+0000","last_active":"2024-09-12T03:19:20.374938+0000","last_peered":"2024-09-12T03:19:20.374938+0000","last_clean":"2024-09-12T03:19:20.374938+0000","last_became_active":"2024-09-12T03:19:07.211509+0000","last_became_peered":"2024-09-12T03:19:07.211509+0000","last_unstale":"2024-09-12T03:19:20.374938+0000","last_undegraded":"2024-09-12T03:19:20.374938+0000","last_fullsized":"2024-09-12T03:19:20.374938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:27.461924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.244962+0000","last_change":"2024-09-12T03:19:09.331168+0000","last_active":"2024-09-12T03:20:49.244962+0000","last_peered":"2024-09-12T03:20:49.244962+0000","last_clean":"2024-09-12T03:20:49.244962+0000","last_became_active":"2024-09-12T03:19:09.329592+0000","last_became_peered":"2024-09-12T03:19:09.329592+0000","last_unstale":"2024-09-12T03:20:49.244962+0000","last_undegraded":"2024-09-12T03:20:49.244962+0000","last_fullsized":"2024-09-12T03:20:49.244962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:46.127861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373572+0000","last_change":"2024-09-12T03:19:07.207802+0000","last_active":"2024-09-12T03:19:20.373572+0000","last_peered":"2024-09-12T03:19:20.373572+0000","last_clean":"2024-09-12T03:19:20.373572+0000","last_became_active":"2024-09-12T03:19:07.207428+0000","last_became_peered":"2024-09-12T03:19:07.207428+0000","last_unstale":"2024-09-12T03:19:20.373572+0000","last_undegraded":"2024-09-12T03:19:20.373572+0000","last_fullsized":"2024-09-12T03:19:20.373572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:45.135019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.255394+0000","last_change":"2024-09-12T03:19:09.336094+0000","last_active":"2024-09-12T03:20:49.255394+0000","last_peered":"2024-09-12T03:20:49.255394+0000","last_clean":"2024-09-12T03:20:49.255394+0000","last_became_active":"2024-09-12T03:19:09.335779+0000","last_became_peered":"2024-09-12T03:19:09.335779+0000","last_unstale":"2024-09-12T03:20:49.255394+0000","last_undegraded":"2024-09-12T03:20:49.255394+0000","last_fullsized":"2024-09-12T03:20:49.255394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:37.019082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.254523+0000","last_change":"2024-09-12T03:19:09.344537+0000","last_active":"2024-09-12T03:20:49.254523+0000","last_peered":"2024-09-12T03:20:49.254523+0000","last_clean":"2024-09-12T03:20:49.254523+0000","last_became_active":"2024-09-12T03:19:09.343879+0000","last_became_peered":"2024-09-12T03:19:09.343879+0000","last_unstale":"2024-09-12T03:20:49.254523+0000","last_undegraded":"2024-09-12T03:20:49.254523+0000","last_fullsized":"2024-09-12T03:20:49.254523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:40.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.373973+0000","last_change":"2024-09-12T03:19:07.208040+0000","last_active":"2024-09-12T03:19:20.373973+0000","last_peered":"2024-09-12T03:19:20.373973+0000","last_clean":"2024-09-12T03:19:20.373973+0000","last_became_active":"2024-09-12T03:19:07.207658+0000","last_became_peered":"2024-09-12T03:19:07.207658+0000","last_unstale":"2024-09-12T03:19:20.373973+0000","last_undegraded":"2024-09-12T03:19:20.373973+0000","last_fullsized":"2024-09-12T03:19:20.373973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:48.620707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265759+0000","last_change":"2024-09-12T03:19:09.333837+0000","last_active":"2024-09-12T03:20:49.265759+0000","last_peered":"2024-09-12T03:20:49.265759+0000","last_clean":"2024-09-12T03:20:49.265759+0000","last_became_active":"2024-09-12T03:19:09.333194+0000","last_became_peered":"2024-09-12T03:19:09.333194+0000","last_unstale":"2024-09-12T03:20:49.265759+0000","last_undegraded":"2024-09-12T03:20:49.265759+0000","last_fullsized":"2024-09-12T03:20:49.265759+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:18.766181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.385455+0000","last_change":"2024-09-12T03:19:20.384978+0000","last_active":"2024-09-12T03:19:20.385455+0000","last_peered":"2024-09-12T03:19:20.385455+0000","last_clean":"2024-09-12T03:19:20.385455+0000","last_became_active":"2024-09-12T03:19:20.376883+0000","last_became_peered":"2024-09-12T03:19:20.376883+0000","last_unstale":"2024-09-12T03:19:20.385455+0000","last_undegraded":"2024-09-12T03:19:20.385455+0000","last_fullsized":"2024-09-12T03:19:20.385455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:32.542334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.265022+0000","last_change":"2024-09-12T03:19:09.334684+0000","last_active":"2024-09-12T03:20:49.265022+0000","last_peered":"2024-09-12T03:20:49.265022+0000","last_clean":"2024-09-12T03:20:49.265022+0000","last_became_active":"2024-09-12T03:19:09.334281+0000","last_became_peered":"2024-09-12T03:19:09.334281+0000","last_unstale":"2024-09-12T03:20:49.265022+0000","last_undegraded":"2024-09-12T03:20:49.265022+0000","last_fullsized":"2024-09-12T03:20:49.265022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:44.837067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.372829+0000","last_change":"2024-09-12T03:19:07.388555+0000","last_active":"2024-09-12T03:19:20.372829+0000","last_peered":"2024-09-12T03:19:20.372829+0000","last_clean":"2024-09-12T03:19:20.372829+0000","last_became_active":"2024-09-12T03:19:07.388206+0000","last_became_peered":"2024-09-12T03:19:07.388206+0000","last_unstale":"2024-09-12T03:19:20.372829+0000","last_undegraded":"2024-09-12T03:19:20.372829+0000","last_fullsized":"2024-09-12T03:19:20.372829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:33.628294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:20:49.248105+0000","last_change":"2024-09-12T03:19:09.339108+0000","last_active":"2024-09-12T03:20:49.248105+0000","last_peered":"2024-09-12T03:20:49.248105+0000","last_clean":"2024-09-12T03:20:49.248105+0000","last_became_active":"2024-09-12T03:19:09.338819+0000","last_became_peered":"2024-09-12T03:19:09.338819+0000","last_unstale":"2024-09-12T03:20:49.248105+0000","last_undegraded":"2024-09-12T03:20:49.248105+0000","last_fullsized":"2024-09-12T03:20:49.248105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:08.305737+0000","last_clean_scrub_stamp":"2024-09-12T03:19:08.305737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:32.864900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.379495+0000","last_change":"2024-09-12T03:19:07.207197+0000","last_active":"2024-09-12T03:19:20.379495+0000","last_peered":"2024-09-12T03:19:20.379495+0000","last_clean":"2024-09-12T03:19:20.379495+0000","last_became_active":"2024-09-12T03:19:07.206879+0000","last_became_peered":"2024-09-12T03:19:07.206879+0000","last_unstale":"2024-09-12T03:19:20.379495+0000","last_undegraded":"2024-09-12T03:19:20.379495+0000","last_fullsized":"2024-09-12T03:19:20.379495+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:06.188595+0000","last_clean_scrub_stamp":"2024-09-12T03:19:06.188595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:00:46.269636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73845,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":665,"num_read_kb":408,"num_write":182,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":221535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579940,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":433387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36900,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708252,"statfs":{"total":95995035648,"available":95957250048,"internally_reserved":0,"allocated":1593344,"data_stored":1093834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972397,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":433589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201331,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":433509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430264,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1499136,"data_stored":1026676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593789,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":432978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1511424,"data_stored":1025967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":897024,"data_stored":433808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":433289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":503929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40444,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704708,"statfs":{"total":95995035648,"available":95953620992,"internally_reserved":0,"allocated":962560,"data_stored":501378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":434249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:21:08.140 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:21:08.409 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:08.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.981+0000 7f104d448640 1 -- 172.21.15.6:0/804517753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1048103780 msgr2=0x7f1048107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:08.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.981+0000 7f104d448640 1 --2- 172.21.15.6:0/804517753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1048103780 0x7f1048107bc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f103c009930 tx=0x7f103c02a420 comp rx=0 tx=0).stop 2024-09-12T03:21:08.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.981+0000 7f104d448640 1 -- 172.21.15.6:0/804517753 shutdown_connections 2024-09-12T03:21:08.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.981+0000 7f104d448640 1 --2- 172.21.15.6:0/804517753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1048108300 0x7f104810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.981+0000 7f104d448640 1 --2- 172.21.15.6:0/804517753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1048103780 0x7f1048107bc0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.981+0000 7f104d448640 1 --2- 172.21.15.6:0/804517753 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1048102dd0 0x7f10481031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.981+0000 7f104d448640 1 -- 172.21.15.6:0/804517753 >> 172.21.15.6:0/804517753 conn(0x7f10480fc7d0 msgr2=0x7f10480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:08.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.981+0000 7f104d448640 1 -- 172.21.15.6:0/804517753 shutdown_connections 2024-09-12T03:21:08.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.981+0000 7f104d448640 1 -- 172.21.15.6:0/804517753 wait complete. 2024-09-12T03:21:08.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.982+0000 7f104d448640 1 Processor -- start 2024-09-12T03:21:08.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.982+0000 7f104d448640 1 -- start start 2024-09-12T03:21:08.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.982+0000 7f104d448640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1048102dd0 0x7f10481a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:08.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.982+0000 7f104d448640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1048103780 0x7f10481a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:08.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.982+0000 7f104d448640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1048108300 0x7f104819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:08.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.982+0000 7f104d448640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1048112050 con 0x7f1048103780 2024-09-12T03:21:08.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.982+0000 7f104d448640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f1048111ed0 con 0x7f1048102dd0 2024-09-12T03:21:08.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.982+0000 7f104d448640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f10481121d0 con 0x7f1048108300 2024-09-12T03:21:08.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.982+0000 7f10467fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1048103780 0x7f10481a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:08.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.982+0000 7f10467fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1048103780 0x7f10481a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42106/0 (socket says 172.21.15.6:42106) 2024-09-12T03:21:08.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.982+0000 7f10467fc640 1 -- 172.21.15.6:0/2553707134 learned_addr learned my addr 172.21.15.6:0/2553707134 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:08.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.982+0000 7f1046ffd640 1 --2- 172.21.15.6:0/2553707134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1048102dd0 0x7f10481a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:08.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.982+0000 7f10477fe640 1 --2- 172.21.15.6:0/2553707134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1048108300 0x7f104819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:08.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.983+0000 7f10467fc640 1 -- 172.21.15.6:0/2553707134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1048108300 msgr2=0x7f104819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:08.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.983+0000 7f10467fc640 1 --2- 172.21.15.6:0/2553707134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1048108300 0x7f104819a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.983+0000 7f10467fc640 1 -- 172.21.15.6:0/2553707134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1048102dd0 msgr2=0x7f10481a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:08.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.983+0000 7f10467fc640 1 --2- 172.21.15.6:0/2553707134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1048102dd0 0x7f10481a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.983+0000 7f10467fc640 1 -- 172.21.15.6:0/2553707134 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104819acf0 con 0x7f1048103780 2024-09-12T03:21:08.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.983+0000 7f10477fe640 1 --2- 172.21.15.6:0/2553707134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1048108300 0x7f104819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:08.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.983+0000 7f10467fc640 1 --2- 172.21.15.6:0/2553707134 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1048103780 0x7f10481a0940 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f103c005940 tx=0x7f103c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:08.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.983+0000 7f102ffff640 1 -- 172.21.15.6:0/2553707134 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c03f030 con 0x7f1048103780 2024-09-12T03:21:08.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.983+0000 7f102ffff640 1 -- 172.21.15.6:0/2553707134 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f103c003ce0 con 0x7f1048103780 2024-09-12T03:21:08.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.983+0000 7f104d448640 1 -- 172.21.15.6:0/2553707134 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104819af80 con 0x7f1048103780 2024-09-12T03:21:08.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.983+0000 7f102ffff640 1 -- 172.21.15.6:0/2553707134 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c035070 con 0x7f1048103780 2024-09-12T03:21:08.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.983+0000 7f104d448640 1 -- 172.21.15.6:0/2553707134 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10481a72d0 con 0x7f1048103780 2024-09-12T03:21:08.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.984+0000 7f104d448640 1 -- 172.21.15.6:0/2553707134 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10481031b0 con 0x7f1048103780 2024-09-12T03:21:08.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.985+0000 7f102ffff640 1 -- 172.21.15.6:0/2553707134 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f103c042030 con 0x7f1048103780 2024-09-12T03:21:08.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.986+0000 7f102ffff640 1 --2- 172.21.15.6:0/2553707134 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f10300a2c90 0x7f10300a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:08.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.986+0000 7f1046ffd640 1 --2- 172.21.15.6:0/2553707134 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f10300a2c90 0x7f10300a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:08.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.986+0000 7f102ffff640 1 -- 172.21.15.6:0/2553707134 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f103c0ef8a0 con 0x7f1048103780 2024-09-12T03:21:08.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.987+0000 7f1046ffd640 1 --2- 172.21.15.6:0/2553707134 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f10300a2c90 0x7f10300a5150 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f103800abe0 tx=0x7f103800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:08.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:08.990+0000 7f102ffff640 1 -- 172.21.15.6:0/2553707134 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f103c0bc670 con 0x7f1048103780 2024-09-12T03:21:09.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:09.200+0000 7f104d448640 1 -- 172.21.15.6:0/2553707134 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f104819baf0 con 0x7f1048103780 2024-09-12T03:21:09.200 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:09 smithi006 ceph-mon[30991]: pgmap v336: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:21:09.200 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:09 smithi006 ceph-mon[30991]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:21:09.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:09.200+0000 7f102ffff640 1 -- 172.21.15.6:0/2553707134 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11908 (secure 0 0 0) 0x7f104819baf0 con 0x7f1048103780 2024-09-12T03:21:09.201 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:21:09.201 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"btime":"2024-09-12T03:20:52:974598+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:20:52.261388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24463]},"id":1}]} 2024-09-12T03:21:09.201 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 25 2024-09-12T03:21:09.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:09.203+0000 7f104d448640 1 -- 172.21.15.6:0/2553707134 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f10300a2c90 msgr2=0x7f10300a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:09.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:09.203+0000 7f104d448640 1 --2- 172.21.15.6:0/2553707134 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f10300a2c90 0x7f10300a5150 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f103800abe0 tx=0x7f103800a2d0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:09.203+0000 7f104d448640 1 -- 172.21.15.6:0/2553707134 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1048103780 msgr2=0x7f10481a0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:09.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:09.203+0000 7f104d448640 1 --2- 172.21.15.6:0/2553707134 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1048103780 0x7f10481a0940 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f103c005940 tx=0x7f103c0042f0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:09.203+0000 7f104d448640 1 -- 172.21.15.6:0/2553707134 shutdown_connections 2024-09-12T03:21:09.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:09.203+0000 7f104d448640 1 --2- 172.21.15.6:0/2553707134 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f10300a2c90 0x7f10300a5150 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:09.203+0000 7f104d448640 1 --2- 172.21.15.6:0/2553707134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1048108300 0x7f104819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:09.203+0000 7f104d448640 1 --2- 172.21.15.6:0/2553707134 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1048103780 0x7f10481a0940 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:09.203+0000 7f104d448640 1 --2- 172.21.15.6:0/2553707134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1048102dd0 0x7f10481a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:09.203+0000 7f104d448640 1 -- 172.21.15.6:0/2553707134 >> 172.21.15.6:0/2553707134 conn(0x7f10480fc7d0 msgr2=0x7f10480fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:09.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:09.203+0000 7f104d448640 1 -- 172.21.15.6:0/2553707134 shutdown_connections 2024-09-12T03:21:09.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:09.203+0000 7f104d448640 1 -- 172.21.15.6:0/2553707134 wait complete. 2024-09-12T03:21:09.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:09 smithi161 ceph-mon[35485]: pgmap v336: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:21:09.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:09 smithi161 ceph-mon[35485]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:21:09.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:09 smithi118 ceph-mon[35048]: pgmap v336: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:21:09.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:09 smithi118 ceph-mon[35048]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:21:09.930 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:21:10.193 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:10.314 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:10 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2553707134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:10.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:10 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2553707134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:10.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:10 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2553707134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:10.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.670+0000 7f934d22b640 1 -- 172.21.15.6:0/3664182710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9348101ba0 msgr2=0x7f934810d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.670+0000 7f934d22b640 1 --2- 172.21.15.6:0/3664182710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9348101ba0 0x7f934810d970 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9340009ca0 tx=0x7f934002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.670+0000 7f934d22b640 1 -- 172.21.15.6:0/3664182710 shutdown_connections 2024-09-12T03:21:10.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.670+0000 7f934d22b640 1 --2- 172.21.15.6:0/3664182710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9348101ba0 0x7f934810d970 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.670+0000 7f934d22b640 1 --2- 172.21.15.6:0/3664182710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9348101200 0x7f9348101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.670+0000 7f934d22b640 1 --2- 172.21.15.6:0/3664182710 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9348105ad0 0x7f9348105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.670+0000 7f934d22b640 1 -- 172.21.15.6:0/3664182710 >> 172.21.15.6:0/3664182710 conn(0x7f9348077210 msgr2=0x7f93480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:10.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.670+0000 7f934d22b640 1 -- 172.21.15.6:0/3664182710 shutdown_connections 2024-09-12T03:21:10.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.670+0000 7f934d22b640 1 -- 172.21.15.6:0/3664182710 wait complete. 2024-09-12T03:21:10.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.671+0000 7f934d22b640 1 Processor -- start 2024-09-12T03:21:10.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.671+0000 7f934d22b640 1 -- start start 2024-09-12T03:21:10.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.671+0000 7f934d22b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9348101200 0x7f93481983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.671+0000 7f934d22b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9348101ba0 0x7f9348198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.671+0000 7f934d22b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9348105ad0 0x7f934819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.671+0000 7f934d22b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9348111fe0 con 0x7f9348105ad0 2024-09-12T03:21:10.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.671+0000 7f934d22b640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f9348111e60 con 0x7f9348101200 2024-09-12T03:21:10.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.671+0000 7f934d22b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9348112160 con 0x7f9348101ba0 2024-09-12T03:21:10.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.671+0000 7f934759e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9348105ad0 0x7f934819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.671+0000 7f934759e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9348105ad0 0x7f934819cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42120/0 (socket says 172.21.15.6:42120) 2024-09-12T03:21:10.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.671+0000 7f934759e640 1 -- 172.21.15.6:0/1893614033 learned_addr learned my addr 172.21.15.6:0/1893614033 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:10.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.672+0000 7f9346d9d640 1 --2- 172.21.15.6:0/1893614033 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9348101200 0x7f93481983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.672+0000 7f934659c640 1 --2- 172.21.15.6:0/1893614033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9348101ba0 0x7f9348198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.672+0000 7f934759e640 1 -- 172.21.15.6:0/1893614033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9348101ba0 msgr2=0x7f9348198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.672+0000 7f934759e640 1 --2- 172.21.15.6:0/1893614033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9348101ba0 0x7f9348198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.672+0000 7f934759e640 1 -- 172.21.15.6:0/1893614033 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9348101200 msgr2=0x7f93481983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.672+0000 7f934759e640 1 --2- 172.21.15.6:0/1893614033 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9348101200 0x7f93481983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.672+0000 7f934759e640 1 -- 172.21.15.6:0/1893614033 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934819d370 con 0x7f9348105ad0 2024-09-12T03:21:10.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.672+0000 7f934759e640 1 --2- 172.21.15.6:0/1893614033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9348105ad0 0x7f934819cc90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9340003f40 tx=0x7f9340003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:10.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.672+0000 7f932ffff640 1 -- 172.21.15.6:0/1893614033 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340031790 con 0x7f9348105ad0 2024-09-12T03:21:10.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.672+0000 7f934d22b640 1 -- 172.21.15.6:0/1893614033 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f934819d600 con 0x7f9348105ad0 2024-09-12T03:21:10.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.672+0000 7f932ffff640 1 -- 172.21.15.6:0/1893614033 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9340031930 con 0x7f9348105ad0 2024-09-12T03:21:10.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.672+0000 7f932ffff640 1 -- 172.21.15.6:0/1893614033 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340030bf0 con 0x7f9348105ad0 2024-09-12T03:21:10.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.672+0000 7f934d22b640 1 -- 172.21.15.6:0/1893614033 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93481a4f40 con 0x7f9348105ad0 2024-09-12T03:21:10.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.673+0000 7f934d22b640 1 -- 172.21.15.6:0/1893614033 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9348101660 con 0x7f9348105ad0 2024-09-12T03:21:10.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.675+0000 7f932ffff640 1 -- 172.21.15.6:0/1893614033 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f9340004b40 con 0x7f9348105ad0 2024-09-12T03:21:10.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.675+0000 7f932ffff640 1 --2- 172.21.15.6:0/1893614033 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f93300a2c10 0x7f93300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.676+0000 7f9346d9d640 1 --2- 172.21.15.6:0/1893614033 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f93300a2c10 0x7f93300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.676+0000 7f932ffff640 1 -- 172.21.15.6:0/1893614033 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f93400f0690 con 0x7f9348105ad0 2024-09-12T03:21:10.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.676+0000 7f9346d9d640 1 --2- 172.21.15.6:0/1893614033 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f93300a2c10 0x7f93300a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f933800bbe0 tx=0x7f933800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:10.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.679+0000 7f932ffff640 1 -- 172.21.15.6:0/1893614033 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f93400bd660 con 0x7f9348105ad0 2024-09-12T03:21:10.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.893+0000 7f934d22b640 1 -- 172.21.15.6:0/1893614033 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93481a5200 con 0x7f9348105ad0 2024-09-12T03:21:10.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.893+0000 7f932ffff640 1 -- 172.21.15.6:0/1893614033 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11908 (secure 0 0 0) 0x7f93400bcf60 con 0x7f9348105ad0 2024-09-12T03:21:10.894 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:21:10.894 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"btime":"2024-09-12T03:20:52:974598+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:20:52.261388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24463]},"id":1}]} 2024-09-12T03:21:10.894 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 25 2024-09-12T03:21:10.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.895+0000 7f934d22b640 1 -- 172.21.15.6:0/1893614033 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f93300a2c10 msgr2=0x7f93300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.895+0000 7f934d22b640 1 --2- 172.21.15.6:0/1893614033 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f93300a2c10 0x7f93300a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f933800bbe0 tx=0x7f933800b390 comp rx=0 tx=0).stop 2024-09-12T03:21:10.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.895+0000 7f934d22b640 1 -- 172.21.15.6:0/1893614033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9348105ad0 msgr2=0x7f934819cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.895+0000 7f934d22b640 1 --2- 172.21.15.6:0/1893614033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9348105ad0 0x7f934819cc90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9340003f40 tx=0x7f9340003f70 comp rx=0 tx=0).stop 2024-09-12T03:21:10.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.895+0000 7f934d22b640 1 -- 172.21.15.6:0/1893614033 shutdown_connections 2024-09-12T03:21:10.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.895+0000 7f934d22b640 1 --2- 172.21.15.6:0/1893614033 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f93300a2c10 0x7f93300a50d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.895+0000 7f934d22b640 1 --2- 172.21.15.6:0/1893614033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9348105ad0 0x7f934819cc90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.895+0000 7f934d22b640 1 --2- 172.21.15.6:0/1893614033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9348101ba0 0x7f9348198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.895+0000 7f934d22b640 1 --2- 172.21.15.6:0/1893614033 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9348101200 0x7f93481983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.895+0000 7f934d22b640 1 -- 172.21.15.6:0/1893614033 >> 172.21.15.6:0/1893614033 conn(0x7f9348077210 msgr2=0x7f9348103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:10.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.896+0000 7f934d22b640 1 -- 172.21.15.6:0/1893614033 shutdown_connections 2024-09-12T03:21:10.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:10.896+0000 7f934d22b640 1 -- 172.21.15.6:0/1893614033 wait complete. 2024-09-12T03:21:11.354 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:11 smithi006 ceph-mon[30991]: pgmap v337: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-12T03:21:11.354 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:11 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1893614033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:11.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:11 smithi161 ceph-mon[35485]: pgmap v337: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-12T03:21:11.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:11 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1893614033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:11.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:11 smithi118 ceph-mon[35048]: pgmap v337: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-12T03:21:11.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:11 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1893614033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:11.588 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:21:11.849 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:12.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.305+0000 7f90c87a3640 1 -- 172.21.15.6:0/671044468 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90c0100500 msgr2=0x7f90c01008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:12.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.305+0000 7f90c87a3640 1 --2- 172.21.15.6:0/671044468 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90c0100500 0x7f90c01008e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009900 tx=0x7f90bc02a410 comp rx=0 tx=0).stop 2024-09-12T03:21:12.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.305+0000 7f90c87a3640 1 -- 172.21.15.6:0/671044468 shutdown_connections 2024-09-12T03:21:12.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.305+0000 7f90c87a3640 1 --2- 172.21.15.6:0/671044468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90c0106ed0 0x7f90c010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.305+0000 7f90c87a3640 1 --2- 172.21.15.6:0/671044468 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90c0100eb0 0x7f90c01067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.305+0000 7f90c87a3640 1 --2- 172.21.15.6:0/671044468 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90c0100500 0x7f90c01008e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.305+0000 7f90c87a3640 1 -- 172.21.15.6:0/671044468 >> 172.21.15.6:0/671044468 conn(0x7f90c00fc5c0 msgr2=0x7f90c00fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:12.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.305+0000 7f90c87a3640 1 -- 172.21.15.6:0/671044468 shutdown_connections 2024-09-12T03:21:12.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.305+0000 7f90c87a3640 1 -- 172.21.15.6:0/671044468 wait complete. 2024-09-12T03:21:12.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.306+0000 7f90c87a3640 1 Processor -- start 2024-09-12T03:21:12.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.306+0000 7f90c87a3640 1 -- start start 2024-09-12T03:21:12.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.306+0000 7f90c87a3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90c0100500 0x7f90c0101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:12.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.306+0000 7f90c87a3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90c0100eb0 0x7f90c01019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:12.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.306+0000 7f90c87a3640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90c0106ed0 0x7f90c0101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:12.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.306+0000 7f90c87a3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f90c0111f80 con 0x7f90c0100500 2024-09-12T03:21:12.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.306+0000 7f90c87a3640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f90c0111e00 con 0x7f90c0106ed0 2024-09-12T03:21:12.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.306+0000 7f90c87a3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f90c0112100 con 0x7f90c0100eb0 2024-09-12T03:21:12.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.306+0000 7f90c653f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90c0100500 0x7f90c0101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:12.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.306+0000 7f90c653f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90c0100500 0x7f90c0101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42132/0 (socket says 172.21.15.6:42132) 2024-09-12T03:21:12.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.306+0000 7f90c5d3e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90c0100eb0 0x7f90c01019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:12.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.306+0000 7f90c653f640 1 -- 172.21.15.6:0/4067938436 learned_addr learned my addr 172.21.15.6:0/4067938436 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:12.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.306+0000 7f90c6d40640 1 --2- 172.21.15.6:0/4067938436 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90c0106ed0 0x7f90c0101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:12.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.306+0000 7f90c653f640 1 -- 172.21.15.6:0/4067938436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90c0100eb0 msgr2=0x7f90c01019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:12.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.306+0000 7f90c653f640 1 --2- 172.21.15.6:0/4067938436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90c0100eb0 0x7f90c01019c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.306+0000 7f90c653f640 1 -- 172.21.15.6:0/4067938436 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90c0106ed0 msgr2=0x7f90c0101f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:12.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.306+0000 7f90c653f640 1 --2- 172.21.15.6:0/4067938436 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90c0106ed0 0x7f90c0101f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.306+0000 7f90c653f640 1 -- 172.21.15.6:0/4067938436 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c01a68e0 con 0x7f90c0100500 2024-09-12T03:21:12.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.306+0000 7f90c6d40640 1 --2- 172.21.15.6:0/4067938436 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90c0106ed0 0x7f90c0101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:12.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.307+0000 7f90c653f640 1 --2- 172.21.15.6:0/4067938436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90c0100500 0x7f90c0101480 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f90bc006010 tx=0x7f90bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:12.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.307+0000 7f90b77fe640 1 -- 172.21.15.6:0/4067938436 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc03f030 con 0x7f90c0100500 2024-09-12T03:21:12.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.307+0000 7f90c87a3640 1 -- 172.21.15.6:0/4067938436 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c01a6b70 con 0x7f90c0100500 2024-09-12T03:21:12.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.307+0000 7f90b77fe640 1 -- 172.21.15.6:0/4067938436 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90bc003ce0 con 0x7f90c0100500 2024-09-12T03:21:12.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.307+0000 7f90b77fe640 1 -- 172.21.15.6:0/4067938436 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc035070 con 0x7f90c0100500 2024-09-12T03:21:12.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.307+0000 7f90c87a3640 1 -- 172.21.15.6:0/4067938436 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c01a70a0 con 0x7f90c0100500 2024-09-12T03:21:12.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.308+0000 7f90c87a3640 1 -- 172.21.15.6:0/4067938436 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90c01008e0 con 0x7f90c0100500 2024-09-12T03:21:12.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.309+0000 7f90b77fe640 1 -- 172.21.15.6:0/4067938436 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f90bc042030 con 0x7f90c0100500 2024-09-12T03:21:12.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.310+0000 7f90b77fe640 1 --2- 172.21.15.6:0/4067938436 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f90ac0a2c10 0x7f90ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:12.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.310+0000 7f90c5d3e640 1 --2- 172.21.15.6:0/4067938436 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f90ac0a2c10 0x7f90ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:12.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.310+0000 7f90b77fe640 1 -- 172.21.15.6:0/4067938436 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f90bc0efaa0 con 0x7f90c0100500 2024-09-12T03:21:12.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.311+0000 7f90c5d3e640 1 --2- 172.21.15.6:0/4067938436 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f90ac0a2c10 0x7f90ac0a50d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f90c0102f10 tx=0x7f90b000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:12.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.313+0000 7f90b77fe640 1 -- 172.21.15.6:0/4067938436 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90bc0bc930 con 0x7f90c0100500 2024-09-12T03:21:12.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.491+0000 7f90c87a3640 1 -- 172.21.15.6:0/4067938436 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90c010a140 con 0x7f90c0100500 2024-09-12T03:21:12.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.492+0000 7f90b77fe640 1 -- 172.21.15.6:0/4067938436 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22607 (secure 0 0 0) 0x7f90bc0bc230 con 0x7f90c0100500 2024-09-12T03:21:12.493 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:21:12.493 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":82,"fsid":"89d99938-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:11:29.530545+0000","modified":"2024-09-12T03:19:20.366710+0000","last_up_change":"2024-09-12T03:18:21.292572+0000","last_in_change":"2024-09-12T03:17:58.013800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:14:37.662010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:06.143398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:08.305279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f55d68b-95a0-4ee2-a46f-bc422b38471e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6803","nonce":726729286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6805","nonce":726729286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6809","nonce":726729286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6807","nonce":726729286}]},"public_addr":"172.21.15.6:6803/726729286","cluster_addr":"172.21.15.6:6805/726729286","heartbeat_back_addr":"172.21.15.6:6809/726729286","heartbeat_front_addr":"172.21.15.6:6807/726729286","state":["exists","up"]},{"osd":1,"uuid":"83afdb01-c731-4de1-96c6-3f798549b1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6801","nonce":3377555415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6803","nonce":3377555415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6807","nonce":3377555415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6805","nonce":3377555415}]},"public_addr":"172.21.15.118:6801/3377555415","cluster_addr":"172.21.15.118:6803/3377555415","heartbeat_back_addr":"172.21.15.118:6807/3377555415","heartbeat_front_addr":"172.21.15.118:6805/3377555415","state":["exists","up"]},{"osd":2,"uuid":"aed718b4-b849-403a-b81c-3a140b5c380c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6801","nonce":392541009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6803","nonce":392541009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6807","nonce":392541009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6805","nonce":392541009}]},"public_addr":"172.21.15.161:6801/392541009","cluster_addr":"172.21.15.161:6803/392541009","heartbeat_back_addr":"172.21.15.161:6807/392541009","heartbeat_front_addr":"172.21.15.161:6805/392541009","state":["exists","up"]},{"osd":3,"uuid":"25ced4c1-bf31-402d-8397-d42fd57f615f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6811","nonce":1892039151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6813","nonce":1892039151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6817","nonce":1892039151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6815","nonce":1892039151}]},"public_addr":"172.21.15.6:6811/1892039151","cluster_addr":"172.21.15.6:6813/1892039151","heartbeat_back_addr":"172.21.15.6:6817/1892039151","heartbeat_front_addr":"172.21.15.6:6815/1892039151","state":["exists","up"]},{"osd":4,"uuid":"1d884604-920b-4c35-b41e-d629ad9f9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6809","nonce":150177024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6811","nonce":150177024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6815","nonce":150177024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6813","nonce":150177024}]},"public_addr":"172.21.15.118:6809/150177024","cluster_addr":"172.21.15.118:6811/150177024","heartbeat_back_addr":"172.21.15.118:6815/150177024","heartbeat_front_addr":"172.21.15.118:6813/150177024","state":["exists","up"]},{"osd":5,"uuid":"1d40c0ef-99f6-426f-90c7-27579edc66af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6809","nonce":3758167854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6811","nonce":3758167854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6815","nonce":3758167854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6813","nonce":3758167854}]},"public_addr":"172.21.15.161:6809/3758167854","cluster_addr":"172.21.15.161:6811/3758167854","heartbeat_back_addr":"172.21.15.161:6815/3758167854","heartbeat_front_addr":"172.21.15.161:6813/3758167854","state":["exists","up"]},{"osd":6,"uuid":"9d9f9510-b107-40c2-9a98-9012715944b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6819","nonce":2129224874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6821","nonce":2129224874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6825","nonce":2129224874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6823","nonce":2129224874}]},"public_addr":"172.21.15.6:6819/2129224874","cluster_addr":"172.21.15.6:6821/2129224874","heartbeat_back_addr":"172.21.15.6:6825/2129224874","heartbeat_front_addr":"172.21.15.6:6823/2129224874","state":["exists","up"]},{"osd":7,"uuid":"84635ae1-3240-4954-af14-76bb3aa162e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6817","nonce":269980353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6819","nonce":269980353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6823","nonce":269980353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6821","nonce":269980353}]},"public_addr":"172.21.15.118:6817/269980353","cluster_addr":"172.21.15.118:6819/269980353","heartbeat_back_addr":"172.21.15.118:6823/269980353","heartbeat_front_addr":"172.21.15.118:6821/269980353","state":["exists","up"]},{"osd":8,"uuid":"6fd58e55-1492-4595-8647-823ea3e3c5d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6817","nonce":1783416134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6819","nonce":1783416134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6823","nonce":1783416134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6821","nonce":1783416134}]},"public_addr":"172.21.15.161:6817/1783416134","cluster_addr":"172.21.15.161:6819/1783416134","heartbeat_back_addr":"172.21.15.161:6823/1783416134","heartbeat_front_addr":"172.21.15.161:6821/1783416134","state":["exists","up"]},{"osd":9,"uuid":"90508a6d-a49c-43c6-a600-5bbccd8702ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6827","nonce":2987332365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6829","nonce":2987332365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6833","nonce":2987332365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6831","nonce":2987332365}]},"public_addr":"172.21.15.6:6827/2987332365","cluster_addr":"172.21.15.6:6829/2987332365","heartbeat_back_addr":"172.21.15.6:6833/2987332365","heartbeat_front_addr":"172.21.15.6:6831/2987332365","state":["exists","up"]},{"osd":10,"uuid":"911e86d1-669a-4935-82c0-7da51d141df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6825","nonce":2167146341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6827","nonce":2167146341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6831","nonce":2167146341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6829","nonce":2167146341}]},"public_addr":"172.21.15.118:6825/2167146341","cluster_addr":"172.21.15.118:6827/2167146341","heartbeat_back_addr":"172.21.15.118:6831/2167146341","heartbeat_front_addr":"172.21.15.118:6829/2167146341","state":["exists","up"]},{"osd":11,"uuid":"5a15c6a7-ff43-473c-bb48-0c129c316df5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6825","nonce":2889795796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6827","nonce":2889795796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6831","nonce":2889795796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6829","nonce":2889795796}]},"public_addr":"172.21.15.161:6825/2889795796","cluster_addr":"172.21.15.161:6827/2889795796","heartbeat_back_addr":"172.21.15.161:6831/2889795796","heartbeat_front_addr":"172.21.15.161:6829/2889795796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:42.912205+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:08.410669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:34.299134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:59.093181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:24.116635+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:49.566433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:15.719320+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:40.541017+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:05.178178+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:31.163877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:55.405110+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:19.417156+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6800/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/2620491916":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/4037656392":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/1377116573":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/3062800125":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:6800/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/4145338968":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/825701128":"2024-09-13T03:11:55.244257+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:21:12.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.495+0000 7f90c87a3640 1 -- 172.21.15.6:0/4067938436 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f90ac0a2c10 msgr2=0x7f90ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:12.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.495+0000 7f90c87a3640 1 --2- 172.21.15.6:0/4067938436 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f90ac0a2c10 0x7f90ac0a50d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f90c0102f10 tx=0x7f90b000b390 comp rx=0 tx=0).stop 2024-09-12T03:21:12.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.495+0000 7f90c87a3640 1 -- 172.21.15.6:0/4067938436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90c0100500 msgr2=0x7f90c0101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:12.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.495+0000 7f90c87a3640 1 --2- 172.21.15.6:0/4067938436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90c0100500 0x7f90c0101480 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f90bc006010 tx=0x7f90bc0042f0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.495+0000 7f90c87a3640 1 -- 172.21.15.6:0/4067938436 shutdown_connections 2024-09-12T03:21:12.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.495+0000 7f90c87a3640 1 --2- 172.21.15.6:0/4067938436 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f90ac0a2c10 0x7f90ac0a50d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.495+0000 7f90c87a3640 1 --2- 172.21.15.6:0/4067938436 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90c0106ed0 0x7f90c0101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.495+0000 7f90c87a3640 1 --2- 172.21.15.6:0/4067938436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90c0100eb0 0x7f90c01019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.495+0000 7f90c87a3640 1 --2- 172.21.15.6:0/4067938436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90c0100500 0x7f90c0101480 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.495+0000 7f90c87a3640 1 -- 172.21.15.6:0/4067938436 >> 172.21.15.6:0/4067938436 conn(0x7f90c00fc5c0 msgr2=0x7f90c01056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:12.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.495+0000 7f90c87a3640 1 -- 172.21.15.6:0/4067938436 shutdown_connections 2024-09-12T03:21:12.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:12.495+0000 7f90c87a3640 1 -- 172.21.15.6:0/4067938436 wait complete. 2024-09-12T03:21:13.073 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:13 smithi006 ceph-mon[30991]: pgmap v338: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T03:21:13.073 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:13 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/4067938436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:21:13.074 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:13 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:13.174 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-12T03:21:13.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:13 smithi161 ceph-mon[35485]: pgmap v338: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T03:21:13.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:13 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/4067938436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:21:13.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:13 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:13.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:13 smithi118 ceph-mon[35048]: pgmap v338: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T03:21:13.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:13 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/4067938436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:21:13.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:13 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:14.078 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:15.249 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:15 smithi006 ceph-mon[30991]: pgmap v339: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-12T03:21:15.250 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:15 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:21:15.250 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:15 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:21:15.250 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:15 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:21:15.250 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:15 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:21:15.282 INFO:teuthology.orchestra.run.smithi006.stdout:{"run_dir":"/var/run/ceph"} 2024-09-12T03:21:15.283 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-12T03:21:15.283 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 481 2024-09-12T03:21:15.283 DEBUG:tasks.quiescer:Yielding 2024-09-12T03:21:15.283 INFO:teuthology.run_tasks:Running task workunit... 2024-09-12T03:21:15.299 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T03:21:15.299 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-12T03:21:15.299 INFO:tasks.workunit:timeout=3h 2024-09-12T03:21:15.299 INFO:tasks.workunit:cleanup=True 2024-09-12T03:21:15.300 DEBUG:teuthology.orchestra.run.smithi006:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-12T03:21:15.300 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:21:15.318 INFO:teuthology.orchestra.run.smithi006.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-12T03:21:15.318 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-12T03:21:15.318 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 6ch/108d Inode: 1099511628283 Links: 2 2024-09-12T03:21:15.318 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:21:15.318 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-12T03:21:15.318 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-09-12 03:19:27.673315691 +0000 2024-09-12T03:21:15.318 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-09-12 03:19:27.673315691 +0000 2024-09-12T03:21:15.318 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-09-12 03:20:51.562891180 +0000 2024-09-12T03:21:15.318 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-09-12 03:19:27.673315691 +0000 2024-09-12T03:21:15.318 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-12T03:21:15.319 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-12T03:21:15.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:15 smithi161 ceph-mon[35485]: pgmap v339: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-12T03:21:15.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:15 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:21:15.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:15 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:21:15.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:15 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:21:15.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:15 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:21:15.393 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T03:21:15.452 INFO:tasks.workunit.client.0.smithi006.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-12T03:21:15.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:15 smithi118 ceph-mon[35048]: pgmap v339: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-12T03:21:15.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:15 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:21:15.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:15 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:21:15.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:15 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:21:15.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:15 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:21:15.599 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:16.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.098+0000 7f6666590640 1 -- 172.21.15.6:0/2449795896 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6660102570 msgr2=0x7f66601029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:16.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.098+0000 7f6666590640 1 --2- 172.21.15.6:0/2449795896 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6660102570 0x7f66601029b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6648009900 tx=0x7f664802a480 comp rx=0 tx=0).stop 2024-09-12T03:21:16.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.098+0000 7f6666590640 1 -- 172.21.15.6:0/2449795896 shutdown_connections 2024-09-12T03:21:16.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.098+0000 7f6666590640 1 --2- 172.21.15.6:0/2449795896 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6660102ef0 0x7f666010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.098+0000 7f6666590640 1 --2- 172.21.15.6:0/2449795896 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6660102570 0x7f66601029b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.098+0000 7f6666590640 1 --2- 172.21.15.6:0/2449795896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6660100e20 0x7f6660101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.098+0000 7f6666590640 1 -- 172.21.15.6:0/2449795896 >> 172.21.15.6:0/2449795896 conn(0x7f66600fc7d0 msgr2=0x7f66600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:16.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.098+0000 7f6666590640 1 -- 172.21.15.6:0/2449795896 shutdown_connections 2024-09-12T03:21:16.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.098+0000 7f6666590640 1 -- 172.21.15.6:0/2449795896 wait complete. 2024-09-12T03:21:16.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.099+0000 7f6666590640 1 Processor -- start 2024-09-12T03:21:16.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.099+0000 7f6666590640 1 -- start start 2024-09-12T03:21:16.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.099+0000 7f6666590640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6660100e20 0x7f666019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.099+0000 7f6666590640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6660102570 0x7f666019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.099+0000 7f6666590640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6660102ef0 0x7f666019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.099+0000 7f6666590640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6660110fc0 con 0x7f6660100e20 2024-09-12T03:21:16.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.099+0000 7f6666590640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f6660110e40 con 0x7f6660102570 2024-09-12T03:21:16.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.099+0000 7f6666590640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6660111140 con 0x7f6660102ef0 2024-09-12T03:21:16.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.099+0000 7f665ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6660100e20 0x7f666019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:16.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.099+0000 7f665ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6660100e20 0x7f666019a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42148/0 (socket says 172.21.15.6:42148) 2024-09-12T03:21:16.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.099+0000 7f665ffff640 1 -- 172.21.15.6:0/1879370758 learned_addr learned my addr 172.21.15.6:0/1879370758 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:16.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.099+0000 7f6664b2d640 1 --2- 172.21.15.6:0/1879370758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6660102ef0 0x7f666019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:16.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.099+0000 7f6657fff640 1 --2- 172.21.15.6:0/1879370758 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6660102570 0x7f666019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:16.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.100+0000 7f665ffff640 1 -- 172.21.15.6:0/1879370758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6660102ef0 msgr2=0x7f666019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:16.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.100+0000 7f665ffff640 1 --2- 172.21.15.6:0/1879370758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6660102ef0 0x7f666019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.100+0000 7f665ffff640 1 -- 172.21.15.6:0/1879370758 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6660102570 msgr2=0x7f666019ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:16.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.100+0000 7f665ffff640 1 --2- 172.21.15.6:0/1879370758 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6660102570 0x7f666019ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.100+0000 7f665ffff640 1 -- 172.21.15.6:0/1879370758 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666019f6e0 con 0x7f6660100e20 2024-09-12T03:21:16.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.100+0000 7f6657fff640 1 --2- 172.21.15.6:0/1879370758 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6660102570 0x7f666019ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:16.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.100+0000 7f665ffff640 1 --2- 172.21.15.6:0/1879370758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6660100e20 0x7f666019a730 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f665000d770 tx=0x7f665000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:16.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.100+0000 7f665dffb640 1 -- 172.21.15.6:0/1879370758 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650017070 con 0x7f6660100e20 2024-09-12T03:21:16.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.100+0000 7f665dffb640 1 -- 172.21.15.6:0/1879370758 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6650005500 con 0x7f6660100e20 2024-09-12T03:21:16.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.100+0000 7f6666590640 1 -- 172.21.15.6:0/1879370758 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666019f9d0 con 0x7f6660100e20 2024-09-12T03:21:16.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.100+0000 7f665dffb640 1 -- 172.21.15.6:0/1879370758 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650004e00 con 0x7f6660100e20 2024-09-12T03:21:16.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.100+0000 7f6666590640 1 -- 172.21.15.6:0/1879370758 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66601a72b0 con 0x7f6660100e20 2024-09-12T03:21:16.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.101+0000 7f6666590640 1 -- 172.21.15.6:0/1879370758 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6660101200 con 0x7f6660100e20 2024-09-12T03:21:16.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.103+0000 7f665dffb640 1 -- 172.21.15.6:0/1879370758 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f6650004fa0 con 0x7f6660100e20 2024-09-12T03:21:16.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.104+0000 7f665dffb640 1 --2- 172.21.15.6:0/1879370758 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f664c0a2c10 0x7f664c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.104+0000 7f6657fff640 1 --2- 172.21.15.6:0/1879370758 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f664c0a2c10 0x7f664c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:16.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.104+0000 7f665dffb640 1 -- 172.21.15.6:0/1879370758 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f66500cefb0 con 0x7f6660100e20 2024-09-12T03:21:16.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.105+0000 7f6657fff640 1 --2- 172.21.15.6:0/1879370758 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f664c0a2c10 0x7f664c0a50d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6648000c00 tx=0x7f6648004100 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:16.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.107+0000 7f665dffb640 1 -- 172.21.15.6:0/1879370758 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f665009be80 con 0x7f6660100e20 2024-09-12T03:21:16.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.326+0000 7f6666590640 1 -- 172.21.15.6:0/1879370758 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66601a7480 con 0x7f6660100e20 2024-09-12T03:21:16.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.327+0000 7f665dffb640 1 -- 172.21.15.6:0/1879370758 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11908 (secure 0 0 0) 0x7f665009b780 con 0x7f6660100e20 2024-09-12T03:21:16.327 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:21:16.327 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"btime":"2024-09-12T03:20:52:974598+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:20:52.261388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24463]},"id":1}]} 2024-09-12T03:21:16.327 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 25 2024-09-12T03:21:16.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.331+0000 7f6666590640 1 -- 172.21.15.6:0/1879370758 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f664c0a2c10 msgr2=0x7f664c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:16.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.331+0000 7f6666590640 1 --2- 172.21.15.6:0/1879370758 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f664c0a2c10 0x7f664c0a50d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6648000c00 tx=0x7f6648004100 comp rx=0 tx=0).stop 2024-09-12T03:21:16.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.331+0000 7f6666590640 1 -- 172.21.15.6:0/1879370758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6660100e20 msgr2=0x7f666019a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:16.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.331+0000 7f6666590640 1 --2- 172.21.15.6:0/1879370758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6660100e20 0x7f666019a730 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f665000d770 tx=0x7f665000dc30 comp rx=0 tx=0).stop 2024-09-12T03:21:16.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.331+0000 7f6666590640 1 -- 172.21.15.6:0/1879370758 shutdown_connections 2024-09-12T03:21:16.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.331+0000 7f6666590640 1 --2- 172.21.15.6:0/1879370758 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f664c0a2c10 0x7f664c0a50d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.331+0000 7f6666590640 1 --2- 172.21.15.6:0/1879370758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6660102ef0 0x7f666019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.331+0000 7f6666590640 1 --2- 172.21.15.6:0/1879370758 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6660102570 0x7f666019ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.331+0000 7f6666590640 1 --2- 172.21.15.6:0/1879370758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6660100e20 0x7f666019a730 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.331+0000 7f6666590640 1 -- 172.21.15.6:0/1879370758 >> 172.21.15.6:0/1879370758 conn(0x7f66600fc7d0 msgr2=0x7f666006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:16.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.331+0000 7f6666590640 1 -- 172.21.15.6:0/1879370758 shutdown_connections 2024-09-12T03:21:16.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:16.331+0000 7f6666590640 1 -- 172.21.15.6:0/1879370758 wait complete. 2024-09-12T03:21:17.054 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T03:19:09.851402+0000', 'modified': '2024-09-12T03:20:52.261388+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24494, 'mds_1': 24464, 'mds_2': 24463}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24439': {'gid': 24439, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/955127607', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 955127607}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 955127607}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.118:6835/1389654073', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6834', 'nonce': 1389654073}, {'type': 'v1', 'addr': '172.21.15.118:6835', 'nonce': 1389654073}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6837/283024944', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 283024944}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 283024944}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.161:6839/3645555254', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 3645555254}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 3645555254}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.118:6837/3879191727', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6836', 'nonce': 3879191727}, {'type': 'v1', 'addr': '172.21.15.118:6837', 'nonce': 3879191727}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.118:6839/3035878957', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6838', 'nonce': 3035878957}, {'type': 'v1', 'addr': '172.21.15.118:6839', 'nonce': 3035878957}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24494, 24464, 24463]} 2024-09-12T03:21:17.055 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T03:21:17.055 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T03:21:17.349 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:17.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:17 smithi006 ceph-mon[30991]: pgmap v340: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-12T03:21:17.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:17 smithi006 ceph-mon[30991]: scrub summary: idle 2024-09-12T03:21:17.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:21:17.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:21:17.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:17 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1879370758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:17.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:17.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:17.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:21:17.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:17 smithi161 ceph-mon[35485]: pgmap v340: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-12T03:21:17.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:17 smithi161 ceph-mon[35485]: scrub summary: idle 2024-09-12T03:21:17.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:21:17.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:21:17.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:17 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1879370758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:17.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:17.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:17.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:21:17.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:17 smithi118 ceph-mon[35048]: pgmap v340: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-12T03:21:17.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:17 smithi118 ceph-mon[35048]: scrub summary: idle 2024-09-12T03:21:17.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:21:17.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:21:17.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:17 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1879370758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:17.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:17.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:17.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:21:17.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.794+0000 7fd5aa577640 1 -- 172.21.15.6:0/175393010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5a4102570 msgr2=0x7fd5a41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:17.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.794+0000 7fd5aa577640 1 --2- 172.21.15.6:0/175393010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5a4102570 0x7fd5a41029b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd598009900 tx=0x7fd59802a480 comp rx=0 tx=0).stop 2024-09-12T03:21:17.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.794+0000 7fd5aa577640 1 -- 172.21.15.6:0/175393010 shutdown_connections 2024-09-12T03:21:17.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.794+0000 7fd5aa577640 1 --2- 172.21.15.6:0/175393010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5a4102ef0 0x7fd5a410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.794+0000 7fd5aa577640 1 --2- 172.21.15.6:0/175393010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5a4102570 0x7fd5a41029b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.794+0000 7fd5aa577640 1 --2- 172.21.15.6:0/175393010 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5a4100e20 0x7fd5a4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.794+0000 7fd5aa577640 1 -- 172.21.15.6:0/175393010 >> 172.21.15.6:0/175393010 conn(0x7fd5a40fc7d0 msgr2=0x7fd5a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:17.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.794+0000 7fd5aa577640 1 -- 172.21.15.6:0/175393010 shutdown_connections 2024-09-12T03:21:17.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.794+0000 7fd5aa577640 1 -- 172.21.15.6:0/175393010 wait complete. 2024-09-12T03:21:17.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.795+0000 7fd5aa577640 1 Processor -- start 2024-09-12T03:21:17.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.795+0000 7fd5aa577640 1 -- start start 2024-09-12T03:21:17.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.795+0000 7fd5aa577640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5a4100e20 0x7fd5a41984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:17.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.795+0000 7fd5aa577640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5a4102570 0x7fd5a4198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:17.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.795+0000 7fd5aa577640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5a4102ef0 0x7fd5a419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:17.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.795+0000 7fd5aa577640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a4110f80 con 0x7fd5a4100e20 2024-09-12T03:21:17.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.795+0000 7fd5aa577640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a4110e00 con 0x7fd5a4102570 2024-09-12T03:21:17.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.795+0000 7fd5aa577640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a4111100 con 0x7fd5a4102ef0 2024-09-12T03:21:17.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.795+0000 7fd5a3fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5a4100e20 0x7fd5a41984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:17.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.796+0000 7fd5a3fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5a4100e20 0x7fd5a41984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42164/0 (socket says 172.21.15.6:42164) 2024-09-12T03:21:17.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.796+0000 7fd5a3fff640 1 -- 172.21.15.6:0/640325413 learned_addr learned my addr 172.21.15.6:0/640325413 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:17.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.796+0000 7fd5a8b14640 1 --2- 172.21.15.6:0/640325413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5a4102ef0 0x7fd5a419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:17.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.796+0000 7fd5a37fe640 1 --2- 172.21.15.6:0/640325413 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5a4102570 0x7fd5a4198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:17.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.796+0000 7fd5a3fff640 1 -- 172.21.15.6:0/640325413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5a4102ef0 msgr2=0x7fd5a419cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:17.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.796+0000 7fd5a3fff640 1 --2- 172.21.15.6:0/640325413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5a4102ef0 0x7fd5a419cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.796+0000 7fd5a3fff640 1 -- 172.21.15.6:0/640325413 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5a4102570 msgr2=0x7fd5a4198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:17.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.796+0000 7fd5a3fff640 1 --2- 172.21.15.6:0/640325413 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5a4102570 0x7fd5a4198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.796+0000 7fd5a3fff640 1 -- 172.21.15.6:0/640325413 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a419d4a0 con 0x7fd5a4100e20 2024-09-12T03:21:17.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.796+0000 7fd5a37fe640 1 --2- 172.21.15.6:0/640325413 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5a4102570 0x7fd5a4198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:17.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.796+0000 7fd5a3fff640 1 --2- 172.21.15.6:0/640325413 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5a4100e20 0x7fd5a41984f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd59400d950 tx=0x7fd59400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:17.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.796+0000 7fd5a17fa640 1 -- 172.21.15.6:0/640325413 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594017070 con 0x7fd5a4100e20 2024-09-12T03:21:17.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.796+0000 7fd5a17fa640 1 -- 172.21.15.6:0/640325413 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd594009c30 con 0x7fd5a4100e20 2024-09-12T03:21:17.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.796+0000 7fd5aa577640 1 -- 172.21.15.6:0/640325413 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a419d790 con 0x7fd5a4100e20 2024-09-12T03:21:17.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.796+0000 7fd5a17fa640 1 -- 172.21.15.6:0/640325413 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59400f6b0 con 0x7fd5a4100e20 2024-09-12T03:21:17.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.796+0000 7fd5aa577640 1 -- 172.21.15.6:0/640325413 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a41a5070 con 0x7fd5a4100e20 2024-09-12T03:21:17.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.799+0000 7fd5a17fa640 1 -- 172.21.15.6:0/640325413 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fd5940109d0 con 0x7fd5a4100e20 2024-09-12T03:21:17.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.799+0000 7fd5ac950740 1 Processor -- start 2024-09-12T03:21:17.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.799+0000 7fd5ac950740 1 -- start start 2024-09-12T03:21:17.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.799+0000 7fd5ac950740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e5f9d0 0x55e770e5fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:17.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.799+0000 7fd5ac950740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e602f0 0x55e770e6a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:17.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.799+0000 7fd5ac950740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770e6cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:17.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.799+0000 7fd5ac950740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e770dc6090 con 0x55e770e6abb0 2024-09-12T03:21:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.799+0000 7fd5ac950740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55e770d6da70 con 0x55e770e602f0 2024-09-12T03:21:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.799+0000 7fd5ac950740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e770dc2050 con 0x55e770e5f9d0 2024-09-12T03:21:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.799+0000 7fd5a37fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770e6cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.799+0000 7fd5a17fa640 1 --2- 172.21.15.6:0/640325413 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd58c0a2c90 0x7fd58c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.799+0000 7fd5a37fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770e6cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42166/0 (socket says 172.21.15.6:42166) 2024-09-12T03:21:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.799+0000 7fd5a37fe640 1 -- 172.21.15.6:0/681345019 learned_addr learned my addr 172.21.15.6:0/681345019 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.799+0000 7fd5a8b14640 1 --2- 172.21.15.6:0/681345019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e5f9d0 0x55e770e5fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.800+0000 7fd5a3fff640 1 --2- 172.21.15.6:0/681345019 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e602f0 0x55e770e6a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.800+0000 7fd5a8b14640 1 --2- 172.21.15.6:0/640325413 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd58c0a2c90 0x7fd58c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.800+0000 7fd5a17fa640 1 -- 172.21.15.6:0/640325413 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd5940cedf0 con 0x7fd5a4100e20 2024-09-12T03:21:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.800+0000 7fd5a37fe640 1 -- 172.21.15.6:0/681345019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e5f9d0 msgr2=0x55e770e5fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.800+0000 7fd5a37fe640 1 --2- 172.21.15.6:0/681345019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e5f9d0 0x55e770e5fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.800+0000 7fd5a37fe640 1 -- 172.21.15.6:0/681345019 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e602f0 msgr2=0x55e770e6a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:17.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.800+0000 7fd5a37fe640 1 --2- 172.21.15.6:0/681345019 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e602f0 0x55e770e6a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.800+0000 7fd5a37fe640 1 -- 172.21.15.6:0/681345019 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e770d7a130 con 0x55e770e6abb0 2024-09-12T03:21:17.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.800+0000 7fd5a8b14640 1 --2- 172.21.15.6:0/681345019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e5f9d0 0x55e770e5fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:17.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.800+0000 7fd5a37fe640 1 --2- 172.21.15.6:0/681345019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770e6cfa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd5980314e0 tx=0x7fd598004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd56ed3cb69ddc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:17.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.800+0000 7fd5a8b14640 1 --2- 172.21.15.6:0/640325413 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd58c0a2c90 0x7fd58c0a5150 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0077f0 tx=0x7fd59c007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:17.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.800+0000 7fd58a7fc640 1 -- 172.21.15.6:0/681345019 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598035030 con 0x55e770e6abb0 2024-09-12T03:21:17.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.800+0000 7fd58a7fc640 1 -- 172.21.15.6:0/681345019 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd598005520 con 0x55e770e6abb0 2024-09-12T03:21:17.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.800+0000 7fd58a7fc640 1 -- 172.21.15.6:0/681345019 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598003850 con 0x55e770e6abb0 2024-09-12T03:21:17.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.801+0000 7fd5ac950740 1 -- 172.21.15.6:0/681345019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 msgr2=0x55e770e6cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:17.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.801+0000 7fd5ac950740 1 --2- 172.21.15.6:0/681345019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770e6cfa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd5980314e0 tx=0x7fd598004200 comp rx=0 tx=0).stop 2024-09-12T03:21:17.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.801+0000 7fd5ac950740 1 -- 172.21.15.6:0/681345019 shutdown_connections 2024-09-12T03:21:17.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.801+0000 7fd5ac950740 1 --2- 172.21.15.6:0/681345019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770e6cfa0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.801+0000 7fd5ac950740 1 --2- 172.21.15.6:0/681345019 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e602f0 0x55e770e6a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.801+0000 7fd5ac950740 1 --2- 172.21.15.6:0/681345019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e5f9d0 0x55e770e5fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.801+0000 7fd5ac950740 1 -- 172.21.15.6:0/681345019 >> 172.21.15.6:0/681345019 conn(0x55e770e5bf60 msgr2=0x55e770e5c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:17.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.801+0000 7fd5ac950740 1 -- 172.21.15.6:0/681345019 shutdown_connections 2024-09-12T03:21:17.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.801+0000 7fd5ac950740 1 -- 172.21.15.6:0/681345019 wait complete. 2024-09-12T03:21:17.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.801+0000 7fd5ac950740 1 -- start start 2024-09-12T03:21:17.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.802+0000 7fd5ac950740 1 Processor -- start 2024-09-12T03:21:17.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.802+0000 7fd5ac950740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e5f9d0 0x55e770e6a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:17.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.802+0000 7fd5ac950740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e602f0 0x55e770e67790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:17.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.802+0000 7fd5ac950740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770e67cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:17.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.802+0000 7fd5ac950740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e770e71b90 con 0x55e770e6abb0 2024-09-12T03:21:17.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.802+0000 7fd5ac950740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55e770d80f20 con 0x55e770e602f0 2024-09-12T03:21:17.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.802+0000 7fd5ac950740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e770e71d10 con 0x55e770e5f9d0 2024-09-12T03:21:17.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.802+0000 7fd5ac950740 10 client.0 _ll_register_callbacks cb 0x55e770d4a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:17.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.802+0000 7fd5a8b14640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770e67cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:17.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.802+0000 7fd5a8b14640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770e67cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42180/0 (socket says 172.21.15.6:42180) 2024-09-12T03:21:17.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.802+0000 7fd5a8b14640 1 -- 172.21.15.6:0/1684930388 learned_addr learned my addr 172.21.15.6:0/1684930388 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:17.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.802+0000 7fd5a3fff640 1 --2- 172.21.15.6:0/1684930388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e5f9d0 0x55e770e6a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:17.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.802+0000 7fd5a37fe640 1 --2- 172.21.15.6:0/1684930388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e602f0 0x55e770e67790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:17.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.803+0000 7fd5a8b14640 1 -- 172.21.15.6:0/1684930388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e5f9d0 msgr2=0x55e770e6a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:17.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.803+0000 7fd5a8b14640 1 --2- 172.21.15.6:0/1684930388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e5f9d0 0x55e770e6a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.803+0000 7fd5a8b14640 1 -- 172.21.15.6:0/1684930388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e602f0 msgr2=0x55e770e67790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:17.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.803+0000 7fd5a8b14640 1 --2- 172.21.15.6:0/1684930388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e602f0 0x55e770e67790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.803+0000 7fd5a8b14640 1 -- 172.21.15.6:0/1684930388 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e770b372b0 con 0x55e770e6abb0 2024-09-12T03:21:17.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.803+0000 7fd5a3fff640 1 --2- 172.21.15.6:0/1684930388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e5f9d0 0x55e770e6a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:17.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.803+0000 7fd5a37fe640 1 --2- 172.21.15.6:0/1684930388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e602f0 0x55e770e67790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:17.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.803+0000 7fd5a8b14640 1 --2- 172.21.15.6:0/1684930388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770e67cd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd59c004390 tx=0x7fd59c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec874eb09c64d546 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:17.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.803+0000 7fd5837fe640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:21:17.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.803+0000 7fd5837fe640 1 -- 172.21.15.6:0/1684930388 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c011d10 con 0x55e770e6abb0 2024-09-12T03:21:17.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.803+0000 7fd5837fe640 1 -- 172.21.15.6:0/1684930388 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd59c004b20 con 0x55e770e6abb0 2024-09-12T03:21:17.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.803+0000 7fd5ac950740 1 -- 172.21.15.6:0/1684930388 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd598005520 con 0x55e770e6abb0 2024-09-12T03:21:17.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.803+0000 7fd5837fe640 1 -- 172.21.15.6:0/1684930388 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd59c004d20 con 0x55e770e6abb0 2024-09-12T03:21:17.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.803+0000 7fd5ac950740 10 client.15279 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:17.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.803+0000 7fd5ac950740 1 -- 172.21.15.6:0/1684930388 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e770e7d4a0 con 0x55e770e6abb0 2024-09-12T03:21:17.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.803+0000 7fd5837fe640 1 -- 172.21.15.6:0/1684930388 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd598005520 con 0x55e770e6abb0 2024-09-12T03:21:17.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.804+0000 7fd5837fe640 1 -- 172.21.15.6:0/1684930388 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd59c00f450 con 0x55e770e6abb0 2024-09-12T03:21:17.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.804+0000 7fd5ac950740 10 client.15279 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:17.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.804+0000 7fd5ac950740 10 client.15279 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:21:17.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.804+0000 7fd5ac950740 20 client.15279 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:17.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.804+0000 7fd5ac950740 1 --2- 172.21.15.6:0/1684930388 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55e770e7fb70 0x55e770e80fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:17.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.804+0000 7fd5ac950740 4 client.15279 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:17.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.804+0000 7fd5ac950740 1 -- 172.21.15.6:0/1684930388 --> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e770dd3600 con 0x55e770e7fb70 2024-09-12T03:21:17.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.804+0000 7fd5a3fff640 1 --2- 172.21.15.6:0/1684930388 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55e770e7fb70 0x55e770e80fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:17.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.805+0000 7fd5837fe640 10 client.15279 ms_handle_connect on v2:172.21.15.161:6838/3645555254 2024-09-12T03:21:17.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.805+0000 7fd5a3fff640 1 --2- 172.21.15.6:0/1684930388 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55e770e7fb70 0x55e770e80fe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a72a256a6356c74c server_cookie=91c4445811b72a80 in_seq=0 out_seq=0 2024-09-12T03:21:17.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.805+0000 7fd5837fe640 1 -- 172.21.15.6:0/1684930388 <== mds.2 v2:172.21.15.161:6838/3645555254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e770dd3600 con 0x55e770e7fb70 2024-09-12T03:21:17.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.805+0000 7fd5837fe640 10 client.15279 handle_command_reply: tid=0 2024-09-12T03:21:17.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.805+0000 7fd5ac950740 1 client.15279 shutdown 2024-09-12T03:21:17.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.805+0000 7fd5ac950740 1 -- 172.21.15.6:0/1684930388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 msgr2=0x55e770e67cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:17.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.805+0000 7fd5ac950740 1 --2- 172.21.15.6:0/1684930388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770e67cd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd59c004390 tx=0x7fd59c004a40 comp rx=0 tx=0).stop 2024-09-12T03:21:17.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.805+0000 7fd5ac950740 1 -- 172.21.15.6:0/1684930388 shutdown_connections 2024-09-12T03:21:17.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.805+0000 7fd5ac950740 1 --2- 172.21.15.6:0/1684930388 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55e770e7fb70 0x55e770e80fe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.806+0000 7fd5ac950740 1 --2- 172.21.15.6:0/1684930388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770e67cd0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.806+0000 7fd5ac950740 1 --2- 172.21.15.6:0/1684930388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e602f0 0x55e770e67790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.806+0000 7fd5837fe640 0 client.15279 ms_handle_reset on v2:172.21.15.161:6838/3645555254 2024-09-12T03:21:17.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.806+0000 7fd5ac950740 1 --2- 172.21.15.6:0/1684930388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e5f9d0 0x55e770e6a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.806+0000 7fd5ac950740 1 -- 172.21.15.6:0/1684930388 >> 172.21.15.6:0/1684930388 conn(0x55e770e5bf60 msgr2=0x55e770e61c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:17.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.806+0000 7fd5ac950740 1 -- 172.21.15.6:0/1684930388 shutdown_connections 2024-09-12T03:21:17.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.806+0000 7fd5ac950740 1 -- 172.21.15.6:0/1684930388 wait complete. 2024-09-12T03:21:17.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.806+0000 7fd5ac950740 20 client.15279 trim_cache size 0 max 16384 2024-09-12T03:21:17.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.823+0000 7fd5ac950740 1 Processor -- start 2024-09-12T03:21:17.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.823+0000 7fd5ac950740 1 -- start start 2024-09-12T03:21:17.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.824+0000 7fd5ac950740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e5f9d0 0x55e770e5c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:17.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.824+0000 7fd5ac950740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e602f0 0x55e770e5c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:17.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.824+0000 7fd5ac950740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770e66060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:17.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.824+0000 7fd5ac950740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e770e7f320 con 0x55e770e6abb0 2024-09-12T03:21:17.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.824+0000 7fd5ac950740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55e770e7f9f0 con 0x55e770e602f0 2024-09-12T03:21:17.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.824+0000 7fd5ac950740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e770e800c0 con 0x55e770e5f9d0 2024-09-12T03:21:17.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.824+0000 7fd5a8b14640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770e66060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:17.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.824+0000 7fd5a3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e5f9d0 0x55e770e5c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:17.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.824+0000 7fd5a8b14640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770e66060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42192/0 (socket says 172.21.15.6:42192) 2024-09-12T03:21:17.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.824+0000 7fd5a8b14640 1 -- 172.21.15.6:0/2410481337 learned_addr learned my addr 172.21.15.6:0/2410481337 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:17.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.824+0000 7fd5a37fe640 1 --2- 172.21.15.6:0/2410481337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e602f0 0x55e770e5c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:17.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.824+0000 7fd5a3fff640 1 -- 172.21.15.6:0/2410481337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e602f0 msgr2=0x55e770e5c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:17.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.824+0000 7fd5a3fff640 1 --2- 172.21.15.6:0/2410481337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e602f0 0x55e770e5c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.824+0000 7fd5a3fff640 1 -- 172.21.15.6:0/2410481337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 msgr2=0x55e770e66060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:17.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.824+0000 7fd5a3fff640 1 --2- 172.21.15.6:0/2410481337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770e66060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.824+0000 7fd5a3fff640 1 -- 172.21.15.6:0/2410481337 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e770e5d0f0 con 0x55e770e5f9d0 2024-09-12T03:21:17.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.824+0000 7fd5a8b14640 1 --2- 172.21.15.6:0/2410481337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770e66060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:17.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.824+0000 7fd5a3fff640 1 --2- 172.21.15.6:0/2410481337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e5f9d0 0x55e770e5c340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd5940cc770 tx=0x7fd59400dfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=380370bcf3b00a63 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:17.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.824+0000 7fd589ffb640 1 -- 172.21.15.6:0/2410481337 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594020040 con 0x55e770e5f9d0 2024-09-12T03:21:17.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.825+0000 7fd589ffb640 1 -- 172.21.15.6:0/2410481337 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5940a4130 con 0x55e770e5f9d0 2024-09-12T03:21:17.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.825+0000 7fd589ffb640 1 -- 172.21.15.6:0/2410481337 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59409ab90 con 0x55e770e5f9d0 2024-09-12T03:21:17.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.825+0000 7fd5ac950740 1 -- 172.21.15.6:0/2410481337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e5f9d0 msgr2=0x55e770e5c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:17.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.825+0000 7fd5ac950740 1 --2- 172.21.15.6:0/2410481337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e5f9d0 0x55e770e5c340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd5940cc770 tx=0x7fd59400dfa0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.825+0000 7fd5ac950740 1 -- 172.21.15.6:0/2410481337 shutdown_connections 2024-09-12T03:21:17.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.825+0000 7fd5ac950740 1 --2- 172.21.15.6:0/2410481337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770e66060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.825+0000 7fd5ac950740 1 --2- 172.21.15.6:0/2410481337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e602f0 0x55e770e5c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.825+0000 7fd5ac950740 1 --2- 172.21.15.6:0/2410481337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e5f9d0 0x55e770e5c340 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.825+0000 7fd5ac950740 1 -- 172.21.15.6:0/2410481337 >> 172.21.15.6:0/2410481337 conn(0x55e770e5bf60 msgr2=0x55e770e67750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:17.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.825+0000 7fd5ac950740 1 -- 172.21.15.6:0/2410481337 shutdown_connections 2024-09-12T03:21:17.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.825+0000 7fd5ac950740 1 -- 172.21.15.6:0/2410481337 wait complete. 2024-09-12T03:21:17.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.825+0000 7fd5ac950740 1 -- start start 2024-09-12T03:21:17.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.826+0000 7fd5ac950740 1 Processor -- start 2024-09-12T03:21:17.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.826+0000 7fd5ac950740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e5f9d0 0x55e770e82890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:17.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.826+0000 7fd5ac950740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e602f0 0x55e770e82dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:17.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.826+0000 7fd5ac950740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770ee8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:17.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.826+0000 7fd5ac950740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e770e80e60 con 0x55e770e6abb0 2024-09-12T03:21:17.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.826+0000 7fd5ac950740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55e770e80790 con 0x55e770e5f9d0 2024-09-12T03:21:17.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.826+0000 7fd5ac950740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e770e81530 con 0x55e770e602f0 2024-09-12T03:21:17.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.826+0000 7fd5ac950740 10 client.0 _ll_register_callbacks cb 0x7fd59c007ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:17.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.826+0000 7fd5a8b14640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770ee8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:17.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.826+0000 7fd5a8b14640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770ee8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42208/0 (socket says 172.21.15.6:42208) 2024-09-12T03:21:17.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.826+0000 7fd5a8b14640 1 -- 172.21.15.6:0/1525840738 learned_addr learned my addr 172.21.15.6:0/1525840738 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:17.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.826+0000 7fd5a37fe640 1 --2- 172.21.15.6:0/1525840738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e602f0 0x55e770e82dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:17.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.826+0000 7fd5a3fff640 1 --2- 172.21.15.6:0/1525840738 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e5f9d0 0x55e770e82890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:17.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.827+0000 7fd5a8b14640 1 -- 172.21.15.6:0/1525840738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e602f0 msgr2=0x55e770e82dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:17.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.827+0000 7fd5a8b14640 1 --2- 172.21.15.6:0/1525840738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e602f0 0x55e770e82dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.827+0000 7fd5a8b14640 1 -- 172.21.15.6:0/1525840738 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e5f9d0 msgr2=0x55e770e82890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:17.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.827+0000 7fd5a8b14640 1 --2- 172.21.15.6:0/1525840738 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e5f9d0 0x55e770e82890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.827+0000 7fd5a8b14640 1 -- 172.21.15.6:0/1525840738 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e770ee87b0 con 0x55e770e6abb0 2024-09-12T03:21:17.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.827+0000 7fd5a37fe640 1 --2- 172.21.15.6:0/1525840738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e602f0 0x55e770e82dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:17.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.827+0000 7fd5a3fff640 1 --2- 172.21.15.6:0/1525840738 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e5f9d0 0x55e770e82890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:17.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.827+0000 7fd5a8b14640 1 --2- 172.21.15.6:0/1525840738 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770ee8270 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd59c004390 tx=0x7fd59c0119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425d915b106edffa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:17.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.827+0000 7fd5837fe640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:21:17.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.827+0000 7fd5837fe640 1 -- 172.21.15.6:0/1525840738 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c03a070 con 0x55e770e6abb0 2024-09-12T03:21:17.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.827+0000 7fd5837fe640 1 -- 172.21.15.6:0/1525840738 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd59c00f450 con 0x55e770e6abb0 2024-09-12T03:21:17.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.827+0000 7fd5ac950740 1 -- 172.21.15.6:0/1525840738 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5940a4130 con 0x55e770e6abb0 2024-09-12T03:21:17.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.827+0000 7fd5837fe640 1 -- 172.21.15.6:0/1525840738 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd59c018030 con 0x55e770e6abb0 2024-09-12T03:21:17.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.827+0000 7fd5ac950740 10 client.15285 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:17.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.827+0000 7fd5ac950740 1 -- 172.21.15.6:0/1525840738 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e770ee99f0 con 0x55e770e6abb0 2024-09-12T03:21:17.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.827+0000 7fd5837fe640 1 -- 172.21.15.6:0/1525840738 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5940a4130 con 0x55e770e6abb0 2024-09-12T03:21:17.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.828+0000 7fd5837fe640 1 -- 172.21.15.6:0/1525840738 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd59c03d030 con 0x55e770e6abb0 2024-09-12T03:21:17.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.828+0000 7fd5ac950740 10 client.15285 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:17.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.828+0000 7fd5ac950740 10 client.15285 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:21:17.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.828+0000 7fd5ac950740 20 client.15285 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:17.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.828+0000 7fd5ac950740 1 --2- 172.21.15.6:0/1525840738 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55e770eee5d0 0x55e770eed2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:17.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.828+0000 7fd5ac950740 4 client.15285 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:21:17.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.828+0000 7fd5ac950740 1 -- 172.21.15.6:0/1525840738 --> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] -- command(tid 0: {"prefix": "status"}) -- 0x55e770dcdb10 con 0x55e770eee5d0 2024-09-12T03:21:17.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.828+0000 7fd5a3fff640 1 --2- 172.21.15.6:0/1525840738 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55e770eee5d0 0x55e770eed2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:17.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.828+0000 7fd5837fe640 10 client.15285 ms_handle_connect on v2:172.21.15.161:6838/3645555254 2024-09-12T03:21:17.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.828+0000 7fd5a3fff640 1 --2- 172.21.15.6:0/1525840738 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55e770eee5d0 0x55e770eed2e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cccf2e253883a844 server_cookie=88d84a413c8711b3 in_seq=0 out_seq=0 2024-09-12T03:21:17.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.829+0000 7fd5837fe640 1 -- 172.21.15.6:0/1525840738 <== mds.2 v2:172.21.15.161:6838/3645555254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e770dcdb10 con 0x55e770eee5d0 2024-09-12T03:21:17.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.829+0000 7fd5837fe640 10 client.15285 handle_command_reply: tid=0 2024-09-12T03:21:17.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.829+0000 7fd5ac950740 1 client.15285 shutdown 2024-09-12T03:21:17.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.829+0000 7fd5ac950740 1 -- 172.21.15.6:0/1525840738 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 msgr2=0x55e770ee8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:17.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.829+0000 7fd5ac950740 1 --2- 172.21.15.6:0/1525840738 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770ee8270 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd59c004390 tx=0x7fd59c0119d0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.829+0000 7fd5ac950740 1 -- 172.21.15.6:0/1525840738 shutdown_connections 2024-09-12T03:21:17.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.829+0000 7fd5ac950740 1 --2- 172.21.15.6:0/1525840738 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55e770eee5d0 0x55e770eed2e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.829+0000 7fd5837fe640 0 client.15285 ms_handle_reset on v2:172.21.15.161:6838/3645555254 2024-09-12T03:21:17.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.829+0000 7fd5ac950740 1 --2- 172.21.15.6:0/1525840738 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e770e6abb0 0x55e770ee8270 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.829+0000 7fd5ac950740 1 --2- 172.21.15.6:0/1525840738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e770e602f0 0x55e770e82dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.829+0000 7fd5ac950740 1 --2- 172.21.15.6:0/1525840738 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e770e5f9d0 0x55e770e82890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.829+0000 7fd5ac950740 1 -- 172.21.15.6:0/1525840738 >> 172.21.15.6:0/1525840738 conn(0x55e770e5bf60 msgr2=0x55e770e67750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:17.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.829+0000 7fd5ac950740 1 -- 172.21.15.6:0/1525840738 shutdown_connections 2024-09-12T03:21:17.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.829+0000 7fd5ac950740 1 -- 172.21.15.6:0/1525840738 wait complete. 2024-09-12T03:21:17.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.829+0000 7fd5ac950740 20 client.15285 trim_cache size 0 max 16384 2024-09-12T03:21:17.835 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:21:17.835 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "89d99938-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:21:17.835 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 2, 2024-09-12T03:21:17.835 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24463, 2024-09-12T03:21:17.835 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-09-12T03:21:17.835 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-09-12T03:21:17.835 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-09-12T03:21:17.836 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 119.467051259, 2024-09-12T03:21:17.836 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 25, 2024-09-12T03:21:17.836 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 82, 2024-09-12T03:21:17.836 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T03:21:17.836 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 145.09385322599999 2024-09-12T03:21:17.836 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:21:17.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.830+0000 7fd5aa577640 1 -- 172.21.15.6:0/640325413 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd58c0a2c90 msgr2=0x7fd58c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:17.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.830+0000 7fd5aa577640 1 --2- 172.21.15.6:0/640325413 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd58c0a2c90 0x7fd58c0a5150 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0077f0 tx=0x7fd59c007520 comp rx=0 tx=0).stop 2024-09-12T03:21:17.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.830+0000 7fd5aa577640 1 -- 172.21.15.6:0/640325413 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5a4100e20 msgr2=0x7fd5a41984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:17.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.830+0000 7fd5aa577640 1 --2- 172.21.15.6:0/640325413 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5a4100e20 0x7fd5a41984f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd59400d950 tx=0x7fd59400de10 comp rx=0 tx=0).stop 2024-09-12T03:21:17.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.830+0000 7fd5aa577640 1 -- 172.21.15.6:0/640325413 shutdown_connections 2024-09-12T03:21:17.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.830+0000 7fd5aa577640 1 --2- 172.21.15.6:0/640325413 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd58c0a2c90 0x7fd58c0a5150 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.830+0000 7fd5aa577640 1 --2- 172.21.15.6:0/640325413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5a4102ef0 0x7fd5a419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.830+0000 7fd5aa577640 1 --2- 172.21.15.6:0/640325413 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5a4102570 0x7fd5a4198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.830+0000 7fd5aa577640 1 --2- 172.21.15.6:0/640325413 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5a4100e20 0x7fd5a41984f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.830+0000 7fd5aa577640 1 -- 172.21.15.6:0/640325413 >> 172.21.15.6:0/640325413 conn(0x7fd5a40fc7d0 msgr2=0x7fd5a406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:17.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.830+0000 7fd5aa577640 1 -- 172.21.15.6:0/640325413 shutdown_connections 2024-09-12T03:21:17.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:17.830+0000 7fd5aa577640 1 -- 172.21.15.6:0/640325413 wait complete. 2024-09-12T03:21:18.541 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-12T03:21:18.798 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:19.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:19 smithi006 ceph-mon[30991]: pgmap v341: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:21:19.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.280+0000 7f73e70d2640 1 -- 172.21.15.6:0/248531072 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73e0102570 msgr2=0x7f73e01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:19.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.280+0000 7f73e70d2640 1 --2- 172.21.15.6:0/248531072 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73e0102570 0x7f73e01029b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f73c8009900 tx=0x7f73c802a480 comp rx=0 tx=0).stop 2024-09-12T03:21:19.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.280+0000 7f73e70d2640 1 -- 172.21.15.6:0/248531072 shutdown_connections 2024-09-12T03:21:19.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.280+0000 7f73e70d2640 1 --2- 172.21.15.6:0/248531072 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73e0102ef0 0x7f73e010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.280+0000 7f73e70d2640 1 --2- 172.21.15.6:0/248531072 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73e0102570 0x7f73e01029b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.280+0000 7f73e70d2640 1 --2- 172.21.15.6:0/248531072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73e0100e20 0x7f73e0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.280+0000 7f73e70d2640 1 -- 172.21.15.6:0/248531072 >> 172.21.15.6:0/248531072 conn(0x7f73e00fc7d0 msgr2=0x7f73e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:19.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.280+0000 7f73e70d2640 1 -- 172.21.15.6:0/248531072 shutdown_connections 2024-09-12T03:21:19.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.280+0000 7f73e70d2640 1 -- 172.21.15.6:0/248531072 wait complete. 2024-09-12T03:21:19.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.280+0000 7f73e70d2640 1 Processor -- start 2024-09-12T03:21:19.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.281+0000 7f73e70d2640 1 -- start start 2024-09-12T03:21:19.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.281+0000 7f73e70d2640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73e0100e20 0x7f73e0196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:19.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.281+0000 7f73e70d2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73e0102570 0x7f73e0197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:19.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.281+0000 7f73e70d2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73e0102ef0 0x7f73e01a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:19.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.281+0000 7f73e70d2640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f73e0111010 con 0x7f73e0102ef0 2024-09-12T03:21:19.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.281+0000 7f73e70d2640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f73e0110e90 con 0x7f73e0100e20 2024-09-12T03:21:19.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.281+0000 7f73e70d2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f73e0111190 con 0x7f73e0102570 2024-09-12T03:21:19.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.281+0000 7f73e566f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73e0102ef0 0x7f73e01a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:19.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.281+0000 7f73e566f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73e0102ef0 0x7f73e01a63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56116/0 (socket says 172.21.15.6:56116) 2024-09-12T03:21:19.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.281+0000 7f73e566f640 1 -- 172.21.15.6:0/2577850267 learned_addr learned my addr 172.21.15.6:0/2577850267 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:19.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.282+0000 7f73dffff640 1 --2- 172.21.15.6:0/2577850267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73e0102570 0x7f73e0197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:19.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.282+0000 7f73e4e6e640 1 --2- 172.21.15.6:0/2577850267 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73e0100e20 0x7f73e0196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:19.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.282+0000 7f73e566f640 1 -- 172.21.15.6:0/2577850267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73e0102570 msgr2=0x7f73e0197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:19.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.282+0000 7f73e566f640 1 --2- 172.21.15.6:0/2577850267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73e0102570 0x7f73e0197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.282+0000 7f73e566f640 1 -- 172.21.15.6:0/2577850267 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73e0100e20 msgr2=0x7f73e0196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:19.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.282+0000 7f73e566f640 1 --2- 172.21.15.6:0/2577850267 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73e0100e20 0x7f73e0196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.282+0000 7f73e566f640 1 -- 172.21.15.6:0/2577850267 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e01a6a30 con 0x7f73e0102ef0 2024-09-12T03:21:19.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.282+0000 7f73e566f640 1 --2- 172.21.15.6:0/2577850267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73e0102ef0 0x7f73e01a63b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f73d800b970 tx=0x7f73d800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:19.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.282+0000 7f73ddffb640 1 -- 172.21.15.6:0/2577850267 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8019070 con 0x7f73e0102ef0 2024-09-12T03:21:19.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.282+0000 7f73e70d2640 1 -- 172.21.15.6:0/2577850267 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e01a6cc0 con 0x7f73e0102ef0 2024-09-12T03:21:19.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.282+0000 7f73ddffb640 1 -- 172.21.15.6:0/2577850267 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d800cb40 con 0x7f73e0102ef0 2024-09-12T03:21:19.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.282+0000 7f73ddffb640 1 -- 172.21.15.6:0/2577850267 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8007650 con 0x7f73e0102ef0 2024-09-12T03:21:19.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.282+0000 7f73e70d2640 1 -- 172.21.15.6:0/2577850267 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e01a7250 con 0x7f73e0102ef0 2024-09-12T03:21:19.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.284+0000 7f73ddffb640 1 -- 172.21.15.6:0/2577850267 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f73d801c070 con 0x7f73e0102ef0 2024-09-12T03:21:19.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.285+0000 7f73e94ab740 1 Processor -- start 2024-09-12T03:21:19.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.285+0000 7f73e94ab740 1 -- start start 2024-09-12T03:21:19.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.285+0000 7f73e94ab740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55df681aa9d0 0x55df681aadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:19.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.285+0000 7f73e94ab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681ab2f0 0x55df681b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:19.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.285+0000 7f73ddffb640 1 --2- 172.21.15.6:0/2577850267 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f73cc0a2c10 0x7f73cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:19.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.285+0000 7f73e94ab740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681b5bb0 0x55df681b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:19.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.285+0000 7f73e94ab740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55df68111090 con 0x55df681b5bb0 2024-09-12T03:21:19.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.285+0000 7f73e94ab740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55df680b8a70 con 0x55df681aa9d0 2024-09-12T03:21:19.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.285+0000 7f73e94ab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55df6810d050 con 0x55df681ab2f0 2024-09-12T03:21:19.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.285+0000 7f73dffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681b5bb0 0x55df681b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:19.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.285+0000 7f73dffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681b5bb0 0x55df681b7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56122/0 (socket says 172.21.15.6:56122) 2024-09-12T03:21:19.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.285+0000 7f73e566f640 1 --2- 172.21.15.6:0/2577850267 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f73cc0a2c10 0x7f73cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:19.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.285+0000 7f73dffff640 1 -- 172.21.15.6:0/1611698433 learned_addr learned my addr 172.21.15.6:0/1611698433 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:19.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.286+0000 7f73dffff640 1 --2- 172.21.15.6:0/1611698433 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55df681aa9d0 0x55df681aadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:19.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.286+0000 7f73e4e6e640 1 --2- 172.21.15.6:0/1611698433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681ab2f0 0x55df681b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:19.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.286+0000 7f73dffff640 1 -- 172.21.15.6:0/1611698433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681ab2f0 msgr2=0x55df681b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:19.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.286+0000 7f73dffff640 1 --2- 172.21.15.6:0/1611698433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681ab2f0 0x55df681b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.286+0000 7f73dffff640 1 -- 172.21.15.6:0/1611698433 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55df681aa9d0 msgr2=0x55df681aadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:19.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.286+0000 7f73dffff640 1 --2- 172.21.15.6:0/1611698433 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55df681aa9d0 0x55df681aadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.286+0000 7f73dffff640 1 -- 172.21.15.6:0/1611698433 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df680c5130 con 0x55df681b5bb0 2024-09-12T03:21:19.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.286+0000 7f73e4e6e640 1 --2- 172.21.15.6:0/1611698433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681ab2f0 0x55df681b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:19.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.286+0000 7f73ddffb640 1 -- 172.21.15.6:0/2577850267 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f73d80ce640 con 0x7f73e0102ef0 2024-09-12T03:21:19.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.286+0000 7f73e566f640 1 --2- 172.21.15.6:0/2577850267 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f73cc0a2c10 0x7f73cc0a50d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f73d80a35c0 tx=0x7f73d8096aa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:19.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.286+0000 7f73dffff640 1 --2- 172.21.15.6:0/1611698433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681b5bb0 0x55df681b7fa0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f73c8009230 tx=0x7f73c8005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801bde7acae94ddc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:19.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.286+0000 7f73d6ffd640 1 -- 172.21.15.6:0/1611698433 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c8035030 con 0x55df681b5bb0 2024-09-12T03:21:19.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.286+0000 7f73d6ffd640 1 -- 172.21.15.6:0/1611698433 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73c80038c0 con 0x55df681b5bb0 2024-09-12T03:21:19.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.286+0000 7f73d6ffd640 1 -- 172.21.15.6:0/1611698433 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c80049b0 con 0x55df681b5bb0 2024-09-12T03:21:19.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.287+0000 7f73e94ab740 1 -- 172.21.15.6:0/1611698433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681b5bb0 msgr2=0x55df681b7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:19.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.287+0000 7f73e94ab740 1 --2- 172.21.15.6:0/1611698433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681b5bb0 0x55df681b7fa0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f73c8009230 tx=0x7f73c8005d70 comp rx=0 tx=0).stop 2024-09-12T03:21:19.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.287+0000 7f73e94ab740 1 -- 172.21.15.6:0/1611698433 shutdown_connections 2024-09-12T03:21:19.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.287+0000 7f73e94ab740 1 --2- 172.21.15.6:0/1611698433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681b5bb0 0x55df681b7fa0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.287+0000 7f73e94ab740 1 --2- 172.21.15.6:0/1611698433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681ab2f0 0x55df681b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.287+0000 7f73e94ab740 1 --2- 172.21.15.6:0/1611698433 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55df681aa9d0 0x55df681aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.287+0000 7f73e94ab740 1 -- 172.21.15.6:0/1611698433 >> 172.21.15.6:0/1611698433 conn(0x55df681a6f60 msgr2=0x55df681a7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:19.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.287+0000 7f73e94ab740 1 -- 172.21.15.6:0/1611698433 shutdown_connections 2024-09-12T03:21:19.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.287+0000 7f73e94ab740 1 -- 172.21.15.6:0/1611698433 wait complete. 2024-09-12T03:21:19.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.287+0000 7f73e94ab740 1 -- start start 2024-09-12T03:21:19.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.288+0000 7f73e94ab740 1 Processor -- start 2024-09-12T03:21:19.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.288+0000 7f73e94ab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681aa9d0 0x55df681b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:19.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.288+0000 7f73e94ab740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681ab2f0 0x55df681b2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:19.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.288+0000 7f73e94ab740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55df681b5bb0 0x55df681b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:19.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.288+0000 7f73e94ab740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55df681b8780 con 0x55df681ab2f0 2024-09-12T03:21:19.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.288+0000 7f73e94ab740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55df680cbf20 con 0x55df681b5bb0 2024-09-12T03:21:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.288+0000 7f73e94ab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55df681b8900 con 0x55df681aa9d0 2024-09-12T03:21:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.288+0000 7f73e94ab740 10 client.0 _ll_register_callbacks cb 0x55df68095030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.288+0000 7f73e4e6e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681ab2f0 0x55df681b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.288+0000 7f73e4e6e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681ab2f0 0x55df681b2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56126/0 (socket says 172.21.15.6:56126) 2024-09-12T03:21:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.288+0000 7f73e4e6e640 1 -- 172.21.15.6:0/4230904779 learned_addr learned my addr 172.21.15.6:0/4230904779 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.288+0000 7f73dffff640 1 --2- 172.21.15.6:0/4230904779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681aa9d0 0x55df681b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.288+0000 7f73dffff640 1 --2- 172.21.15.6:0/4230904779 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55df681b5bb0 0x55df681b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.288+0000 7f73e4e6e640 1 -- 172.21.15.6:0/4230904779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681aa9d0 msgr2=0x55df681b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.288+0000 7f73e4e6e640 1 --2- 172.21.15.6:0/4230904779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681aa9d0 0x55df681b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.288+0000 7f73e4e6e640 1 -- 172.21.15.6:0/4230904779 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55df681b5bb0 msgr2=0x55df681b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.288+0000 7f73e4e6e640 1 --2- 172.21.15.6:0/4230904779 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55df681b5bb0 0x55df681b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.289+0000 7f73e4e6e640 1 -- 172.21.15.6:0/4230904779 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df67e822b0 con 0x55df681ab2f0 2024-09-12T03:21:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.289+0000 7f73dffff640 1 --2- 172.21.15.6:0/4230904779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681aa9d0 0x55df681b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.289+0000 7f73e4e6e640 1 --2- 172.21.15.6:0/4230904779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681ab2f0 0x55df681b2740 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f73d000bed0 tx=0x7f73d00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478d51fbffcef6d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.289+0000 7f73bffff640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:21:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.289+0000 7f73bffff640 1 -- 172.21.15.6:0/4230904779 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0012650 con 0x55df681ab2f0 2024-09-12T03:21:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.289+0000 7f73e94ab740 1 -- 172.21.15.6:0/4230904779 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73c80038c0 con 0x55df681ab2f0 2024-09-12T03:21:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.289+0000 7f73bffff640 1 -- 172.21.15.6:0/4230904779 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f73d00127f0 con 0x55df681ab2f0 2024-09-12T03:21:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.289+0000 7f73bffff640 1 -- 172.21.15.6:0/4230904779 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f73d00129f0 con 0x55df681ab2f0 2024-09-12T03:21:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.289+0000 7f73e94ab740 10 client.15297 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.289+0000 7f73e94ab740 1 -- 172.21.15.6:0/4230904779 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55df681c4120 con 0x55df681ab2f0 2024-09-12T03:21:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.289+0000 7f73bffff640 1 -- 172.21.15.6:0/4230904779 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73c80038c0 con 0x55df681ab2f0 2024-09-12T03:21:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.290+0000 7f73bffff640 1 -- 172.21.15.6:0/4230904779 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f73d0026420 con 0x55df681ab2f0 2024-09-12T03:21:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.290+0000 7f73e94ab740 10 client.15297 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.290+0000 7f73e94ab740 10 client.15297 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T03:21:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.290+0000 7f73e94ab740 20 client.15297 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.290+0000 7f73e94ab740 1 --2- 172.21.15.6:0/4230904779 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55df681c67c0 0x55df681c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:19.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.290+0000 7f73e94ab740 4 client.15297 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:19.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.290+0000 7f73e94ab740 1 -- 172.21.15.6:0/4230904779 --> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df6811e600 con 0x55df681c67c0 2024-09-12T03:21:19.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.290+0000 7f73dffff640 1 --2- 172.21.15.6:0/4230904779 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55df681c67c0 0x55df681c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:19.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.290+0000 7f73bffff640 10 client.15297 ms_handle_connect on v2:172.21.15.118:6836/3879191727 2024-09-12T03:21:19.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.290+0000 7f73dffff640 1 --2- 172.21.15.6:0/4230904779 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55df681c67c0 0x55df681c7c30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7dfafc8059c4881d server_cookie=39b69ae9200f027f in_seq=0 out_seq=0 2024-09-12T03:21:19.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.291+0000 7f73bffff640 1 -- 172.21.15.6:0/4230904779 <== mds.1 v2:172.21.15.118:6836/3879191727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df6811e600 con 0x55df681c67c0 2024-09-12T03:21:19.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.291+0000 7f73bffff640 10 client.15297 handle_command_reply: tid=0 2024-09-12T03:21:19.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.291+0000 7f73e94ab740 1 client.15297 shutdown 2024-09-12T03:21:19.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.291+0000 7f73e94ab740 1 -- 172.21.15.6:0/4230904779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681ab2f0 msgr2=0x55df681b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:19.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.291+0000 7f73e94ab740 1 --2- 172.21.15.6:0/4230904779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681ab2f0 0x55df681b2740 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f73d000bed0 tx=0x7f73d00117b0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.291+0000 7f73e94ab740 1 -- 172.21.15.6:0/4230904779 shutdown_connections 2024-09-12T03:21:19.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.291+0000 7f73e94ab740 1 --2- 172.21.15.6:0/4230904779 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55df681c67c0 0x55df681c7c30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.291+0000 7f73e94ab740 1 --2- 172.21.15.6:0/4230904779 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55df681b5bb0 0x55df681b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.291+0000 7f73e94ab740 1 --2- 172.21.15.6:0/4230904779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681ab2f0 0x55df681b2740 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.291+0000 7f73e94ab740 1 --2- 172.21.15.6:0/4230904779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681aa9d0 0x55df681b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.291+0000 7f73e94ab740 1 -- 172.21.15.6:0/4230904779 >> 172.21.15.6:0/4230904779 conn(0x55df681a6f60 msgr2=0x55df681acc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:19.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.291+0000 7f73bffff640 0 client.15297 ms_handle_reset on v2:172.21.15.118:6836/3879191727 2024-09-12T03:21:19.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.292+0000 7f73e94ab740 1 -- 172.21.15.6:0/4230904779 shutdown_connections 2024-09-12T03:21:19.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.292+0000 7f73e94ab740 1 -- 172.21.15.6:0/4230904779 wait complete. 2024-09-12T03:21:19.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.292+0000 7f73e94ab740 20 client.15297 trim_cache size 0 max 16384 2024-09-12T03:21:19.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.309+0000 7f73e94ab740 1 Processor -- start 2024-09-12T03:21:19.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.309+0000 7f73e94ab740 1 -- start start 2024-09-12T03:21:19.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.310+0000 7f73e94ab740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55df681aa9d0 0x55df681c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:19.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.310+0000 7f73e94ab740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681ab2f0 0x55df681c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:19.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.310+0000 7f73e94ab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681b5bb0 0x55df681af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:19.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.310+0000 7f73e94ab740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55df681c5f70 con 0x55df681ab2f0 2024-09-12T03:21:19.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.310+0000 7f73e94ab740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55df681c6640 con 0x55df681aa9d0 2024-09-12T03:21:19.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.310+0000 7f73e94ab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55df681c6d10 con 0x55df681b5bb0 2024-09-12T03:21:19.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.310+0000 7f73e4e6e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681ab2f0 0x55df681c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:19.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.310+0000 7f73e4e6e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681ab2f0 0x55df681c7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56128/0 (socket says 172.21.15.6:56128) 2024-09-12T03:21:19.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.310+0000 7f73e4e6e640 1 -- 172.21.15.6:0/3008930201 learned_addr learned my addr 172.21.15.6:0/3008930201 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:19.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.310+0000 7f73dffff640 1 --2- 172.21.15.6:0/3008930201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681b5bb0 0x55df681af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:19.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.310+0000 7f73e4e6e640 1 -- 172.21.15.6:0/3008930201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681b5bb0 msgr2=0x55df681af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:19.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.310+0000 7f73dffff640 1 --2- 172.21.15.6:0/3008930201 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55df681aa9d0 0x55df681c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:19.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.310+0000 7f73e4e6e640 1 --2- 172.21.15.6:0/3008930201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681b5bb0 0x55df681af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.310+0000 7f73e4e6e640 1 -- 172.21.15.6:0/3008930201 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55df681aa9d0 msgr2=0x55df681c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:19.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.310+0000 7f73e4e6e640 1 --2- 172.21.15.6:0/3008930201 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55df681aa9d0 0x55df681c60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.310+0000 7f73e4e6e640 1 -- 172.21.15.6:0/3008930201 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df681ab120 con 0x55df681ab2f0 2024-09-12T03:21:19.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.310+0000 7f73e4e6e640 1 --2- 172.21.15.6:0/3008930201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681ab2f0 0x55df681c7c30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f73d0026910 tx=0x7f73d0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d91945a0b335eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:19.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.310+0000 7f73d67fc640 1 -- 172.21.15.6:0/3008930201 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d002b070 con 0x55df681ab2f0 2024-09-12T03:21:19.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.311+0000 7f73d67fc640 1 -- 172.21.15.6:0/3008930201 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d0011cb0 con 0x55df681ab2f0 2024-09-12T03:21:19.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.311+0000 7f73d67fc640 1 -- 172.21.15.6:0/3008930201 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d00052c0 con 0x55df681ab2f0 2024-09-12T03:21:19.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.311+0000 7f73e94ab740 1 -- 172.21.15.6:0/3008930201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681ab2f0 msgr2=0x55df681c7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:19.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.311+0000 7f73e94ab740 1 --2- 172.21.15.6:0/3008930201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681ab2f0 0x55df681c7c30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f73d0026910 tx=0x7f73d0005c50 comp rx=0 tx=0).stop 2024-09-12T03:21:19.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.311+0000 7f73e94ab740 1 -- 172.21.15.6:0/3008930201 shutdown_connections 2024-09-12T03:21:19.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.311+0000 7f73e94ab740 1 --2- 172.21.15.6:0/3008930201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681b5bb0 0x55df681af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.311+0000 7f73e94ab740 1 --2- 172.21.15.6:0/3008930201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681ab2f0 0x55df681c7c30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.311+0000 7f73e94ab740 1 --2- 172.21.15.6:0/3008930201 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55df681aa9d0 0x55df681c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.311+0000 7f73e94ab740 1 -- 172.21.15.6:0/3008930201 >> 172.21.15.6:0/3008930201 conn(0x55df681a6f60 msgr2=0x55df681a7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:19.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.311+0000 7f73e94ab740 1 -- 172.21.15.6:0/3008930201 shutdown_connections 2024-09-12T03:21:19.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.311+0000 7f73e94ab740 1 -- 172.21.15.6:0/3008930201 wait complete. 2024-09-12T03:21:19.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.311+0000 7f73e94ab740 1 -- start start 2024-09-12T03:21:19.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.312+0000 7f73e94ab740 1 Processor -- start 2024-09-12T03:21:19.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.312+0000 7f73e94ab740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55df681aa9d0 0x55df681cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:19.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.312+0000 7f73e94ab740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681ab2f0 0x55df681ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:19.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.312+0000 7f73e94ab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681b5bb0 0x55df681c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:19.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.312+0000 7f73e94ab740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55df681c7ab0 con 0x55df681ab2f0 2024-09-12T03:21:19.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.312+0000 7f73e94ab740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55df681c73e0 con 0x55df681aa9d0 2024-09-12T03:21:19.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.312+0000 7f73e94ab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55df681c8180 con 0x55df681b5bb0 2024-09-12T03:21:19.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.312+0000 7f73e94ab740 10 client.0 _ll_register_callbacks cb 0x7f73d000b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:19.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.312+0000 7f73e4e6e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681ab2f0 0x55df681ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:19.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.312+0000 7f73e4e6e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681ab2f0 0x55df681ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56134/0 (socket says 172.21.15.6:56134) 2024-09-12T03:21:19.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.312+0000 7f73e4e6e640 1 -- 172.21.15.6:0/929684042 learned_addr learned my addr 172.21.15.6:0/929684042 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:19.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.312+0000 7f73dffff640 1 --2- 172.21.15.6:0/929684042 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55df681aa9d0 0x55df681cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:19.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.312+0000 7f73dffff640 1 --2- 172.21.15.6:0/929684042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681b5bb0 0x55df681c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:19.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.312+0000 7f73e4e6e640 1 -- 172.21.15.6:0/929684042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681b5bb0 msgr2=0x55df681c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:19.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.313+0000 7f73e4e6e640 1 --2- 172.21.15.6:0/929684042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681b5bb0 0x55df681c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.313+0000 7f73e4e6e640 1 -- 172.21.15.6:0/929684042 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55df681aa9d0 msgr2=0x55df681cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:19.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.313+0000 7f73e4e6e640 1 --2- 172.21.15.6:0/929684042 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55df681aa9d0 0x55df681cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.313+0000 7f73e4e6e640 1 -- 172.21.15.6:0/929684042 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df681c98a0 con 0x55df681ab2f0 2024-09-12T03:21:19.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.313+0000 7f73dffff640 1 --2- 172.21.15.6:0/929684042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681b5bb0 0x55df681c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:19.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.313+0000 7f73e4e6e640 1 --2- 172.21.15.6:0/929684042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681ab2f0 0x55df681ce220 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f73d0026a00 tx=0x7f73d0012d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95612369e373acb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:19.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.313+0000 7f73bffff640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:21:19.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.313+0000 7f73bffff640 1 -- 172.21.15.6:0/929684042 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d002b070 con 0x55df681ab2f0 2024-09-12T03:21:19.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.313+0000 7f73bffff640 1 -- 172.21.15.6:0/929684042 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f73d0012450 con 0x55df681ab2f0 2024-09-12T03:21:19.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.313+0000 7f73e94ab740 1 -- 172.21.15.6:0/929684042 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73d0011cb0 con 0x55df681ab2f0 2024-09-12T03:21:19.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.313+0000 7f73bffff640 1 -- 172.21.15.6:0/929684042 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f73d0012650 con 0x55df681ab2f0 2024-09-12T03:21:19.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.313+0000 7f73e94ab740 10 client.15303 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:19.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.313+0000 7f73e94ab740 1 -- 172.21.15.6:0/929684042 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55df681cd3a0 con 0x55df681ab2f0 2024-09-12T03:21:19.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.313+0000 7f73bffff640 1 -- 172.21.15.6:0/929684042 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d0011cb0 con 0x55df681ab2f0 2024-09-12T03:21:19.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.314+0000 7f73bffff640 1 -- 172.21.15.6:0/929684042 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f73d0004710 con 0x55df681ab2f0 2024-09-12T03:21:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.314+0000 7f73e94ab740 10 client.15303 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.314+0000 7f73e94ab740 10 client.15303 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T03:21:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.314+0000 7f73e94ab740 20 client.15303 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.314+0000 7f73e94ab740 1 --2- 172.21.15.6:0/929684042 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55df68235160 0x55df68233e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.314+0000 7f73e94ab740 4 client.15303 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:21:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.314+0000 7f73e94ab740 1 -- 172.21.15.6:0/929684042 --> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] -- command(tid 0: {"prefix": "status"}) -- 0x55df68118b10 con 0x55df68235160 2024-09-12T03:21:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.314+0000 7f73dffff640 1 --2- 172.21.15.6:0/929684042 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55df68235160 0x55df68233e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.314+0000 7f73dffff640 1 --2- 172.21.15.6:0/929684042 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55df68235160 0x55df68233e70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c47aac8d91fed4 server_cookie=b51c23f1c08a00eb in_seq=0 out_seq=0 2024-09-12T03:21:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.314+0000 7f73bffff640 10 client.15303 ms_handle_connect on v2:172.21.15.118:6836/3879191727 2024-09-12T03:21:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.315+0000 7f73bffff640 1 -- 172.21.15.6:0/929684042 <== mds.1 v2:172.21.15.118:6836/3879191727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55df68118b10 con 0x55df68235160 2024-09-12T03:21:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.315+0000 7f73bffff640 10 client.15303 handle_command_reply: tid=0 2024-09-12T03:21:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.315+0000 7f73e94ab740 1 client.15303 shutdown 2024-09-12T03:21:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.315+0000 7f73e94ab740 1 -- 172.21.15.6:0/929684042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681ab2f0 msgr2=0x55df681ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.315+0000 7f73e94ab740 1 --2- 172.21.15.6:0/929684042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681ab2f0 0x55df681ce220 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f73d0026a00 tx=0x7f73d0012d30 comp rx=0 tx=0).stop 2024-09-12T03:21:19.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.315+0000 7f73e94ab740 1 -- 172.21.15.6:0/929684042 shutdown_connections 2024-09-12T03:21:19.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.315+0000 7f73e94ab740 1 --2- 172.21.15.6:0/929684042 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55df68235160 0x55df68233e70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.315+0000 7f73e94ab740 1 --2- 172.21.15.6:0/929684042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df681b5bb0 0x55df681c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.315+0000 7f73e94ab740 1 --2- 172.21.15.6:0/929684042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55df681ab2f0 0x55df681ce220 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.315+0000 7f73e94ab740 1 --2- 172.21.15.6:0/929684042 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55df681aa9d0 0x55df681cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.315+0000 7f73e94ab740 1 -- 172.21.15.6:0/929684042 >> 172.21.15.6:0/929684042 conn(0x55df681a6f60 msgr2=0x55df681a57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:19.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.315+0000 7f73bffff640 0 client.15303 ms_handle_reset on v2:172.21.15.118:6836/3879191727 2024-09-12T03:21:19.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.315+0000 7f73e94ab740 1 -- 172.21.15.6:0/929684042 shutdown_connections 2024-09-12T03:21:19.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.315+0000 7f73e94ab740 1 -- 172.21.15.6:0/929684042 wait complete. 2024-09-12T03:21:19.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.315+0000 7f73e94ab740 20 client.15303 trim_cache size 0 max 16384 2024-09-12T03:21:19.321 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:21:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "89d99938-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:21:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 1, 2024-09-12T03:21:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24464, 2024-09-12T03:21:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-09-12T03:21:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-09-12T03:21:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-09-12T03:21:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 121.958410759, 2024-09-12T03:21:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 25, 2024-09-12T03:21:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 82, 2024-09-12T03:21:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T03:21:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 149.22665440899999 2024-09-12T03:21:19.321 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:21:19.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.315+0000 7f73e70d2640 1 -- 172.21.15.6:0/2577850267 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f73cc0a2c10 msgr2=0x7f73cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:19.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.315+0000 7f73e70d2640 1 --2- 172.21.15.6:0/2577850267 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f73cc0a2c10 0x7f73cc0a50d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f73d80a35c0 tx=0x7f73d8096aa0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.315+0000 7f73e70d2640 1 -- 172.21.15.6:0/2577850267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73e0102ef0 msgr2=0x7f73e01a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:19.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.315+0000 7f73e70d2640 1 --2- 172.21.15.6:0/2577850267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73e0102ef0 0x7f73e01a63b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f73d800b970 tx=0x7f73d800be30 comp rx=0 tx=0).stop 2024-09-12T03:21:19.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.316+0000 7f73e70d2640 1 -- 172.21.15.6:0/2577850267 shutdown_connections 2024-09-12T03:21:19.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.316+0000 7f73e70d2640 1 --2- 172.21.15.6:0/2577850267 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f73cc0a2c10 0x7f73cc0a50d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.316+0000 7f73e70d2640 1 --2- 172.21.15.6:0/2577850267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73e0102ef0 0x7f73e01a63b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.316+0000 7f73e70d2640 1 --2- 172.21.15.6:0/2577850267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73e0102570 0x7f73e0197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.316+0000 7f73e70d2640 1 --2- 172.21.15.6:0/2577850267 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73e0100e20 0x7f73e0196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:19.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.316+0000 7f73e70d2640 1 -- 172.21.15.6:0/2577850267 >> 172.21.15.6:0/2577850267 conn(0x7f73e00fc7d0 msgr2=0x7f73e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:19.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.316+0000 7f73e70d2640 1 -- 172.21.15.6:0/2577850267 shutdown_connections 2024-09-12T03:21:19.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:19.316+0000 7f73e70d2640 1 -- 172.21.15.6:0/2577850267 wait complete. 2024-09-12T03:21:19.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:19 smithi161 ceph-mon[35485]: pgmap v341: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:21:19.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:19 smithi118 ceph-mon[35048]: pgmap v341: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:21:20.039 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T03:21:20.296 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:20.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.788+0000 7efc9f326640 1 -- 172.21.15.6:0/2468185838 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc98106f10 msgr2=0x7efc9810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:20.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.788+0000 7efc9f326640 1 --2- 172.21.15.6:0/2468185838 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc98106f10 0x7efc9810ea90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efc90009ca0 tx=0x7efc9002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.788+0000 7efc9f326640 1 -- 172.21.15.6:0/2468185838 shutdown_connections 2024-09-12T03:21:20.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.788+0000 7efc9f326640 1 --2- 172.21.15.6:0/2468185838 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc98106f10 0x7efc9810ea90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.788+0000 7efc9f326640 1 --2- 172.21.15.6:0/2468185838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc98102790 0x7efc981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.788+0000 7efc9f326640 1 --2- 172.21.15.6:0/2468185838 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efc98101de0 0x7efc981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.788+0000 7efc9f326640 1 -- 172.21.15.6:0/2468185838 >> 172.21.15.6:0/2468185838 conn(0x7efc980fd8c0 msgr2=0x7efc980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:20.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.788+0000 7efc9f326640 1 -- 172.21.15.6:0/2468185838 shutdown_connections 2024-09-12T03:21:20.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.788+0000 7efc9f326640 1 -- 172.21.15.6:0/2468185838 wait complete. 2024-09-12T03:21:20.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.789+0000 7efc9f326640 1 Processor -- start 2024-09-12T03:21:20.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.789+0000 7efc9f326640 1 -- start start 2024-09-12T03:21:20.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.789+0000 7efc9f326640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efc98101de0 0x7efc981983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.789+0000 7efc9f326640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc98102790 0x7efc981988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.789+0000 7efc9f326640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc98106f10 0x7efc9819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.789+0000 7efc9f326640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7efc98111f90 con 0x7efc98102790 2024-09-12T03:21:20.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.789+0000 7efc9f326640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7efc98111e10 con 0x7efc98101de0 2024-09-12T03:21:20.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.789+0000 7efc9f326640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efc98112110 con 0x7efc98106f10 2024-09-12T03:21:20.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.789+0000 7efc9c8c1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc98102790 0x7efc981988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.789+0000 7efc9c8c1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc98102790 0x7efc981988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56146/0 (socket says 172.21.15.6:56146) 2024-09-12T03:21:20.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.789+0000 7efc9c8c1640 1 -- 172.21.15.6:0/150261223 learned_addr learned my addr 172.21.15.6:0/150261223 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:20.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.789+0000 7efc9d0c2640 1 --2- 172.21.15.6:0/150261223 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efc98101de0 0x7efc981983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.789+0000 7efc9d8c3640 1 --2- 172.21.15.6:0/150261223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc98106f10 0x7efc9819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.790+0000 7efc9c8c1640 1 -- 172.21.15.6:0/150261223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc98106f10 msgr2=0x7efc9819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:20.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.790+0000 7efc9c8c1640 1 --2- 172.21.15.6:0/150261223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc98106f10 0x7efc9819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.790+0000 7efc9c8c1640 1 -- 172.21.15.6:0/150261223 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efc98101de0 msgr2=0x7efc981983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:20.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.790+0000 7efc9c8c1640 1 --2- 172.21.15.6:0/150261223 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efc98101de0 0x7efc981983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.790+0000 7efc9c8c1640 1 -- 172.21.15.6:0/150261223 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9819d350 con 0x7efc98102790 2024-09-12T03:21:20.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.790+0000 7efc9d8c3640 1 --2- 172.21.15.6:0/150261223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc98106f10 0x7efc9819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:20.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.790+0000 7efc9c8c1640 1 --2- 172.21.15.6:0/150261223 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc98102790 0x7efc981988e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7efc8800c920 tx=0x7efc8800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:20.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.790+0000 7efc8e7fc640 1 -- 172.21.15.6:0/150261223 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88007bb0 con 0x7efc98102790 2024-09-12T03:21:20.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.790+0000 7efc8e7fc640 1 -- 172.21.15.6:0/150261223 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc88007d50 con 0x7efc98102790 2024-09-12T03:21:20.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.790+0000 7efc9f326640 1 -- 172.21.15.6:0/150261223 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc9819d640 con 0x7efc98102790 2024-09-12T03:21:20.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.790+0000 7efc8e7fc640 1 -- 172.21.15.6:0/150261223 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc880187d0 con 0x7efc98102790 2024-09-12T03:21:20.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.790+0000 7efc9f326640 1 -- 172.21.15.6:0/150261223 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc981a4f20 con 0x7efc98102790 2024-09-12T03:21:20.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.792+0000 7efc8e7fc640 1 -- 172.21.15.6:0/150261223 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7efc8800f450 con 0x7efc98102790 2024-09-12T03:21:20.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.793+0000 7efca16ff740 1 Processor -- start 2024-09-12T03:21:20.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.793+0000 7efca16ff740 1 -- start start 2024-09-12T03:21:20.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.793+0000 7efca16ff740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c4188b0 0x55a32c418c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.793+0000 7efca16ff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c4191d0 0x55a32c427a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.793+0000 7efca16ff740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c427fc0 0x55a32c42a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.793+0000 7efca16ff740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a32c37f090 con 0x55a32c4188b0 2024-09-12T03:21:20.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.793+0000 7efca16ff740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55a32c326a70 con 0x55a32c427fc0 2024-09-12T03:21:20.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.793+0000 7efca16ff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a32c37b050 con 0x55a32c4191d0 2024-09-12T03:21:20.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.793+0000 7efc9d8c3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c4188b0 0x55a32c418c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.793+0000 7efc8e7fc640 1 --2- 172.21.15.6:0/150261223 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efc840a2c10 0x7efc840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.793+0000 7efc9d8c3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c4188b0 0x55a32c418c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56150/0 (socket says 172.21.15.6:56150) 2024-09-12T03:21:20.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.793+0000 7efc9d8c3640 1 -- 172.21.15.6:0/1661808507 learned_addr learned my addr 172.21.15.6:0/1661808507 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:20.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.793+0000 7efc9d0c2640 1 --2- 172.21.15.6:0/1661808507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c4191d0 0x55a32c427a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.793+0000 7efc9c8c1640 1 --2- 172.21.15.6:0/1661808507 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c427fc0 0x55a32c42a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.794+0000 7efc8e7fc640 1 -- 172.21.15.6:0/150261223 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7efc880cf2e0 con 0x7efc98102790 2024-09-12T03:21:20.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.794+0000 7efc9d8c3640 1 --2- 172.21.15.6:0/150261223 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efc840a2c10 0x7efc840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.794+0000 7efc9d0c2640 1 -- 172.21.15.6:0/1661808507 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c427fc0 msgr2=0x55a32c42a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:20.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.794+0000 7efc9d0c2640 1 --2- 172.21.15.6:0/1661808507 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c427fc0 0x55a32c42a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.794+0000 7efc9d0c2640 1 -- 172.21.15.6:0/1661808507 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c4188b0 msgr2=0x55a32c418c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:20.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.794+0000 7efc9d0c2640 1 --2- 172.21.15.6:0/1661808507 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c4188b0 0x55a32c418c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.794+0000 7efc9d0c2640 1 -- 172.21.15.6:0/1661808507 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a32c333130 con 0x55a32c4191d0 2024-09-12T03:21:20.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.794+0000 7efc9c8c1640 1 --2- 172.21.15.6:0/1661808507 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c427fc0 0x55a32c42a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:20.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.794+0000 7efc9d0c2640 1 --2- 172.21.15.6:0/1661808507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c4191d0 0x55a32c427a80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efc940099a0 tx=0x7efc94009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ae6f68976ed0416 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:20.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.794+0000 7efc9d8c3640 1 --2- 172.21.15.6:0/150261223 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efc840a2c10 0x7efc840a50d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efc90009430 tx=0x7efc9002a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:20.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.794+0000 7efc837fe640 1 -- 172.21.15.6:0/1661808507 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc94017070 con 0x55a32c4191d0 2024-09-12T03:21:20.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.794+0000 7efc837fe640 1 -- 172.21.15.6:0/1661808507 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc94013470 con 0x55a32c4191d0 2024-09-12T03:21:20.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.794+0000 7efc837fe640 1 -- 172.21.15.6:0/1661808507 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc94012610 con 0x55a32c4191d0 2024-09-12T03:21:20.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.795+0000 7efca16ff740 1 -- 172.21.15.6:0/1661808507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c4191d0 msgr2=0x55a32c427a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:20.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.795+0000 7efca16ff740 1 --2- 172.21.15.6:0/1661808507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c4191d0 0x55a32c427a80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efc940099a0 tx=0x7efc94009e60 comp rx=0 tx=0).stop 2024-09-12T03:21:20.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.795+0000 7efca16ff740 1 -- 172.21.15.6:0/1661808507 shutdown_connections 2024-09-12T03:21:20.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.795+0000 7efca16ff740 1 --2- 172.21.15.6:0/1661808507 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c427fc0 0x55a32c42a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.795+0000 7efca16ff740 1 --2- 172.21.15.6:0/1661808507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c4191d0 0x55a32c427a80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.795+0000 7efca16ff740 1 --2- 172.21.15.6:0/1661808507 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c4188b0 0x55a32c418c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.795+0000 7efca16ff740 1 -- 172.21.15.6:0/1661808507 >> 172.21.15.6:0/1661808507 conn(0x55a32c414f60 msgr2=0x55a32c415340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:20.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.795+0000 7efca16ff740 1 -- 172.21.15.6:0/1661808507 shutdown_connections 2024-09-12T03:21:20.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.795+0000 7efca16ff740 1 -- 172.21.15.6:0/1661808507 wait complete. 2024-09-12T03:21:20.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.795+0000 7efca16ff740 1 -- start start 2024-09-12T03:21:20.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.795+0000 7efca16ff740 1 Processor -- start 2024-09-12T03:21:20.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.796+0000 7efca16ff740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c4188b0 0x55a32c427800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.796+0000 7efca16ff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c4191d0 0x55a32c424b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.796+0000 7efca16ff740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c427fc0 0x55a32c425090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.796+0000 7efca16ff740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a32c42ab90 con 0x55a32c427fc0 2024-09-12T03:21:20.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.796+0000 7efca16ff740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55a32c339f20 con 0x55a32c4188b0 2024-09-12T03:21:20.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.796+0000 7efca16ff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a32c42ad10 con 0x55a32c4191d0 2024-09-12T03:21:20.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.796+0000 7efca16ff740 10 client.0 _ll_register_callbacks cb 0x55a32c303030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:20.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.796+0000 7efc9d8c3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c427fc0 0x55a32c425090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.796+0000 7efc9d8c3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c427fc0 0x55a32c425090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56164/0 (socket says 172.21.15.6:56164) 2024-09-12T03:21:20.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.796+0000 7efc9d8c3640 1 -- 172.21.15.6:0/3966341070 learned_addr learned my addr 172.21.15.6:0/3966341070 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:20.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.796+0000 7efc9c8c1640 1 --2- 172.21.15.6:0/3966341070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c4191d0 0x55a32c424b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.796+0000 7efc9d0c2640 1 --2- 172.21.15.6:0/3966341070 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c4188b0 0x55a32c427800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.796+0000 7efc9d8c3640 1 -- 172.21.15.6:0/3966341070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c4191d0 msgr2=0x55a32c424b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:20.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.796+0000 7efc9d8c3640 1 --2- 172.21.15.6:0/3966341070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c4191d0 0x55a32c424b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.796+0000 7efc9d8c3640 1 -- 172.21.15.6:0/3966341070 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c4188b0 msgr2=0x55a32c427800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:20.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.796+0000 7efc9d8c3640 1 --2- 172.21.15.6:0/3966341070 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c4188b0 0x55a32c427800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.796+0000 7efc9d8c3640 1 -- 172.21.15.6:0/3966341070 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a32c0f02b0 con 0x55a32c427fc0 2024-09-12T03:21:20.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.796+0000 7efc9d0c2640 1 --2- 172.21.15.6:0/3966341070 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c4188b0 0x55a32c427800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:20.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.796+0000 7efc9d8c3640 1 --2- 172.21.15.6:0/3966341070 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c427fc0 0x55a32c425090 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7efc9002ade0 tx=0x7efc900095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458d893a0823a6b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:20.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.796+0000 7efc80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:21:20.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.797+0000 7efc80ff9640 1 -- 172.21.15.6:0/3966341070 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90050070 con 0x55a32c427fc0 2024-09-12T03:21:20.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.797+0000 7efc80ff9640 1 -- 172.21.15.6:0/3966341070 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7efc9002f610 con 0x55a32c427fc0 2024-09-12T03:21:20.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.797+0000 7efca16ff740 1 -- 172.21.15.6:0/3966341070 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc94013470 con 0x55a32c427fc0 2024-09-12T03:21:20.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.797+0000 7efc80ff9640 1 -- 172.21.15.6:0/3966341070 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efc9002f810 con 0x55a32c427fc0 2024-09-12T03:21:20.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.797+0000 7efca16ff740 10 client.15315 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:20.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.797+0000 7efca16ff740 1 -- 172.21.15.6:0/3966341070 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a32c436400 con 0x55a32c427fc0 2024-09-12T03:21:20.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.797+0000 7efc80ff9640 1 -- 172.21.15.6:0/3966341070 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc94013470 con 0x55a32c427fc0 2024-09-12T03:21:20.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.797+0000 7efc80ff9640 1 -- 172.21.15.6:0/3966341070 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efc90049050 con 0x55a32c427fc0 2024-09-12T03:21:20.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.797+0000 7efca16ff740 10 client.15315 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:20.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.797+0000 7efca16ff740 10 client.15315 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:21:20.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.797+0000 7efca16ff740 20 client.15315 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:20.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.798+0000 7efca16ff740 1 --2- 172.21.15.6:0/3966341070 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a32c438ab0 0x55a32c439f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.798+0000 7efca16ff740 4 client.15315 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:20.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.798+0000 7efca16ff740 1 -- 172.21.15.6:0/3966341070 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a32c38c600 con 0x55a32c438ab0 2024-09-12T03:21:20.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.798+0000 7efc9d0c2640 1 --2- 172.21.15.6:0/3966341070 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a32c438ab0 0x55a32c439f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.798+0000 7efc80ff9640 10 client.15315 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:20.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.798+0000 7efc9d0c2640 1 --2- 172.21.15.6:0/3966341070 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a32c438ab0 0x55a32c439f20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd4e01cee37ef39 server_cookie=601fb75d040142ba in_seq=0 out_seq=0 2024-09-12T03:21:20.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.799+0000 7efc80ff9640 1 -- 172.21.15.6:0/3966341070 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a32c38c600 con 0x55a32c438ab0 2024-09-12T03:21:20.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.799+0000 7efc80ff9640 10 client.15315 handle_command_reply: tid=0 2024-09-12T03:21:20.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.799+0000 7efca16ff740 1 client.15315 shutdown 2024-09-12T03:21:20.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.799+0000 7efca16ff740 1 -- 172.21.15.6:0/3966341070 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c427fc0 msgr2=0x55a32c425090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:20.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.799+0000 7efca16ff740 1 --2- 172.21.15.6:0/3966341070 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c427fc0 0x55a32c425090 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7efc9002ade0 tx=0x7efc900095c0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.799+0000 7efca16ff740 1 -- 172.21.15.6:0/3966341070 shutdown_connections 2024-09-12T03:21:20.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.799+0000 7efca16ff740 1 --2- 172.21.15.6:0/3966341070 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a32c438ab0 0x55a32c439f20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.799+0000 7efca16ff740 1 --2- 172.21.15.6:0/3966341070 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c427fc0 0x55a32c425090 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.799+0000 7efca16ff740 1 --2- 172.21.15.6:0/3966341070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c4191d0 0x55a32c424b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.799+0000 7efca16ff740 1 --2- 172.21.15.6:0/3966341070 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c4188b0 0x55a32c427800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.799+0000 7efca16ff740 1 -- 172.21.15.6:0/3966341070 >> 172.21.15.6:0/3966341070 conn(0x55a32c414f60 msgr2=0x55a32c41acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:20.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.799+0000 7efc80ff9640 0 client.15315 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:20.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.799+0000 7efca16ff740 1 -- 172.21.15.6:0/3966341070 shutdown_connections 2024-09-12T03:21:20.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.799+0000 7efca16ff740 1 -- 172.21.15.6:0/3966341070 wait complete. 2024-09-12T03:21:20.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.799+0000 7efca16ff740 20 client.15315 trim_cache size 0 max 16384 2024-09-12T03:21:20.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.817+0000 7efca16ff740 1 Processor -- start 2024-09-12T03:21:20.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.817+0000 7efca16ff740 1 -- start start 2024-09-12T03:21:20.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.817+0000 7efca16ff740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c4188b0 0x55a32c434480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.817+0000 7efca16ff740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c4191d0 0x55a32c423420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.817+0000 7efca16ff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c427fc0 0x55a32c423960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.817+0000 7efca16ff740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a32c438260 con 0x55a32c4188b0 2024-09-12T03:21:20.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.817+0000 7efca16ff740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55a32c438930 con 0x55a32c4191d0 2024-09-12T03:21:20.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.817+0000 7efca16ff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a32c439000 con 0x55a32c427fc0 2024-09-12T03:21:20.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.818+0000 7efc9d0c2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c4188b0 0x55a32c434480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.818+0000 7efc9d0c2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c4188b0 0x55a32c434480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56166/0 (socket says 172.21.15.6:56166) 2024-09-12T03:21:20.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.818+0000 7efc9d0c2640 1 -- 172.21.15.6:0/2654696524 learned_addr learned my addr 172.21.15.6:0/2654696524 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:20.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.818+0000 7efc9c8c1640 1 --2- 172.21.15.6:0/2654696524 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c4191d0 0x55a32c423420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.818+0000 7efc9d8c3640 1 --2- 172.21.15.6:0/2654696524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c427fc0 0x55a32c423960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.818+0000 7efc9d0c2640 1 -- 172.21.15.6:0/2654696524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c427fc0 msgr2=0x55a32c423960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:20.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.818+0000 7efc9d0c2640 1 --2- 172.21.15.6:0/2654696524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c427fc0 0x55a32c423960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.818+0000 7efc9d0c2640 1 -- 172.21.15.6:0/2654696524 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c4191d0 msgr2=0x55a32c423420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:20.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.818+0000 7efc9d0c2640 1 --2- 172.21.15.6:0/2654696524 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c4191d0 0x55a32c423420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.818+0000 7efc9d0c2640 1 -- 172.21.15.6:0/2654696524 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a32c4394b0 con 0x55a32c4188b0 2024-09-12T03:21:20.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.818+0000 7efc9d0c2640 1 --2- 172.21.15.6:0/2654696524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c4188b0 0x55a32c434480 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efc940098c0 tx=0x7efc94013e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61dead433d090b00 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:20.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.818+0000 7efc82ffd640 1 -- 172.21.15.6:0/2654696524 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9402a070 con 0x55a32c4188b0 2024-09-12T03:21:20.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.818+0000 7efc82ffd640 1 -- 172.21.15.6:0/2654696524 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc94013950 con 0x55a32c4188b0 2024-09-12T03:21:20.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.818+0000 7efc82ffd640 1 -- 172.21.15.6:0/2654696524 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc94017040 con 0x55a32c4188b0 2024-09-12T03:21:20.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.819+0000 7efca16ff740 1 -- 172.21.15.6:0/2654696524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c4188b0 msgr2=0x55a32c434480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:20.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.819+0000 7efca16ff740 1 --2- 172.21.15.6:0/2654696524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c4188b0 0x55a32c434480 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efc940098c0 tx=0x7efc94013e20 comp rx=0 tx=0).stop 2024-09-12T03:21:20.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.819+0000 7efca16ff740 1 -- 172.21.15.6:0/2654696524 shutdown_connections 2024-09-12T03:21:20.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.819+0000 7efca16ff740 1 --2- 172.21.15.6:0/2654696524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c427fc0 0x55a32c423960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.819+0000 7efca16ff740 1 --2- 172.21.15.6:0/2654696524 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c4191d0 0x55a32c423420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.819+0000 7efca16ff740 1 --2- 172.21.15.6:0/2654696524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c4188b0 0x55a32c434480 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.819+0000 7efca16ff740 1 -- 172.21.15.6:0/2654696524 >> 172.21.15.6:0/2654696524 conn(0x55a32c414f60 msgr2=0x55a32c437750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:20.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.819+0000 7efca16ff740 1 -- 172.21.15.6:0/2654696524 shutdown_connections 2024-09-12T03:21:20.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.819+0000 7efca16ff740 1 -- 172.21.15.6:0/2654696524 wait complete. 2024-09-12T03:21:20.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.819+0000 7efca16ff740 1 -- start start 2024-09-12T03:21:20.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.819+0000 7efca16ff740 1 Processor -- start 2024-09-12T03:21:20.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.820+0000 7efca16ff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c4188b0 0x55a32c442410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.820+0000 7efca16ff740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c4191d0 0x55a32c442950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.820+0000 7efca16ff740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c427fc0 0x55a32c43b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.820+0000 7efca16ff740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a32c439da0 con 0x55a32c427fc0 2024-09-12T03:21:20.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.820+0000 7efca16ff740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55a32c4396d0 con 0x55a32c4191d0 2024-09-12T03:21:20.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.820+0000 7efca16ff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a32c43a470 con 0x55a32c4188b0 2024-09-12T03:21:20.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.820+0000 7efca16ff740 10 client.0 _ll_register_callbacks cb 0x7efc90009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:20.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.820+0000 7efc9d8c3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c427fc0 0x55a32c43b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.820+0000 7efc9d8c3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c427fc0 0x55a32c43b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56168/0 (socket says 172.21.15.6:56168) 2024-09-12T03:21:20.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.820+0000 7efc9d8c3640 1 -- 172.21.15.6:0/2271645796 learned_addr learned my addr 172.21.15.6:0/2271645796 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:20.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.820+0000 7efc9d0c2640 1 --2- 172.21.15.6:0/2271645796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c4188b0 0x55a32c442410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.820+0000 7efc9c8c1640 1 --2- 172.21.15.6:0/2271645796 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c4191d0 0x55a32c442950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.820+0000 7efc9d8c3640 1 -- 172.21.15.6:0/2271645796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c4188b0 msgr2=0x55a32c442410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:20.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.820+0000 7efc9d8c3640 1 --2- 172.21.15.6:0/2271645796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c4188b0 0x55a32c442410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.820+0000 7efc9d8c3640 1 -- 172.21.15.6:0/2271645796 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c4191d0 msgr2=0x55a32c442950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:20.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.820+0000 7efc9d8c3640 1 --2- 172.21.15.6:0/2271645796 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c4191d0 0x55a32c442950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.820+0000 7efc9d8c3640 1 -- 172.21.15.6:0/2271645796 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a32c43bf70 con 0x55a32c427fc0 2024-09-12T03:21:20.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.820+0000 7efc9d0c2640 1 --2- 172.21.15.6:0/2271645796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c4188b0 0x55a32c442410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:20.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.820+0000 7efc9c8c1640 1 --2- 172.21.15.6:0/2271645796 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c4191d0 0x55a32c442950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:20.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.821+0000 7efc9d8c3640 1 --2- 172.21.15.6:0/2271645796 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c427fc0 0x55a32c43b740 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efc90006010 tx=0x7efc90030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7412fe65c5ad7acb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:20.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.821+0000 7efc80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:21:20.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.821+0000 7efc80ff9640 1 -- 172.21.15.6:0/2271645796 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90050040 con 0x55a32c427fc0 2024-09-12T03:21:20.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.821+0000 7efc80ff9640 1 -- 172.21.15.6:0/2271645796 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7efc900307d0 con 0x55a32c427fc0 2024-09-12T03:21:20.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.821+0000 7efca16ff740 1 -- 172.21.15.6:0/2271645796 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc94013950 con 0x55a32c427fc0 2024-09-12T03:21:20.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.821+0000 7efc80ff9640 1 -- 172.21.15.6:0/2271645796 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efc900309d0 con 0x55a32c427fc0 2024-09-12T03:21:20.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.821+0000 7efca16ff740 10 client.15321 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:20.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.821+0000 7efca16ff740 1 -- 172.21.15.6:0/2271645796 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a32c440a10 con 0x55a32c427fc0 2024-09-12T03:21:20.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.821+0000 7efc80ff9640 1 -- 172.21.15.6:0/2271645796 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc94013950 con 0x55a32c427fc0 2024-09-12T03:21:20.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.821+0000 7efc80ff9640 1 -- 172.21.15.6:0/2271645796 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efc90049070 con 0x55a32c427fc0 2024-09-12T03:21:20.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.821+0000 7efca16ff740 10 client.15321 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:20.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.821+0000 7efca16ff740 10 client.15321 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:21:20.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.821+0000 7efca16ff740 20 client.15321 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:20.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.821+0000 7efca16ff740 1 --2- 172.21.15.6:0/2271645796 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a32c4a7860 0x55a32c4a6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.821+0000 7efca16ff740 4 client.15321 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:21:20.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.821+0000 7efca16ff740 1 -- 172.21.15.6:0/2271645796 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "status"}) -- 0x55a32c425ca0 con 0x55a32c4a7860 2024-09-12T03:21:20.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.822+0000 7efc9d0c2640 1 --2- 172.21.15.6:0/2271645796 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a32c4a7860 0x55a32c4a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.822+0000 7efc9d0c2640 1 --2- 172.21.15.6:0/2271645796 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a32c4a7860 0x55a32c4a6570 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48cd8460b626f6ab server_cookie=c8dcdaa893beccbb in_seq=0 out_seq=0 2024-09-12T03:21:20.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.822+0000 7efc80ff9640 10 client.15321 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:20.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.823+0000 7efc80ff9640 1 -- 172.21.15.6:0/2271645796 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55a32c425ca0 con 0x55a32c4a7860 2024-09-12T03:21:20.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.823+0000 7efc80ff9640 10 client.15321 handle_command_reply: tid=0 2024-09-12T03:21:20.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.823+0000 7efca16ff740 1 client.15321 shutdown 2024-09-12T03:21:20.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.823+0000 7efca16ff740 1 -- 172.21.15.6:0/2271645796 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c427fc0 msgr2=0x55a32c43b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:20.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.823+0000 7efca16ff740 1 --2- 172.21.15.6:0/2271645796 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c427fc0 0x55a32c43b740 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efc90006010 tx=0x7efc90030ec0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.823+0000 7efca16ff740 1 -- 172.21.15.6:0/2271645796 shutdown_connections 2024-09-12T03:21:20.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.823+0000 7efca16ff740 1 --2- 172.21.15.6:0/2271645796 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a32c4a7860 0x55a32c4a6570 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.823+0000 7efca16ff740 1 --2- 172.21.15.6:0/2271645796 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a32c427fc0 0x55a32c43b740 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.823+0000 7efca16ff740 1 --2- 172.21.15.6:0/2271645796 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a32c4191d0 0x55a32c442950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.823+0000 7efca16ff740 1 --2- 172.21.15.6:0/2271645796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a32c4188b0 0x55a32c442410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.823+0000 7efca16ff740 1 -- 172.21.15.6:0/2271645796 >> 172.21.15.6:0/2271645796 conn(0x55a32c414f60 msgr2=0x55a32c4232b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:20.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.823+0000 7efc80ff9640 0 client.15321 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:20.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.823+0000 7efca16ff740 1 -- 172.21.15.6:0/2271645796 shutdown_connections 2024-09-12T03:21:20.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.823+0000 7efca16ff740 1 -- 172.21.15.6:0/2271645796 wait complete. 2024-09-12T03:21:20.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.823+0000 7efca16ff740 20 client.15321 trim_cache size 0 max 16384 2024-09-12T03:21:20.831 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:21:20.831 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "89d99938-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:21:20.831 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 0, 2024-09-12T03:21:20.831 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24494, 2024-09-12T03:21:20.831 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-09-12T03:21:20.831 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-09-12T03:21:20.831 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-09-12T03:21:20.831 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 130.50043101, 2024-09-12T03:21:20.832 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 25, 2024-09-12T03:21:20.832 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 82, 2024-09-12T03:21:20.832 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T03:21:20.832 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 139.76569994100001 2024-09-12T03:21:20.832 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:21:20.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.823+0000 7efc9f326640 1 -- 172.21.15.6:0/150261223 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efc840a2c10 msgr2=0x7efc840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:20.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.824+0000 7efc9f326640 1 --2- 172.21.15.6:0/150261223 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efc840a2c10 0x7efc840a50d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efc90009430 tx=0x7efc9002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.824+0000 7efc9f326640 1 -- 172.21.15.6:0/150261223 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc98102790 msgr2=0x7efc981988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:20.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.824+0000 7efc9f326640 1 --2- 172.21.15.6:0/150261223 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc98102790 0x7efc981988e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7efc8800c920 tx=0x7efc8800cd20 comp rx=0 tx=0).stop 2024-09-12T03:21:20.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.824+0000 7efc9f326640 1 -- 172.21.15.6:0/150261223 shutdown_connections 2024-09-12T03:21:20.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.824+0000 7efc9f326640 1 --2- 172.21.15.6:0/150261223 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efc840a2c10 0x7efc840a50d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.824+0000 7efc9f326640 1 --2- 172.21.15.6:0/150261223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc98106f10 0x7efc9819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.824+0000 7efc9f326640 1 --2- 172.21.15.6:0/150261223 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc98102790 0x7efc981988e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.824+0000 7efc9f326640 1 --2- 172.21.15.6:0/150261223 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efc98101de0 0x7efc981983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.824+0000 7efc9f326640 1 -- 172.21.15.6:0/150261223 >> 172.21.15.6:0/150261223 conn(0x7efc980fd8c0 msgr2=0x7efc980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:20.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.824+0000 7efc9f326640 1 -- 172.21.15.6:0/150261223 shutdown_connections 2024-09-12T03:21:20.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:20.824+0000 7efc9f326640 1 -- 172.21.15.6:0/150261223 wait complete. 2024-09-12T03:21:21.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:21 smithi006 ceph-mon[30991]: pgmap v342: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T03:21:21.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:21 smithi161 ceph-mon[35485]: pgmap v342: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T03:21:21.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:21 smithi118 ceph-mon[35048]: pgmap v342: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T03:21:21.539 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-12T03:21:23.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:23 smithi006 ceph-mon[30991]: pgmap v343: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-12T03:21:23.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:23 smithi161 ceph-mon[35485]: pgmap v343: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-12T03:21:23.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:23 smithi118 ceph-mon[35048]: pgmap v343: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-12T03:21:25.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:25 smithi006 ceph-mon[30991]: pgmap v344: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T03:21:25.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:25 smithi161 ceph-mon[35485]: pgmap v344: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T03:21:25.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:25 smithi118 ceph-mon[35048]: pgmap v344: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T03:21:27.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:27 smithi006 ceph-mon[30991]: pgmap v345: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-12T03:21:27.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:27 smithi161 ceph-mon[35485]: pgmap v345: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-12T03:21:27.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:27 smithi118 ceph-mon[35048]: pgmap v345: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-12T03:21:29.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:29 smithi006 ceph-mon[30991]: pgmap v346: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-12T03:21:29.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:29 smithi161 ceph-mon[35485]: pgmap v346: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-12T03:21:29.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:29 smithi118 ceph-mon[35048]: pgmap v346: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-12T03:21:31.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:31 smithi006 ceph-mon[30991]: pgmap v347: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-12T03:21:31.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:31 smithi161 ceph-mon[35485]: pgmap v347: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-12T03:21:31.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:31 smithi118 ceph-mon[35048]: pgmap v347: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-12T03:21:33.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:33 smithi006 ceph-mon[30991]: pgmap v348: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:21:33.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:33 smithi161 ceph-mon[35485]: pgmap v348: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:21:33.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:33 smithi118 ceph-mon[35048]: pgmap v348: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:21:35.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:35 smithi006 ceph-mon[30991]: pgmap v349: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:21:35.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:35 smithi161 ceph-mon[35485]: pgmap v349: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:21:35.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:35 smithi118 ceph-mon[35048]: pgmap v349: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:21:36.540 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-12T03:21:36.541 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:21:36.583 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:21:36.827 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:37.100 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:37.230 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:37 smithi006 ceph-mon[30991]: pgmap v350: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:21:37.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:37 smithi161 ceph-mon[35485]: pgmap v350: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:21:37.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:37 smithi118 ceph-mon[35048]: pgmap v350: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:21:37.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.574+0000 7feb6092a640 1 -- 172.21.15.6:0/349865536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb5c100e20 msgr2=0x7feb5c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.574+0000 7feb6092a640 1 --2- 172.21.15.6:0/349865536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb5c100e20 0x7feb5c101200 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009900 tx=0x7feb4c02a480 comp rx=0 tx=0).stop 2024-09-12T03:21:37.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.574+0000 7feb6092a640 1 -- 172.21.15.6:0/349865536 shutdown_connections 2024-09-12T03:21:37.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.574+0000 7feb6092a640 1 --2- 172.21.15.6:0/349865536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb5c102ef0 0x7feb5c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.574+0000 7feb6092a640 1 --2- 172.21.15.6:0/349865536 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb5c102570 0x7feb5c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.574+0000 7feb6092a640 1 --2- 172.21.15.6:0/349865536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb5c100e20 0x7feb5c101200 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.574+0000 7feb6092a640 1 -- 172.21.15.6:0/349865536 >> 172.21.15.6:0/349865536 conn(0x7feb5c0fc7d0 msgr2=0x7feb5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:37.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.574+0000 7feb6092a640 1 -- 172.21.15.6:0/349865536 shutdown_connections 2024-09-12T03:21:37.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.574+0000 7feb6092a640 1 -- 172.21.15.6:0/349865536 wait complete. 2024-09-12T03:21:37.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.575+0000 7feb6092a640 1 Processor -- start 2024-09-12T03:21:37.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.575+0000 7feb6092a640 1 -- start start 2024-09-12T03:21:37.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.575+0000 7feb6092a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb5c100e20 0x7feb5c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.575+0000 7feb6092a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb5c102570 0x7feb5c19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.575+0000 7feb6092a640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb5c102ef0 0x7feb5c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.575+0000 7feb6092a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c110f80 con 0x7feb5c100e20 2024-09-12T03:21:37.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.575+0000 7feb6092a640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c110e00 con 0x7feb5c102ef0 2024-09-12T03:21:37.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.575+0000 7feb6092a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c111100 con 0x7feb5c102570 2024-09-12T03:21:37.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.575+0000 7feb5a59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb5c100e20 0x7feb5c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.575+0000 7feb5a59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb5c100e20 0x7feb5c19a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38464/0 (socket says 172.21.15.6:38464) 2024-09-12T03:21:37.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.575+0000 7feb5a59c640 1 -- 172.21.15.6:0/3742117936 learned_addr learned my addr 172.21.15.6:0/3742117936 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:37.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.575+0000 7feb53fff640 1 --2- 172.21.15.6:0/3742117936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb5c102570 0x7feb5c19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.575+0000 7feb5ad9d640 1 --2- 172.21.15.6:0/3742117936 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb5c102ef0 0x7feb5c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.575+0000 7feb5a59c640 1 -- 172.21.15.6:0/3742117936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb5c102570 msgr2=0x7feb5c19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.575+0000 7feb5a59c640 1 --2- 172.21.15.6:0/3742117936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb5c102570 0x7feb5c19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.575+0000 7feb5a59c640 1 -- 172.21.15.6:0/3742117936 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb5c102ef0 msgr2=0x7feb5c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.575+0000 7feb5a59c640 1 --2- 172.21.15.6:0/3742117936 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb5c102ef0 0x7feb5c19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.575+0000 7feb5a59c640 1 -- 172.21.15.6:0/3742117936 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb5c19f660 con 0x7feb5c100e20 2024-09-12T03:21:37.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.575+0000 7feb5ad9d640 1 --2- 172.21.15.6:0/3742117936 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb5c102ef0 0x7feb5c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:37.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.576+0000 7feb5a59c640 1 --2- 172.21.15.6:0/3742117936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb5c100e20 0x7feb5c19a6b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7feb4c006010 tx=0x7feb4c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:37.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.576+0000 7feb537fe640 1 -- 172.21.15.6:0/3742117936 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c03f070 con 0x7feb5c100e20 2024-09-12T03:21:37.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.576+0000 7feb537fe640 1 -- 172.21.15.6:0/3742117936 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb4c031c30 con 0x7feb5c100e20 2024-09-12T03:21:37.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.576+0000 7feb6092a640 1 -- 172.21.15.6:0/3742117936 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb5c19f8f0 con 0x7feb5c100e20 2024-09-12T03:21:37.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.576+0000 7feb537fe640 1 -- 172.21.15.6:0/3742117936 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c035030 con 0x7feb5c100e20 2024-09-12T03:21:37.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.576+0000 7feb6092a640 1 -- 172.21.15.6:0/3742117936 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb5c1a7230 con 0x7feb5c100e20 2024-09-12T03:21:37.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.577+0000 7feb6092a640 1 -- 172.21.15.6:0/3742117936 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb5c1021b0 con 0x7feb5c100e20 2024-09-12T03:21:37.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.579+0000 7feb537fe640 1 -- 172.21.15.6:0/3742117936 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7feb4c042070 con 0x7feb5c100e20 2024-09-12T03:21:37.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.579+0000 7feb537fe640 1 --2- 172.21.15.6:0/3742117936 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7feb480a2c10 0x7feb480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.580+0000 7feb53fff640 1 --2- 172.21.15.6:0/3742117936 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7feb480a2c10 0x7feb480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.580+0000 7feb537fe640 1 -- 172.21.15.6:0/3742117936 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7feb4c0efe70 con 0x7feb5c100e20 2024-09-12T03:21:37.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.580+0000 7feb53fff640 1 --2- 172.21.15.6:0/3742117936 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7feb480a2c10 0x7feb480a50d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7feb5c104bf0 tx=0x7feb44005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:37.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.583+0000 7feb537fe640 1 -- 172.21.15.6:0/3742117936 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feb4c0bcca0 con 0x7feb5c100e20 2024-09-12T03:21:37.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.681+0000 7fa253325640 1 -- 172.21.15.6:0/1141002791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa24c102ef0 msgr2=0x7fa24c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.681+0000 7fa253325640 1 --2- 172.21.15.6:0/1141002791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa24c102ef0 0x7fa24c10da40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa244009ca0 tx=0x7fa24402a950 comp rx=0 tx=0).stop 2024-09-12T03:21:37.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.681+0000 7fa253325640 1 -- 172.21.15.6:0/1141002791 shutdown_connections 2024-09-12T03:21:37.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.681+0000 7fa253325640 1 --2- 172.21.15.6:0/1141002791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa24c102ef0 0x7fa24c10da40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.681+0000 7fa253325640 1 --2- 172.21.15.6:0/1141002791 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa24c102570 0x7fa24c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.681+0000 7fa253325640 1 --2- 172.21.15.6:0/1141002791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa24c100e20 0x7fa24c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.681+0000 7fa253325640 1 -- 172.21.15.6:0/1141002791 >> 172.21.15.6:0/1141002791 conn(0x7fa24c0fc7d0 msgr2=0x7fa24c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:37.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.681+0000 7fa253325640 1 -- 172.21.15.6:0/1141002791 shutdown_connections 2024-09-12T03:21:37.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.681+0000 7fa253325640 1 -- 172.21.15.6:0/1141002791 wait complete. 2024-09-12T03:21:37.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.682+0000 7fa253325640 1 Processor -- start 2024-09-12T03:21:37.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.682+0000 7fa253325640 1 -- start start 2024-09-12T03:21:37.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.682+0000 7fa253325640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa24c100e20 0x7fa24c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.682+0000 7fa253325640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa24c102570 0x7fa24c19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.682+0000 7fa253325640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa24c102ef0 0x7fa24c19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.682+0000 7fa253325640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c110f80 con 0x7fa24c102570 2024-09-12T03:21:37.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.682+0000 7fa253325640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c110e00 con 0x7fa24c102ef0 2024-09-12T03:21:37.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.682+0000 7fa253325640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c111100 con 0x7fa24c100e20 2024-09-12T03:21:37.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.682+0000 7fa2508c0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa24c102570 0x7fa24c19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.682+0000 7fa2508c0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa24c102570 0x7fa24c19ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38486/0 (socket says 172.21.15.6:38486) 2024-09-12T03:21:37.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.682+0000 7fa2510c1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa24c100e20 0x7fa24c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.682+0000 7fa2508c0640 1 -- 172.21.15.6:0/2544569115 learned_addr learned my addr 172.21.15.6:0/2544569115 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:37.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.683+0000 7fa2518c2640 1 --2- 172.21.15.6:0/2544569115 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa24c102ef0 0x7fa24c19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.683+0000 7fa2508c0640 1 -- 172.21.15.6:0/2544569115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa24c100e20 msgr2=0x7fa24c19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.683+0000 7fa2508c0640 1 --2- 172.21.15.6:0/2544569115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa24c100e20 0x7fa24c19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.683+0000 7fa2508c0640 1 -- 172.21.15.6:0/2544569115 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa24c102ef0 msgr2=0x7fa24c19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.683+0000 7fa2508c0640 1 --2- 172.21.15.6:0/2544569115 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa24c102ef0 0x7fa24c19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.683+0000 7fa2508c0640 1 -- 172.21.15.6:0/2544569115 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24c19f6d0 con 0x7fa24c102570 2024-09-12T03:21:37.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.683+0000 7fa2508c0640 1 --2- 172.21.15.6:0/2544569115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa24c102570 0x7fa24c19ac60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa23c00d950 tx=0x7fa23c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:37.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.683+0000 7fa2427fc640 1 -- 172.21.15.6:0/2544569115 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c017070 con 0x7fa24c102570 2024-09-12T03:21:37.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.683+0000 7fa253325640 1 -- 172.21.15.6:0/2544569115 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24c19f9c0 con 0x7fa24c102570 2024-09-12T03:21:37.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.683+0000 7fa253325640 1 -- 172.21.15.6:0/2544569115 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa24c1a72a0 con 0x7fa24c102570 2024-09-12T03:21:37.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.683+0000 7fa2427fc640 1 -- 172.21.15.6:0/2544569115 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa23c009c30 con 0x7fa24c102570 2024-09-12T03:21:37.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.683+0000 7fa2427fc640 1 -- 172.21.15.6:0/2544569115 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c00f6b0 con 0x7fa24c102570 2024-09-12T03:21:37.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.686+0000 7fa2556fe740 1 Processor -- start 2024-09-12T03:21:37.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.686+0000 7fa2556fe740 1 -- start start 2024-09-12T03:21:37.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.686+0000 7fa2556fe740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1f4a0 0x55f7b5d1f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.686+0000 7fa2556fe740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d2a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.686+0000 7fa2556fe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7b5d2a680 0x55f7b5d2ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.686+0000 7fa2556fe740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55f7b5c852e0 con 0x55f7b5d1f4a0 2024-09-12T03:21:37.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.686+0000 7fa2556fe740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55f7b5c2cce0 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.686+0000 7fa2556fe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f7b5c812a0 con 0x55f7b5d2a680 2024-09-12T03:21:37.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.687+0000 7fa2518c2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1f4a0 0x55f7b5d1f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.687+0000 7fa2510c1640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d2a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.687+0000 7fa2510c1640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d2a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:47490/0 (socket says 172.21.15.6:47490) 2024-09-12T03:21:37.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.687+0000 7fa2518c2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1f4a0 0x55f7b5d1f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38498/0 (socket says 172.21.15.6:38498) 2024-09-12T03:21:37.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.687+0000 7fa2510c1640 1 -- 172.21.15.6:0/1074894591 learned_addr learned my addr 172.21.15.6:0/1074894591 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:37.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.687+0000 7fa2427fc640 1 -- 172.21.15.6:0/2544569115 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fa23c00f850 con 0x7fa24c102570 2024-09-12T03:21:37.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.687+0000 7fa2518c2640 1 -- 172.21.15.6:0/1074894591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7b5d2a680 msgr2=0x55f7b5d2ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:37.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.687+0000 7fa2518c2640 1 --2- 172.21.15.6:0/1074894591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7b5d2a680 0x55f7b5d2ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.687+0000 7fa2518c2640 1 -- 172.21.15.6:0/1074894591 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f7b5d1fdc0 msgr2=0x55f7b5d2a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:37.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.687+0000 7fa2518c2640 1 --2- 172.21.15.6:0/1074894591 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d2a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.687+0000 7fa2518c2640 1 -- 172.21.15.6:0/1074894591 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7b5c11440 con 0x55f7b5d1f4a0 2024-09-12T03:21:37.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.687+0000 7fa2510c1640 1 --2- 172.21.15.6:0/1074894591 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d2a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:37.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.687+0000 7fa2518c2640 1 --2- 172.21.15.6:0/1074894591 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1f4a0 0x55f7b5d1f880 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa24402ae70 tx=0x7fa244009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d3172ac9e17382 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:37.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.688+0000 7fa2377fe640 1 -- 172.21.15.6:0/1074894591 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244034040 con 0x55f7b5d1f4a0 2024-09-12T03:21:37.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.688+0000 7fa2377fe640 1 -- 172.21.15.6:0/1074894591 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa24402fb30 con 0x55f7b5d1f4a0 2024-09-12T03:21:37.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.688+0000 7fa2377fe640 1 -- 172.21.15.6:0/1074894591 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2440043c0 con 0x55f7b5d1f4a0 2024-09-12T03:21:37.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.688+0000 7fa2427fc640 1 --2- 172.21.15.6:0/2544569115 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa2380a2c90 0x7fa2380a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.688+0000 7fa2510c1640 1 --2- 172.21.15.6:0/2544569115 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa2380a2c90 0x7fa2380a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.688+0000 7fa2427fc640 1 -- 172.21.15.6:0/2544569115 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa23c0cefc0 con 0x7fa24c102570 2024-09-12T03:21:37.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.689+0000 7fa2556fe740 1 -- 172.21.15.6:0/1074894591 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1f4a0 msgr2=0x55f7b5d1f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.689+0000 7fa2510c1640 1 --2- 172.21.15.6:0/2544569115 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa2380a2c90 0x7fa2380a5150 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55f7b5d1c890 tx=0x7fa248005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:37.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.689+0000 7fa2556fe740 1 --2- 172.21.15.6:0/1074894591 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1f4a0 0x55f7b5d1f880 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa24402ae70 tx=0x7fa244009360 comp rx=0 tx=0).stop 2024-09-12T03:21:37.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.689+0000 7fa2556fe740 1 -- 172.21.15.6:0/1074894591 shutdown_connections 2024-09-12T03:21:37.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.689+0000 7fa2556fe740 1 --2- 172.21.15.6:0/1074894591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7b5d2a680 0x55f7b5d2ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.689+0000 7fa2556fe740 1 --2- 172.21.15.6:0/1074894591 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d2a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.689+0000 7fa2556fe740 1 --2- 172.21.15.6:0/1074894591 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1f4a0 0x55f7b5d1f880 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.689+0000 7fa2556fe740 1 -- 172.21.15.6:0/1074894591 >> 172.21.15.6:0/1074894591 conn(0x55f7b5d1ba30 msgr2=0x55f7b5d1be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:37.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.689+0000 7fa2556fe740 1 -- 172.21.15.6:0/1074894591 shutdown_connections 2024-09-12T03:21:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.689+0000 7fa2556fe740 1 -- 172.21.15.6:0/1074894591 wait complete. 2024-09-12T03:21:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.689+0000 7fa2556fe740 1 -- start start 2024-09-12T03:21:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.689+0000 7fa2556fe740 1 Processor -- start 2024-09-12T03:21:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.690+0000 7fa2556fe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7b5d1f4a0 0x55f7b5d29fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.690+0000 7fa2556fe740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d27270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.690+0000 7fa2556fe740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f7b5d2a680 0x55f7b5d277b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.690+0000 7fa2556fe740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55f7b5d2d2a0 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.690+0000 7fa2556fe740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55f7b5c40050 con 0x55f7b5d2a680 2024-09-12T03:21:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.690+0000 7fa2556fe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f7b5d2d420 con 0x55f7b5d1f4a0 2024-09-12T03:21:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.690+0000 7fa2508c0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d27270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.690+0000 7fa2556fe740 10 client.0 _ll_register_callbacks cb 0x55f7b5c80160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.690+0000 7fa2508c0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d27270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38510/0 (socket says 172.21.15.6:38510) 2024-09-12T03:21:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.690+0000 7fa2508c0640 1 -- 172.21.15.6:0/4204355443 learned_addr learned my addr 172.21.15.6:0/4204355443 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.690+0000 7fa2510c1640 1 --2- 172.21.15.6:0/4204355443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7b5d1f4a0 0x55f7b5d29fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.690+0000 7fa2518c2640 1 --2- 172.21.15.6:0/4204355443 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f7b5d2a680 0x55f7b5d277b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.690+0000 7fa2508c0640 1 -- 172.21.15.6:0/4204355443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7b5d1f4a0 msgr2=0x55f7b5d29fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.690+0000 7fa2508c0640 1 --2- 172.21.15.6:0/4204355443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7b5d1f4a0 0x55f7b5d29fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.690+0000 7fa2508c0640 1 -- 172.21.15.6:0/4204355443 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f7b5d2a680 msgr2=0x55f7b5d277b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.690+0000 7fa2508c0640 1 --2- 172.21.15.6:0/4204355443 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f7b5d2a680 0x55f7b5d277b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.690+0000 7fa2508c0640 1 -- 172.21.15.6:0/4204355443 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7b59f63c0 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.691+0000 7fa2508c0640 1 --2- 172.21.15.6:0/4204355443 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d27270 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa23c0a3dd0 tx=0x7fa23c09bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4d5a4fc39576ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:37.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.691+0000 7fa234ff9640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:21:37.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.691+0000 7fa234ff9640 1 -- 172.21.15.6:0/4204355443 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c020070 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.691+0000 7fa234ff9640 1 -- 172.21.15.6:0/4204355443 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa23c0a3e70 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.691+0000 7fa2556fe740 1 -- 172.21.15.6:0/4204355443 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa24402fb30 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.691+0000 7fa234ff9640 1 -- 172.21.15.6:0/4204355443 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa23c09b690 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.691+0000 7fa2556fe740 10 client.15339 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:37.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.691+0000 7fa2556fe740 1 -- 172.21.15.6:0/4204355443 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f7b5d414f0 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.691+0000 7fa234ff9640 1 -- 172.21.15.6:0/4204355443 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa24402fb30 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.691+0000 7fa234ff9640 1 -- 172.21.15.6:0/4204355443 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa23c0afbb0 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.691+0000 7fa2556fe740 10 client.15339 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:37.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.691+0000 7fa2556fe740 10 client.15339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:21:37.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.691+0000 7fa2556fe740 20 client.15339 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:37.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.691+0000 7fa2556fe740 1 --2- 172.21.15.6:0/4204355443 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55f7b5d42950 0x55f7b5d44d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.691+0000 7fa2556fe740 4 client.15339 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.691+0000 7fa2556fe740 1 -- 172.21.15.6:0/4204355443 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7b5c92850 con 0x55f7b5d42950 2024-09-12T03:21:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.692+0000 7fa2510c1640 1 --2- 172.21.15.6:0/4204355443 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55f7b5d42950 0x55f7b5d44d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.692+0000 7fa2510c1640 1 --2- 172.21.15.6:0/4204355443 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55f7b5d42950 0x55f7b5d44d70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ba633b02a7fb78a server_cookie=9fee4e0dacd629a4 in_seq=0 out_seq=0 2024-09-12T03:21:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.692+0000 7fa234ff9640 10 client.15339 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.693+0000 7fa234ff9640 1 -- 172.21.15.6:0/4204355443 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7b5c92850 con 0x55f7b5d42950 2024-09-12T03:21:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.693+0000 7fa234ff9640 10 client.15339 handle_command_reply: tid=0 2024-09-12T03:21:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.693+0000 7fa2556fe740 1 client.15339 shutdown 2024-09-12T03:21:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.693+0000 7fa2556fe740 1 -- 172.21.15.6:0/4204355443 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1fdc0 msgr2=0x55f7b5d27270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.693+0000 7fa2556fe740 1 --2- 172.21.15.6:0/4204355443 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d27270 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa23c0a3dd0 tx=0x7fa23c09bd90 comp rx=0 tx=0).stop 2024-09-12T03:21:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.693+0000 7fa2556fe740 1 -- 172.21.15.6:0/4204355443 shutdown_connections 2024-09-12T03:21:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.693+0000 7fa2556fe740 1 --2- 172.21.15.6:0/4204355443 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55f7b5d42950 0x55f7b5d44d70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.693+0000 7fa234ff9640 0 client.15339 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.693+0000 7fa2556fe740 1 --2- 172.21.15.6:0/4204355443 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f7b5d2a680 0x55f7b5d277b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.693+0000 7fa2556fe740 1 --2- 172.21.15.6:0/4204355443 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d27270 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.693+0000 7fa2556fe740 1 --2- 172.21.15.6:0/4204355443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7b5d1f4a0 0x55f7b5d29fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.693+0000 7fa2556fe740 1 -- 172.21.15.6:0/4204355443 >> 172.21.15.6:0/4204355443 conn(0x55f7b5d1ba30 msgr2=0x55f7b5d21760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:37.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.694+0000 7fa2556fe740 1 -- 172.21.15.6:0/4204355443 shutdown_connections 2024-09-12T03:21:37.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.694+0000 7fa2556fe740 1 -- 172.21.15.6:0/4204355443 wait complete. 2024-09-12T03:21:37.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.694+0000 7fa2556fe740 20 client.15339 trim_cache size 0 max 16384 2024-09-12T03:21:37.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.712+0000 7fa2556fe740 1 Processor -- start 2024-09-12T03:21:37.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.712+0000 7fa2556fe740 1 -- start start 2024-09-12T03:21:37.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.712+0000 7fa2556fe740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1f4a0 0x55f7b5d3ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.712+0000 7fa2556fe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d25b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.712+0000 7fa2556fe740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f7b5d2a680 0x55f7b5d4a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.712+0000 7fa2556fe740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa230010ba0 con 0x55f7b5d1f4a0 2024-09-12T03:21:37.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.712+0000 7fa2556fe740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa23000e470 con 0x55f7b5d2a680 2024-09-12T03:21:37.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.712+0000 7fa2556fe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa23000c2c0 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.713+0000 7fa2510c1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1f4a0 0x55f7b5d3ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.713+0000 7fa2510c1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1f4a0 0x55f7b5d3ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38514/0 (socket says 172.21.15.6:38514) 2024-09-12T03:21:37.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.713+0000 7fa2510c1640 1 -- 172.21.15.6:0/2283189144 learned_addr learned my addr 172.21.15.6:0/2283189144 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:37.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.713+0000 7fa2508c0640 1 --2- 172.21.15.6:0/2283189144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d25b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.713+0000 7fa2518c2640 1 --2- 172.21.15.6:0/2283189144 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f7b5d2a680 0x55f7b5d4a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.713+0000 7fa2508c0640 1 -- 172.21.15.6:0/2283189144 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f7b5d2a680 msgr2=0x55f7b5d4a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:37.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.713+0000 7fa2508c0640 1 --2- 172.21.15.6:0/2283189144 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f7b5d2a680 0x55f7b5d4a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.713+0000 7fa2508c0640 1 -- 172.21.15.6:0/2283189144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1f4a0 msgr2=0x55f7b5d3ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:37.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.713+0000 7fa2508c0640 1 --2- 172.21.15.6:0/2283189144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1f4a0 0x55f7b5d3ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.713+0000 7fa2508c0640 1 -- 172.21.15.6:0/2283189144 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7b5d257c0 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.713+0000 7fa2510c1640 1 --2- 172.21.15.6:0/2283189144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1f4a0 0x55f7b5d3ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:37.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.713+0000 7fa2508c0640 1 --2- 172.21.15.6:0/2283189144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d25b40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009610 tx=0x7fa23c00bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10b9b5a05d9ff4b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:37.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.713+0000 7fa236ffd640 1 -- 172.21.15.6:0/2283189144 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c020070 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.713+0000 7fa236ffd640 1 -- 172.21.15.6:0/2283189144 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa23c0a4300 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.713+0000 7fa236ffd640 1 -- 172.21.15.6:0/2283189144 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c0a3e70 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.714+0000 7fa2556fe740 1 -- 172.21.15.6:0/2283189144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7b5d1fdc0 msgr2=0x55f7b5d25b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.714+0000 7fa2556fe740 1 --2- 172.21.15.6:0/2283189144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d25b40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009610 tx=0x7fa23c00bf60 comp rx=0 tx=0).stop 2024-09-12T03:21:37.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.714+0000 7fa2556fe740 1 -- 172.21.15.6:0/2283189144 shutdown_connections 2024-09-12T03:21:37.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.714+0000 7fa2556fe740 1 --2- 172.21.15.6:0/2283189144 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f7b5d2a680 0x55f7b5d4a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.714+0000 7fa2556fe740 1 --2- 172.21.15.6:0/2283189144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d25b40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.714+0000 7fa2556fe740 1 --2- 172.21.15.6:0/2283189144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1f4a0 0x55f7b5d3ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.714+0000 7fa2556fe740 1 -- 172.21.15.6:0/2283189144 >> 172.21.15.6:0/2283189144 conn(0x55f7b5d1ba30 msgr2=0x55f7b5d21710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:37.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.714+0000 7fa2556fe740 1 -- 172.21.15.6:0/2283189144 shutdown_connections 2024-09-12T03:21:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.714+0000 7fa2556fe740 1 -- 172.21.15.6:0/2283189144 wait complete. 2024-09-12T03:21:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.715+0000 7fa2556fe740 1 -- start start 2024-09-12T03:21:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.715+0000 7fa2556fe740 1 Processor -- start 2024-09-12T03:21:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.715+0000 7fa2556fe740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f7b5d1f4a0 0x55f7b5d36cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.715+0000 7fa2556fe740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d371f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.715+0000 7fa2556fe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7b5d2a680 0x55f7b5d37730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.715+0000 7fa2556fe740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa23000c930 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.715+0000 7fa2556fe740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa23000dda0 con 0x55f7b5d1f4a0 2024-09-12T03:21:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.715+0000 7fa2556fe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa23000d000 con 0x55f7b5d2a680 2024-09-12T03:21:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.715+0000 7fa2556fe740 10 client.0 _ll_register_callbacks cb 0x7fa23c09b1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.715+0000 7fa2508c0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.715+0000 7fa2508c0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d371f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38526/0 (socket says 172.21.15.6:38526) 2024-09-12T03:21:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.715+0000 7fa2508c0640 1 -- 172.21.15.6:0/2747907408 learned_addr learned my addr 172.21.15.6:0/2747907408 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.715+0000 7fa2518c2640 1 --2- 172.21.15.6:0/2747907408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7b5d2a680 0x55f7b5d37730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.716+0000 7fa2508c0640 1 -- 172.21.15.6:0/2747907408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7b5d2a680 msgr2=0x55f7b5d37730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.716+0000 7fa2508c0640 1 --2- 172.21.15.6:0/2747907408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7b5d2a680 0x55f7b5d37730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.716+0000 7fa2508c0640 1 -- 172.21.15.6:0/2747907408 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f7b5d1f4a0 msgr2=0x55f7b5d36cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.716+0000 7fa2508c0640 1 --2- 172.21.15.6:0/2747907408 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f7b5d1f4a0 0x55f7b5d36cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.716+0000 7fa2508c0640 1 -- 172.21.15.6:0/2747907408 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7b5dac5d0 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.716+0000 7fa2508c0640 1 --2- 172.21.15.6:0/2747907408 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d371f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa23c09b620 tx=0x7fa23c010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa04924b9c330090 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.716+0000 7fa234ff9640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:21:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.716+0000 7fa234ff9640 1 -- 172.21.15.6:0/2747907408 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c020040 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.716+0000 7fa234ff9640 1 -- 172.21.15.6:0/2747907408 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa23c0a4080 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.716+0000 7fa2556fe740 1 -- 172.21.15.6:0/2747907408 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa23c0a4300 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.716+0000 7fa234ff9640 1 -- 172.21.15.6:0/2747907408 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa23c09fa40 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.716+0000 7fa2556fe740 10 client.15345 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.716+0000 7fa2556fe740 1 -- 172.21.15.6:0/2747907408 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f7b5dad890 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.716+0000 7fa234ff9640 1 -- 172.21.15.6:0/2747907408 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa23c0a4300 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.717+0000 7fa234ff9640 1 -- 172.21.15.6:0/2747907408 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa23c0975e0 con 0x55f7b5d1fdc0 2024-09-12T03:21:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.717+0000 7fa2556fe740 10 client.15345 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.717+0000 7fa2556fe740 10 client.15345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:21:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.717+0000 7fa2556fe740 20 client.15345 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.717+0000 7fa2556fe740 1 --2- 172.21.15.6:0/2747907408 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55f7b5dadda0 0x55f7b5db01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.717+0000 7fa2556fe740 4 client.15345 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:21:37.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.717+0000 7fa2556fe740 1 -- 172.21.15.6:0/2747907408 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f7b5c8cd60 con 0x55f7b5dadda0 2024-09-12T03:21:37.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.717+0000 7fa2510c1640 1 --2- 172.21.15.6:0/2747907408 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55f7b5dadda0 0x55f7b5db01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.718+0000 7fa2510c1640 1 --2- 172.21.15.6:0/2747907408 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55f7b5dadda0 0x55f7b5db01c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=359f838ba06d2320 server_cookie=bd605f5a0ea9af1b in_seq=0 out_seq=0 2024-09-12T03:21:37.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.718+0000 7fa234ff9640 10 client.15345 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:37.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.718+0000 7fa234ff9640 1 -- 172.21.15.6:0/2747907408 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f7b5c8cd60 con 0x55f7b5dadda0 2024-09-12T03:21:37.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.718+0000 7fa234ff9640 10 client.15345 handle_command_reply: tid=0 2024-09-12T03:21:37.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.718+0000 7fa2556fe740 1 client.15345 shutdown 2024-09-12T03:21:37.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.718+0000 7fa2556fe740 1 -- 172.21.15.6:0/2747907408 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1fdc0 msgr2=0x55f7b5d371f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.718+0000 7fa2556fe740 1 --2- 172.21.15.6:0/2747907408 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d371f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa23c09b620 tx=0x7fa23c010f90 comp rx=0 tx=0).stop 2024-09-12T03:21:37.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.718+0000 7fa2556fe740 1 -- 172.21.15.6:0/2747907408 shutdown_connections 2024-09-12T03:21:37.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.718+0000 7fa2556fe740 1 --2- 172.21.15.6:0/2747907408 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55f7b5dadda0 0x55f7b5db01c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.718+0000 7fa2556fe740 1 --2- 172.21.15.6:0/2747907408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7b5d2a680 0x55f7b5d37730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.822 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:21:37.822 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2024-09-12T03:21:37.823 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2024-09-12T03:21:37.823 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:21:37.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.718+0000 7fa2556fe740 1 --2- 172.21.15.6:0/2747907408 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f7b5d1fdc0 0x55f7b5d371f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.718+0000 7fa234ff9640 0 client.15345 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:37.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.718+0000 7fa2556fe740 1 --2- 172.21.15.6:0/2747907408 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f7b5d1f4a0 0x55f7b5d36cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.718+0000 7fa2556fe740 1 -- 172.21.15.6:0/2747907408 >> 172.21.15.6:0/2747907408 conn(0x55f7b5d1ba30 msgr2=0x55f7b5d26b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:37.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.718+0000 7fa2556fe740 1 -- 172.21.15.6:0/2747907408 shutdown_connections 2024-09-12T03:21:37.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.718+0000 7fa2556fe740 1 -- 172.21.15.6:0/2747907408 wait complete. 2024-09-12T03:21:37.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.719+0000 7fa2556fe740 20 client.15345 trim_cache size 0 max 16384 2024-09-12T03:21:37.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.719+0000 7fa253325640 1 -- 172.21.15.6:0/2544569115 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa2380a2c90 msgr2=0x7fa2380a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.719+0000 7fa253325640 1 --2- 172.21.15.6:0/2544569115 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa2380a2c90 0x7fa2380a5150 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55f7b5d1c890 tx=0x7fa248005f90 comp rx=0 tx=0).stop 2024-09-12T03:21:37.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.719+0000 7fa253325640 1 -- 172.21.15.6:0/2544569115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa24c102570 msgr2=0x7fa24c19ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.719+0000 7fa253325640 1 --2- 172.21.15.6:0/2544569115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa24c102570 0x7fa24c19ac60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa23c00d950 tx=0x7fa23c00de10 comp rx=0 tx=0).stop 2024-09-12T03:21:37.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.719+0000 7fa253325640 1 -- 172.21.15.6:0/2544569115 shutdown_connections 2024-09-12T03:21:37.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.719+0000 7fa253325640 1 --2- 172.21.15.6:0/2544569115 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa2380a2c90 0x7fa2380a5150 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.719+0000 7fa253325640 1 --2- 172.21.15.6:0/2544569115 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa24c102ef0 0x7fa24c19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.719+0000 7fa253325640 1 --2- 172.21.15.6:0/2544569115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa24c102570 0x7fa24c19ac60 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.719+0000 7fa253325640 1 --2- 172.21.15.6:0/2544569115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa24c100e20 0x7fa24c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.719+0000 7fa253325640 1 -- 172.21.15.6:0/2544569115 >> 172.21.15.6:0/2544569115 conn(0x7fa24c0fc7d0 msgr2=0x7fa24c06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:37.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.719+0000 7fa253325640 1 -- 172.21.15.6:0/2544569115 shutdown_connections 2024-09-12T03:21:37.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.719+0000 7fa253325640 1 -- 172.21.15.6:0/2544569115 wait complete. 2024-09-12T03:21:37.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.816+0000 7feb6092a640 1 -- 172.21.15.6:0/3742117936 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb5c101200 con 0x7feb5c100e20 2024-09-12T03:21:37.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.817+0000 7feb537fe640 1 -- 172.21.15.6:0/3742117936 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11908 (secure 0 0 0) 0x7feb5c101200 con 0x7feb5c100e20 2024-09-12T03:21:37.826 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:21:37.826 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"btime":"2024-09-12T03:20:52:974598+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:20:52.261388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24463]},"id":1}]} 2024-09-12T03:21:37.827 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 25 2024-09-12T03:21:37.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.819+0000 7feb6092a640 1 -- 172.21.15.6:0/3742117936 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7feb480a2c10 msgr2=0x7feb480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.819+0000 7feb6092a640 1 --2- 172.21.15.6:0/3742117936 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7feb480a2c10 0x7feb480a50d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7feb5c104bf0 tx=0x7feb44005f90 comp rx=0 tx=0).stop 2024-09-12T03:21:37.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.819+0000 7feb6092a640 1 -- 172.21.15.6:0/3742117936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb5c100e20 msgr2=0x7feb5c19a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.819+0000 7feb6092a640 1 --2- 172.21.15.6:0/3742117936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb5c100e20 0x7feb5c19a6b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7feb4c006010 tx=0x7feb4c003c10 comp rx=0 tx=0).stop 2024-09-12T03:21:37.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.819+0000 7feb6092a640 1 -- 172.21.15.6:0/3742117936 shutdown_connections 2024-09-12T03:21:37.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.819+0000 7feb6092a640 1 --2- 172.21.15.6:0/3742117936 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7feb480a2c10 0x7feb480a50d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.819+0000 7feb6092a640 1 --2- 172.21.15.6:0/3742117936 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb5c102ef0 0x7feb5c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.819+0000 7feb6092a640 1 --2- 172.21.15.6:0/3742117936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb5c102570 0x7feb5c19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.819+0000 7feb6092a640 1 --2- 172.21.15.6:0/3742117936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb5c100e20 0x7feb5c19a6b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.819+0000 7feb6092a640 1 -- 172.21.15.6:0/3742117936 >> 172.21.15.6:0/3742117936 conn(0x7feb5c0fc7d0 msgr2=0x7feb5c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:37.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.819+0000 7feb6092a640 1 -- 172.21.15.6:0/3742117936 shutdown_connections 2024-09-12T03:21:37.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:37.819+0000 7feb6092a640 1 -- 172.21.15.6:0/3742117936 wait complete. 2024-09-12T03:21:38.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:38 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3742117936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:38.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:38 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3742117936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:38.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:38 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3742117936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:38.578 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:21:38.579 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:21:39.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:39 smithi006 ceph-mon[30991]: pgmap v351: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:21:39.243 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-12T03:21:39.243 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-12T03:21:39.313 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:39.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:39 smithi161 ceph-mon[35485]: pgmap v351: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:21:39.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:39 smithi118 ceph-mon[35048]: pgmap v351: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:21:39.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.843+0000 7fa1843e5640 1 -- 172.21.15.6:0/1071755132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa17c1036e0 msgr2=0x7fa17c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.843+0000 7fa1843e5640 1 --2- 172.21.15.6:0/1071755132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa17c1036e0 0x7fa17c107b90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa16c009930 tx=0x7fa16c02a420 comp rx=0 tx=0).stop 2024-09-12T03:21:39.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.843+0000 7fa1843e5640 1 -- 172.21.15.6:0/1071755132 shutdown_connections 2024-09-12T03:21:39.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.843+0000 7fa1843e5640 1 --2- 172.21.15.6:0/1071755132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa17c1082a0 0x7fa17c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.843+0000 7fa1843e5640 1 --2- 172.21.15.6:0/1071755132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa17c1036e0 0x7fa17c107b90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.843+0000 7fa1843e5640 1 --2- 172.21.15.6:0/1071755132 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa17c102d30 0x7fa17c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.843+0000 7fa1843e5640 1 -- 172.21.15.6:0/1071755132 >> 172.21.15.6:0/1071755132 conn(0x7fa17c0fc7d0 msgr2=0x7fa17c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:39.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.843+0000 7fa1843e5640 1 -- 172.21.15.6:0/1071755132 shutdown_connections 2024-09-12T03:21:39.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.843+0000 7fa1843e5640 1 -- 172.21.15.6:0/1071755132 wait complete. 2024-09-12T03:21:39.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.844+0000 7fa1843e5640 1 Processor -- start 2024-09-12T03:21:39.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.844+0000 7fa1843e5640 1 -- start start 2024-09-12T03:21:39.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.844+0000 7fa1843e5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa17c102d30 0x7fa17c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.844+0000 7fa1843e5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa17c1036e0 0x7fa17c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.844+0000 7fa1843e5640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa17c1082a0 0x7fa17c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.844+0000 7fa1843e5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c1120a0 con 0x7fa17c102d30 2024-09-12T03:21:39.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.844+0000 7fa1843e5640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c111f20 con 0x7fa17c1082a0 2024-09-12T03:21:39.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.844+0000 7fa1843e5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c112220 con 0x7fa17c1036e0 2024-09-12T03:21:39.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.844+0000 7fa182181640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa17c102d30 0x7fa17c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.844+0000 7fa182181640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa17c102d30 0x7fa17c100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43028/0 (socket says 172.21.15.6:43028) 2024-09-12T03:21:39.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.845+0000 7fa182181640 1 -- 172.21.15.6:0/857304331 learned_addr learned my addr 172.21.15.6:0/857304331 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:39.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.845+0000 7fa181980640 1 --2- 172.21.15.6:0/857304331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa17c1036e0 0x7fa17c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.845+0000 7fa182982640 1 --2- 172.21.15.6:0/857304331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa17c1082a0 0x7fa17c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.845+0000 7fa182181640 1 -- 172.21.15.6:0/857304331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa17c1036e0 msgr2=0x7fa17c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.845+0000 7fa182181640 1 --2- 172.21.15.6:0/857304331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa17c1036e0 0x7fa17c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.845+0000 7fa182181640 1 -- 172.21.15.6:0/857304331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa17c1082a0 msgr2=0x7fa17c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.845+0000 7fa182181640 1 --2- 172.21.15.6:0/857304331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa17c1082a0 0x7fa17c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.845+0000 7fa182181640 1 -- 172.21.15.6:0/857304331 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17c1a6a80 con 0x7fa17c102d30 2024-09-12T03:21:39.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.845+0000 7fa182982640 1 --2- 172.21.15.6:0/857304331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa17c1082a0 0x7fa17c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:39.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.845+0000 7fa181980640 1 --2- 172.21.15.6:0/857304331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa17c1036e0 0x7fa17c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:39.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.845+0000 7fa182181640 1 --2- 172.21.15.6:0/857304331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa17c102d30 0x7fa17c100990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa17800e9f0 tx=0x7fa17800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:39.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.845+0000 7fa1737fe640 1 -- 172.21.15.6:0/857304331 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17800cd80 con 0x7fa17c102d30 2024-09-12T03:21:39.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.846+0000 7fa1737fe640 1 -- 172.21.15.6:0/857304331 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa178010910 con 0x7fa17c102d30 2024-09-12T03:21:39.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.846+0000 7fa1843e5640 1 -- 172.21.15.6:0/857304331 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa17c1a6d70 con 0x7fa17c102d30 2024-09-12T03:21:39.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.846+0000 7fa1737fe640 1 -- 172.21.15.6:0/857304331 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa178018990 con 0x7fa17c102d30 2024-09-12T03:21:39.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.846+0000 7fa1843e5640 1 -- 172.21.15.6:0/857304331 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa17c1a7270 con 0x7fa17c102d30 2024-09-12T03:21:39.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.847+0000 7fa1843e5640 1 -- 172.21.15.6:0/857304331 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa17c103110 con 0x7fa17c102d30 2024-09-12T03:21:39.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.849+0000 7fa1737fe640 1 -- 172.21.15.6:0/857304331 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fa178010ac0 con 0x7fa17c102d30 2024-09-12T03:21:39.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.850+0000 7fa1737fe640 1 --2- 172.21.15.6:0/857304331 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa1680a2c90 0x7fa1680a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.850+0000 7fa181980640 1 --2- 172.21.15.6:0/857304331 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa1680a2c90 0x7fa1680a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.850+0000 7fa1737fe640 1 -- 172.21.15.6:0/857304331 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa1780cf570 con 0x7fa17c102d30 2024-09-12T03:21:39.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.851+0000 7fa181980640 1 --2- 172.21.15.6:0/857304331 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa1680a2c90 0x7fa1680a5150 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa16c009900 tx=0x7fa16c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:39.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:39.855+0000 7fa1737fe640 1 -- 172.21.15.6:0/857304331 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa17809c3a0 con 0x7fa17c102d30 2024-09-12T03:21:39.885 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:40.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:40.088+0000 7fa1843e5640 1 -- 172.21.15.6:0/857304331 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa17c10a2b0 con 0x7fa17c102d30 2024-09-12T03:21:40.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:40.089+0000 7fa1737fe640 1 -- 172.21.15.6:0/857304331 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11908 (secure 0 0 0) 0x7fa17809bca0 con 0x7fa17c102d30 2024-09-12T03:21:40.089 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:21:40.089 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"btime":"2024-09-12T03:20:52:974598+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:20:52.261388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24463]},"id":1}]} 2024-09-12T03:21:40.089 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 25 2024-09-12T03:21:40.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:40.091+0000 7fa1843e5640 1 -- 172.21.15.6:0/857304331 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa1680a2c90 msgr2=0x7fa1680a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:40.091+0000 7fa1843e5640 1 --2- 172.21.15.6:0/857304331 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa1680a2c90 0x7fa1680a5150 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa16c009900 tx=0x7fa16c005a20 comp rx=0 tx=0).stop 2024-09-12T03:21:40.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:40.091+0000 7fa1843e5640 1 -- 172.21.15.6:0/857304331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa17c102d30 msgr2=0x7fa17c100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:40.091+0000 7fa1843e5640 1 --2- 172.21.15.6:0/857304331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa17c102d30 0x7fa17c100990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa17800e9f0 tx=0x7fa17800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:40.091+0000 7fa1843e5640 1 -- 172.21.15.6:0/857304331 shutdown_connections 2024-09-12T03:21:40.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:40.091+0000 7fa1843e5640 1 --2- 172.21.15.6:0/857304331 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa1680a2c90 0x7fa1680a5150 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:40.091+0000 7fa1843e5640 1 --2- 172.21.15.6:0/857304331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa17c1082a0 0x7fa17c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:40.091+0000 7fa1843e5640 1 --2- 172.21.15.6:0/857304331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa17c1036e0 0x7fa17c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:40.091+0000 7fa1843e5640 1 --2- 172.21.15.6:0/857304331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa17c102d30 0x7fa17c100990 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:40.091+0000 7fa1843e5640 1 -- 172.21.15.6:0/857304331 >> 172.21.15.6:0/857304331 conn(0x7fa17c0fc7d0 msgr2=0x7fa17c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:40.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:40.091+0000 7fa1843e5640 1 -- 172.21.15.6:0/857304331 shutdown_connections 2024-09-12T03:21:40.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:40.091+0000 7fa1843e5640 1 -- 172.21.15.6:0/857304331 wait complete. 2024-09-12T03:21:40.801 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:21:41.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:41 smithi006 ceph-mon[30991]: pgmap v352: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:21:41.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:41 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/857304331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:41.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:41 smithi161 ceph-mon[35485]: pgmap v352: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:21:41.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:41 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/857304331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:41.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:41 smithi118 ceph-mon[35048]: pgmap v352: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:21:41.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:41 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/857304331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:42.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.544+0000 7efcb8f6a640 1 -- 172.21.15.6:0/3667447726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcb4100e20 msgr2=0x7efcb4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:42.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.544+0000 7efcb8f6a640 1 --2- 172.21.15.6:0/3667447726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcb4100e20 0x7efcb4101200 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efca4009900 tx=0x7efca402a480 comp rx=0 tx=0).stop 2024-09-12T03:21:42.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.544+0000 7efcb8f6a640 1 -- 172.21.15.6:0/3667447726 shutdown_connections 2024-09-12T03:21:42.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.544+0000 7efcb8f6a640 1 --2- 172.21.15.6:0/3667447726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcb4102ef0 0x7efcb410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.544+0000 7efcb8f6a640 1 --2- 172.21.15.6:0/3667447726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcb4102570 0x7efcb41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.544+0000 7efcb8f6a640 1 --2- 172.21.15.6:0/3667447726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcb4100e20 0x7efcb4101200 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.544+0000 7efcb8f6a640 1 -- 172.21.15.6:0/3667447726 >> 172.21.15.6:0/3667447726 conn(0x7efcb40fc7d0 msgr2=0x7efcb40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:42.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.544+0000 7efcb8f6a640 1 -- 172.21.15.6:0/3667447726 shutdown_connections 2024-09-12T03:21:42.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.544+0000 7efcb8f6a640 1 -- 172.21.15.6:0/3667447726 wait complete. 2024-09-12T03:21:42.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.545+0000 7efcb8f6a640 1 Processor -- start 2024-09-12T03:21:42.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.545+0000 7efcb8f6a640 1 -- start start 2024-09-12T03:21:42.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.545+0000 7efcb8f6a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcb4100e20 0x7efcb419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.545+0000 7efcb8f6a640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcb4102570 0x7efcb419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.545+0000 7efcb8f6a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcb4102ef0 0x7efcb419dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.545+0000 7efcb8f6a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4110fd0 con 0x7efcb4100e20 2024-09-12T03:21:42.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.545+0000 7efcb8f6a640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4110e50 con 0x7efcb4102570 2024-09-12T03:21:42.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.545+0000 7efcb8f6a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4111150 con 0x7efcb4102ef0 2024-09-12T03:21:42.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.545+0000 7efcb259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcb4100e20 0x7efcb419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.545+0000 7efcb259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcb4100e20 0x7efcb419a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43048/0 (socket says 172.21.15.6:43048) 2024-09-12T03:21:42.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.545+0000 7efcb259c640 1 -- 172.21.15.6:0/3324276313 learned_addr learned my addr 172.21.15.6:0/3324276313 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:42.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.545+0000 7efcb1d9b640 1 --2- 172.21.15.6:0/3324276313 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcb4102570 0x7efcb419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.545+0000 7efcb2d9d640 1 --2- 172.21.15.6:0/3324276313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcb4102ef0 0x7efcb419dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.545+0000 7efcb259c640 1 -- 172.21.15.6:0/3324276313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcb4102ef0 msgr2=0x7efcb419dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:42.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.545+0000 7efcb259c640 1 --2- 172.21.15.6:0/3324276313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcb4102ef0 0x7efcb419dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.545+0000 7efcb259c640 1 -- 172.21.15.6:0/3324276313 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcb4102570 msgr2=0x7efcb419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:42.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.545+0000 7efcb259c640 1 --2- 172.21.15.6:0/3324276313 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcb4102570 0x7efcb419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.545+0000 7efcb259c640 1 -- 172.21.15.6:0/3324276313 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb419e720 con 0x7efcb4100e20 2024-09-12T03:21:42.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.546+0000 7efcb259c640 1 --2- 172.21.15.6:0/3324276313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcb4100e20 0x7efcb419a7d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efca402aa70 tx=0x7efca4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:42.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.546+0000 7efca37fe640 1 -- 172.21.15.6:0/3324276313 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca403f070 con 0x7efcb4100e20 2024-09-12T03:21:42.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.546+0000 7efcb8f6a640 1 -- 172.21.15.6:0/3324276313 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb419e9b0 con 0x7efcb4100e20 2024-09-12T03:21:42.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.546+0000 7efca37fe640 1 -- 172.21.15.6:0/3324276313 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca4031c30 con 0x7efcb4100e20 2024-09-12T03:21:42.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.546+0000 7efca37fe640 1 -- 172.21.15.6:0/3324276313 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4035030 con 0x7efcb4100e20 2024-09-12T03:21:42.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.546+0000 7efcb8f6a640 1 -- 172.21.15.6:0/3324276313 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb41a7380 con 0x7efcb4100e20 2024-09-12T03:21:42.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.548+0000 7efca37fe640 1 -- 172.21.15.6:0/3324276313 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7efca4042070 con 0x7efcb4100e20 2024-09-12T03:21:42.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.548+0000 7efcbb343740 1 Processor -- start 2024-09-12T03:21:42.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.549+0000 7efcbb343740 1 -- start start 2024-09-12T03:21:42.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.549+0000 7efcbb343740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212600760 0x562212600b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.549+0000 7efcbb343740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562212601080 0x56221260b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.549+0000 7efcbb343740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56221260b940 0x56221260dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.549+0000 7efcbb343740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5622125662a0 con 0x562212600760 2024-09-12T03:21:42.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.549+0000 7efcbb343740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x56221250db10 con 0x56221260b940 2024-09-12T03:21:42.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.549+0000 7efcbb343740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562212562260 con 0x562212601080 2024-09-12T03:21:42.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.549+0000 7efcb2d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212600760 0x562212600b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.549+0000 7efcb2d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212600760 0x562212600b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43062/0 (socket says 172.21.15.6:43062) 2024-09-12T03:21:42.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.549+0000 7efcb2d9d640 1 -- 172.21.15.6:0/3155650810 learned_addr learned my addr 172.21.15.6:0/3155650810 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:42.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.549+0000 7efca37fe640 1 --2- 172.21.15.6:0/3324276313 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efc9c0a2c10 0x7efc9c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.549+0000 7efcb1d9b640 1 --2- 172.21.15.6:0/3155650810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56221260b940 0x56221260dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.549+0000 7efcb2d9d640 1 -- 172.21.15.6:0/3155650810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562212601080 msgr2=0x56221260b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:42.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.549+0000 7efcb2d9d640 1 --2- 172.21.15.6:0/3155650810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562212601080 0x56221260b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.549+0000 7efcb2d9d640 1 -- 172.21.15.6:0/3155650810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56221260b940 msgr2=0x56221260dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:42.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.549+0000 7efcb2d9d640 1 --2- 172.21.15.6:0/3155650810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56221260b940 0x56221260dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.550+0000 7efcb2d9d640 1 -- 172.21.15.6:0/3155650810 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622122d7b90 con 0x562212600760 2024-09-12T03:21:42.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.550+0000 7efca37fe640 1 -- 172.21.15.6:0/3324276313 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7efca40ef590 con 0x7efcb4100e20 2024-09-12T03:21:42.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.550+0000 7efcb2d9d640 1 --2- 172.21.15.6:0/3324276313 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efc9c0a2c10 0x7efc9c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.550+0000 7efcb2d9d640 1 --2- 172.21.15.6:0/3155650810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212600760 0x562212600b40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efcac009fc0 tx=0x7efcac010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63f6655dfb5031d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:42.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.550+0000 7efca0ff9640 1 -- 172.21.15.6:0/3155650810 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac01c900 con 0x562212600760 2024-09-12T03:21:42.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.550+0000 7efcb2d9d640 1 --2- 172.21.15.6:0/3324276313 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efc9c0a2c10 0x7efc9c0a50d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efcac020000 tx=0x7efcac01d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:42.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.550+0000 7efca0ff9640 1 -- 172.21.15.6:0/3155650810 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcac0113e0 con 0x562212600760 2024-09-12T03:21:42.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.550+0000 7efca0ff9640 1 -- 172.21.15.6:0/3155650810 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac013640 con 0x562212600760 2024-09-12T03:21:42.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.551+0000 7efcbb343740 1 -- 172.21.15.6:0/3155650810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212600760 msgr2=0x562212600b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:42.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.551+0000 7efcbb343740 1 --2- 172.21.15.6:0/3155650810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212600760 0x562212600b40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efcac009fc0 tx=0x7efcac010800 comp rx=0 tx=0).stop 2024-09-12T03:21:42.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.551+0000 7efcbb343740 1 -- 172.21.15.6:0/3155650810 shutdown_connections 2024-09-12T03:21:42.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.551+0000 7efcbb343740 1 --2- 172.21.15.6:0/3155650810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56221260b940 0x56221260dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.551+0000 7efcbb343740 1 --2- 172.21.15.6:0/3155650810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562212601080 0x56221260b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.551+0000 7efcbb343740 1 --2- 172.21.15.6:0/3155650810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212600760 0x562212600b40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.551+0000 7efcbb343740 1 -- 172.21.15.6:0/3155650810 >> 172.21.15.6:0/3155650810 conn(0x5622125fccf0 msgr2=0x5622125fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:42.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.551+0000 7efcbb343740 1 -- 172.21.15.6:0/3155650810 shutdown_connections 2024-09-12T03:21:42.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.551+0000 7efcbb343740 1 -- 172.21.15.6:0/3155650810 wait complete. 2024-09-12T03:21:42.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.551+0000 7efcbb343740 1 -- start start 2024-09-12T03:21:42.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.551+0000 7efcbb343740 1 Processor -- start 2024-09-12T03:21:42.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.552+0000 7efcbb343740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212600760 0x562212608890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.552+0000 7efcbb343740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562212601080 0x562212608dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.552+0000 7efcbb343740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56221260b940 0x562212609310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.552+0000 7efcbb343740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56221260e510 con 0x562212600760 2024-09-12T03:21:42.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.552+0000 7efcbb343740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5622124ee6e0 con 0x562212601080 2024-09-12T03:21:42.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.552+0000 7efcbb343740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56221260e690 con 0x56221260b940 2024-09-12T03:21:42.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.552+0000 7efcbb343740 10 client.0 _ll_register_callbacks cb 0x562212512a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:42.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.552+0000 7efcb259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212600760 0x562212608890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.552+0000 7efcb259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212600760 0x562212608890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43076/0 (socket says 172.21.15.6:43076) 2024-09-12T03:21:42.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.552+0000 7efcb259c640 1 -- 172.21.15.6:0/1580183602 learned_addr learned my addr 172.21.15.6:0/1580183602 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:42.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.552+0000 7efcb1d9b640 1 --2- 172.21.15.6:0/1580183602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562212601080 0x562212608dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.552+0000 7efcb259c640 1 -- 172.21.15.6:0/1580183602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56221260b940 msgr2=0x562212609310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:42.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.552+0000 7efcb259c640 1 --2- 172.21.15.6:0/1580183602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56221260b940 0x562212609310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.552+0000 7efcb259c640 1 -- 172.21.15.6:0/1580183602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562212601080 msgr2=0x562212608dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:42.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.552+0000 7efcb259c640 1 --2- 172.21.15.6:0/1580183602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562212601080 0x562212608dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.552+0000 7efcb259c640 1 -- 172.21.15.6:0/1580183602 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562212617ab0 con 0x562212600760 2024-09-12T03:21:42.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.552+0000 7efcb259c640 1 --2- 172.21.15.6:0/1580183602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212600760 0x562212608890 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7efca40ecfa0 tx=0x7efca40bb800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc5156614f134c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:42.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.552+0000 7efc99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:21:42.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.553+0000 7efc99ffb640 1 -- 172.21.15.6:0/1580183602 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4049030 con 0x562212600760 2024-09-12T03:21:42.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.553+0000 7efc99ffb640 1 -- 172.21.15.6:0/1580183602 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7efca403b460 con 0x562212600760 2024-09-12T03:21:42.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.553+0000 7efcbb343740 1 -- 172.21.15.6:0/1580183602 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcac0113e0 con 0x562212600760 2024-09-12T03:21:42.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.553+0000 7efc99ffb640 1 -- 172.21.15.6:0/1580183602 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efca40bc3c0 con 0x562212600760 2024-09-12T03:21:42.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.553+0000 7efcbb343740 10 client.15363 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:42.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.553+0000 7efcbb343740 1 -- 172.21.15.6:0/1580183602 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56221261a420 con 0x562212600760 2024-09-12T03:21:42.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.553+0000 7efc99ffb640 1 -- 172.21.15.6:0/1580183602 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcac0113e0 con 0x562212600760 2024-09-12T03:21:42.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.553+0000 7efc99ffb640 1 -- 172.21.15.6:0/1580183602 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efca404c060 con 0x562212600760 2024-09-12T03:21:42.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.553+0000 7efcbb343740 10 client.15363 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:42.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.553+0000 7efcbb343740 10 client.15363 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:21:42.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.553+0000 7efcbb343740 20 client.15363 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:42.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.553+0000 7efcbb343740 1 --2- 172.21.15.6:0/1580183602 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56221261b880 0x56221261dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.553+0000 7efcbb343740 4 client.15363 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:42.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.553+0000 7efcbb343740 1 -- 172.21.15.6:0/1580183602 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622125739d0 con 0x56221261b880 2024-09-12T03:21:42.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.553+0000 7efcb1d9b640 1 --2- 172.21.15.6:0/1580183602 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56221261b880 0x56221261dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.554+0000 7efc99ffb640 10 client.15363 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:42.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.554+0000 7efcb1d9b640 1 --2- 172.21.15.6:0/1580183602 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56221261b880 0x56221261dcf0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=145d1f6d8a0074b1 server_cookie=7269dc21be40c56f in_seq=0 out_seq=0 2024-09-12T03:21:42.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.555+0000 7efc99ffb640 1 -- 172.21.15.6:0/1580183602 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5622125739d0 con 0x56221261b880 2024-09-12T03:21:42.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.555+0000 7efc99ffb640 10 client.15363 handle_command_reply: tid=0 2024-09-12T03:21:42.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.555+0000 7efcbb343740 1 client.15363 shutdown 2024-09-12T03:21:42.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.555+0000 7efcbb343740 1 -- 172.21.15.6:0/1580183602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212600760 msgr2=0x562212608890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:42.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.555+0000 7efcbb343740 1 --2- 172.21.15.6:0/1580183602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212600760 0x562212608890 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7efca40ecfa0 tx=0x7efca40bb800 comp rx=0 tx=0).stop 2024-09-12T03:21:42.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.555+0000 7efcbb343740 1 -- 172.21.15.6:0/1580183602 shutdown_connections 2024-09-12T03:21:42.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.555+0000 7efcbb343740 1 --2- 172.21.15.6:0/1580183602 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56221261b880 0x56221261dcf0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.555+0000 7efcbb343740 1 --2- 172.21.15.6:0/1580183602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56221260b940 0x562212609310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.555+0000 7efcbb343740 1 --2- 172.21.15.6:0/1580183602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562212601080 0x562212608dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.555+0000 7efc99ffb640 0 client.15363 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.555+0000 7efcbb343740 1 --2- 172.21.15.6:0/1580183602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212600760 0x562212608890 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.555+0000 7efcbb343740 1 -- 172.21.15.6:0/1580183602 >> 172.21.15.6:0/1580183602 conn(0x5622125fccf0 msgr2=0x5622126029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.555+0000 7efcbb343740 1 -- 172.21.15.6:0/1580183602 shutdown_connections 2024-09-12T03:21:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.555+0000 7efcbb343740 1 -- 172.21.15.6:0/1580183602 wait complete. 2024-09-12T03:21:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.555+0000 7efcbb343740 20 client.15363 trim_cache size 0 max 16384 2024-09-12T03:21:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.576+0000 7efcbb343740 1 Processor -- start 2024-09-12T03:21:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.576+0000 7efcbb343740 1 -- start start 2024-09-12T03:21:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.576+0000 7efcbb343740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562212600760 0x56221260a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.576+0000 7efcbb343740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212601080 0x562212607160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.576+0000 7efcbb343740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56221260b940 0x5622126031c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.576+0000 7efcbb343740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41b2030 con 0x562212601080 2024-09-12T03:21:42.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.576+0000 7efcbb343740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41af900 con 0x562212600760 2024-09-12T03:21:42.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.576+0000 7efcbb343740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41ad750 con 0x56221260b940 2024-09-12T03:21:42.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.576+0000 7efcb1d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212601080 0x562212607160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.576+0000 7efcb1d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212601080 0x562212607160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43078/0 (socket says 172.21.15.6:43078) 2024-09-12T03:21:42.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.576+0000 7efcb1d9b640 1 -- 172.21.15.6:0/344595814 learned_addr learned my addr 172.21.15.6:0/344595814 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:42.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.576+0000 7efcb259c640 1 --2- 172.21.15.6:0/344595814 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562212600760 0x56221260a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.576+0000 7efcb2d9d640 1 --2- 172.21.15.6:0/344595814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56221260b940 0x5622126031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.576+0000 7efcb1d9b640 1 -- 172.21.15.6:0/344595814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56221260b940 msgr2=0x5622126031c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:42.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.576+0000 7efcb1d9b640 1 --2- 172.21.15.6:0/344595814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56221260b940 0x5622126031c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.576+0000 7efcb1d9b640 1 -- 172.21.15.6:0/344595814 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562212600760 msgr2=0x56221260a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:42.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.576+0000 7efcb1d9b640 1 --2- 172.21.15.6:0/344595814 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562212600760 0x56221260a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.576+0000 7efcb1d9b640 1 -- 172.21.15.6:0/344595814 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622125f8260 con 0x562212601080 2024-09-12T03:21:42.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.577+0000 7efcb259c640 1 --2- 172.21.15.6:0/344595814 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562212600760 0x56221260a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:42.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.577+0000 7efcb1d9b640 1 --2- 172.21.15.6:0/344595814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212601080 0x562212607160 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x5622125fe070 tx=0x7efca8004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8daff41602d16c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:42.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.577+0000 7efc9bfff640 1 -- 172.21.15.6:0/344595814 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8003d80 con 0x562212601080 2024-09-12T03:21:42.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.577+0000 7efc9bfff640 1 -- 172.21.15.6:0/344595814 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca8005500 con 0x562212601080 2024-09-12T03:21:42.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.577+0000 7efc9bfff640 1 -- 172.21.15.6:0/344595814 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8004e60 con 0x562212601080 2024-09-12T03:21:42.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.578+0000 7efcbb343740 1 -- 172.21.15.6:0/344595814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212601080 msgr2=0x562212607160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:42.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.578+0000 7efcbb343740 1 --2- 172.21.15.6:0/344595814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212601080 0x562212607160 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x5622125fe070 tx=0x7efca8004410 comp rx=0 tx=0).stop 2024-09-12T03:21:42.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.578+0000 7efcbb343740 1 -- 172.21.15.6:0/344595814 shutdown_connections 2024-09-12T03:21:42.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.578+0000 7efcbb343740 1 --2- 172.21.15.6:0/344595814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56221260b940 0x5622126031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.578+0000 7efcbb343740 1 --2- 172.21.15.6:0/344595814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212601080 0x562212607160 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.578+0000 7efcbb343740 1 --2- 172.21.15.6:0/344595814 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562212600760 0x56221260a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.578+0000 7efcbb343740 1 -- 172.21.15.6:0/344595814 >> 172.21.15.6:0/344595814 conn(0x5622125fccf0 msgr2=0x5622125fbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:42.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.578+0000 7efcbb343740 1 -- 172.21.15.6:0/344595814 shutdown_connections 2024-09-12T03:21:42.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.578+0000 7efcbb343740 1 -- 172.21.15.6:0/344595814 wait complete. 2024-09-12T03:21:42.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.578+0000 7efcbb343740 1 -- start start 2024-09-12T03:21:42.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.578+0000 7efcbb343740 1 Processor -- start 2024-09-12T03:21:42.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efcbb343740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562212600760 0x5622126059d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efcbb343740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212601080 0x562212605f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efcbb343740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56221260b940 0x562212606450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efcbb343740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41addc0 con 0x562212601080 2024-09-12T03:21:42.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efcbb343740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41af230 con 0x562212600760 2024-09-12T03:21:42.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efcbb343740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41ae490 con 0x56221260b940 2024-09-12T03:21:42.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efcbb343740 10 client.0 _ll_register_callbacks cb 0x7efca4009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:42.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efcb1d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212601080 0x562212605f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efcb1d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212601080 0x562212605f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43090/0 (socket says 172.21.15.6:43090) 2024-09-12T03:21:42.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efcb1d9b640 1 -- 172.21.15.6:0/3829056185 learned_addr learned my addr 172.21.15.6:0/3829056185 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:42.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efcb259c640 1 --2- 172.21.15.6:0/3829056185 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562212600760 0x5622126059d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efcb2d9d640 1 --2- 172.21.15.6:0/3829056185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56221260b940 0x562212606450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efcb1d9b640 1 -- 172.21.15.6:0/3829056185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56221260b940 msgr2=0x562212606450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:42.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efcb1d9b640 1 --2- 172.21.15.6:0/3829056185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56221260b940 0x562212606450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efcb1d9b640 1 -- 172.21.15.6:0/3829056185 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562212600760 msgr2=0x5622126059d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:42.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efcb1d9b640 1 --2- 172.21.15.6:0/3829056185 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562212600760 0x5622126059d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efcb1d9b640 1 -- 172.21.15.6:0/3829056185 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622126856c0 con 0x562212601080 2024-09-12T03:21:42.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efcb2d9d640 1 --2- 172.21.15.6:0/3829056185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56221260b940 0x562212606450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:42.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efcb1d9b640 1 --2- 172.21.15.6:0/3829056185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212601080 0x562212605f10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efca8009a10 tx=0x7efca8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f624ff831575b37c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:42.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efc99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:21:42.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efc99ffb640 1 -- 172.21.15.6:0/3829056185 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8031650 con 0x562212601080 2024-09-12T03:21:42.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efc99ffb640 1 -- 172.21.15.6:0/3829056185 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7efca801d430 con 0x562212601080 2024-09-12T03:21:42.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.579+0000 7efcbb343740 1 -- 172.21.15.6:0/3829056185 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efca8005500 con 0x562212601080 2024-09-12T03:21:42.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.580+0000 7efc99ffb640 1 -- 172.21.15.6:0/3829056185 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efca80317f0 con 0x562212601080 2024-09-12T03:21:42.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.580+0000 7efcbb343740 10 client.15369 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:42.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.580+0000 7efcbb343740 1 -- 172.21.15.6:0/3829056185 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562212686920 con 0x562212601080 2024-09-12T03:21:42.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.580+0000 7efc99ffb640 1 -- 172.21.15.6:0/3829056185 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca8005500 con 0x562212601080 2024-09-12T03:21:42.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.580+0000 7efc99ffb640 1 -- 172.21.15.6:0/3829056185 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efca8010430 con 0x562212601080 2024-09-12T03:21:42.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.580+0000 7efcbb343740 10 client.15369 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:42.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.580+0000 7efcbb343740 10 client.15369 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:21:42.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.580+0000 7efcbb343740 20 client.15369 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:42.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.580+0000 7efcbb343740 1 --2- 172.21.15.6:0/3829056185 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x562212687060 0x562212689480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.580+0000 7efcbb343740 4 client.15369 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-12T03:21:42.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.580+0000 7efcbb343740 1 -- 172.21.15.6:0/3829056185 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x56221256dd20 con 0x562212687060 2024-09-12T03:21:42.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.580+0000 7efcb259c640 1 --2- 172.21.15.6:0/3829056185 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x562212687060 0x562212689480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.581+0000 7efcb259c640 1 --2- 172.21.15.6:0/3829056185 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x562212687060 0x562212689480 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a738fa5c7ef377 server_cookie=402f33e33533710d in_seq=0 out_seq=0 2024-09-12T03:21:42.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.581+0000 7efc99ffb640 10 client.15369 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:42.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.592+0000 7efc99ffb640 1 -- 172.21.15.6:0/3829056185 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56221256dd20 con 0x562212687060 2024-09-12T03:21:42.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.592+0000 7efc99ffb640 10 client.15369 handle_command_reply: tid=0 2024-09-12T03:21:42.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.592+0000 7efcbb343740 1 client.15369 shutdown 2024-09-12T03:21:42.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.592+0000 7efcbb343740 1 -- 172.21.15.6:0/3829056185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212601080 msgr2=0x562212605f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:42.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.592+0000 7efcbb343740 1 --2- 172.21.15.6:0/3829056185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212601080 0x562212605f10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efca8009a10 tx=0x7efca8005780 comp rx=0 tx=0).stop 2024-09-12T03:21:42.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.592+0000 7efcbb343740 1 -- 172.21.15.6:0/3829056185 shutdown_connections 2024-09-12T03:21:42.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.592+0000 7efcbb343740 1 --2- 172.21.15.6:0/3829056185 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x562212687060 0x562212689480 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.592+0000 7efc99ffb640 0 client.15369 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:42.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.592+0000 7efcbb343740 1 --2- 172.21.15.6:0/3829056185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56221260b940 0x562212606450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.592+0000 7efcbb343740 1 --2- 172.21.15.6:0/3829056185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562212601080 0x562212605f10 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.592+0000 7efcbb343740 1 --2- 172.21.15.6:0/3829056185 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562212600760 0x5622126059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.592+0000 7efcbb343740 1 -- 172.21.15.6:0/3829056185 >> 172.21.15.6:0/3829056185 conn(0x5622125fccf0 msgr2=0x562212616790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:42.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.592+0000 7efcbb343740 1 -- 172.21.15.6:0/3829056185 shutdown_connections 2024-09-12T03:21:42.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.592+0000 7efcbb343740 1 -- 172.21.15.6:0/3829056185 wait complete. 2024-09-12T03:21:42.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.592+0000 7efcbb343740 20 client.15369 trim_cache size 0 max 16384 2024-09-12T03:21:42.351 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:21:42.351 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 25, 2024-09-12T03:21:42.352 INFO:teuthology.orchestra.run.smithi006.stdout: "leader": 24494, 2024-09-12T03:21:42.352 INFO:teuthology.orchestra.run.smithi006.stdout: "set_version": 2, 2024-09-12T03:21:42.352 INFO:teuthology.orchestra.run.smithi006.stdout: "sets": { 2024-09-12T03:21:42.352 INFO:teuthology.orchestra.run.smithi006.stdout: "2de2c24f": { 2024-09-12T03:21:42.352 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 2, 2024-09-12T03:21:42.352 INFO:teuthology.orchestra.run.smithi006.stdout: "age_ref": 0.0, 2024-09-12T03:21:42.352 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:21:42.352 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "QUIESCED", 2024-09-12T03:21:42.352 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:21:42.352 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:21:42.352 INFO:teuthology.orchestra.run.smithi006.stdout: "timeout": 90.0, 2024-09-12T03:21:42.352 INFO:teuthology.orchestra.run.smithi006.stdout: "expiration": 144.5, 2024-09-12T03:21:42.353 INFO:teuthology.orchestra.run.smithi006.stdout: "members": { 2024-09-12T03:21:42.353 INFO:teuthology.orchestra.run.smithi006.stdout: "file:/": { 2024-09-12T03:21:42.353 INFO:teuthology.orchestra.run.smithi006.stdout: "excluded": false, 2024-09-12T03:21:42.353 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:21:42.353 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "QUIESCED", 2024-09-12T03:21:42.353 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:21:42.353 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:21:42.353 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:21:42.353 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:21:42.353 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:21:42.353 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:21:42.354 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:21:42.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.593+0000 7efcb8f6a640 1 -- 172.21.15.6:0/3324276313 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efc9c0a2c10 msgr2=0x7efc9c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:42.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.593+0000 7efcb8f6a640 1 --2- 172.21.15.6:0/3324276313 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efc9c0a2c10 0x7efc9c0a50d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efcac020000 tx=0x7efcac01d000 comp rx=0 tx=0).stop 2024-09-12T03:21:42.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.593+0000 7efcb8f6a640 1 -- 172.21.15.6:0/3324276313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcb4100e20 msgr2=0x7efcb419a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:42.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.593+0000 7efcb8f6a640 1 --2- 172.21.15.6:0/3324276313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcb4100e20 0x7efcb419a7d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efca402aa70 tx=0x7efca4003c10 comp rx=0 tx=0).stop 2024-09-12T03:21:42.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.593+0000 7efcb8f6a640 1 -- 172.21.15.6:0/3324276313 shutdown_connections 2024-09-12T03:21:42.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.593+0000 7efcb8f6a640 1 --2- 172.21.15.6:0/3324276313 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efc9c0a2c10 0x7efc9c0a50d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.593+0000 7efcb8f6a640 1 --2- 172.21.15.6:0/3324276313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcb4102ef0 0x7efcb419dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.593+0000 7efcb8f6a640 1 --2- 172.21.15.6:0/3324276313 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcb4102570 0x7efcb419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.593+0000 7efcb8f6a640 1 --2- 172.21.15.6:0/3324276313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcb4100e20 0x7efcb419a7d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.593+0000 7efcb8f6a640 1 -- 172.21.15.6:0/3324276313 >> 172.21.15.6:0/3324276313 conn(0x7efcb40fc7d0 msgr2=0x7efcb406a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:42.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.593+0000 7efcb8f6a640 1 -- 172.21.15.6:0/3324276313 shutdown_connections 2024-09-12T03:21:42.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:41.593+0000 7efcb8f6a640 1 -- 172.21.15.6:0/3324276313 wait complete. 2024-09-12T03:21:43.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:43 smithi006 ceph-mon[30991]: pgmap v353: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:21:43.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:43 smithi161 ceph-mon[35485]: pgmap v353: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:21:43.466 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:43 smithi118 ceph-mon[35048]: pgmap v353: 97 pgs: 97 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:21:44.448 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:45.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:45 smithi006 ceph-mon[30991]: pgmap v354: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:21:45.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:45 smithi161 ceph-mon[35485]: pgmap v354: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:21:45.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:45 smithi118 ceph-mon[35048]: pgmap v354: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:21:46.036 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2de2c24f', quiesce took 9.5 seconds. Will release after: 15.0 2024-09-12T03:21:46.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.630+0000 7f03e5b4e640 1 -- 172.21.15.6:0/1832241010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03e01036e0 msgr2=0x7f03e0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:46.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.630+0000 7f03e5b4e640 1 --2- 172.21.15.6:0/1832241010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03e01036e0 0x7f03e0107b90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009930 tx=0x7f03d402a420 comp rx=0 tx=0).stop 2024-09-12T03:21:46.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.630+0000 7f03e5b4e640 1 -- 172.21.15.6:0/1832241010 shutdown_connections 2024-09-12T03:21:46.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.630+0000 7f03e5b4e640 1 --2- 172.21.15.6:0/1832241010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e01082a0 0x7f03e010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.630+0000 7f03e5b4e640 1 --2- 172.21.15.6:0/1832241010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03e01036e0 0x7f03e0107b90 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.630+0000 7f03e5b4e640 1 --2- 172.21.15.6:0/1832241010 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03e0102d30 0x7f03e0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.630+0000 7f03e5b4e640 1 -- 172.21.15.6:0/1832241010 >> 172.21.15.6:0/1832241010 conn(0x7f03e00fc7d0 msgr2=0x7f03e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:46.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.631+0000 7f03e5b4e640 1 -- 172.21.15.6:0/1832241010 shutdown_connections 2024-09-12T03:21:46.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.631+0000 7f03e5b4e640 1 -- 172.21.15.6:0/1832241010 wait complete. 2024-09-12T03:21:46.633 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.631+0000 7f03e5b4e640 1 Processor -- start 2024-09-12T03:21:46.633 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.632+0000 7f03e5b4e640 1 -- start start 2024-09-12T03:21:46.633 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.632+0000 7f03e5b4e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e0102d30 0x7f03e01a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.632+0000 7f03e5b4e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03e01036e0 0x7f03e01a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.632+0000 7f03e5b4e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03e01082a0 0x7f03e019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.632+0000 7f03e5b4e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01120b0 con 0x7f03e01082a0 2024-09-12T03:21:46.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.632+0000 7f03e5b4e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f03e0111f30 con 0x7f03e01036e0 2024-09-12T03:21:46.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.632+0000 7f03e5b4e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f03e0112230 con 0x7f03e0102d30 2024-09-12T03:21:46.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.632+0000 7f03dffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03e01082a0 0x7f03e019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.632+0000 7f03dffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03e01082a0 0x7f03e019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43114/0 (socket says 172.21.15.6:43114) 2024-09-12T03:21:46.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.632+0000 7f03dffff640 1 -- 172.21.15.6:0/357279355 learned_addr learned my addr 172.21.15.6:0/357279355 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:46.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.632+0000 7f03deffd640 1 --2- 172.21.15.6:0/357279355 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03e01036e0 0x7f03e01a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.633+0000 7f03df7fe640 1 --2- 172.21.15.6:0/357279355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e0102d30 0x7f03e01a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.633+0000 7f03dffff640 1 -- 172.21.15.6:0/357279355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e0102d30 msgr2=0x7f03e01a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:46.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.633+0000 7f03dffff640 1 --2- 172.21.15.6:0/357279355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e0102d30 0x7f03e01a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.633+0000 7f03dffff640 1 -- 172.21.15.6:0/357279355 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03e01036e0 msgr2=0x7f03e01a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:46.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.633+0000 7f03dffff640 1 --2- 172.21.15.6:0/357279355 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03e01036e0 0x7f03e01a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.633+0000 7f03dffff640 1 -- 172.21.15.6:0/357279355 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e019ad20 con 0x7f03e01082a0 2024-09-12T03:21:46.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.633+0000 7f03df7fe640 1 --2- 172.21.15.6:0/357279355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e0102d30 0x7f03e01a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:46.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.633+0000 7f03dffff640 1 --2- 172.21.15.6:0/357279355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03e01082a0 0x7f03e019a5f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f03d800eb10 tx=0x7f03d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:46.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.633+0000 7f03dcff9640 1 -- 172.21.15.6:0/357279355 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8004b20 con 0x7f03e01082a0 2024-09-12T03:21:46.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.633+0000 7f03e5b4e640 1 -- 172.21.15.6:0/357279355 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e019b010 con 0x7f03e01082a0 2024-09-12T03:21:46.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.633+0000 7f03dcff9640 1 -- 172.21.15.6:0/357279355 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03d8004cc0 con 0x7f03e01082a0 2024-09-12T03:21:46.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.633+0000 7f03dcff9640 1 -- 172.21.15.6:0/357279355 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8010760 con 0x7f03e01082a0 2024-09-12T03:21:46.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.633+0000 7f03e5b4e640 1 -- 172.21.15.6:0/357279355 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e01a7300 con 0x7f03e01082a0 2024-09-12T03:21:46.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.636+0000 7f03dcff9640 1 -- 172.21.15.6:0/357279355 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f03d8005020 con 0x7f03e01082a0 2024-09-12T03:21:46.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.636+0000 7f03e7f27740 1 Processor -- start 2024-09-12T03:21:46.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.636+0000 7f03e7f27740 1 -- start start 2024-09-12T03:21:46.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.636+0000 7f03e7f27740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564315317b40 0x564315316090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.636+0000 7f03e7f27740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643153165d0 0x564315316a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.636+0000 7f03e7f27740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564315321370 0x564315323760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.637+0000 7f03e7f27740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56431527c2f0 con 0x5643153165d0 2024-09-12T03:21:46.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.637+0000 7f03e7f27740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x564315223cf0 con 0x564315317b40 2024-09-12T03:21:46.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.637+0000 7f03e7f27740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5643152782b0 con 0x564315321370 2024-09-12T03:21:46.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.637+0000 7f03deffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564315321370 0x564315323760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.637+0000 7f03deffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564315321370 0x564315323760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:36582/0 (socket says 172.21.15.6:36582) 2024-09-12T03:21:46.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.637+0000 7f03deffd640 1 -- 172.21.15.6:0/2897856157 learned_addr learned my addr 172.21.15.6:0/2897856157 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:46.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.637+0000 7f03dcff9640 1 --2- 172.21.15.6:0/357279355 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f03c80a2c90 0x7f03c80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.637+0000 7f03deffd640 1 --2- 172.21.15.6:0/2897856157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564315317b40 0x564315316090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.637+0000 7f03df7fe640 1 --2- 172.21.15.6:0/2897856157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643153165d0 0x564315316a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.637+0000 7f03dffff640 1 --2- 172.21.15.6:0/357279355 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f03c80a2c90 0x7f03c80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.637+0000 7f03deffd640 1 -- 172.21.15.6:0/2897856157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564315317b40 msgr2=0x564315316090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:46.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.637+0000 7f03deffd640 1 --2- 172.21.15.6:0/2897856157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564315317b40 0x564315316090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.637+0000 7f03deffd640 1 -- 172.21.15.6:0/2897856157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643153165d0 msgr2=0x564315316a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:46.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.637+0000 7f03deffd640 1 --2- 172.21.15.6:0/2897856157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643153165d0 0x564315316a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.637+0000 7f03deffd640 1 -- 172.21.15.6:0/2897856157 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643152083a0 con 0x564315321370 2024-09-12T03:21:46.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.637+0000 7f03dcff9640 1 -- 172.21.15.6:0/357279355 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f03d80cf860 con 0x7f03e01082a0 2024-09-12T03:21:46.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.638+0000 7f03dffff640 1 --2- 172.21.15.6:0/357279355 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f03c80a2c90 0x7f03c80a5150 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f03d8097c10 tx=0x7f03d8097b20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:46.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.638+0000 7f03deffd640 1 --2- 172.21.15.6:0/2897856157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564315321370 0x564315323760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f03d4006010 tx=0x7f03d4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95a93a995be9e1ee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:46.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.638+0000 7f03c5ffb640 1 -- 172.21.15.6:0/2897856157 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4035030 con 0x564315321370 2024-09-12T03:21:46.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.638+0000 7f03c5ffb640 1 -- 172.21.15.6:0/2897856157 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03d40047a0 con 0x564315321370 2024-09-12T03:21:46.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.638+0000 7f03c5ffb640 1 -- 172.21.15.6:0/2897856157 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4003900 con 0x564315321370 2024-09-12T03:21:46.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.639+0000 7f03e7f27740 1 -- 172.21.15.6:0/2897856157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564315321370 msgr2=0x564315323760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:46.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.639+0000 7f03e7f27740 1 --2- 172.21.15.6:0/2897856157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564315321370 0x564315323760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f03d4006010 tx=0x7f03d4000bc0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.639+0000 7f03e7f27740 1 -- 172.21.15.6:0/2897856157 shutdown_connections 2024-09-12T03:21:46.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.639+0000 7f03e7f27740 1 --2- 172.21.15.6:0/2897856157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564315321370 0x564315323760 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.639+0000 7f03e7f27740 1 --2- 172.21.15.6:0/2897856157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643153165d0 0x564315316a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.639+0000 7f03e7f27740 1 --2- 172.21.15.6:0/2897856157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564315317b40 0x564315316090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.639+0000 7f03e7f27740 1 -- 172.21.15.6:0/2897856157 >> 172.21.15.6:0/2897856157 conn(0x5643152df310 msgr2=0x564315312c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:46.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.639+0000 7f03e7f27740 1 -- 172.21.15.6:0/2897856157 shutdown_connections 2024-09-12T03:21:46.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.639+0000 7f03e7f27740 1 -- 172.21.15.6:0/2897856157 wait complete. 2024-09-12T03:21:46.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.640+0000 7f03e7f27740 1 -- start start 2024-09-12T03:21:46.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.640+0000 7f03e7f27740 1 Processor -- start 2024-09-12T03:21:46.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.640+0000 7f03e7f27740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5643153165d0 0x564315320d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.640+0000 7f03e7f27740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564315317b40 0x56431531e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.640+0000 7f03e7f27740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564315321370 0x56431531e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.640+0000 7f03e7f27740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x564315324010 con 0x564315321370 2024-09-12T03:21:46.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.640+0000 7f03e7f27740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x564315237060 con 0x5643153165d0 2024-09-12T03:21:46.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.640+0000 7f03e7f27740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564315324190 con 0x564315317b40 2024-09-12T03:21:46.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.640+0000 7f03e7f27740 10 client.0 _ll_register_callbacks cb 0x564314f8a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:46.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.640+0000 7f03deffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564315321370 0x56431531e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.640+0000 7f03deffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564315321370 0x56431531e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43138/0 (socket says 172.21.15.6:43138) 2024-09-12T03:21:46.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.640+0000 7f03deffd640 1 -- 172.21.15.6:0/3497805412 learned_addr learned my addr 172.21.15.6:0/3497805412 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:46.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.641+0000 7f03deffd640 1 --2- 172.21.15.6:0/3497805412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5643153165d0 0x564315320d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.641+0000 7f03df7fe640 1 --2- 172.21.15.6:0/3497805412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564315317b40 0x56431531e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.641+0000 7f03deffd640 1 -- 172.21.15.6:0/3497805412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564315317b40 msgr2=0x56431531e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:46.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.641+0000 7f03deffd640 1 --2- 172.21.15.6:0/3497805412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564315317b40 0x56431531e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.641+0000 7f03deffd640 1 -- 172.21.15.6:0/3497805412 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564315321370 msgr2=0x56431531e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:46.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.641+0000 7f03deffd640 1 --2- 172.21.15.6:0/3497805412 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564315321370 0x56431531e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.641+0000 7f03deffd640 1 -- 172.21.15.6:0/3497805412 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564314fed3c0 con 0x5643153165d0 2024-09-12T03:21:46.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.641+0000 7f03df7fe640 1 --2- 172.21.15.6:0/3497805412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564315317b40 0x56431531e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:46.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.641+0000 7f03deffd640 1 --2- 172.21.15.6:0/3497805412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5643153165d0 0x564315320d50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f03d4004570 tx=0x7f03d402a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8110f0a8fa10276 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:46.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.641+0000 7f03beffd640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:21:46.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.641+0000 7f03beffd640 1 -- 172.21.15.6:0/3497805412 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4035040 con 0x5643153165d0 2024-09-12T03:21:46.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.641+0000 7f03e7f27740 1 -- 172.21.15.6:0/3497805412 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03d40047a0 con 0x5643153165d0 2024-09-12T03:21:46.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.642+0000 7f03beffd640 1 -- 172.21.15.6:0/3497805412 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f03d4005430 con 0x5643153165d0 2024-09-12T03:21:46.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.642+0000 7f03beffd640 1 -- 172.21.15.6:0/3497805412 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f03d40315b0 con 0x5643153165d0 2024-09-12T03:21:46.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.642+0000 7f03e7f27740 10 client.24977 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:46.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.642+0000 7f03e7f27740 1 -- 172.21.15.6:0/3497805412 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56431532fac0 con 0x5643153165d0 2024-09-12T03:21:46.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.642+0000 7f03beffd640 1 -- 172.21.15.6:0/3497805412 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03d40047a0 con 0x5643153165d0 2024-09-12T03:21:46.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.642+0000 7f03beffd640 1 -- 172.21.15.6:0/3497805412 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f03d4003940 con 0x5643153165d0 2024-09-12T03:21:46.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.642+0000 7f03e7f27740 10 client.24977 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:46.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.642+0000 7f03e7f27740 10 client.24977 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-12T03:21:46.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.642+0000 7f03e7f27740 20 client.24977 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:46.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.642+0000 7f03e7f27740 1 --2- 172.21.15.6:0/3497805412 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x564315330e10 0x564315333200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.642+0000 7f03e7f27740 4 client.24977 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:46.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.642+0000 7f03e7f27740 1 -- 172.21.15.6:0/3497805412 --> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564315289860 con 0x564315330e10 2024-09-12T03:21:46.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.643+0000 7f03df7fe640 1 --2- 172.21.15.6:0/3497805412 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x564315330e10 0x564315333200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.643+0000 7f03df7fe640 1 --2- 172.21.15.6:0/3497805412 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x564315330e10 0x564315333200 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=750d9114c72b6f62 server_cookie=3f9b0fc7bbe6ac5b in_seq=0 out_seq=0 2024-09-12T03:21:46.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.643+0000 7f03beffd640 10 client.24977 ms_handle_connect on v2:172.21.15.161:6838/3645555254 2024-09-12T03:21:46.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.644+0000 7f03beffd640 1 -- 172.21.15.6:0/3497805412 <== mds.2 v2:172.21.15.161:6838/3645555254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564315289860 con 0x564315330e10 2024-09-12T03:21:46.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.644+0000 7f03beffd640 10 client.24977 handle_command_reply: tid=0 2024-09-12T03:21:46.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.644+0000 7f03e7f27740 1 client.24977 shutdown 2024-09-12T03:21:46.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.644+0000 7f03e7f27740 1 -- 172.21.15.6:0/3497805412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5643153165d0 msgr2=0x564315320d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:46.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.644+0000 7f03e7f27740 1 --2- 172.21.15.6:0/3497805412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5643153165d0 0x564315320d50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f03d4004570 tx=0x7f03d402a420 comp rx=0 tx=0).stop 2024-09-12T03:21:46.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.644+0000 7f03e7f27740 1 -- 172.21.15.6:0/3497805412 shutdown_connections 2024-09-12T03:21:46.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.644+0000 7f03e7f27740 1 --2- 172.21.15.6:0/3497805412 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x564315330e10 0x564315333200 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.644+0000 7f03e7f27740 1 --2- 172.21.15.6:0/3497805412 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564315321370 0x56431531e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.644+0000 7f03e7f27740 1 --2- 172.21.15.6:0/3497805412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564315317b40 0x56431531e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.644+0000 7f03e7f27740 1 --2- 172.21.15.6:0/3497805412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5643153165d0 0x564315320d50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.644+0000 7f03beffd640 0 client.24977 ms_handle_reset on v2:172.21.15.161:6838/3645555254 2024-09-12T03:21:46.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.644+0000 7f03e7f27740 1 -- 172.21.15.6:0/3497805412 >> 172.21.15.6:0/3497805412 conn(0x5643152df310 msgr2=0x564315318930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:46.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.644+0000 7f03e7f27740 1 -- 172.21.15.6:0/3497805412 shutdown_connections 2024-09-12T03:21:46.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.645+0000 7f03e7f27740 1 -- 172.21.15.6:0/3497805412 wait complete. 2024-09-12T03:21:46.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.645+0000 7f03e7f27740 20 client.24977 trim_cache size 0 max 16384 2024-09-12T03:21:46.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.658+0000 7f03e7f27740 1 Processor -- start 2024-09-12T03:21:46.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.658+0000 7f03e7f27740 1 -- start start 2024-09-12T03:21:46.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.658+0000 7f03e7f27740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5643153165d0 0x56431532d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.658+0000 7f03e7f27740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564315317b40 0x564315319120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.658+0000 7f03e7f27740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564315321370 0x564315319660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.658+0000 7f03e7f27740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01b1fa0 con 0x564315317b40 2024-09-12T03:21:46.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.658+0000 7f03e7f27740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01af870 con 0x5643153165d0 2024-09-12T03:21:46.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.658+0000 7f03e7f27740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01ad6e0 con 0x564315321370 2024-09-12T03:21:46.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.658+0000 7f03df7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564315317b40 0x564315319120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.658+0000 7f03df7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564315317b40 0x564315319120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43154/0 (socket says 172.21.15.6:43154) 2024-09-12T03:21:46.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.658+0000 7f03df7fe640 1 -- 172.21.15.6:0/92288066 learned_addr learned my addr 172.21.15.6:0/92288066 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:46.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.658+0000 7f03deffd640 1 --2- 172.21.15.6:0/92288066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564315321370 0x564315319660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.658+0000 7f03deffd640 1 --2- 172.21.15.6:0/92288066 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5643153165d0 0x56431532d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.658+0000 7f03df7fe640 1 -- 172.21.15.6:0/92288066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564315321370 msgr2=0x564315319660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:46.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.658+0000 7f03df7fe640 1 --2- 172.21.15.6:0/92288066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564315321370 0x564315319660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.658+0000 7f03df7fe640 1 -- 172.21.15.6:0/92288066 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5643153165d0 msgr2=0x56431532d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:46.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.658+0000 7f03df7fe640 1 --2- 172.21.15.6:0/92288066 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5643153165d0 0x56431532d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.658+0000 7f03df7fe640 1 -- 172.21.15.6:0/92288066 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643153240d0 con 0x564315317b40 2024-09-12T03:21:46.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.659+0000 7f03deffd640 1 --2- 172.21.15.6:0/92288066 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5643153165d0 0x56431532d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:46.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.659+0000 7f03df7fe640 1 --2- 172.21.15.6:0/92288066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564315317b40 0x564315319120 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f03d001bfd0 tx=0x7f03d00137d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925599c99a72f264 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:46.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.659+0000 7f03c57fa640 1 -- 172.21.15.6:0/92288066 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0005220 con 0x564315317b40 2024-09-12T03:21:46.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.659+0000 7f03c57fa640 1 -- 172.21.15.6:0/92288066 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03d0004c80 con 0x564315317b40 2024-09-12T03:21:46.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.659+0000 7f03c57fa640 1 -- 172.21.15.6:0/92288066 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d001b7b0 con 0x564315317b40 2024-09-12T03:21:46.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.660+0000 7f03e7f27740 1 -- 172.21.15.6:0/92288066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564315317b40 msgr2=0x564315319120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:46.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.660+0000 7f03e7f27740 1 --2- 172.21.15.6:0/92288066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564315317b40 0x564315319120 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f03d001bfd0 tx=0x7f03d00137d0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.660+0000 7f03e7f27740 1 -- 172.21.15.6:0/92288066 shutdown_connections 2024-09-12T03:21:46.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.660+0000 7f03e7f27740 1 --2- 172.21.15.6:0/92288066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564315321370 0x564315319660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.660+0000 7f03e7f27740 1 --2- 172.21.15.6:0/92288066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564315317b40 0x564315319120 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.660+0000 7f03e7f27740 1 --2- 172.21.15.6:0/92288066 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5643153165d0 0x56431532d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.660+0000 7f03e7f27740 1 -- 172.21.15.6:0/92288066 >> 172.21.15.6:0/92288066 conn(0x5643152df310 msgr2=0x56431531cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:46.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.660+0000 7f03e7f27740 1 -- 172.21.15.6:0/92288066 shutdown_connections 2024-09-12T03:21:46.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.660+0000 7f03e7f27740 1 -- 172.21.15.6:0/92288066 wait complete. 2024-09-12T03:21:46.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.661+0000 7f03e7f27740 1 -- start start 2024-09-12T03:21:46.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.661+0000 7f03e7f27740 1 Processor -- start 2024-09-12T03:21:46.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.662+0000 7f03e7f27740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643153165d0 0x5643153244f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.662+0000 7f03e7f27740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564315317b40 0x564315324a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.662+0000 7f03e7f27740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564315321370 0x564315399790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.662+0000 7f03deffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643153165d0 0x5643153244f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.662+0000 7f03df7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564315317b40 0x564315324a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.662+0000 7f03deffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643153165d0 0x5643153244f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:36612/0 (socket says 172.21.15.6:36612) 2024-09-12T03:21:46.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.662+0000 7f03df7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564315317b40 0x564315324a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43170/0 (socket says 172.21.15.6:43170) 2024-09-12T03:21:46.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.662+0000 7f03deffd640 1 -- 172.21.15.6:0/2767318877 learned_addr learned my addr 172.21.15.6:0/2767318877 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:46.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.662+0000 7f03e7f27740 1 -- 172.21.15.6:0/2767318877 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01add30 con 0x564315317b40 2024-09-12T03:21:46.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.662+0000 7f03e7f27740 1 -- 172.21.15.6:0/2767318877 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01af1a0 con 0x564315321370 2024-09-12T03:21:46.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.662+0000 7f03e7f27740 1 -- 172.21.15.6:0/2767318877 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01ae400 con 0x5643153165d0 2024-09-12T03:21:46.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.662+0000 7f03e7f27740 10 client.0 _ll_register_callbacks cb 0x7f03d40035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:46.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.663+0000 7f03deffd640 1 --2- 172.21.15.6:0/2767318877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564315321370 0x564315399790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.663+0000 7f03deffd640 1 -- 172.21.15.6:0/2767318877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564315321370 msgr2=0x564315399790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:46.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.663+0000 7f03deffd640 1 --2- 172.21.15.6:0/2767318877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564315321370 0x564315399790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.663+0000 7f03deffd640 1 -- 172.21.15.6:0/2767318877 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564315317b40 msgr2=0x564315324a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:46.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.663+0000 7f03deffd640 1 --2- 172.21.15.6:0/2767318877 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564315317b40 0x564315324a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.663+0000 7f03deffd640 1 -- 172.21.15.6:0/2767318877 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564315399cd0 con 0x5643153165d0 2024-09-12T03:21:46.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.663+0000 7f03deffd640 1 --2- 172.21.15.6:0/2767318877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643153165d0 0x5643153244f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f03d4004650 tx=0x7f03d4004330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6815fe06575a32be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:46.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.663+0000 7f03beffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:21:46.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.663+0000 7f03beffd640 1 -- 172.21.15.6:0/2767318877 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d406c060 con 0x5643153165d0 2024-09-12T03:21:46.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.664+0000 7f03e7f27740 1 -- 172.21.15.6:0/2767318877 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03d0004c80 con 0x5643153165d0 2024-09-12T03:21:46.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.664+0000 7f03beffd640 1 -- 172.21.15.6:0/2767318877 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f03d4005430 con 0x5643153165d0 2024-09-12T03:21:46.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.664+0000 7f03beffd640 1 -- 172.21.15.6:0/2767318877 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f03d4005630 con 0x5643153165d0 2024-09-12T03:21:46.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.664+0000 7f03e7f27740 10 client.25003 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:46.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.664+0000 7f03e7f27740 1 -- 172.21.15.6:0/2767318877 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56431539ae20 con 0x5643153165d0 2024-09-12T03:21:46.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.664+0000 7f03beffd640 1 -- 172.21.15.6:0/2767318877 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03d0004c80 con 0x5643153165d0 2024-09-12T03:21:46.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.664+0000 7f03beffd640 1 -- 172.21.15.6:0/2767318877 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f03d4031bd0 con 0x5643153165d0 2024-09-12T03:21:46.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.664+0000 7f03e7f27740 10 client.25003 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:46.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.664+0000 7f03e7f27740 10 client.25003 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-12T03:21:46.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.664+0000 7f03e7f27740 20 client.25003 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:46.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.665+0000 7f03e7f27740 1 --2- 172.21.15.6:0/2767318877 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x56431539b2d0 0x56431539d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.665+0000 7f03e7f27740 4 client.25003 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:21:46.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.665+0000 7f03e7f27740 1 -- 172.21.15.6:0/2767318877 --> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564315283d70 con 0x56431539b2d0 2024-09-12T03:21:46.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.665+0000 7f03df7fe640 1 --2- 172.21.15.6:0/2767318877 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x56431539b2d0 0x56431539d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.665+0000 7f03beffd640 10 client.25003 ms_handle_connect on v2:172.21.15.161:6838/3645555254 2024-09-12T03:21:46.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.665+0000 7f03df7fe640 1 --2- 172.21.15.6:0/2767318877 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x56431539b2d0 0x56431539d6f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=deb149e7981489fb server_cookie=71c9fec941c67fb6 in_seq=0 out_seq=0 2024-09-12T03:21:46.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.665+0000 7f03beffd640 1 -- 172.21.15.6:0/2767318877 <== mds.2 v2:172.21.15.161:6838/3645555254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564315283d70 con 0x56431539b2d0 2024-09-12T03:21:46.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.665+0000 7f03beffd640 10 client.25003 handle_command_reply: tid=0 2024-09-12T03:21:46.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.665+0000 7f03e7f27740 1 client.25003 shutdown 2024-09-12T03:21:46.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.666+0000 7f03e7f27740 1 -- 172.21.15.6:0/2767318877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643153165d0 msgr2=0x5643153244f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:46.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.666+0000 7f03e7f27740 1 --2- 172.21.15.6:0/2767318877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643153165d0 0x5643153244f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f03d4004650 tx=0x7f03d4004330 comp rx=0 tx=0).stop 2024-09-12T03:21:46.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.666+0000 7f03e7f27740 1 -- 172.21.15.6:0/2767318877 shutdown_connections 2024-09-12T03:21:46.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.666+0000 7f03e7f27740 1 --2- 172.21.15.6:0/2767318877 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x56431539b2d0 0x56431539d6f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.666+0000 7f03e7f27740 1 --2- 172.21.15.6:0/2767318877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564315321370 0x564315399790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.666+0000 7f03beffd640 0 client.25003 ms_handle_reset on v2:172.21.15.161:6838/3645555254 2024-09-12T03:21:46.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.666+0000 7f03e7f27740 1 --2- 172.21.15.6:0/2767318877 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564315317b40 0x564315324a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.666+0000 7f03e7f27740 1 --2- 172.21.15.6:0/2767318877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643153165d0 0x5643153244f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.666+0000 7f03e7f27740 1 -- 172.21.15.6:0/2767318877 >> 172.21.15.6:0/2767318877 conn(0x5643152df310 msgr2=0x564315311310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:46.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.666+0000 7f03e7f27740 1 -- 172.21.15.6:0/2767318877 shutdown_connections 2024-09-12T03:21:46.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.666+0000 7f03e7f27740 1 -- 172.21.15.6:0/2767318877 wait complete. 2024-09-12T03:21:46.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.666+0000 7f03e7f27740 20 client.25003 trim_cache size 0 max 16384 2024-09-12T03:21:46.670 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-09-12T03:21:46.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.667+0000 7f03e5b4e640 1 -- 172.21.15.6:0/357279355 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f03c80a2c90 msgr2=0x7f03c80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:46.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.667+0000 7f03e5b4e640 1 --2- 172.21.15.6:0/357279355 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f03c80a2c90 0x7f03c80a5150 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f03d8097c10 tx=0x7f03d8097b20 comp rx=0 tx=0).stop 2024-09-12T03:21:46.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.667+0000 7f03e5b4e640 1 -- 172.21.15.6:0/357279355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03e01082a0 msgr2=0x7f03e019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:46.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.667+0000 7f03e5b4e640 1 --2- 172.21.15.6:0/357279355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03e01082a0 0x7f03e019a5f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f03d800eb10 tx=0x7f03d800c2d0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.667+0000 7f03e5b4e640 1 -- 172.21.15.6:0/357279355 shutdown_connections 2024-09-12T03:21:46.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.667+0000 7f03e5b4e640 1 --2- 172.21.15.6:0/357279355 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f03c80a2c90 0x7f03c80a5150 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.667+0000 7f03e5b4e640 1 --2- 172.21.15.6:0/357279355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03e01082a0 0x7f03e019a5f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.667+0000 7f03e5b4e640 1 --2- 172.21.15.6:0/357279355 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03e01036e0 0x7f03e01a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.667+0000 7f03e5b4e640 1 --2- 172.21.15.6:0/357279355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e0102d30 0x7f03e01a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.667+0000 7f03e5b4e640 1 -- 172.21.15.6:0/357279355 >> 172.21.15.6:0/357279355 conn(0x7f03e00fc7d0 msgr2=0x7f03e0100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:46.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.668+0000 7f03e5b4e640 1 -- 172.21.15.6:0/357279355 shutdown_connections 2024-09-12T03:21:46.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:46.668+0000 7f03e5b4e640 1 -- 172.21.15.6:0/357279355 wait complete. 2024-09-12T03:21:47.348 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:47 smithi006 ceph-mon[30991]: pgmap v355: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T03:21:47.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:47 smithi161 ceph-mon[35485]: pgmap v355: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T03:21:47.376 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:21:47.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:47 smithi118 ceph-mon[35048]: pgmap v355: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T03:21:47.666 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:48.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.175+0000 7f532353b640 1 -- 172.21.15.6:0/3100196678 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f531c102dc0 msgr2=0x7f531c1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.175+0000 7f532353b640 1 --2- 172.21.15.6:0/3100196678 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f531c102dc0 0x7f531c1031a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5318009900 tx=0x7f531802a480 comp rx=0 tx=0).stop 2024-09-12T03:21:48.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.175+0000 7f532353b640 1 -- 172.21.15.6:0/3100196678 shutdown_connections 2024-09-12T03:21:48.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.175+0000 7f532353b640 1 --2- 172.21.15.6:0/3100196678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531c10c160 0x7f531c10e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.175+0000 7f532353b640 1 --2- 172.21.15.6:0/3100196678 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f531c1036e0 0x7f531c10bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.175+0000 7f532353b640 1 --2- 172.21.15.6:0/3100196678 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f531c102dc0 0x7f531c1031a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.175+0000 7f532353b640 1 -- 172.21.15.6:0/3100196678 >> 172.21.15.6:0/3100196678 conn(0x7f531c0fc780 msgr2=0x7f531c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:48.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.175+0000 7f532353b640 1 -- 172.21.15.6:0/3100196678 shutdown_connections 2024-09-12T03:21:48.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.175+0000 7f532353b640 1 -- 172.21.15.6:0/3100196678 wait complete. 2024-09-12T03:21:48.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.176+0000 7f532353b640 1 Processor -- start 2024-09-12T03:21:48.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.176+0000 7f532353b640 1 -- start start 2024-09-12T03:21:48.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.176+0000 7f532353b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531c102dc0 0x7f531c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.176+0000 7f532353b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f531c1036e0 0x7f531c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.176+0000 7f532353b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f531c10c160 0x7f531c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.176+0000 7f532353b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f531c111f50 con 0x7f531c1036e0 2024-09-12T03:21:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.176+0000 7f532353b640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f531c111dd0 con 0x7f531c10c160 2024-09-12T03:21:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.176+0000 7f532353b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f531c1120d0 con 0x7f531c102dc0 2024-09-12T03:21:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.176+0000 7f5320ad6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f531c1036e0 0x7f531c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.176+0000 7f5320ad6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f531c1036e0 0x7f531c198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43186/0 (socket says 172.21.15.6:43186) 2024-09-12T03:21:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.176+0000 7f5320ad6640 1 -- 172.21.15.6:0/3676502157 learned_addr learned my addr 172.21.15.6:0/3676502157 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.176+0000 7f53212d7640 1 --2- 172.21.15.6:0/3676502157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531c102dc0 0x7f531c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.177+0000 7f5321ad8640 1 --2- 172.21.15.6:0/3676502157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f531c10c160 0x7f531c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.177+0000 7f5320ad6640 1 -- 172.21.15.6:0/3676502157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531c102dc0 msgr2=0x7f531c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.177+0000 7f5320ad6640 1 --2- 172.21.15.6:0/3676502157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531c102dc0 0x7f531c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.177+0000 7f5320ad6640 1 -- 172.21.15.6:0/3676502157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f531c10c160 msgr2=0x7f531c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.177+0000 7f5320ad6640 1 --2- 172.21.15.6:0/3676502157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f531c10c160 0x7f531c19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.177+0000 7f5320ad6640 1 -- 172.21.15.6:0/3676502157 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f531c19d400 con 0x7f531c1036e0 2024-09-12T03:21:48.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.177+0000 7f5321ad8640 1 --2- 172.21.15.6:0/3676502157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f531c10c160 0x7f531c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:48.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.177+0000 7f53212d7640 1 --2- 172.21.15.6:0/3676502157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531c102dc0 0x7f531c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:48.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.177+0000 7f5320ad6640 1 --2- 172.21.15.6:0/3676502157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f531c1036e0 0x7f531c198990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f530c00e9f0 tx=0x7f530c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:48.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.177+0000 7f53127fc640 1 -- 172.21.15.6:0/3676502157 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c00cd80 con 0x7f531c1036e0 2024-09-12T03:21:48.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.177+0000 7f532353b640 1 -- 172.21.15.6:0/3676502157 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f531c19d6f0 con 0x7f531c1036e0 2024-09-12T03:21:48.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.178+0000 7f532353b640 1 -- 172.21.15.6:0/3676502157 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f531c1a4fd0 con 0x7f531c1036e0 2024-09-12T03:21:48.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.178+0000 7f53127fc640 1 -- 172.21.15.6:0/3676502157 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f530c010910 con 0x7f531c1036e0 2024-09-12T03:21:48.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.178+0000 7f53127fc640 1 -- 172.21.15.6:0/3676502157 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c018990 con 0x7f531c1036e0 2024-09-12T03:21:48.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.181+0000 7f5325914740 1 Processor -- start 2024-09-12T03:21:48.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.181+0000 7f53127fc640 1 -- 172.21.15.6:0/3676502157 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f530c018b30 con 0x7f531c1036e0 2024-09-12T03:21:48.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.181+0000 7f5325914740 1 -- start start 2024-09-12T03:21:48.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.181+0000 7f5325914740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabb0b40 0x564ccabaf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.181+0000 7f5325914740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabaf5d0 0x564ccabafa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.182+0000 7f5325914740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabbe780 0x564ccabc0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.182+0000 7f5325914740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x564ccab152f0 con 0x564ccabbe780 2024-09-12T03:21:48.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.182+0000 7f5325914740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x564ccaabccf0 con 0x564ccabaf5d0 2024-09-12T03:21:48.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.182+0000 7f5325914740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564ccab112b0 con 0x564ccabb0b40 2024-09-12T03:21:48.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.182+0000 7f5320ad6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabbe780 0x564ccabc0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.182+0000 7f5321ad8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabb0b40 0x564ccabaf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.182+0000 7f5321ad8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabb0b40 0x564ccabaf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:36636/0 (socket says 172.21.15.6:36636) 2024-09-12T03:21:48.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.182+0000 7f5321ad8640 1 -- 172.21.15.6:0/2410933501 learned_addr learned my addr 172.21.15.6:0/2410933501 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:48.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.182+0000 7f5320ad6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabbe780 0x564ccabc0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43202/0 (socket says 172.21.15.6:43202) 2024-09-12T03:21:48.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.182+0000 7f53212d7640 1 --2- 172.21.15.6:0/2410933501 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabaf5d0 0x564ccabafa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.182+0000 7f53212d7640 1 -- 172.21.15.6:0/2410933501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabb0b40 msgr2=0x564ccabaf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:48.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.182+0000 7f53212d7640 1 --2- 172.21.15.6:0/2410933501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabb0b40 0x564ccabaf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.182+0000 7f53212d7640 1 -- 172.21.15.6:0/2410933501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabbe780 msgr2=0x564ccabc0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:48.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.182+0000 7f53212d7640 1 --2- 172.21.15.6:0/2410933501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabbe780 0x564ccabc0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.182+0000 7f53212d7640 1 -- 172.21.15.6:0/2410933501 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ccaaa13a0 con 0x564ccabaf5d0 2024-09-12T03:21:48.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.182+0000 7f5320ad6640 1 --2- 172.21.15.6:0/2410933501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabbe780 0x564ccabc0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:48.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.182+0000 7f5321ad8640 1 --2- 172.21.15.6:0/2410933501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabb0b40 0x564ccabaf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:48.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.183+0000 7f53212d7640 1 --2- 172.21.15.6:0/2410933501 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabaf5d0 0x564ccabafa10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5318005940 tx=0x7f5318005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7203df38f05a6eac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:48.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.183+0000 7f53077fe640 1 -- 172.21.15.6:0/2410933501 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5318035040 con 0x564ccabaf5d0 2024-09-12T03:21:48.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.183+0000 7f53077fe640 1 -- 172.21.15.6:0/2410933501 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5318031830 con 0x564ccabaf5d0 2024-09-12T03:21:48.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.183+0000 7f53077fe640 1 -- 172.21.15.6:0/2410933501 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5318004750 con 0x564ccabaf5d0 2024-09-12T03:21:48.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.184+0000 7f5325914740 1 -- 172.21.15.6:0/2410933501 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabaf5d0 msgr2=0x564ccabafa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.184+0000 7f5325914740 1 --2- 172.21.15.6:0/2410933501 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabaf5d0 0x564ccabafa10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5318005940 tx=0x7f5318005d50 comp rx=0 tx=0).stop 2024-09-12T03:21:48.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.184+0000 7f5325914740 1 -- 172.21.15.6:0/2410933501 shutdown_connections 2024-09-12T03:21:48.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.184+0000 7f5325914740 1 --2- 172.21.15.6:0/2410933501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabbe780 0x564ccabc0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.184+0000 7f5325914740 1 --2- 172.21.15.6:0/2410933501 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabaf5d0 0x564ccabafa10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.184+0000 7f5325914740 1 --2- 172.21.15.6:0/2410933501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabb0b40 0x564ccabaf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.184+0000 7f5325914740 1 -- 172.21.15.6:0/2410933501 >> 172.21.15.6:0/2410933501 conn(0x564ccab78310 msgr2=0x564ccababc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:48.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.184+0000 7f5325914740 1 -- 172.21.15.6:0/2410933501 shutdown_connections 2024-09-12T03:21:48.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.184+0000 7f5325914740 1 -- 172.21.15.6:0/2410933501 wait complete. 2024-09-12T03:21:48.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.184+0000 7f53127fc640 1 --2- 172.21.15.6:0/3676502157 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f53080a2ba0 0x7f53080a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.185+0000 7f53212d7640 1 --2- 172.21.15.6:0/3676502157 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f53080a2ba0 0x7f53080a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.185+0000 7f5325914740 1 -- start start 2024-09-12T03:21:48.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.185+0000 7f5325914740 1 Processor -- start 2024-09-12T03:21:48.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.185+0000 7f53127fc640 1 -- 172.21.15.6:0/3676502157 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f530c0cf400 con 0x7f531c1036e0 2024-09-12T03:21:48.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.185+0000 7f53212d7640 1 --2- 172.21.15.6:0/3676502157 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f53080a2ba0 0x7f53080a5060 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f53180096c0 tx=0x7f531802a480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:48.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.186+0000 7f5325914740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabaf5d0 0x564ccabbe0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.186+0000 7f5325914740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabb0b40 0x564ccabbb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.186+0000 7f5325914740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabbe780 0x564ccabbb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.186+0000 7f5325914740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x564ccabc1420 con 0x564ccabb0b40 2024-09-12T03:21:48.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.186+0000 7f5325914740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x564ccaad0060 con 0x564ccabbe780 2024-09-12T03:21:48.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.186+0000 7f5325914740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564ccabc15a0 con 0x564ccabaf5d0 2024-09-12T03:21:48.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.186+0000 7f5325914740 10 client.0 _ll_register_callbacks cb 0x564cca8238c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:48.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.186+0000 7f5320ad6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabb0b40 0x564ccabbb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.186+0000 7f5320ad6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabb0b40 0x564ccabbb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43216/0 (socket says 172.21.15.6:43216) 2024-09-12T03:21:48.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.186+0000 7f5320ad6640 1 -- 172.21.15.6:0/1536773444 learned_addr learned my addr 172.21.15.6:0/1536773444 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:48.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.186+0000 7f5321ad8640 1 --2- 172.21.15.6:0/1536773444 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabbe780 0x564ccabbb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.186+0000 7f53212d7640 1 --2- 172.21.15.6:0/1536773444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabaf5d0 0x564ccabbe0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.186+0000 7f5320ad6640 1 -- 172.21.15.6:0/1536773444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabaf5d0 msgr2=0x564ccabbe0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.186+0000 7f5320ad6640 1 --2- 172.21.15.6:0/1536773444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabaf5d0 0x564ccabbe0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.186+0000 7f5320ad6640 1 -- 172.21.15.6:0/1536773444 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabbe780 msgr2=0x564ccabbb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.186+0000 7f5320ad6640 1 --2- 172.21.15.6:0/1536773444 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabbe780 0x564ccabbb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.186+0000 7f5320ad6640 1 -- 172.21.15.6:0/1536773444 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cca8863c0 con 0x564ccabb0b40 2024-09-12T03:21:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.186+0000 7f5320ad6640 1 --2- 172.21.15.6:0/1536773444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabb0b40 0x564ccabbb380 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f530c097f30 tx=0x7f530c018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43743c91fa9983fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.186+0000 7f5304ff9640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:21:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.187+0000 7f5304ff9640 1 -- 172.21.15.6:0/1536773444 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c09b430 con 0x564ccabb0b40 2024-09-12T03:21:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.187+0000 7f5304ff9640 1 -- 172.21.15.6:0/1536773444 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f530c09af20 con 0x564ccabb0b40 2024-09-12T03:21:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.187+0000 7f5325914740 1 -- 172.21.15.6:0/1536773444 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5318031830 con 0x564ccabb0b40 2024-09-12T03:21:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.187+0000 7f5304ff9640 1 -- 172.21.15.6:0/1536773444 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f530c09c1d0 con 0x564ccabb0b40 2024-09-12T03:21:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.187+0000 7f5325914740 10 client.15399 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.187+0000 7f5325914740 1 -- 172.21.15.6:0/1536773444 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564ccabcce90 con 0x564ccabb0b40 2024-09-12T03:21:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.187+0000 7f5304ff9640 1 -- 172.21.15.6:0/1536773444 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5318031830 con 0x564ccabb0b40 2024-09-12T03:21:48.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.187+0000 7f5304ff9640 1 -- 172.21.15.6:0/1536773444 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f530c097930 con 0x564ccabb0b40 2024-09-12T03:21:48.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.187+0000 7f5325914740 10 client.15399 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:48.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.187+0000 7f5325914740 10 client.15399 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-12T03:21:48.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.187+0000 7f5325914740 20 client.15399 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:48.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.187+0000 7f5325914740 1 --2- 172.21.15.6:0/1536773444 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x564ccabce8e0 0x564ccabd0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.187+0000 7f5325914740 4 client.15399 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:48.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.187+0000 7f5325914740 1 -- 172.21.15.6:0/1536773444 --> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ccab22860 con 0x564ccabce8e0 2024-09-12T03:21:48.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.188+0000 7f53212d7640 1 --2- 172.21.15.6:0/1536773444 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x564ccabce8e0 0x564ccabd0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.188+0000 7f5304ff9640 10 client.15399 ms_handle_connect on v2:172.21.15.118:6836/3879191727 2024-09-12T03:21:48.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.188+0000 7f53212d7640 1 --2- 172.21.15.6:0/1536773444 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x564ccabce8e0 0x564ccabd0d40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f57b0558e94fe4aa server_cookie=eab6c1445dbdae32 in_seq=0 out_seq=0 2024-09-12T03:21:48.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.189+0000 7f5304ff9640 1 -- 172.21.15.6:0/1536773444 <== mds.1 v2:172.21.15.118:6836/3879191727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ccab22860 con 0x564ccabce8e0 2024-09-12T03:21:48.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.189+0000 7f5304ff9640 10 client.15399 handle_command_reply: tid=0 2024-09-12T03:21:48.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.189+0000 7f5325914740 1 client.15399 shutdown 2024-09-12T03:21:48.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.189+0000 7f5325914740 1 -- 172.21.15.6:0/1536773444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabb0b40 msgr2=0x564ccabbb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.189+0000 7f5325914740 1 --2- 172.21.15.6:0/1536773444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabb0b40 0x564ccabbb380 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f530c097f30 tx=0x7f530c018420 comp rx=0 tx=0).stop 2024-09-12T03:21:48.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.189+0000 7f5325914740 1 -- 172.21.15.6:0/1536773444 shutdown_connections 2024-09-12T03:21:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.189+0000 7f5325914740 1 --2- 172.21.15.6:0/1536773444 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x564ccabce8e0 0x564ccabd0d40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.189+0000 7f5325914740 1 --2- 172.21.15.6:0/1536773444 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabbe780 0x564ccabbb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.189+0000 7f5325914740 1 --2- 172.21.15.6:0/1536773444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabb0b40 0x564ccabbb380 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.189+0000 7f5325914740 1 --2- 172.21.15.6:0/1536773444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabaf5d0 0x564ccabbe0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.189+0000 7f5325914740 1 -- 172.21.15.6:0/1536773444 >> 172.21.15.6:0/1536773444 conn(0x564ccab78310 msgr2=0x564ccabb1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.189+0000 7f5304ff9640 0 client.15399 ms_handle_reset on v2:172.21.15.118:6836/3879191727 2024-09-12T03:21:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.189+0000 7f5325914740 1 -- 172.21.15.6:0/1536773444 shutdown_connections 2024-09-12T03:21:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.190+0000 7f5325914740 1 -- 172.21.15.6:0/1536773444 wait complete. 2024-09-12T03:21:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.190+0000 7f5325914740 20 client.15399 trim_cache size 0 max 16384 2024-09-12T03:21:48.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.203+0000 7f5325914740 1 Processor -- start 2024-09-12T03:21:48.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.203+0000 7f5325914740 1 -- start start 2024-09-12T03:21:48.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.203+0000 7f5325914740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabaf5d0 0x564ccababe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.203+0000 7f5325914740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabb0b40 0x564ccabba2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.203+0000 7f5325914740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabbe780 0x564ccabca720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.203+0000 7f5325914740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5300010c90 con 0x564ccabaf5d0 2024-09-12T03:21:48.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.203+0000 7f5325914740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f530000e560 con 0x564ccabbe780 2024-09-12T03:21:48.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.203+0000 7f53212d7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabaf5d0 0x564ccababe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.203+0000 7f53212d7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabaf5d0 0x564ccababe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43220/0 (socket says 172.21.15.6:43220) 2024-09-12T03:21:48.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.203+0000 7f53212d7640 1 -- 172.21.15.6:0/3160068716 learned_addr learned my addr 172.21.15.6:0/3160068716 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:48.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.203+0000 7f5325914740 1 -- 172.21.15.6:0/3160068716 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f530000c3b0 con 0x564ccabb0b40 2024-09-12T03:21:48.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.203+0000 7f5320ad6640 1 --2- 172.21.15.6:0/3160068716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabb0b40 0x564ccabba2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.204+0000 7f5321ad8640 1 --2- 172.21.15.6:0/3160068716 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabbe780 0x564ccabca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.204+0000 7f5320ad6640 1 -- 172.21.15.6:0/3160068716 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabbe780 msgr2=0x564ccabca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:48.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.204+0000 7f5320ad6640 1 --2- 172.21.15.6:0/3160068716 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabbe780 0x564ccabca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.204+0000 7f5320ad6640 1 -- 172.21.15.6:0/3160068716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabaf5d0 msgr2=0x564ccababe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:48.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.204+0000 7f5320ad6640 1 --2- 172.21.15.6:0/3160068716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabaf5d0 0x564ccababe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.204+0000 7f5320ad6640 1 -- 172.21.15.6:0/3160068716 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ccabd13c0 con 0x564ccabb0b40 2024-09-12T03:21:48.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.204+0000 7f5320ad6640 1 --2- 172.21.15.6:0/3160068716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabb0b40 0x564ccabba2e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x564ccabbc930 tx=0x7f530c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98a26599b8f194a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:48.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.204+0000 7f5306ffd640 1 -- 172.21.15.6:0/3160068716 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c09b430 con 0x564ccabb0b40 2024-09-12T03:21:48.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.204+0000 7f5306ffd640 1 -- 172.21.15.6:0/3160068716 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f530c09bad0 con 0x564ccabb0b40 2024-09-12T03:21:48.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.204+0000 7f5306ffd640 1 -- 172.21.15.6:0/3160068716 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c09b8a0 con 0x564ccabb0b40 2024-09-12T03:21:48.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.205+0000 7f5325914740 1 -- 172.21.15.6:0/3160068716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabb0b40 msgr2=0x564ccabba2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.205+0000 7f5325914740 1 --2- 172.21.15.6:0/3160068716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabb0b40 0x564ccabba2e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x564ccabbc930 tx=0x7f530c000bc0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.205+0000 7f5325914740 1 -- 172.21.15.6:0/3160068716 shutdown_connections 2024-09-12T03:21:48.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.205+0000 7f5325914740 1 --2- 172.21.15.6:0/3160068716 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabbe780 0x564ccabca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.205+0000 7f5325914740 1 --2- 172.21.15.6:0/3160068716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabb0b40 0x564ccabba2e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.205+0000 7f5325914740 1 --2- 172.21.15.6:0/3160068716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabaf5d0 0x564ccababe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.205+0000 7f5325914740 1 -- 172.21.15.6:0/3160068716 >> 172.21.15.6:0/3160068716 conn(0x564ccab78310 msgr2=0x564ccabab700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:48.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.205+0000 7f5325914740 1 -- 172.21.15.6:0/3160068716 shutdown_connections 2024-09-12T03:21:48.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.206+0000 7f5325914740 1 -- 172.21.15.6:0/3160068716 wait complete. 2024-09-12T03:21:48.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.206+0000 7f5325914740 1 -- start start 2024-09-12T03:21:48.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.206+0000 7f5325914740 1 Processor -- start 2024-09-12T03:21:48.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.207+0000 7f5325914740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabaf5d0 0x564ccabb3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.207+0000 7f53212d7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabaf5d0 0x564ccabb3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.207+0000 7f53212d7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabaf5d0 0x564ccabb3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43234/0 (socket says 172.21.15.6:43234) 2024-09-12T03:21:48.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.207+0000 7f53212d7640 1 -- 172.21.15.6:0/271807204 learned_addr learned my addr 172.21.15.6:0/271807204 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:48.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.207+0000 7f5325914740 1 --2- 172.21.15.6:0/271807204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabb0b40 0x564ccabb3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.207+0000 7f5325914740 1 --2- 172.21.15.6:0/271807204 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabbe780 0x564ccabc5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.208+0000 7f5325914740 1 -- 172.21.15.6:0/271807204 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f530000ca20 con 0x564ccabaf5d0 2024-09-12T03:21:48.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.208+0000 7f5325914740 1 -- 172.21.15.6:0/271807204 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f530000de90 con 0x564ccabbe780 2024-09-12T03:21:48.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.208+0000 7f5325914740 1 -- 172.21.15.6:0/271807204 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f530000d0f0 con 0x564ccabb0b40 2024-09-12T03:21:48.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.208+0000 7f5320ad6640 1 --2- 172.21.15.6:0/271807204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabb0b40 0x564ccabb3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.208+0000 7f5321ad8640 1 --2- 172.21.15.6:0/271807204 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabbe780 0x564ccabc5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.208+0000 7f5325914740 10 client.0 _ll_register_callbacks cb 0x7f530c0d2ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:48.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.208+0000 7f5321ad8640 1 -- 172.21.15.6:0/271807204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabb0b40 msgr2=0x564ccabb3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:48.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.208+0000 7f5321ad8640 1 --2- 172.21.15.6:0/271807204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabb0b40 0x564ccabb3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.208+0000 7f5321ad8640 1 -- 172.21.15.6:0/271807204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabaf5d0 msgr2=0x564ccabb3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:48.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.208+0000 7f5321ad8640 1 --2- 172.21.15.6:0/271807204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabaf5d0 0x564ccabb3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.208+0000 7f5321ad8640 1 -- 172.21.15.6:0/271807204 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ccabc5dc0 con 0x564ccabbe780 2024-09-12T03:21:48.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.208+0000 7f5320ad6640 1 --2- 172.21.15.6:0/271807204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabb0b40 0x564ccabb3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:48.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.208+0000 7f5321ad8640 1 --2- 172.21.15.6:0/271807204 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabbe780 0x564ccabc5880 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5314005bf0 tx=0x7f5314010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c93607314f4f860b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:48.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.208+0000 7f5304ff9640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:21:48.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.209+0000 7f5304ff9640 1 -- 172.21.15.6:0/271807204 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314004b20 con 0x564ccabbe780 2024-09-12T03:21:48.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.209+0000 7f5304ff9640 1 -- 172.21.15.6:0/271807204 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f5314004cc0 con 0x564ccabbe780 2024-09-12T03:21:48.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.209+0000 7f5325914740 1 -- 172.21.15.6:0/271807204 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f530c09bad0 con 0x564ccabbe780 2024-09-12T03:21:48.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.209+0000 7f5304ff9640 1 -- 172.21.15.6:0/271807204 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f531401c5f0 con 0x564ccabbe780 2024-09-12T03:21:48.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.209+0000 7f5325914740 10 client.25001 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:48.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.209+0000 7f5325914740 1 -- 172.21.15.6:0/271807204 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564ccabc70e0 con 0x564ccabbe780 2024-09-12T03:21:48.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.209+0000 7f5304ff9640 1 -- 172.21.15.6:0/271807204 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f530c09bad0 con 0x564ccabbe780 2024-09-12T03:21:48.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.209+0000 7f5304ff9640 1 -- 172.21.15.6:0/271807204 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f531401d8c0 con 0x564ccabbe780 2024-09-12T03:21:48.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.210+0000 7f5325914740 10 client.25001 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:48.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.210+0000 7f5325914740 10 client.25001 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-12T03:21:48.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.210+0000 7f5325914740 20 client.25001 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:48.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.210+0000 7f5325914740 1 --2- 172.21.15.6:0/271807204 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x564ccac3e020 0x564ccac40440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.210+0000 7f5325914740 4 client.25001 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:21:48.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.210+0000 7f5325914740 1 -- 172.21.15.6:0/271807204 --> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ccab1cd70 con 0x564ccac3e020 2024-09-12T03:21:48.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.210+0000 7f53212d7640 1 --2- 172.21.15.6:0/271807204 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x564ccac3e020 0x564ccac40440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.210+0000 7f53212d7640 1 --2- 172.21.15.6:0/271807204 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x564ccac3e020 0x564ccac40440 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=45f874135d3894dc server_cookie=dc076ee7e7ea393f in_seq=0 out_seq=0 2024-09-12T03:21:48.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.210+0000 7f5304ff9640 10 client.25001 ms_handle_connect on v2:172.21.15.118:6836/3879191727 2024-09-12T03:21:48.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.210+0000 7f5304ff9640 1 -- 172.21.15.6:0/271807204 <== mds.1 v2:172.21.15.118:6836/3879191727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ccab1cd70 con 0x564ccac3e020 2024-09-12T03:21:48.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.210+0000 7f5304ff9640 10 client.25001 handle_command_reply: tid=0 2024-09-12T03:21:48.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.211+0000 7f5325914740 1 client.25001 shutdown 2024-09-12T03:21:48.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.211+0000 7f5325914740 1 -- 172.21.15.6:0/271807204 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabbe780 msgr2=0x564ccabc5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.211+0000 7f5325914740 1 --2- 172.21.15.6:0/271807204 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabbe780 0x564ccabc5880 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5314005bf0 tx=0x7f5314010800 comp rx=0 tx=0).stop 2024-09-12T03:21:48.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.211+0000 7f5325914740 1 -- 172.21.15.6:0/271807204 shutdown_connections 2024-09-12T03:21:48.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.211+0000 7f5325914740 1 --2- 172.21.15.6:0/271807204 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x564ccac3e020 0x564ccac40440 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.211+0000 7f5304ff9640 0 client.25001 ms_handle_reset on v2:172.21.15.118:6836/3879191727 2024-09-12T03:21:48.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.211+0000 7f5325914740 1 --2- 172.21.15.6:0/271807204 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564ccabbe780 0x564ccabc5880 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.211+0000 7f5325914740 1 --2- 172.21.15.6:0/271807204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ccabb0b40 0x564ccabb3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.211+0000 7f5325914740 1 --2- 172.21.15.6:0/271807204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564ccabaf5d0 0x564ccabb3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.211+0000 7f5325914740 1 -- 172.21.15.6:0/271807204 >> 172.21.15.6:0/271807204 conn(0x564ccab78310 msgr2=0x564ccabbcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:48.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.211+0000 7f5325914740 1 -- 172.21.15.6:0/271807204 shutdown_connections 2024-09-12T03:21:48.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.211+0000 7f5325914740 1 -- 172.21.15.6:0/271807204 wait complete. 2024-09-12T03:21:48.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.211+0000 7f5325914740 20 client.25001 trim_cache size 0 max 16384 2024-09-12T03:21:48.215 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-09-12T03:21:48.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.212+0000 7f532353b640 1 -- 172.21.15.6:0/3676502157 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f53080a2ba0 msgr2=0x7f53080a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.212+0000 7f532353b640 1 --2- 172.21.15.6:0/3676502157 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f53080a2ba0 0x7f53080a5060 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f53180096c0 tx=0x7f531802a480 comp rx=0 tx=0).stop 2024-09-12T03:21:48.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.212+0000 7f532353b640 1 -- 172.21.15.6:0/3676502157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f531c1036e0 msgr2=0x7f531c198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.212+0000 7f532353b640 1 --2- 172.21.15.6:0/3676502157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f531c1036e0 0x7f531c198990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f530c00e9f0 tx=0x7f530c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.212+0000 7f532353b640 1 -- 172.21.15.6:0/3676502157 shutdown_connections 2024-09-12T03:21:48.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.212+0000 7f532353b640 1 --2- 172.21.15.6:0/3676502157 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f53080a2ba0 0x7f53080a5060 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.212+0000 7f532353b640 1 --2- 172.21.15.6:0/3676502157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f531c10c160 0x7f531c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.212+0000 7f532353b640 1 --2- 172.21.15.6:0/3676502157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f531c1036e0 0x7f531c198990 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.212+0000 7f532353b640 1 --2- 172.21.15.6:0/3676502157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531c102dc0 0x7f531c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.212+0000 7f532353b640 1 -- 172.21.15.6:0/3676502157 >> 172.21.15.6:0/3676502157 conn(0x7f531c0fc780 msgr2=0x7f531c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:48.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.213+0000 7f532353b640 1 -- 172.21.15.6:0/3676502157 shutdown_connections 2024-09-12T03:21:48.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:48.213+0000 7f532353b640 1 -- 172.21.15.6:0/3676502157 wait complete. 2024-09-12T03:21:48.839 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:21:49.092 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:49 smithi006 ceph-mon[30991]: pgmap v356: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:21:49.190 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:49.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:49 smithi161 ceph-mon[35485]: pgmap v356: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:21:49.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:49 smithi118 ceph-mon[35048]: pgmap v356: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:21:49.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.732+0000 7f6683fae640 1 -- 172.21.15.6:0/2562735036 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f667c10e4d0 msgr2=0x7f667c1108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.732+0000 7f6683fae640 1 --2- 172.21.15.6:0/2562735036 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f667c10e4d0 0x7f667c1108e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6674009ca0 tx=0x7f667402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.733+0000 7f6683fae640 1 -- 172.21.15.6:0/2562735036 shutdown_connections 2024-09-12T03:21:49.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.733+0000 7f6683fae640 1 --2- 172.21.15.6:0/2562735036 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f667c10e4d0 0x7f667c1108e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.733+0000 7f6683fae640 1 --2- 172.21.15.6:0/2562735036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f667c101360 0x7f667c10df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.733+0000 7f6683fae640 1 --2- 172.21.15.6:0/2562735036 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f667c1009b0 0x7f667c100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.733+0000 7f6683fae640 1 -- 172.21.15.6:0/2562735036 >> 172.21.15.6:0/2562735036 conn(0x7f667c0fc7d0 msgr2=0x7f667c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:49.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.733+0000 7f6683fae640 1 -- 172.21.15.6:0/2562735036 shutdown_connections 2024-09-12T03:21:49.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.733+0000 7f6683fae640 1 -- 172.21.15.6:0/2562735036 wait complete. 2024-09-12T03:21:49.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.734+0000 7f6683fae640 1 Processor -- start 2024-09-12T03:21:49.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.734+0000 7f6683fae640 1 -- start start 2024-09-12T03:21:49.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.734+0000 7f6683fae640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f667c1009b0 0x7f667c1a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.734+0000 7f6683fae640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f667c101360 0x7f667c1a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.734+0000 7f6683fae640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f667c10e4d0 0x7f667c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.734+0000 7f6683fae640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f667c114280 con 0x7f667c1009b0 2024-09-12T03:21:49.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.734+0000 7f6683fae640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f667c114100 con 0x7f667c10e4d0 2024-09-12T03:21:49.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.734+0000 7f6683fae640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f667c114400 con 0x7f667c101360 2024-09-12T03:21:49.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.734+0000 7f6681d4a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f667c1009b0 0x7f667c1a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.735+0000 7f6681d4a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f667c1009b0 0x7f667c1a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35060/0 (socket says 172.21.15.6:35060) 2024-09-12T03:21:49.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.735+0000 7f6681d4a640 1 -- 172.21.15.6:0/3242735122 learned_addr learned my addr 172.21.15.6:0/3242735122 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:49.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.735+0000 7f668254b640 1 --2- 172.21.15.6:0/3242735122 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f667c10e4d0 0x7f667c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.735+0000 7f6681549640 1 --2- 172.21.15.6:0/3242735122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f667c101360 0x7f667c1a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.735+0000 7f668254b640 1 -- 172.21.15.6:0/3242735122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f667c101360 msgr2=0x7f667c1a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.735+0000 7f668254b640 1 --2- 172.21.15.6:0/3242735122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f667c101360 0x7f667c1a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.735+0000 7f668254b640 1 -- 172.21.15.6:0/3242735122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f667c1009b0 msgr2=0x7f667c1a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.735+0000 7f668254b640 1 --2- 172.21.15.6:0/3242735122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f667c1009b0 0x7f667c1a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.735+0000 7f668254b640 1 -- 172.21.15.6:0/3242735122 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667c19ace0 con 0x7f667c10e4d0 2024-09-12T03:21:49.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.735+0000 7f6681d4a640 1 --2- 172.21.15.6:0/3242735122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f667c1009b0 0x7f667c1a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:49.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.735+0000 7f668254b640 1 --2- 172.21.15.6:0/3242735122 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f667c10e4d0 0x7f667c19a5b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f667402ade0 tx=0x7f66740319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:49.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.736+0000 7f6672ffd640 1 -- 172.21.15.6:0/3242735122 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6674003b70 con 0x7f667c10e4d0 2024-09-12T03:21:49.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.736+0000 7f6683fae640 1 -- 172.21.15.6:0/3242735122 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667c19af70 con 0x7f667c10e4d0 2024-09-12T03:21:49.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.736+0000 7f6683fae640 1 -- 172.21.15.6:0/3242735122 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f667c102b30 con 0x7f667c10e4d0 2024-09-12T03:21:49.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.736+0000 7f6672ffd640 1 -- 172.21.15.6:0/3242735122 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6674003d10 con 0x7f667c10e4d0 2024-09-12T03:21:49.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.736+0000 7f6672ffd640 1 -- 172.21.15.6:0/3242735122 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667402fcb0 con 0x7f667c10e4d0 2024-09-12T03:21:49.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.739+0000 7f6672ffd640 1 -- 172.21.15.6:0/3242735122 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f6674004050 con 0x7f667c10e4d0 2024-09-12T03:21:49.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.739+0000 7f6686387740 1 Processor -- start 2024-09-12T03:21:49.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.740+0000 7f6686387740 1 -- start start 2024-09-12T03:21:49.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.740+0000 7f6686387740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6affb40 0x557cd6afe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.740+0000 7f6686387740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557cd6afe5d0 0x557cd6afea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.740+0000 7f6686387740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6b09370 0x557cd6b0b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.740+0000 7f6686387740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x557cd6a642f0 con 0x557cd6b09370 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.740+0000 7f6686387740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x557cd6a0bcf0 con 0x557cd6afe5d0 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.740+0000 7f6686387740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557cd6a602b0 con 0x557cd6affb40 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.740+0000 7f6681549640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6b09370 0x557cd6b0b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.740+0000 7f6681549640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6b09370 0x557cd6b0b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35076/0 (socket says 172.21.15.6:35076) 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.740+0000 7f6681549640 1 -- 172.21.15.6:0/1785851527 learned_addr learned my addr 172.21.15.6:0/1785851527 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.740+0000 7f6681d4a640 1 --2- 172.21.15.6:0/1785851527 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557cd6afe5d0 0x557cd6afea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.740+0000 7f6672ffd640 1 --2- 172.21.15.6:0/3242735122 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f66680a2c90 0x7f66680a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.740+0000 7f6681549640 1 --2- 172.21.15.6:0/1785851527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6affb40 0x557cd6afe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.741+0000 7f6672ffd640 1 -- 172.21.15.6:0/3242735122 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f66740f01d0 con 0x7f667c10e4d0 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.741+0000 7f6681d4a640 1 -- 172.21.15.6:0/1785851527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6affb40 msgr2=0x557cd6afe090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:49.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.741+0000 7f6681d4a640 1 --2- 172.21.15.6:0/1785851527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6affb40 0x557cd6afe090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.741+0000 7f6681d4a640 1 -- 172.21.15.6:0/1785851527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6b09370 msgr2=0x557cd6b0b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:49.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.741+0000 7f6681d4a640 1 --2- 172.21.15.6:0/1785851527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6b09370 0x557cd6b0b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.741+0000 7f6681d4a640 1 -- 172.21.15.6:0/1785851527 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cd69f03a0 con 0x557cd6afe5d0 2024-09-12T03:21:49.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.741+0000 7f6681549640 1 --2- 172.21.15.6:0/1785851527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6b09370 0x557cd6b0b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:49.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.741+0000 7f668254b640 1 --2- 172.21.15.6:0/3242735122 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f66680a2c90 0x7f66680a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.741+0000 7f6681d4a640 1 --2- 172.21.15.6:0/1785851527 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557cd6afe5d0 0x557cd6afea10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6678009c40 tx=0x7f6678005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f82a6929701d1e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:49.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.742+0000 7f6667fff640 1 -- 172.21.15.6:0/1785851527 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678022070 con 0x557cd6afe5d0 2024-09-12T03:21:49.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.742+0000 7f6667fff640 1 -- 172.21.15.6:0/1785851527 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f667800bd30 con 0x557cd6afe5d0 2024-09-12T03:21:49.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.742+0000 7f6667fff640 1 -- 172.21.15.6:0/1785851527 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678004a00 con 0x557cd6afe5d0 2024-09-12T03:21:49.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.742+0000 7f6686387740 1 -- 172.21.15.6:0/1785851527 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557cd6afe5d0 msgr2=0x557cd6afea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.742+0000 7f6686387740 1 --2- 172.21.15.6:0/1785851527 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557cd6afe5d0 0x557cd6afea10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6678009c40 tx=0x7f6678005f90 comp rx=0 tx=0).stop 2024-09-12T03:21:49.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.743+0000 7f668254b640 1 --2- 172.21.15.6:0/3242735122 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f66680a2c90 0x7f66680a5150 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6674009bf0 tx=0x7f667403d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:49.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.743+0000 7f6686387740 1 -- 172.21.15.6:0/1785851527 shutdown_connections 2024-09-12T03:21:49.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.743+0000 7f6686387740 1 --2- 172.21.15.6:0/1785851527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6b09370 0x557cd6b0b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.743+0000 7f6686387740 1 --2- 172.21.15.6:0/1785851527 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557cd6afe5d0 0x557cd6afea10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.743+0000 7f6686387740 1 --2- 172.21.15.6:0/1785851527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6affb40 0x557cd6afe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.743+0000 7f6686387740 1 -- 172.21.15.6:0/1785851527 >> 172.21.15.6:0/1785851527 conn(0x557cd6ac7310 msgr2=0x557cd6afac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:49.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.743+0000 7f6686387740 1 -- 172.21.15.6:0/1785851527 shutdown_connections 2024-09-12T03:21:49.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.743+0000 7f6686387740 1 -- 172.21.15.6:0/1785851527 wait complete. 2024-09-12T03:21:49.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.744+0000 7f6686387740 1 -- start start 2024-09-12T03:21:49.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.744+0000 7f6686387740 1 Processor -- start 2024-09-12T03:21:49.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.745+0000 7f6686387740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6afe5d0 0x557cd6b08cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.745+0000 7f6686387740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6affb40 0x557cd6b05f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.745+0000 7f6686387740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557cd6b09370 0x557cd6b064d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.745+0000 7f6681549640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6afe5d0 0x557cd6b08cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.745+0000 7f6681549640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6afe5d0 0x557cd6b08cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35086/0 (socket says 172.21.15.6:35086) 2024-09-12T03:21:49.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.745+0000 7f6681549640 1 -- 172.21.15.6:0/1445901762 learned_addr learned my addr 172.21.15.6:0/1445901762 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:49.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.745+0000 7f6686387740 1 -- 172.21.15.6:0/1445901762 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x557cd6b0c070 con 0x557cd6afe5d0 2024-09-12T03:21:49.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.745+0000 7f6686387740 1 -- 172.21.15.6:0/1445901762 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x557cd6a1f060 con 0x557cd6b09370 2024-09-12T03:21:49.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.745+0000 7f6686387740 1 -- 172.21.15.6:0/1445901762 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557cd6b0c1f0 con 0x557cd6affb40 2024-09-12T03:21:49.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.745+0000 7f6681d4a640 1 --2- 172.21.15.6:0/1445901762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6affb40 0x557cd6b05f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.745+0000 7f6686387740 10 client.0 _ll_register_callbacks cb 0x557cd67728c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:49.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.745+0000 7f6681d4a640 1 -- 172.21.15.6:0/1445901762 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557cd6b09370 msgr2=0x557cd6b064d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:21:49.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.745+0000 7f6681d4a640 1 --2- 172.21.15.6:0/1445901762 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557cd6b09370 0x557cd6b064d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.745+0000 7f6681d4a640 1 -- 172.21.15.6:0/1445901762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6afe5d0 msgr2=0x557cd6b08cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:49.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.746+0000 7f6681d4a640 1 --2- 172.21.15.6:0/1445901762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6afe5d0 0x557cd6b08cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.746+0000 7f6681d4a640 1 -- 172.21.15.6:0/1445901762 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cd67d53c0 con 0x557cd6affb40 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.746+0000 7f6686387740 1 -- 172.21.15.6:0/1445901762 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f667800bd30 con 0x557cd6affb40 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.746+0000 7f6681549640 1 --2- 172.21.15.6:0/1445901762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6afe5d0 0x557cd6b08cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.746+0000 7f6681d4a640 1 --2- 172.21.15.6:0/1445901762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6affb40 0x557cd6b05f70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6678009c40 tx=0x7f66780048d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3c7ddf93e60a2d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.746+0000 7f66657fa640 10 client.25027 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.746+0000 7f66657fa640 1 -- 172.21.15.6:0/1445901762 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678022070 con 0x557cd6affb40 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.746+0000 7f66657fa640 1 -- 172.21.15.6:0/1445901762 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f6678004c40 con 0x557cd6affb40 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.746+0000 7f66657fa640 1 -- 172.21.15.6:0/1445901762 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6678004ec0 con 0x557cd6affb40 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.746+0000 7f6686387740 10 client.25027 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.746+0000 7f6686387740 1 -- 172.21.15.6:0/1445901762 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({fsmap=25}) -- 0x557cd6b17a10 con 0x557cd6affb40 2024-09-12T03:21:49.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.747+0000 7f66657fa640 1 -- 172.21.15.6:0/1445901762 <== mon.1 v2:172.21.15.161:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f667803a470 con 0x557cd6affb40 2024-09-12T03:21:49.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.747+0000 7f6686387740 10 client.25027 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:49.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.747+0000 7f6686387740 10 client.25027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:21:49.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.747+0000 7f6686387740 20 client.25027 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:49.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.747+0000 7f6686387740 1 --2- 172.21.15.6:0/1445901762 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557cd6b18d60 0x557cd6b1b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.747+0000 7f6686387740 4 client.25027 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:49.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.747+0000 7f6686387740 1 -- 172.21.15.6:0/1445901762 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cd6a71860 con 0x557cd6b18d60 2024-09-12T03:21:49.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.747+0000 7f6681549640 1 --2- 172.21.15.6:0/1445901762 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557cd6b18d60 0x557cd6b1b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.748+0000 7f6681549640 1 --2- 172.21.15.6:0/1445901762 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557cd6b18d60 0x557cd6b1b150 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f2c1a2f5427aaec server_cookie=48135a2f3cbf6f1d in_seq=0 out_seq=0 2024-09-12T03:21:49.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.748+0000 7f66657fa640 10 client.25027 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:49.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.749+0000 7f66657fa640 1 -- 172.21.15.6:0/1445901762 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557cd6a71860 con 0x557cd6b18d60 2024-09-12T03:21:49.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.749+0000 7f66657fa640 10 client.25027 handle_command_reply: tid=0 2024-09-12T03:21:49.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.749+0000 7f6686387740 1 client.25027 shutdown 2024-09-12T03:21:49.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.749+0000 7f6686387740 1 -- 172.21.15.6:0/1445901762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6affb40 msgr2=0x557cd6b05f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.749+0000 7f6686387740 1 --2- 172.21.15.6:0/1445901762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6affb40 0x557cd6b05f70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6678009c40 tx=0x7f66780048d0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.749+0000 7f6686387740 1 -- 172.21.15.6:0/1445901762 shutdown_connections 2024-09-12T03:21:49.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.749+0000 7f6686387740 1 --2- 172.21.15.6:0/1445901762 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557cd6b18d60 0x557cd6b1b150 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.749+0000 7f6686387740 1 --2- 172.21.15.6:0/1445901762 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557cd6b09370 0x557cd6b064d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.749+0000 7f66657fa640 0 client.25027 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:49.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.749+0000 7f6686387740 1 --2- 172.21.15.6:0/1445901762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6affb40 0x557cd6b05f70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.749+0000 7f6686387740 1 --2- 172.21.15.6:0/1445901762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6afe5d0 0x557cd6b08cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.749+0000 7f6686387740 1 -- 172.21.15.6:0/1445901762 >> 172.21.15.6:0/1445901762 conn(0x557cd6ac7310 msgr2=0x557cd6b00930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:49.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.750+0000 7f6686387740 1 -- 172.21.15.6:0/1445901762 shutdown_connections 2024-09-12T03:21:49.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.750+0000 7f6686387740 1 -- 172.21.15.6:0/1445901762 wait complete. 2024-09-12T03:21:49.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.750+0000 7f6686387740 20 client.25027 trim_cache size 0 max 16384 2024-09-12T03:21:49.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.763+0000 7f6686387740 1 Processor -- start 2024-09-12T03:21:49.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.763+0000 7f6686387740 1 -- start start 2024-09-12T03:21:49.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.763+0000 7f6686387740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6afe5d0 0x557cd6b153a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.763+0000 7f6686387740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6affb40 0x557cd6b0a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.763+0000 7f6686387740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557cd6b09370 0x557cd6b0aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.763+0000 7f6686387740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6660010bb0 con 0x557cd6affb40 2024-09-12T03:21:49.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.763+0000 7f6686387740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f666000e480 con 0x557cd6b09370 2024-09-12T03:21:49.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.763+0000 7f6686387740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f666000c2d0 con 0x557cd6afe5d0 2024-09-12T03:21:49.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.764+0000 7f6681549640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6afe5d0 0x557cd6b153a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.764+0000 7f6681549640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6afe5d0 0x557cd6b153a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:59808/0 (socket says 172.21.15.6:59808) 2024-09-12T03:21:49.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.764+0000 7f6681549640 1 -- 172.21.15.6:0/2490870025 learned_addr learned my addr 172.21.15.6:0/2490870025 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:49.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.764+0000 7f6681d4a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6affb40 0x557cd6b0a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.764+0000 7f6681549640 1 --2- 172.21.15.6:0/2490870025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557cd6b09370 0x557cd6b0aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.764+0000 7f6681d4a640 1 -- 172.21.15.6:0/2490870025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6afe5d0 msgr2=0x557cd6b153a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:49.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.764+0000 7f6681d4a640 1 --2- 172.21.15.6:0/2490870025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6afe5d0 0x557cd6b153a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.764+0000 7f6681d4a640 1 -- 172.21.15.6:0/2490870025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557cd6b09370 msgr2=0x557cd6b0aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:49.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.764+0000 7f6681d4a640 1 --2- 172.21.15.6:0/2490870025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557cd6b09370 0x557cd6b0aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.764+0000 7f6681d4a640 1 -- 172.21.15.6:0/2490870025 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cd6b06ab0 con 0x557cd6affb40 2024-09-12T03:21:49.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.765+0000 7f6681d4a640 1 --2- 172.21.15.6:0/2490870025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6affb40 0x557cd6b0a760 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x557cd6b07520 tx=0x7f6678029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48b5bcc8045fb97 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:49.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.765+0000 7f66677fe640 1 -- 172.21.15.6:0/2490870025 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678004b40 con 0x557cd6affb40 2024-09-12T03:21:49.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.765+0000 7f66677fe640 1 -- 172.21.15.6:0/2490870025 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f667800bd30 con 0x557cd6affb40 2024-09-12T03:21:49.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.765+0000 7f66677fe640 1 -- 172.21.15.6:0/2490870025 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66780044f0 con 0x557cd6affb40 2024-09-12T03:21:49.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.766+0000 7f6686387740 1 -- 172.21.15.6:0/2490870025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6affb40 msgr2=0x557cd6b0a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.766+0000 7f6686387740 1 --2- 172.21.15.6:0/2490870025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6affb40 0x557cd6b0a760 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x557cd6b07520 tx=0x7f6678029000 comp rx=0 tx=0).stop 2024-09-12T03:21:49.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.766+0000 7f6686387740 1 -- 172.21.15.6:0/2490870025 shutdown_connections 2024-09-12T03:21:49.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.766+0000 7f6686387740 1 --2- 172.21.15.6:0/2490870025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557cd6b09370 0x557cd6b0aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.766+0000 7f6686387740 1 --2- 172.21.15.6:0/2490870025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6affb40 0x557cd6b0a760 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.766+0000 7f6686387740 1 --2- 172.21.15.6:0/2490870025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6afe5d0 0x557cd6b153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.766+0000 7f6686387740 1 -- 172.21.15.6:0/2490870025 >> 172.21.15.6:0/2490870025 conn(0x557cd6ac7310 msgr2=0x557cd6b05f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:49.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.766+0000 7f6686387740 1 -- 172.21.15.6:0/2490870025 shutdown_connections 2024-09-12T03:21:49.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.767+0000 7f6686387740 1 -- 172.21.15.6:0/2490870025 wait complete. 2024-09-12T03:21:49.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.767+0000 7f6686387740 1 -- start start 2024-09-12T03:21:49.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.767+0000 7f6686387740 1 Processor -- start 2024-09-12T03:21:49.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.768+0000 7f6686387740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6afe5d0 0x557cd6b210b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.768+0000 7f6681549640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6afe5d0 0x557cd6b210b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.768+0000 7f6681549640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6afe5d0 0x557cd6b210b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35090/0 (socket says 172.21.15.6:35090) 2024-09-12T03:21:49.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.768+0000 7f6681549640 1 -- 172.21.15.6:0/542118372 learned_addr learned my addr 172.21.15.6:0/542118372 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:49.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.768+0000 7f6686387740 1 --2- 172.21.15.6:0/542118372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6affb40 0x557cd6b11510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.769+0000 7f6686387740 1 --2- 172.21.15.6:0/542118372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557cd6b09370 0x557cd6b11a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.769+0000 7f6686387740 1 -- 172.21.15.6:0/542118372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f666000c940 con 0x557cd6afe5d0 2024-09-12T03:21:49.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.769+0000 7f6686387740 1 -- 172.21.15.6:0/542118372 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f666000ddb0 con 0x557cd6b09370 2024-09-12T03:21:49.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.769+0000 7f6681d4a640 1 --2- 172.21.15.6:0/542118372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6affb40 0x557cd6b11510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.769+0000 7f6686387740 1 -- 172.21.15.6:0/542118372 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f666000d010 con 0x557cd6affb40 2024-09-12T03:21:49.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.769+0000 7f6681549640 1 -- 172.21.15.6:0/542118372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6affb40 msgr2=0x557cd6b11510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:49.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.769+0000 7f6681549640 1 --2- 172.21.15.6:0/542118372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6affb40 0x557cd6b11510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.769+0000 7f6681549640 1 -- 172.21.15.6:0/542118372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557cd6b09370 msgr2=0x557cd6b11a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:49.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.769+0000 7f6681549640 1 --2- 172.21.15.6:0/542118372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557cd6b09370 0x557cd6b11a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.769+0000 7f6681549640 1 -- 172.21.15.6:0/542118372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cd6b03210 con 0x557cd6afe5d0 2024-09-12T03:21:49.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.770+0000 7f6686387740 10 client.0 _ll_register_callbacks cb 0x7f666c00b240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:49.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.770+0000 7f6681549640 1 --2- 172.21.15.6:0/542118372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6afe5d0 0x557cd6b210b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f666c008010 tx=0x7f666c017c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2133475dd6581ee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:49.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.770+0000 7f66657fa640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:21:49.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.770+0000 7f6686387740 1 -- 172.21.15.6:0/542118372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f667800bd30 con 0x557cd6afe5d0 2024-09-12T03:21:49.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.771+0000 7f66657fa640 1 -- 172.21.15.6:0/542118372 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c024070 con 0x557cd6afe5d0 2024-09-12T03:21:49.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.771+0000 7f66657fa640 1 -- 172.21.15.6:0/542118372 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f666c014b80 con 0x557cd6afe5d0 2024-09-12T03:21:49.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.771+0000 7f66657fa640 1 -- 172.21.15.6:0/542118372 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f666c014de0 con 0x557cd6afe5d0 2024-09-12T03:21:49.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.771+0000 7f6686387740 10 client.15423 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:49.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.771+0000 7f6686387740 1 -- 172.21.15.6:0/542118372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({fsmap=25}) -- 0x557cd6b1fad0 con 0x557cd6afe5d0 2024-09-12T03:21:49.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.771+0000 7f66657fa640 1 -- 172.21.15.6:0/542118372 <== mon.0 v2:172.21.15.6:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f666c015040 con 0x557cd6afe5d0 2024-09-12T03:21:49.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.772+0000 7f6686387740 10 client.15423 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:49.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.772+0000 7f6686387740 10 client.15423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:21:49.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.772+0000 7f6686387740 20 client.15423 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:49.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.772+0000 7f6686387740 1 --2- 172.21.15.6:0/542118372 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557cd6b18d60 0x557cd6b200e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.772+0000 7f6686387740 4 client.15423 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:21:49.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.772+0000 7f6686387740 1 -- 172.21.15.6:0/542118372 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557cd6a6bd70 con 0x557cd6b18d60 2024-09-12T03:21:49.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.772+0000 7f6681d4a640 1 --2- 172.21.15.6:0/542118372 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557cd6b18d60 0x557cd6b200e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.772+0000 7f6681d4a640 1 --2- 172.21.15.6:0/542118372 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557cd6b18d60 0x557cd6b200e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd8824a3941da513 server_cookie=78b450bf297319ea in_seq=0 out_seq=0 2024-09-12T03:21:49.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.773+0000 7f66657fa640 10 client.15423 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:49.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.773+0000 7f66657fa640 1 -- 172.21.15.6:0/542118372 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557cd6a6bd70 con 0x557cd6b18d60 2024-09-12T03:21:49.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.773+0000 7f66657fa640 10 client.15423 handle_command_reply: tid=0 2024-09-12T03:21:49.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.773+0000 7f6686387740 1 client.15423 shutdown 2024-09-12T03:21:49.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.773+0000 7f6686387740 1 -- 172.21.15.6:0/542118372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6afe5d0 msgr2=0x557cd6b210b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.773+0000 7f6686387740 1 --2- 172.21.15.6:0/542118372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6afe5d0 0x557cd6b210b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f666c008010 tx=0x7f666c017c90 comp rx=0 tx=0).stop 2024-09-12T03:21:49.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.774+0000 7f6686387740 1 -- 172.21.15.6:0/542118372 shutdown_connections 2024-09-12T03:21:49.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.774+0000 7f6686387740 1 --2- 172.21.15.6:0/542118372 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557cd6b18d60 0x557cd6b200e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.774+0000 7f6686387740 1 --2- 172.21.15.6:0/542118372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557cd6b09370 0x557cd6b11a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.774+0000 7f6686387740 1 --2- 172.21.15.6:0/542118372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd6affb40 0x557cd6b11510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.774+0000 7f6686387740 1 --2- 172.21.15.6:0/542118372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557cd6afe5d0 0x557cd6b210b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.774+0000 7f6686387740 1 -- 172.21.15.6:0/542118372 >> 172.21.15.6:0/542118372 conn(0x557cd6ac7310 msgr2=0x557cd6b077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:49.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.774+0000 7f66657fa640 0 client.15423 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:49.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.774+0000 7f6686387740 1 -- 172.21.15.6:0/542118372 shutdown_connections 2024-09-12T03:21:49.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.774+0000 7f6686387740 1 -- 172.21.15.6:0/542118372 wait complete. 2024-09-12T03:21:49.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.774+0000 7f6686387740 20 client.15423 trim_cache size 0 max 16384 2024-09-12T03:21:49.777 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-09-12T03:21:49.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.775+0000 7f6683fae640 1 -- 172.21.15.6:0/3242735122 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f66680a2c90 msgr2=0x7f66680a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.775+0000 7f6683fae640 1 --2- 172.21.15.6:0/3242735122 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f66680a2c90 0x7f66680a5150 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6674009bf0 tx=0x7f667403d040 comp rx=0 tx=0).stop 2024-09-12T03:21:49.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.775+0000 7f6683fae640 1 -- 172.21.15.6:0/3242735122 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f667c10e4d0 msgr2=0x7f667c19a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.775+0000 7f6683fae640 1 --2- 172.21.15.6:0/3242735122 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f667c10e4d0 0x7f667c19a5b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f667402ade0 tx=0x7f66740319f0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.775+0000 7f6683fae640 1 -- 172.21.15.6:0/3242735122 shutdown_connections 2024-09-12T03:21:49.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.775+0000 7f6683fae640 1 --2- 172.21.15.6:0/3242735122 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f66680a2c90 0x7f66680a5150 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.775+0000 7f6683fae640 1 --2- 172.21.15.6:0/3242735122 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f667c10e4d0 0x7f667c19a5b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.775+0000 7f6683fae640 1 --2- 172.21.15.6:0/3242735122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f667c101360 0x7f667c1a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.775+0000 7f6683fae640 1 --2- 172.21.15.6:0/3242735122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f667c1009b0 0x7f667c1a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.775+0000 7f6683fae640 1 -- 172.21.15.6:0/3242735122 >> 172.21.15.6:0/3242735122 conn(0x7f667c0fc7d0 msgr2=0x7f667c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:49.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.775+0000 7f6683fae640 1 -- 172.21.15.6:0/3242735122 shutdown_connections 2024-09-12T03:21:49.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:49.775+0000 7f6683fae640 1 -- 172.21.15.6:0/3242735122 wait complete. 2024-09-12T03:21:51.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:51 smithi006 ceph-mon[30991]: pgmap v357: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:21:51.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:51 smithi161 ceph-mon[35485]: pgmap v357: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:21:51.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:51 smithi118 ceph-mon[35048]: pgmap v357: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:21:51.541 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:21:51.541 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:21:51.892 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:52.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.390+0000 7fa33728b640 1 -- 172.21.15.6:0/1561541143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3301036e0 msgr2=0x7fa330107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:52.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.390+0000 7fa33728b640 1 --2- 172.21.15.6:0/1561541143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3301036e0 0x7fa330107b90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa320009930 tx=0x7fa32002a440 comp rx=0 tx=0).stop 2024-09-12T03:21:52.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.390+0000 7fa33728b640 1 -- 172.21.15.6:0/1561541143 shutdown_connections 2024-09-12T03:21:52.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.390+0000 7fa33728b640 1 --2- 172.21.15.6:0/1561541143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3301082a0 0x7fa33010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.390+0000 7fa33728b640 1 --2- 172.21.15.6:0/1561541143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3301036e0 0x7fa330107b90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.390+0000 7fa33728b640 1 --2- 172.21.15.6:0/1561541143 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa330102d30 0x7fa330103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.390+0000 7fa33728b640 1 -- 172.21.15.6:0/1561541143 >> 172.21.15.6:0/1561541143 conn(0x7fa3300fc7d0 msgr2=0x7fa3300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:52.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.390+0000 7fa33728b640 1 -- 172.21.15.6:0/1561541143 shutdown_connections 2024-09-12T03:21:52.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.390+0000 7fa33728b640 1 -- 172.21.15.6:0/1561541143 wait complete. 2024-09-12T03:21:52.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.391+0000 7fa33728b640 1 Processor -- start 2024-09-12T03:21:52.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.391+0000 7fa33728b640 1 -- start start 2024-09-12T03:21:52.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.391+0000 7fa33728b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa330102d30 0x7fa330073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:52.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.391+0000 7fa33728b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3301082a0 0x7fa33006cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:52.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.391+0000 7fa33728b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa33006d280 0x7fa33006d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:52.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.391+0000 7fa33728b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa3301120a0 con 0x7fa3301082a0 2024-09-12T03:21:52.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.391+0000 7fa33728b640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa330111f20 con 0x7fa33006d280 2024-09-12T03:21:52.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.391+0000 7fa33728b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa330112220 con 0x7fa330102d30 2024-09-12T03:21:52.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.391+0000 7fa334826640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3301082a0 0x7fa33006cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:52.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.391+0000 7fa335027640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa330102d30 0x7fa330073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:52.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.391+0000 7fa334826640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3301082a0 0x7fa33006cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35100/0 (socket says 172.21.15.6:35100) 2024-09-12T03:21:52.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.391+0000 7fa335027640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa330102d30 0x7fa330073cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:59836/0 (socket says 172.21.15.6:59836) 2024-09-12T03:21:52.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.391+0000 7fa334826640 1 -- 172.21.15.6:0/2572073168 learned_addr learned my addr 172.21.15.6:0/2572073168 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:52.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.391+0000 7fa335828640 1 --2- 172.21.15.6:0/2572073168 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa33006d280 0x7fa33006d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:52.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.392+0000 7fa334826640 1 -- 172.21.15.6:0/2572073168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa330102d30 msgr2=0x7fa330073cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:52.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.392+0000 7fa334826640 1 --2- 172.21.15.6:0/2572073168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa330102d30 0x7fa330073cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.392+0000 7fa334826640 1 -- 172.21.15.6:0/2572073168 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa33006d280 msgr2=0x7fa33006d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:52.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.392+0000 7fa334826640 1 --2- 172.21.15.6:0/2572073168 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa33006d280 0x7fa33006d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.392+0000 7fa334826640 1 -- 172.21.15.6:0/2572073168 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa330071980 con 0x7fa3301082a0 2024-09-12T03:21:52.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.392+0000 7fa335027640 1 --2- 172.21.15.6:0/2572073168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa330102d30 0x7fa330073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:52.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.392+0000 7fa334826640 1 --2- 172.21.15.6:0/2572073168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3301082a0 0x7fa33006cd40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa320009230 tx=0x7fa320004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:52.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.392+0000 7fa3267fc640 1 -- 172.21.15.6:0/2572073168 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa320040030 con 0x7fa3301082a0 2024-09-12T03:21:52.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.392+0000 7fa3267fc640 1 -- 172.21.15.6:0/2572073168 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa320005390 con 0x7fa3301082a0 2024-09-12T03:21:52.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.392+0000 7fa33728b640 1 -- 172.21.15.6:0/2572073168 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa330071c10 con 0x7fa3301082a0 2024-09-12T03:21:52.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.393+0000 7fa3267fc640 1 -- 172.21.15.6:0/2572073168 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32003c800 con 0x7fa3301082a0 2024-09-12T03:21:52.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.393+0000 7fa33728b640 1 -- 172.21.15.6:0/2572073168 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3300720c0 con 0x7fa3301082a0 2024-09-12T03:21:52.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.396+0000 7fa3267fc640 1 -- 172.21.15.6:0/2572073168 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fa320043030 con 0x7fa3301082a0 2024-09-12T03:21:52.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.396+0000 7fa339664740 1 Processor -- start 2024-09-12T03:21:52.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.397+0000 7fa339664740 1 -- start start 2024-09-12T03:21:52.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.397+0000 7fa339664740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2d9615b60 0x55a2d9615f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:52.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.397+0000 7fa339664740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2d9616480 0x55a2d9620800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:52.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.397+0000 7fa339664740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9620d40 0x55a2d9623130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:52.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.397+0000 7fa339664740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a2d957b280 con 0x55a2d9620d40 2024-09-12T03:21:52.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.397+0000 7fa339664740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55a2d9522b10 con 0x55a2d9616480 2024-09-12T03:21:52.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.397+0000 7fa339664740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a2d9577240 con 0x55a2d9615b60 2024-09-12T03:21:52.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.397+0000 7fa334826640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9620d40 0x55a2d9623130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:52.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.397+0000 7fa334826640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9620d40 0x55a2d9623130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35116/0 (socket says 172.21.15.6:35116) 2024-09-12T03:21:52.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.397+0000 7fa334826640 1 -- 172.21.15.6:0/1084341400 learned_addr learned my addr 172.21.15.6:0/1084341400 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:52.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.397+0000 7fa335828640 1 --2- 172.21.15.6:0/1084341400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2d9615b60 0x55a2d9615f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:52.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.398+0000 7fa335027640 1 --2- 172.21.15.6:0/1084341400 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2d9616480 0x55a2d9620800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:52.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.398+0000 7fa334826640 1 -- 172.21.15.6:0/1084341400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2d9615b60 msgr2=0x55a2d9615f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:52.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.398+0000 7fa334826640 1 --2- 172.21.15.6:0/1084341400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2d9615b60 0x55a2d9615f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.398+0000 7fa3267fc640 1 --2- 172.21.15.6:0/2572073168 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa31c0a2af0 0x7fa31c0a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:52.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.398+0000 7fa334826640 1 -- 172.21.15.6:0/1084341400 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2d9616480 msgr2=0x55a2d9620800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:52.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.398+0000 7fa334826640 1 --2- 172.21.15.6:0/1084341400 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2d9616480 0x55a2d9620800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.398+0000 7fa334826640 1 -- 172.21.15.6:0/1084341400 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2d952f1f0 con 0x55a2d9620d40 2024-09-12T03:21:52.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.398+0000 7fa335027640 1 --2- 172.21.15.6:0/1084341400 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2d9616480 0x55a2d9620800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:52.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.398+0000 7fa3267fc640 1 -- 172.21.15.6:0/2572073168 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa3200eeeb0 con 0x7fa3301082a0 2024-09-12T03:21:52.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.398+0000 7fa335828640 1 --2- 172.21.15.6:0/2572073168 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa31c0a2af0 0x7fa31c0a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:52.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.398+0000 7fa334826640 1 --2- 172.21.15.6:0/1084341400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9620d40 0x55a2d9623130 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa3200ec7d0 tx=0x7fa3200b7760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9e8d323416b997 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:52.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.398+0000 7fa335828640 1 --2- 172.21.15.6:0/2572073168 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa31c0a2af0 0x7fa31c0a4fb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa328005fd0 tx=0x7fa328014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:52.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.399+0000 7fa31b7fe640 1 -- 172.21.15.6:0/1084341400 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32004a070 con 0x55a2d9620d40 2024-09-12T03:21:52.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.399+0000 7fa31b7fe640 1 -- 172.21.15.6:0/1084341400 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3200bb690 con 0x55a2d9620d40 2024-09-12T03:21:52.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.399+0000 7fa31b7fe640 1 -- 172.21.15.6:0/1084341400 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3200bb460 con 0x55a2d9620d40 2024-09-12T03:21:52.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.399+0000 7fa339664740 1 -- 172.21.15.6:0/1084341400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9620d40 msgr2=0x55a2d9623130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:52.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.399+0000 7fa339664740 1 --2- 172.21.15.6:0/1084341400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9620d40 0x55a2d9623130 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa3200ec7d0 tx=0x7fa3200b7760 comp rx=0 tx=0).stop 2024-09-12T03:21:52.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.400+0000 7fa339664740 1 -- 172.21.15.6:0/1084341400 shutdown_connections 2024-09-12T03:21:52.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.400+0000 7fa339664740 1 --2- 172.21.15.6:0/1084341400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9620d40 0x55a2d9623130 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.400+0000 7fa339664740 1 --2- 172.21.15.6:0/1084341400 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2d9616480 0x55a2d9620800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.400+0000 7fa339664740 1 --2- 172.21.15.6:0/1084341400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2d9615b60 0x55a2d9615f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.400+0000 7fa339664740 1 -- 172.21.15.6:0/1084341400 >> 172.21.15.6:0/1084341400 conn(0x55a2d96120f0 msgr2=0x55a2d96124d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:52.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.400+0000 7fa339664740 1 -- 172.21.15.6:0/1084341400 shutdown_connections 2024-09-12T03:21:52.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.400+0000 7fa339664740 1 -- 172.21.15.6:0/1084341400 wait complete. 2024-09-12T03:21:52.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.400+0000 7fa339664740 1 -- start start 2024-09-12T03:21:52.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.400+0000 7fa339664740 1 Processor -- start 2024-09-12T03:21:52.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.401+0000 7fa339664740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2d9615b60 0x55a2d961dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:52.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.401+0000 7fa339664740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9616480 0x55a2d961e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:52.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.401+0000 7fa339664740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2d9620d40 0x55a2d961e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:52.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.401+0000 7fa339664740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a2d96236d0 con 0x55a2d9616480 2024-09-12T03:21:52.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.401+0000 7fa339664740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55a2d9535f10 con 0x55a2d9615b60 2024-09-12T03:21:52.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.401+0000 7fa339664740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a2d9623850 con 0x55a2d9620d40 2024-09-12T03:21:52.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.401+0000 7fa334826640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9616480 0x55a2d961e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:52.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.401+0000 7fa334826640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9616480 0x55a2d961e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35130/0 (socket says 172.21.15.6:35130) 2024-09-12T03:21:52.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.401+0000 7fa339664740 10 client.0 _ll_register_callbacks cb 0x55a2d94fea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:52.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.401+0000 7fa334826640 1 -- 172.21.15.6:0/297183382 learned_addr learned my addr 172.21.15.6:0/297183382 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:52.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.401+0000 7fa335027640 1 --2- 172.21.15.6:0/297183382 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2d9615b60 0x55a2d961dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:52.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.401+0000 7fa335828640 1 --2- 172.21.15.6:0/297183382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2d9620d40 0x55a2d961e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:52.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.401+0000 7fa334826640 1 -- 172.21.15.6:0/297183382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2d9620d40 msgr2=0x55a2d961e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:52.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.401+0000 7fa334826640 1 --2- 172.21.15.6:0/297183382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2d9620d40 0x55a2d961e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.401+0000 7fa334826640 1 -- 172.21.15.6:0/297183382 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2d9615b60 msgr2=0x55a2d961dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:52.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.401+0000 7fa334826640 1 --2- 172.21.15.6:0/297183382 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2d9615b60 0x55a2d961dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.401+0000 7fa334826640 1 -- 172.21.15.6:0/297183382 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2d92ec6a0 con 0x55a2d9616480 2024-09-12T03:21:52.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.402+0000 7fa334826640 1 --2- 172.21.15.6:0/297183382 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9616480 0x55a2d961e150 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa3200b7af0 tx=0x7fa320005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9672f9ee5323b04f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:52.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.402+0000 7fa318ff9640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:21:52.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.402+0000 7fa318ff9640 1 -- 172.21.15.6:0/297183382 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32004a040 con 0x55a2d9616480 2024-09-12T03:21:52.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.402+0000 7fa339664740 1 -- 172.21.15.6:0/297183382 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3200bb690 con 0x55a2d9616480 2024-09-12T03:21:52.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.402+0000 7fa318ff9640 1 -- 172.21.15.6:0/297183382 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa3200ecae0 con 0x55a2d9616480 2024-09-12T03:21:52.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.402+0000 7fa318ff9640 1 -- 172.21.15.6:0/297183382 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa3200bdb80 con 0x55a2d9616480 2024-09-12T03:21:52.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.402+0000 7fa339664740 10 client.15435 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:52.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.402+0000 7fa339664740 1 -- 172.21.15.6:0/297183382 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a2d96236d0 con 0x55a2d9616480 2024-09-12T03:21:52.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.402+0000 7fa318ff9640 1 -- 172.21.15.6:0/297183382 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3200bb690 con 0x55a2d9616480 2024-09-12T03:21:52.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.403+0000 7fa318ff9640 1 -- 172.21.15.6:0/297183382 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa320005540 con 0x55a2d9616480 2024-09-12T03:21:52.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.403+0000 7fa339664740 10 client.15435 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:52.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.403+0000 7fa339664740 10 client.15435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:21:52.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.403+0000 7fa339664740 20 client.15435 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:52.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.403+0000 7fa339664740 1 --2- 172.21.15.6:0/297183382 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a2d9630330 0x55a2d9632720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:52.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.403+0000 7fa339664740 4 client.15435 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:52.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.403+0000 7fa339664740 1 -- 172.21.15.6:0/297183382 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2d95887f0 con 0x55a2d9630330 2024-09-12T03:21:52.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.403+0000 7fa335027640 1 --2- 172.21.15.6:0/297183382 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a2d9630330 0x55a2d9632720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:52.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.404+0000 7fa318ff9640 10 client.15435 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:52.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.404+0000 7fa335027640 1 --2- 172.21.15.6:0/297183382 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a2d9630330 0x55a2d9632720 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=743203412e20f338 server_cookie=ccfd3dc7d7d49692 in_seq=0 out_seq=0 2024-09-12T03:21:52.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.405+0000 7fa318ff9640 1 -- 172.21.15.6:0/297183382 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2d95887f0 con 0x55a2d9630330 2024-09-12T03:21:52.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.405+0000 7fa318ff9640 10 client.15435 handle_command_reply: tid=0 2024-09-12T03:21:52.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.405+0000 7fa339664740 1 client.15435 shutdown 2024-09-12T03:21:52.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.405+0000 7fa339664740 1 -- 172.21.15.6:0/297183382 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9616480 msgr2=0x55a2d961e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:52.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.405+0000 7fa339664740 1 --2- 172.21.15.6:0/297183382 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9616480 0x55a2d961e150 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa3200b7af0 tx=0x7fa320005fd0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.405+0000 7fa339664740 1 -- 172.21.15.6:0/297183382 shutdown_connections 2024-09-12T03:21:52.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.405+0000 7fa339664740 1 --2- 172.21.15.6:0/297183382 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a2d9630330 0x55a2d9632720 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.405+0000 7fa339664740 1 --2- 172.21.15.6:0/297183382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2d9620d40 0x55a2d961e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.405+0000 7fa339664740 1 --2- 172.21.15.6:0/297183382 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9616480 0x55a2d961e150 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.405+0000 7fa339664740 1 --2- 172.21.15.6:0/297183382 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2d9615b60 0x55a2d961dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.405+0000 7fa318ff9640 0 client.15435 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:52.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.405+0000 7fa339664740 1 -- 172.21.15.6:0/297183382 >> 172.21.15.6:0/297183382 conn(0x55a2d96120f0 msgr2=0x55a2d9617dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:52.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.405+0000 7fa339664740 1 -- 172.21.15.6:0/297183382 shutdown_connections 2024-09-12T03:21:52.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.405+0000 7fa339664740 1 -- 172.21.15.6:0/297183382 wait complete. 2024-09-12T03:21:52.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.405+0000 7fa339664740 20 client.15435 trim_cache size 0 max 16384 2024-09-12T03:21:52.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.439+0000 7fa339664740 1 Processor -- start 2024-09-12T03:21:52.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.439+0000 7fa339664740 1 -- start start 2024-09-12T03:21:52.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.440+0000 7fa339664740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2d9615b60 0x55a2d9619380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:52.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.440+0000 7fa339664740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2d9616480 0x55a2d9617cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:52.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.440+0000 7fa339664740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9620d40 0x55a2d9637fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:52.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.440+0000 7fa339664740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa3301b6700 con 0x55a2d9620d40 2024-09-12T03:21:52.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.440+0000 7fa339664740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa3301b3fd0 con 0x55a2d9616480 2024-09-12T03:21:52.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.440+0000 7fa339664740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa3301b1e20 con 0x55a2d9615b60 2024-09-12T03:21:52.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.440+0000 7fa335828640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9620d40 0x55a2d9637fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:52.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.440+0000 7fa335828640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9620d40 0x55a2d9637fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35136/0 (socket says 172.21.15.6:35136) 2024-09-12T03:21:52.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.440+0000 7fa335828640 1 -- 172.21.15.6:0/3330889767 learned_addr learned my addr 172.21.15.6:0/3330889767 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:52.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.440+0000 7fa334826640 1 --2- 172.21.15.6:0/3330889767 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2d9616480 0x55a2d9617cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:52.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.440+0000 7fa335027640 1 --2- 172.21.15.6:0/3330889767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2d9615b60 0x55a2d9619380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:52.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.440+0000 7fa335828640 1 -- 172.21.15.6:0/3330889767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2d9615b60 msgr2=0x55a2d9619380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:52.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.440+0000 7fa335828640 1 --2- 172.21.15.6:0/3330889767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2d9615b60 0x55a2d9619380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.440+0000 7fa335828640 1 -- 172.21.15.6:0/3330889767 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2d9616480 msgr2=0x55a2d9617cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:52.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.440+0000 7fa335828640 1 --2- 172.21.15.6:0/3330889767 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2d9616480 0x55a2d9617cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.440+0000 7fa335828640 1 -- 172.21.15.6:0/3330889767 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2d9611b90 con 0x55a2d9620d40 2024-09-12T03:21:52.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.440+0000 7fa335027640 1 --2- 172.21.15.6:0/3330889767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2d9615b60 0x55a2d9619380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:52.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.441+0000 7fa335828640 1 --2- 172.21.15.6:0/3330889767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9620d40 0x55a2d9637fa0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa328003cf0 tx=0x7fa328016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5326aae552a232bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:52.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.441+0000 7fa31affd640 1 -- 172.21.15.6:0/3330889767 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32802c070 con 0x55a2d9620d40 2024-09-12T03:21:52.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.441+0000 7fa31affd640 1 -- 172.21.15.6:0/3330889767 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa328004540 con 0x55a2d9620d40 2024-09-12T03:21:52.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.441+0000 7fa31affd640 1 -- 172.21.15.6:0/3330889767 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328027410 con 0x55a2d9620d40 2024-09-12T03:21:52.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.442+0000 7fa339664740 1 -- 172.21.15.6:0/3330889767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9620d40 msgr2=0x55a2d9637fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:52.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.442+0000 7fa339664740 1 --2- 172.21.15.6:0/3330889767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9620d40 0x55a2d9637fa0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa328003cf0 tx=0x7fa328016000 comp rx=0 tx=0).stop 2024-09-12T03:21:52.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.442+0000 7fa339664740 1 -- 172.21.15.6:0/3330889767 shutdown_connections 2024-09-12T03:21:52.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.442+0000 7fa339664740 1 --2- 172.21.15.6:0/3330889767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9620d40 0x55a2d9637fa0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.442+0000 7fa339664740 1 --2- 172.21.15.6:0/3330889767 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2d9616480 0x55a2d9617cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.442+0000 7fa339664740 1 --2- 172.21.15.6:0/3330889767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2d9615b60 0x55a2d9619380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.442+0000 7fa339664740 1 -- 172.21.15.6:0/3330889767 >> 172.21.15.6:0/3330889767 conn(0x55a2d96120f0 msgr2=0x55a2d96175b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:52.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.442+0000 7fa339664740 1 -- 172.21.15.6:0/3330889767 shutdown_connections 2024-09-12T03:21:52.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.442+0000 7fa339664740 1 -- 172.21.15.6:0/3330889767 wait complete. 2024-09-12T03:21:52.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.443+0000 7fa339664740 1 -- start start 2024-09-12T03:21:52.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.443+0000 7fa339664740 1 Processor -- start 2024-09-12T03:21:52.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.443+0000 7fa339664740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2d9615b60 0x55a2d9624a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:52.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.443+0000 7fa339664740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9616480 0x55a2d9624f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:52.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.443+0000 7fa339664740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2d9620d40 0x55a2d9625480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:52.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.443+0000 7fa339664740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa3301b2490 con 0x55a2d9616480 2024-09-12T03:21:52.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.443+0000 7fa339664740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa3301b3900 con 0x55a2d9615b60 2024-09-12T03:21:52.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.443+0000 7fa339664740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa3301b2b60 con 0x55a2d9620d40 2024-09-12T03:21:52.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.443+0000 7fa339664740 10 client.0 _ll_register_callbacks cb 0x7fa3200f2eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:52.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.443+0000 7fa334826640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9616480 0x55a2d9624f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:52.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.443+0000 7fa334826640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9616480 0x55a2d9624f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35144/0 (socket says 172.21.15.6:35144) 2024-09-12T03:21:52.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.443+0000 7fa334826640 1 -- 172.21.15.6:0/694406427 learned_addr learned my addr 172.21.15.6:0/694406427 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:52.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.444+0000 7fa335027640 1 --2- 172.21.15.6:0/694406427 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2d9615b60 0x55a2d9624a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:52.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.444+0000 7fa335828640 1 --2- 172.21.15.6:0/694406427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2d9620d40 0x55a2d9625480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:52.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.444+0000 7fa334826640 1 -- 172.21.15.6:0/694406427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2d9620d40 msgr2=0x55a2d9625480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:52.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.444+0000 7fa334826640 1 --2- 172.21.15.6:0/694406427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2d9620d40 0x55a2d9625480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.444+0000 7fa334826640 1 -- 172.21.15.6:0/694406427 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2d9615b60 msgr2=0x55a2d9624a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:52.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.444+0000 7fa334826640 1 --2- 172.21.15.6:0/694406427 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2d9615b60 0x55a2d9624a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.444+0000 7fa334826640 1 -- 172.21.15.6:0/694406427 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2d969a210 con 0x55a2d9616480 2024-09-12T03:21:52.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.444+0000 7fa334826640 1 --2- 172.21.15.6:0/694406427 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9616480 0x55a2d9624f40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa3200f2c40 tx=0x7fa3200f2a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee3ee15b0b98cb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:52.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.444+0000 7fa318ff9640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:21:52.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.444+0000 7fa318ff9640 1 -- 172.21.15.6:0/694406427 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32004a060 con 0x55a2d9616480 2024-09-12T03:21:52.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.444+0000 7fa318ff9640 1 -- 172.21.15.6:0/694406427 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa3200c3fc0 con 0x55a2d9616480 2024-09-12T03:21:52.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.445+0000 7fa339664740 1 -- 172.21.15.6:0/694406427 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa328004540 con 0x55a2d9616480 2024-09-12T03:21:52.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.445+0000 7fa318ff9640 1 -- 172.21.15.6:0/694406427 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa3200bddf0 con 0x55a2d9616480 2024-09-12T03:21:52.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.445+0000 7fa339664740 10 client.15441 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:52.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.445+0000 7fa339664740 1 -- 172.21.15.6:0/694406427 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a2d969a210 con 0x55a2d9616480 2024-09-12T03:21:52.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.445+0000 7fa318ff9640 1 -- 172.21.15.6:0/694406427 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3200f27d0 con 0x55a2d9616480 2024-09-12T03:21:52.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.445+0000 7fa318ff9640 1 -- 172.21.15.6:0/694406427 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa3200db2a0 con 0x55a2d9616480 2024-09-12T03:21:52.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.445+0000 7fa339664740 10 client.15441 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:52.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.446+0000 7fa339664740 10 client.15441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:21:52.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.446+0000 7fa339664740 20 client.15441 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:52.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.446+0000 7fa339664740 1 --2- 172.21.15.6:0/694406427 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a2d969b950 0x55a2d969dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:52.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.446+0000 7fa339664740 4 client.15441 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:21:52.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.446+0000 7fa339664740 1 -- 172.21.15.6:0/694406427 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fa328004540 con 0x55a2d969b950 2024-09-12T03:21:52.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.446+0000 7fa335027640 1 --2- 172.21.15.6:0/694406427 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a2d969b950 0x55a2d969dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:52.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.446+0000 7fa335027640 1 --2- 172.21.15.6:0/694406427 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a2d969b950 0x55a2d969dd70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df716c4ba137a7a8 server_cookie=d09e33f742561286 in_seq=0 out_seq=0 2024-09-12T03:21:52.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.446+0000 7fa318ff9640 10 client.15441 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:52.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.448+0000 7fa318ff9640 1 -- 172.21.15.6:0/694406427 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fa328004540 con 0x55a2d969b950 2024-09-12T03:21:52.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.448+0000 7fa318ff9640 10 client.15441 handle_command_reply: tid=0 2024-09-12T03:21:52.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.448+0000 7fa339664740 1 client.15441 shutdown 2024-09-12T03:21:52.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.448+0000 7fa339664740 1 -- 172.21.15.6:0/694406427 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9616480 msgr2=0x55a2d9624f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:52.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.448+0000 7fa339664740 1 --2- 172.21.15.6:0/694406427 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9616480 0x55a2d9624f40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa3200f2c40 tx=0x7fa3200f2a00 comp rx=0 tx=0).stop 2024-09-12T03:21:52.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.448+0000 7fa339664740 1 -- 172.21.15.6:0/694406427 shutdown_connections 2024-09-12T03:21:52.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.448+0000 7fa339664740 1 --2- 172.21.15.6:0/694406427 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a2d969b950 0x55a2d969dd70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.448+0000 7fa318ff9640 0 client.15441 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:52.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.448+0000 7fa339664740 1 --2- 172.21.15.6:0/694406427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2d9620d40 0x55a2d9625480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.448+0000 7fa339664740 1 --2- 172.21.15.6:0/694406427 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2d9616480 0x55a2d9624f40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.448+0000 7fa339664740 1 --2- 172.21.15.6:0/694406427 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2d9615b60 0x55a2d9624a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.449+0000 7fa339664740 1 -- 172.21.15.6:0/694406427 >> 172.21.15.6:0/694406427 conn(0x55a2d96120f0 msgr2=0x55a2d961faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:52.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.449+0000 7fa339664740 1 -- 172.21.15.6:0/694406427 shutdown_connections 2024-09-12T03:21:52.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.449+0000 7fa339664740 1 -- 172.21.15.6:0/694406427 wait complete. 2024-09-12T03:21:52.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.449+0000 7fa339664740 20 client.15441 trim_cache size 0 max 16384 2024-09-12T03:21:52.451 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:21:52.451 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2024-09-12T03:21:52.451 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "3525d3af-7424-48a1-84b9-c551c066120d", 2024-09-12T03:21:52.451 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2024-09-12T03:21:52.452 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:21:52.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.450+0000 7fa33728b640 1 -- 172.21.15.6:0/2572073168 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa31c0a2af0 msgr2=0x7fa31c0a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:52.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.450+0000 7fa33728b640 1 --2- 172.21.15.6:0/2572073168 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa31c0a2af0 0x7fa31c0a4fb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa328005fd0 tx=0x7fa328014040 comp rx=0 tx=0).stop 2024-09-12T03:21:52.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.450+0000 7fa33728b640 1 -- 172.21.15.6:0/2572073168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3301082a0 msgr2=0x7fa33006cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:52.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.450+0000 7fa33728b640 1 --2- 172.21.15.6:0/2572073168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3301082a0 0x7fa33006cd40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa320009230 tx=0x7fa320004f50 comp rx=0 tx=0).stop 2024-09-12T03:21:52.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.450+0000 7fa33728b640 1 -- 172.21.15.6:0/2572073168 shutdown_connections 2024-09-12T03:21:52.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.450+0000 7fa33728b640 1 --2- 172.21.15.6:0/2572073168 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa31c0a2af0 0x7fa31c0a4fb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.450+0000 7fa33728b640 1 --2- 172.21.15.6:0/2572073168 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa33006d280 0x7fa33006d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.450+0000 7fa33728b640 1 --2- 172.21.15.6:0/2572073168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3301082a0 0x7fa33006cd40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.450+0000 7fa33728b640 1 --2- 172.21.15.6:0/2572073168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa330102d30 0x7fa330073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:52.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.450+0000 7fa33728b640 1 -- 172.21.15.6:0/2572073168 >> 172.21.15.6:0/2572073168 conn(0x7fa3300fc7d0 msgr2=0x7fa3300fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:52.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.450+0000 7fa33728b640 1 -- 172.21.15.6:0/2572073168 shutdown_connections 2024-09-12T03:21:52.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:52.450+0000 7fa33728b640 1 -- 172.21.15.6:0/2572073168 wait complete. 2024-09-12T03:21:53.084 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:21:53.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:53 smithi006 ceph-mon[30991]: pgmap v358: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T03:21:53.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:53 smithi006 ceph-mon[30991]: scrub queued for path: / 2024-09-12T03:21:53.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:53 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:53.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:53 smithi006 ceph-mon[30991]: scrub summary: active paths [/] 2024-09-12T03:21:53.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:53 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:53.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:53 smithi161 ceph-mon[35485]: pgmap v358: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T03:21:53.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:53 smithi161 ceph-mon[35485]: scrub queued for path: / 2024-09-12T03:21:53.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:53 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:53.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:53 smithi161 ceph-mon[35485]: scrub summary: active paths [/] 2024-09-12T03:21:53.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:53 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:53.419 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:53.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:53 smithi118 ceph-mon[35048]: pgmap v358: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T03:21:53.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:53 smithi118 ceph-mon[35048]: scrub queued for path: / 2024-09-12T03:21:53.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:53 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:53.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:53 smithi118 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-12T03:21:53.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:53 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:54.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.033+0000 7fe19a42e640 1 -- 172.21.15.6:0/2330036927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe194102570 msgr2=0x7fe1941029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:54.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.033+0000 7fe19a42e640 1 --2- 172.21.15.6:0/2330036927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe194102570 0x7fe1941029b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe188009900 tx=0x7fe18802a480 comp rx=0 tx=0).stop 2024-09-12T03:21:54.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.033+0000 7fe19a42e640 1 -- 172.21.15.6:0/2330036927 shutdown_connections 2024-09-12T03:21:54.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.033+0000 7fe19a42e640 1 --2- 172.21.15.6:0/2330036927 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe194102ef0 0x7fe19410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.033+0000 7fe19a42e640 1 --2- 172.21.15.6:0/2330036927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe194102570 0x7fe1941029b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.033+0000 7fe19a42e640 1 --2- 172.21.15.6:0/2330036927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe194100e20 0x7fe194101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.033+0000 7fe19a42e640 1 -- 172.21.15.6:0/2330036927 >> 172.21.15.6:0/2330036927 conn(0x7fe1940fc7d0 msgr2=0x7fe1940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:54.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.033+0000 7fe19a42e640 1 -- 172.21.15.6:0/2330036927 shutdown_connections 2024-09-12T03:21:54.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.033+0000 7fe19a42e640 1 -- 172.21.15.6:0/2330036927 wait complete. 2024-09-12T03:21:54.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.034+0000 7fe19a42e640 1 Processor -- start 2024-09-12T03:21:54.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.034+0000 7fe19a42e640 1 -- start start 2024-09-12T03:21:54.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.034+0000 7fe19a42e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe194100e20 0x7fe194198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:54.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.034+0000 7fe19a42e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe194102570 0x7fe194198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:54.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.035+0000 7fe19a42e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe194102ef0 0x7fe19419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:54.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.035+0000 7fe19a42e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe194110f80 con 0x7fe194102ef0 2024-09-12T03:21:54.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.035+0000 7fe19a42e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe194110e00 con 0x7fe194102570 2024-09-12T03:21:54.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.035+0000 7fe19a42e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe194111100 con 0x7fe194100e20 2024-09-12T03:21:54.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.035+0000 7fe1989cb640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe194102ef0 0x7fe19419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:54.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.035+0000 7fe1937fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe194102570 0x7fe194198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:54.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.035+0000 7fe1989cb640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe194102ef0 0x7fe19419ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35160/0 (socket says 172.21.15.6:35160) 2024-09-12T03:21:54.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.035+0000 7fe1989cb640 1 -- 172.21.15.6:0/1730207371 learned_addr learned my addr 172.21.15.6:0/1730207371 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:54.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.035+0000 7fe1989cb640 1 -- 172.21.15.6:0/1730207371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe194100e20 msgr2=0x7fe194198580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:21:54.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.035+0000 7fe1989cb640 1 --2- 172.21.15.6:0/1730207371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe194100e20 0x7fe194198580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.035+0000 7fe1989cb640 1 -- 172.21.15.6:0/1730207371 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe194102570 msgr2=0x7fe194198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:54.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.035+0000 7fe1989cb640 1 --2- 172.21.15.6:0/1730207371 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe194102570 0x7fe194198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.035+0000 7fe1989cb640 1 -- 172.21.15.6:0/1730207371 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19419d530 con 0x7fe194102ef0 2024-09-12T03:21:54.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.035+0000 7fe1989cb640 1 --2- 172.21.15.6:0/1730207371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe194102ef0 0x7fe19419ce50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00eaa0 tx=0x7fe18c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:54.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.036+0000 7fe1917fa640 1 -- 172.21.15.6:0/1730207371 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c00cdb0 con 0x7fe194102ef0 2024-09-12T03:21:54.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.036+0000 7fe19a42e640 1 -- 172.21.15.6:0/1730207371 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19419d820 con 0x7fe194102ef0 2024-09-12T03:21:54.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.036+0000 7fe1917fa640 1 -- 172.21.15.6:0/1730207371 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe18c005020 con 0x7fe194102ef0 2024-09-12T03:21:54.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.036+0000 7fe19a42e640 1 -- 172.21.15.6:0/1730207371 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1941a5100 con 0x7fe194102ef0 2024-09-12T03:21:54.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.036+0000 7fe1917fa640 1 -- 172.21.15.6:0/1730207371 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c01f060 con 0x7fe194102ef0 2024-09-12T03:21:54.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.038+0000 7fe1917fa640 1 -- 172.21.15.6:0/1730207371 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fe18c0054b0 con 0x7fe194102ef0 2024-09-12T03:21:54.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.040+0000 7fe1917fa640 1 --2- 172.21.15.6:0/1730207371 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe17c0a70a0 0x7fe17c0a9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:54.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.040+0000 7fe1917fa640 1 -- 172.21.15.6:0/1730207371 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe18c0cf310 con 0x7fe194102ef0 2024-09-12T03:21:54.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.041+0000 7fe193fff640 1 --2- 172.21.15.6:0/1730207371 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe17c0a70a0 0x7fe17c0a9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:54.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.041+0000 7fe19c807740 1 Processor -- start 2024-09-12T03:21:54.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.042+0000 7fe19c807740 1 -- start start 2024-09-12T03:21:54.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.042+0000 7fe193fff640 1 --2- 172.21.15.6:0/1730207371 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe17c0a70a0 0x7fe17c0a9560 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe184009a60 tx=0x7fe184005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:54.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.042+0000 7fe19c807740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0b4a0 0x55609ac0b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:54.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.042+0000 7fe19c807740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55609ac0bdc0 0x55609ac16140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:54.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.042+0000 7fe19c807740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac16680 0x55609ac18a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:54.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.042+0000 7fe19c807740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55609ab712e0 con 0x55609ac16680 2024-09-12T03:21:54.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.042+0000 7fe19c807740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55609ab18ce0 con 0x55609ac0b4a0 2024-09-12T03:21:54.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.042+0000 7fe19c807740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55609ab6d2a0 con 0x55609ac0bdc0 2024-09-12T03:21:54.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.042+0000 7fe1989cb640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac16680 0x55609ac18a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:54.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.042+0000 7fe1989cb640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac16680 0x55609ac18a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35168/0 (socket says 172.21.15.6:35168) 2024-09-12T03:21:54.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.042+0000 7fe1989cb640 1 -- 172.21.15.6:0/2008549329 learned_addr learned my addr 172.21.15.6:0/2008549329 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:54.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.042+0000 7fe1937fe640 1 --2- 172.21.15.6:0/2008549329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55609ac0bdc0 0x55609ac16140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:54.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.043+0000 7fe193fff640 1 --2- 172.21.15.6:0/2008549329 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0b4a0 0x55609ac0b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:54.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.043+0000 7fe1989cb640 1 -- 172.21.15.6:0/2008549329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55609ac0bdc0 msgr2=0x55609ac16140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:54.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.043+0000 7fe1989cb640 1 --2- 172.21.15.6:0/2008549329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55609ac0bdc0 0x55609ac16140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.043+0000 7fe1989cb640 1 -- 172.21.15.6:0/2008549329 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0b4a0 msgr2=0x55609ac0b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:54.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.043+0000 7fe1989cb640 1 --2- 172.21.15.6:0/2008549329 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0b4a0 0x55609ac0b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.043+0000 7fe1989cb640 1 -- 172.21.15.6:0/2008549329 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55609aafd440 con 0x55609ac16680 2024-09-12T03:21:54.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.043+0000 7fe193fff640 1 --2- 172.21.15.6:0/2008549329 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0b4a0 0x55609ac0b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:54.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.043+0000 7fe1989cb640 1 --2- 172.21.15.6:0/2008549329 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac16680 0x55609ac18a70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe18c09b970 tx=0x7fe18c09c220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319b59ad55cc6ad6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:54.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.043+0000 7fe17a7fc640 1 -- 172.21.15.6:0/2008549329 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c0b4070 con 0x55609ac16680 2024-09-12T03:21:54.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.043+0000 7fe17a7fc640 1 -- 172.21.15.6:0/2008549329 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe18c0a4650 con 0x55609ac16680 2024-09-12T03:21:54.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.043+0000 7fe17a7fc640 1 -- 172.21.15.6:0/2008549329 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c09b090 con 0x55609ac16680 2024-09-12T03:21:54.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.044+0000 7fe19c807740 1 -- 172.21.15.6:0/2008549329 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac16680 msgr2=0x55609ac18a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:54.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.044+0000 7fe19c807740 1 --2- 172.21.15.6:0/2008549329 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac16680 0x55609ac18a70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe18c09b970 tx=0x7fe18c09c220 comp rx=0 tx=0).stop 2024-09-12T03:21:54.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.044+0000 7fe19c807740 1 -- 172.21.15.6:0/2008549329 shutdown_connections 2024-09-12T03:21:54.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.044+0000 7fe19c807740 1 --2- 172.21.15.6:0/2008549329 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac16680 0x55609ac18a70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.044+0000 7fe19c807740 1 --2- 172.21.15.6:0/2008549329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55609ac0bdc0 0x55609ac16140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.044+0000 7fe19c807740 1 --2- 172.21.15.6:0/2008549329 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0b4a0 0x55609ac0b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.044+0000 7fe19c807740 1 -- 172.21.15.6:0/2008549329 >> 172.21.15.6:0/2008549329 conn(0x55609ac07a30 msgr2=0x55609ac07e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:54.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.045+0000 7fe19c807740 1 -- 172.21.15.6:0/2008549329 shutdown_connections 2024-09-12T03:21:54.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.045+0000 7fe19c807740 1 -- 172.21.15.6:0/2008549329 wait complete. 2024-09-12T03:21:54.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.045+0000 7fe19c807740 1 -- start start 2024-09-12T03:21:54.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.045+0000 7fe19c807740 1 Processor -- start 2024-09-12T03:21:54.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.046+0000 7fe19c807740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0b4a0 0x55609ac16070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:54.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.046+0000 7fe19c807740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac0bdc0 0x55609ac13320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:54.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.046+0000 7fe19c807740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55609ac16680 0x55609ac13910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:54.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.046+0000 7fe19c807740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55609ac19250 con 0x55609ac0bdc0 2024-09-12T03:21:54.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.046+0000 7fe19c807740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55609ab2c050 con 0x55609ac0b4a0 2024-09-12T03:21:54.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.046+0000 7fe19c807740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55609ac193d0 con 0x55609ac16680 2024-09-12T03:21:54.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.046+0000 7fe19c807740 10 client.0 _ll_register_callbacks cb 0x55609ab6c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:54.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.046+0000 7fe193fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0b4a0 0x55609ac16070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:54.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.046+0000 7fe193fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0b4a0 0x55609ac16070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:53854/0 (socket says 172.21.15.6:53854) 2024-09-12T03:21:54.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.046+0000 7fe193fff640 1 -- 172.21.15.6:0/2899169215 learned_addr learned my addr 172.21.15.6:0/2899169215 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:54.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.046+0000 7fe1989cb640 1 --2- 172.21.15.6:0/2899169215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55609ac16680 0x55609ac13910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:54.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.046+0000 7fe1937fe640 1 --2- 172.21.15.6:0/2899169215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac0bdc0 0x55609ac13320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:54.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.046+0000 7fe193fff640 1 -- 172.21.15.6:0/2899169215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55609ac16680 msgr2=0x55609ac13910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:54.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.046+0000 7fe193fff640 1 --2- 172.21.15.6:0/2899169215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55609ac16680 0x55609ac13910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.046+0000 7fe193fff640 1 -- 172.21.15.6:0/2899169215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac0bdc0 msgr2=0x55609ac13320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:54.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.046+0000 7fe193fff640 1 --2- 172.21.15.6:0/2899169215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac0bdc0 0x55609ac13320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.046+0000 7fe193fff640 1 -- 172.21.15.6:0/2899169215 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55609a8e23c0 con 0x55609ac0b4a0 2024-09-12T03:21:54.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.046+0000 7fe1937fe640 1 --2- 172.21.15.6:0/2899169215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac0bdc0 0x55609ac13320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:54.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.047+0000 7fe193fff640 1 --2- 172.21.15.6:0/2899169215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0b4a0 0x55609ac16070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe184010730 tx=0x7fe184004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48b225a22cc217fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:54.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.047+0000 7fe1737fe640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:21:54.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.047+0000 7fe1737fe640 1 -- 172.21.15.6:0/2899169215 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18401c9c0 con 0x55609ac0b4a0 2024-09-12T03:21:54.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.047+0000 7fe19c807740 1 -- 172.21.15.6:0/2899169215 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe18c0a4650 con 0x55609ac0b4a0 2024-09-12T03:21:54.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.047+0000 7fe1737fe640 1 -- 172.21.15.6:0/2899169215 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe18401cb60 con 0x55609ac0b4a0 2024-09-12T03:21:54.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.047+0000 7fe1737fe640 1 -- 172.21.15.6:0/2899169215 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe18401cd60 con 0x55609ac0b4a0 2024-09-12T03:21:54.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.047+0000 7fe19c807740 10 client.25040 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:54.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.047+0000 7fe19c807740 1 -- 172.21.15.6:0/2899169215 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55609ac24d60 con 0x55609ac0b4a0 2024-09-12T03:21:54.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.047+0000 7fe1737fe640 1 -- 172.21.15.6:0/2899169215 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe184028960 con 0x55609ac0b4a0 2024-09-12T03:21:54.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.048+0000 7fe1737fe640 1 -- 172.21.15.6:0/2899169215 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe18401d7a0 con 0x55609ac0b4a0 2024-09-12T03:21:54.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.048+0000 7fe19c807740 10 client.25040 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:54.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.048+0000 7fe19c807740 10 client.25040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:21:54.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.048+0000 7fe19c807740 20 client.25040 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:54.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.048+0000 7fe19c807740 1 --2- 172.21.15.6:0/2899169215 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55609ac26170 0x55609ac28560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:54.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.048+0000 7fe19c807740 4 client.25040 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:54.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.048+0000 7fe19c807740 1 -- 172.21.15.6:0/2899169215 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe18c0a4650 con 0x55609ac26170 2024-09-12T03:21:54.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.048+0000 7fe1937fe640 1 --2- 172.21.15.6:0/2899169215 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55609ac26170 0x55609ac28560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:54.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.049+0000 7fe1937fe640 1 --2- 172.21.15.6:0/2899169215 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55609ac26170 0x55609ac28560 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7842ec550249932c server_cookie=582939e9148e5961 in_seq=0 out_seq=0 2024-09-12T03:21:54.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.049+0000 7fe1737fe640 10 client.25040 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:54.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.049+0000 7fe1737fe640 1 -- 172.21.15.6:0/2899169215 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe18c0a4650 con 0x55609ac26170 2024-09-12T03:21:54.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.049+0000 7fe1737fe640 10 client.25040 handle_command_reply: tid=0 2024-09-12T03:21:54.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.049+0000 7fe19c807740 1 client.25040 shutdown 2024-09-12T03:21:54.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.050+0000 7fe19c807740 1 -- 172.21.15.6:0/2899169215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0b4a0 msgr2=0x55609ac16070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:54.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.050+0000 7fe19c807740 1 --2- 172.21.15.6:0/2899169215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0b4a0 0x55609ac16070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe184010730 tx=0x7fe184004730 comp rx=0 tx=0).stop 2024-09-12T03:21:54.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.050+0000 7fe19c807740 1 -- 172.21.15.6:0/2899169215 shutdown_connections 2024-09-12T03:21:54.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.050+0000 7fe19c807740 1 --2- 172.21.15.6:0/2899169215 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55609ac26170 0x55609ac28560 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.050+0000 7fe1737fe640 0 client.25040 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:54.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.050+0000 7fe19c807740 1 --2- 172.21.15.6:0/2899169215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55609ac16680 0x55609ac13910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.050+0000 7fe19c807740 1 --2- 172.21.15.6:0/2899169215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac0bdc0 0x55609ac13320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.050+0000 7fe19c807740 1 --2- 172.21.15.6:0/2899169215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0b4a0 0x55609ac16070 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.050+0000 7fe19c807740 1 -- 172.21.15.6:0/2899169215 >> 172.21.15.6:0/2899169215 conn(0x55609ac07a30 msgr2=0x55609ac0d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:54.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.050+0000 7fe19c807740 1 -- 172.21.15.6:0/2899169215 shutdown_connections 2024-09-12T03:21:54.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.050+0000 7fe19c807740 1 -- 172.21.15.6:0/2899169215 wait complete. 2024-09-12T03:21:54.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.050+0000 7fe19c807740 20 client.25040 trim_cache size 0 max 16384 2024-09-12T03:21:54.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.083+0000 7fe19c807740 1 Processor -- start 2024-09-12T03:21:54.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.083+0000 7fe19c807740 1 -- start start 2024-09-12T03:21:54.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.084+0000 7fe19c807740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0b4a0 0x55609ac224b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:54.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.084+0000 7fe19c807740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55609ac0bdc0 0x55609ac2de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:54.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.084+0000 7fe19c807740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac16680 0x55609ac2e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:54.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.084+0000 7fe19c807740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe1941adc40 con 0x55609ac16680 2024-09-12T03:21:54.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.084+0000 7fe19c807740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe1941ab510 con 0x55609ac0b4a0 2024-09-12T03:21:54.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.084+0000 7fe19c807740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe1941a9380 con 0x55609ac0bdc0 2024-09-12T03:21:54.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.084+0000 7fe1989cb640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac16680 0x55609ac2e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:54.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.084+0000 7fe1989cb640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac16680 0x55609ac2e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35192/0 (socket says 172.21.15.6:35192) 2024-09-12T03:21:54.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.084+0000 7fe1989cb640 1 -- 172.21.15.6:0/996253587 learned_addr learned my addr 172.21.15.6:0/996253587 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:54.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.084+0000 7fe193fff640 1 --2- 172.21.15.6:0/996253587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0b4a0 0x55609ac224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:54.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.084+0000 7fe1937fe640 1 --2- 172.21.15.6:0/996253587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55609ac0bdc0 0x55609ac2de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:54.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.084+0000 7fe193fff640 1 -- 172.21.15.6:0/996253587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55609ac0bdc0 msgr2=0x55609ac2de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:54.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.084+0000 7fe193fff640 1 --2- 172.21.15.6:0/996253587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55609ac0bdc0 0x55609ac2de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.084+0000 7fe193fff640 1 -- 172.21.15.6:0/996253587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac16680 msgr2=0x55609ac2e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:54.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.084+0000 7fe193fff640 1 --2- 172.21.15.6:0/996253587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac16680 0x55609ac2e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.084+0000 7fe193fff640 1 -- 172.21.15.6:0/996253587 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55609ac122d0 con 0x55609ac0b4a0 2024-09-12T03:21:54.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.085+0000 7fe193fff640 1 --2- 172.21.15.6:0/996253587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0b4a0 0x55609ac224b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe184013040 tx=0x7fe18401cfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0fdeceff9a8c4a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:54.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.085+0000 7fe179ffb640 1 -- 172.21.15.6:0/996253587 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18401c9c0 con 0x55609ac0b4a0 2024-09-12T03:21:54.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.085+0000 7fe179ffb640 1 -- 172.21.15.6:0/996253587 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe184037070 con 0x55609ac0b4a0 2024-09-12T03:21:54.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.085+0000 7fe179ffb640 1 -- 172.21.15.6:0/996253587 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184026bc0 con 0x55609ac0b4a0 2024-09-12T03:21:54.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.086+0000 7fe19c807740 1 -- 172.21.15.6:0/996253587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0b4a0 msgr2=0x55609ac224b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:54.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.086+0000 7fe19c807740 1 --2- 172.21.15.6:0/996253587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0b4a0 0x55609ac224b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe184013040 tx=0x7fe18401cfd0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.086+0000 7fe19c807740 1 -- 172.21.15.6:0/996253587 shutdown_connections 2024-09-12T03:21:54.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.086+0000 7fe19c807740 1 --2- 172.21.15.6:0/996253587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac16680 0x55609ac2e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.086+0000 7fe19c807740 1 --2- 172.21.15.6:0/996253587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55609ac0bdc0 0x55609ac2de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.086+0000 7fe19c807740 1 --2- 172.21.15.6:0/996253587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0b4a0 0x55609ac224b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.086+0000 7fe19c807740 1 -- 172.21.15.6:0/996253587 >> 172.21.15.6:0/996253587 conn(0x55609ac07a30 msgr2=0x55609ac0e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:54.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.086+0000 7fe19c807740 1 -- 172.21.15.6:0/996253587 shutdown_connections 2024-09-12T03:21:54.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.086+0000 7fe19c807740 1 -- 172.21.15.6:0/996253587 wait complete. 2024-09-12T03:21:54.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.086+0000 7fe19c807740 1 -- start start 2024-09-12T03:21:54.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.086+0000 7fe19c807740 1 Processor -- start 2024-09-12T03:21:54.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.087+0000 7fe19c807740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac0b4a0 0x55609ac1a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:54.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.087+0000 7fe193fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac0b4a0 0x55609ac1a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:54.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.087+0000 7fe193fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac0b4a0 0x55609ac1a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35206/0 (socket says 172.21.15.6:35206) 2024-09-12T03:21:54.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.087+0000 7fe193fff640 1 -- 172.21.15.6:0/2275174394 learned_addr learned my addr 172.21.15.6:0/2275174394 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:21:54.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.087+0000 7fe19c807740 1 --2- 172.21.15.6:0/2275174394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0bdc0 0x55609ac1a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:54.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.087+0000 7fe19c807740 1 --2- 172.21.15.6:0/2275174394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55609ac16680 0x55609ac1ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:54.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.087+0000 7fe19c807740 1 -- 172.21.15.6:0/2275174394 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe1941a99d0 con 0x55609ac0b4a0 2024-09-12T03:21:54.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.087+0000 7fe19c807740 1 -- 172.21.15.6:0/2275174394 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe1941aae40 con 0x55609ac0bdc0 2024-09-12T03:21:54.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.087+0000 7fe19c807740 1 -- 172.21.15.6:0/2275174394 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe1941aa0a0 con 0x55609ac16680 2024-09-12T03:21:54.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.087+0000 7fe19c807740 10 client.0 _ll_register_callbacks cb 0x7fe184014050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:54.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.087+0000 7fe1989cb640 1 --2- 172.21.15.6:0/2275174394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55609ac16680 0x55609ac1ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:54.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.087+0000 7fe193fff640 1 -- 172.21.15.6:0/2275174394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55609ac16680 msgr2=0x55609ac1ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:54.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.087+0000 7fe193fff640 1 --2- 172.21.15.6:0/2275174394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55609ac16680 0x55609ac1ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.087+0000 7fe193fff640 1 -- 172.21.15.6:0/2275174394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0bdc0 msgr2=0x55609ac1a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:54.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.087+0000 7fe193fff640 1 --2- 172.21.15.6:0/2275174394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0bdc0 0x55609ac1a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.087+0000 7fe193fff640 1 -- 172.21.15.6:0/2275174394 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55609ac13c80 con 0x55609ac0b4a0 2024-09-12T03:21:54.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.087+0000 7fe1989cb640 1 --2- 172.21.15.6:0/2275174394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55609ac16680 0x55609ac1ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:54.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.087+0000 7fe193fff640 1 --2- 172.21.15.6:0/2275174394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac0b4a0 0x55609ac1a2b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe1840267a0 tx=0x7fe18401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2b59ae7d6f9ae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:54.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.087+0000 7fe1737fe640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:21:54.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.088+0000 7fe1737fe640 1 -- 172.21.15.6:0/2275174394 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184044020 con 0x55609ac0b4a0 2024-09-12T03:21:54.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.088+0000 7fe1737fe640 1 -- 172.21.15.6:0/2275174394 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe184026d10 con 0x55609ac0b4a0 2024-09-12T03:21:54.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.088+0000 7fe19c807740 1 -- 172.21.15.6:0/2275174394 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe184037070 con 0x55609ac0b4a0 2024-09-12T03:21:54.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.089+0000 7fe1737fe640 1 -- 172.21.15.6:0/2275174394 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe18401d430 con 0x55609ac0b4a0 2024-09-12T03:21:54.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.089+0000 7fe19c807740 10 client.15456 fetch_fsmap learned FSMap version 25 2024-09-12T03:21:54.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.089+0000 7fe19c807740 1 -- 172.21.15.6:0/2275174394 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55609ac90e40 con 0x55609ac0b4a0 2024-09-12T03:21:54.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.089+0000 7fe1737fe640 1 -- 172.21.15.6:0/2275174394 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe184005040 con 0x55609ac0b4a0 2024-09-12T03:21:54.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.089+0000 7fe1737fe640 1 -- 172.21.15.6:0/2275174394 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe184004680 con 0x55609ac0b4a0 2024-09-12T03:21:54.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.089+0000 7fe19c807740 10 client.15456 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:21:54.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.089+0000 7fe19c807740 10 client.15456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:21:54.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.089+0000 7fe19c807740 20 client.15456 populate_metadata read hostname 'smithi006' 2024-09-12T03:21:54.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.089+0000 7fe19c807740 1 --2- 172.21.15.6:0/2275174394 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55609ac914a0 0x55609ac93890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:54.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.089+0000 7fe19c807740 4 client.15456 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:21:54.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.089+0000 7fe19c807740 1 -- 172.21.15.6:0/2275174394 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fe184037070 con 0x55609ac914a0 2024-09-12T03:21:54.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.090+0000 7fe1937fe640 1 --2- 172.21.15.6:0/2275174394 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55609ac914a0 0x55609ac93890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:54.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.090+0000 7fe1737fe640 10 client.15456 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:54.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.091+0000 7fe1937fe640 1 --2- 172.21.15.6:0/2275174394 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55609ac914a0 0x55609ac93890 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9f3a39ead2d264 server_cookie=a8a5faf8af6a6f83 in_seq=0 out_seq=0 2024-09-12T03:21:54.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.091+0000 7fe1737fe640 1 -- 172.21.15.6:0/2275174394 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7fe184037070 con 0x55609ac914a0 2024-09-12T03:21:54.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.091+0000 7fe1737fe640 10 client.15456 handle_command_reply: tid=0 2024-09-12T03:21:54.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.091+0000 7fe19c807740 1 client.15456 shutdown 2024-09-12T03:21:54.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.091+0000 7fe19c807740 1 -- 172.21.15.6:0/2275174394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac0b4a0 msgr2=0x55609ac1a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:54.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.091+0000 7fe19c807740 1 --2- 172.21.15.6:0/2275174394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac0b4a0 0x55609ac1a2b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe1840267a0 tx=0x7fe18401ca70 comp rx=0 tx=0).stop 2024-09-12T03:21:54.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.091+0000 7fe19c807740 1 -- 172.21.15.6:0/2275174394 shutdown_connections 2024-09-12T03:21:54.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.091+0000 7fe19c807740 1 --2- 172.21.15.6:0/2275174394 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55609ac914a0 0x55609ac93890 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.091+0000 7fe19c807740 1 --2- 172.21.15.6:0/2275174394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55609ac16680 0x55609ac1ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.091+0000 7fe1737fe640 0 client.15456 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:21:54.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.091+0000 7fe19c807740 1 --2- 172.21.15.6:0/2275174394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55609ac0bdc0 0x55609ac1a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.091+0000 7fe19c807740 1 --2- 172.21.15.6:0/2275174394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55609ac0b4a0 0x55609ac1a2b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.091+0000 7fe19c807740 1 -- 172.21.15.6:0/2275174394 >> 172.21.15.6:0/2275174394 conn(0x55609ac07a30 msgr2=0x55609ac0cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:54.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.092+0000 7fe19c807740 1 -- 172.21.15.6:0/2275174394 shutdown_connections 2024-09-12T03:21:54.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.092+0000 7fe19c807740 1 -- 172.21.15.6:0/2275174394 wait complete. 2024-09-12T03:21:54.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.092+0000 7fe19c807740 20 client.15456 trim_cache size 0 max 16384 2024-09-12T03:21:54.102 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:21:54.103 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:21:54.103 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2024-09-12T03:21:54.103 INFO:teuthology.orchestra.run.smithi006.stdout: "3525d3af-7424-48a1-84b9-c551c066120d": { 2024-09-12T03:21:54.103 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-09-12T03:21:54.103 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "3525d3af-7424-48a1-84b9-c551c066120d", 2024-09-12T03:21:54.103 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2024-09-12T03:21:54.103 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:21:54.103 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:21:54.103 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:21:54.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.093+0000 7fe19a42e640 1 -- 172.21.15.6:0/1730207371 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe17c0a70a0 msgr2=0x7fe17c0a9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:54.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.093+0000 7fe19a42e640 1 --2- 172.21.15.6:0/1730207371 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe17c0a70a0 0x7fe17c0a9560 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe184009a60 tx=0x7fe184005f90 comp rx=0 tx=0).stop 2024-09-12T03:21:54.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.093+0000 7fe19a42e640 1 -- 172.21.15.6:0/1730207371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe194102ef0 msgr2=0x7fe19419ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:54.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.093+0000 7fe19a42e640 1 --2- 172.21.15.6:0/1730207371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe194102ef0 0x7fe19419ce50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00eaa0 tx=0x7fe18c00ef60 comp rx=0 tx=0).stop 2024-09-12T03:21:54.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.093+0000 7fe19a42e640 1 -- 172.21.15.6:0/1730207371 shutdown_connections 2024-09-12T03:21:54.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.094+0000 7fe19a42e640 1 --2- 172.21.15.6:0/1730207371 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe17c0a70a0 0x7fe17c0a9560 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.094+0000 7fe19a42e640 1 --2- 172.21.15.6:0/1730207371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe194102ef0 0x7fe19419ce50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.094+0000 7fe19a42e640 1 --2- 172.21.15.6:0/1730207371 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe194102570 0x7fe194198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.094+0000 7fe19a42e640 1 --2- 172.21.15.6:0/1730207371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe194100e20 0x7fe194198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:54.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.094+0000 7fe19a42e640 1 -- 172.21.15.6:0/1730207371 >> 172.21.15.6:0/1730207371 conn(0x7fe1940fc7d0 msgr2=0x7fe1940fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:54.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.094+0000 7fe19a42e640 1 -- 172.21.15.6:0/1730207371 shutdown_connections 2024-09-12T03:21:54.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:21:54.094+0000 7fe19a42e640 1 -- 172.21.15.6:0/1730207371 wait complete. 2024-09-12T03:21:54.822 INFO:tasks.cephfs.filesystem:scrub status for tag:3525d3af-7424-48a1-84b9-c551c066120d - {'path': '/', 'tag': '3525d3af-7424-48a1-84b9-c551c066120d', 'options': 'recursive,force'} 2024-09-12T03:21:55.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:55 smithi006 ceph-mon[30991]: pgmap v359: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:21:55.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:55 smithi161 ceph-mon[35485]: pgmap v359: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:21:55.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:55 smithi118 ceph-mon[35048]: pgmap v359: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:21:56.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:56 smithi006 ceph-mon[30991]: scrub summary: idle 2024-09-12T03:21:56.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:56 smithi161 ceph-mon[35485]: scrub summary: idle 2024-09-12T03:21:56.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:56 smithi118 ceph-mon[35048]: scrub summary: idle 2024-09-12T03:21:57.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:57 smithi006 ceph-mon[30991]: pgmap v360: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:21:57.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:57 smithi161 ceph-mon[35485]: pgmap v360: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:21:57.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:57 smithi118 ceph-mon[35048]: pgmap v360: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:21:59.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:21:59 smithi006 ceph-mon[30991]: pgmap v361: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-12T03:21:59.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:21:59 smithi161 ceph-mon[35485]: pgmap v361: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-12T03:21:59.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:21:59 smithi118 ceph-mon[35048]: pgmap v361: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-12T03:22:01.037 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:22:01.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:01 smithi006 ceph-mon[30991]: pgmap v362: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-12T03:22:01.380 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:01.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:01 smithi118 ceph-mon[35048]: pgmap v362: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-12T03:22:01.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:01 smithi161 ceph-mon[35485]: pgmap v362: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-12T03:22:01.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.966+0000 7ff762047640 1 -- 172.21.15.6:0/1612690368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff75c100aa0 msgr2=0x7ff75c10ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:01.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.966+0000 7ff762047640 1 --2- 172.21.15.6:0/1612690368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff75c100aa0 0x7ff75c10ea20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff754009ca0 tx=0x7ff75402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.966+0000 7ff762047640 1 -- 172.21.15.6:0/1612690368 shutdown_connections 2024-09-12T03:22:01.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.966+0000 7ff762047640 1 --2- 172.21.15.6:0/1612690368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff75c100aa0 0x7ff75c10ea20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.967+0000 7ff762047640 1 --2- 172.21.15.6:0/1612690368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff75c100100 0x7ff75c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.967+0000 7ff762047640 1 --2- 172.21.15.6:0/1612690368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff75c106b80 0x7ff75c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.967+0000 7ff762047640 1 -- 172.21.15.6:0/1612690368 >> 172.21.15.6:0/1612690368 conn(0x7ff75c077230 msgr2=0x7ff75c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:01.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.967+0000 7ff762047640 1 -- 172.21.15.6:0/1612690368 shutdown_connections 2024-09-12T03:22:01.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.967+0000 7ff762047640 1 -- 172.21.15.6:0/1612690368 wait complete. 2024-09-12T03:22:01.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.968+0000 7ff762047640 1 Processor -- start 2024-09-12T03:22:01.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.968+0000 7ff762047640 1 -- start start 2024-09-12T03:22:01.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.969+0000 7ff762047640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff75c100100 0x7ff75c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.969+0000 7ff762047640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff75c100aa0 0x7ff75c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.969+0000 7ff762047640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff75c106b80 0x7ff75c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.969+0000 7ff762047640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c1120a0 con 0x7ff75c100aa0 2024-09-12T03:22:01.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.969+0000 7ff75affd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff75c100aa0 0x7ff75c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:01.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.969+0000 7ff75affd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff75c100aa0 0x7ff75c1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51776/0 (socket says 172.21.15.6:51776) 2024-09-12T03:22:01.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.969+0000 7ff75affd640 1 -- 172.21.15.6:0/3440818690 learned_addr learned my addr 172.21.15.6:0/3440818690 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:01.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.969+0000 7ff75b7fe640 1 --2- 172.21.15.6:0/3440818690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff75c100100 0x7ff75c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:01.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.969+0000 7ff762047640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c111f20 con 0x7ff75c106b80 2024-09-12T03:22:01.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.969+0000 7ff762047640 1 -- 172.21.15.6:0/3440818690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c112220 con 0x7ff75c100100 2024-09-12T03:22:01.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.970+0000 7ff75b7fe640 1 -- 172.21.15.6:0/3440818690 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff75c106b80 msgr2=0x7ff75c19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:01.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.970+0000 7ff75b7fe640 1 --2- 172.21.15.6:0/3440818690 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff75c106b80 0x7ff75c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.970+0000 7ff75b7fe640 1 -- 172.21.15.6:0/3440818690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff75c100aa0 msgr2=0x7ff75c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:01.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.970+0000 7ff75b7fe640 1 --2- 172.21.15.6:0/3440818690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff75c100aa0 0x7ff75c1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.970+0000 7ff75b7fe640 1 -- 172.21.15.6:0/3440818690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75c19d430 con 0x7ff75c100100 2024-09-12T03:22:01.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.970+0000 7ff75affd640 1 --2- 172.21.15.6:0/3440818690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff75c100aa0 0x7ff75c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:01.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.970+0000 7ff75b7fe640 1 --2- 172.21.15.6:0/3440818690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff75c100100 0x7ff75c198480 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00e9f0 tx=0x7ff74c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:01.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.970+0000 7ff758ff9640 1 -- 172.21.15.6:0/3440818690 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c00ccc0 con 0x7ff75c100100 2024-09-12T03:22:01.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.970+0000 7ff758ff9640 1 -- 172.21.15.6:0/3440818690 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff74c010910 con 0x7ff75c100100 2024-09-12T03:22:01.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.970+0000 7ff762047640 1 -- 172.21.15.6:0/3440818690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75c19d720 con 0x7ff75c100100 2024-09-12T03:22:01.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.971+0000 7ff762047640 1 -- 172.21.15.6:0/3440818690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff75c1a5000 con 0x7ff75c100100 2024-09-12T03:22:01.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.971+0000 7ff758ff9640 1 -- 172.21.15.6:0/3440818690 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c018980 con 0x7ff75c100100 2024-09-12T03:22:01.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.972+0000 7ff762047640 1 -- 172.21.15.6:0/3440818690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff75c100560 con 0x7ff75c100100 2024-09-12T03:22:01.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.974+0000 7ff758ff9640 1 -- 172.21.15.6:0/3440818690 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7ff74c010ac0 con 0x7ff75c100100 2024-09-12T03:22:01.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.975+0000 7ff758ff9640 1 --2- 172.21.15.6:0/3440818690 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff7440a2d10 0x7ff7440a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.975+0000 7ff758ff9640 1 -- 172.21.15.6:0/3440818690 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff74c0cf650 con 0x7ff75c100100 2024-09-12T03:22:01.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.976+0000 7ff75affd640 1 --2- 172.21.15.6:0/3440818690 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff7440a2d10 0x7ff7440a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:01.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.976+0000 7ff75affd640 1 --2- 172.21.15.6:0/3440818690 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff7440a2d10 0x7ff7440a51d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0ff7f0 tx=0x7ff750005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:01.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:01.979+0000 7ff758ff9640 1 -- 172.21.15.6:0/3440818690 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff74c0541b0 con 0x7ff75c100100 2024-09-12T03:22:02.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:02.390+0000 7ff762047640 1 -- 172.21.15.6:0/3440818690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff75c10a040 con 0x7ff75c100100 2024-09-12T03:22:02.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:02.391+0000 7ff758ff9640 1 -- 172.21.15.6:0/3440818690 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11908 (secure 0 0 0) 0x7ff74c09c420 con 0x7ff75c100100 2024-09-12T03:22:02.391 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:22:02.392 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"btime":"2024-09-12T03:20:52:974598+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:20:52.261388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24463]},"id":1}]} 2024-09-12T03:22:02.392 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 25 2024-09-12T03:22:02.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:02.394+0000 7ff762047640 1 -- 172.21.15.6:0/3440818690 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff7440a2d10 msgr2=0x7ff7440a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:02.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:02.394+0000 7ff762047640 1 --2- 172.21.15.6:0/3440818690 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff7440a2d10 0x7ff7440a51d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0ff7f0 tx=0x7ff750005f90 comp rx=0 tx=0).stop 2024-09-12T03:22:02.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:02.394+0000 7ff762047640 1 -- 172.21.15.6:0/3440818690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff75c100100 msgr2=0x7ff75c198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:02.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:02.394+0000 7ff762047640 1 --2- 172.21.15.6:0/3440818690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff75c100100 0x7ff75c198480 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00e9f0 tx=0x7ff74c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:02.394+0000 7ff762047640 1 -- 172.21.15.6:0/3440818690 shutdown_connections 2024-09-12T03:22:02.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:02.394+0000 7ff762047640 1 --2- 172.21.15.6:0/3440818690 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff7440a2d10 0x7ff7440a51d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:02.394+0000 7ff762047640 1 --2- 172.21.15.6:0/3440818690 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff75c106b80 0x7ff75c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:02.394+0000 7ff762047640 1 --2- 172.21.15.6:0/3440818690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff75c100aa0 0x7ff75c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:02.394+0000 7ff762047640 1 --2- 172.21.15.6:0/3440818690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff75c100100 0x7ff75c198480 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:02.394+0000 7ff762047640 1 -- 172.21.15.6:0/3440818690 >> 172.21.15.6:0/3440818690 conn(0x7ff75c077230 msgr2=0x7ff75c0fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:02.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:02.395+0000 7ff762047640 1 -- 172.21.15.6:0/3440818690 shutdown_connections 2024-09-12T03:22:02.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:02.395+0000 7ff762047640 1 -- 172.21.15.6:0/3440818690 wait complete. 2024-09-12T03:22:03.108 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 2de2c24f --release --await' 2024-09-12T03:22:03.109 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 2de2c24f --release --await 2024-09-12T03:22:03.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:03 smithi006 ceph-mon[30991]: pgmap v363: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:22:03.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:03 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3440818690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:22:03.398 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:03.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:03 smithi118 ceph-mon[35048]: pgmap v363: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:22:03.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:03 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3440818690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:22:03.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:03 smithi161 ceph-mon[35485]: pgmap v363: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:22:03.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:03 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3440818690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:22:03.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.891+0000 7f558528a640 1 -- 172.21.15.6:0/401136466 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55740319b0 con 0x7f5580106b80 2024-09-12T03:22:03.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.892+0000 7f55884f0640 1 -- 172.21.15.6:0/401136466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5580106b80 msgr2=0x7f5580106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.892+0000 7f55884f0640 1 --2- 172.21.15.6:0/401136466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5580106b80 0x7f5580106f60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5574009900 tx=0x7f557402a410 comp rx=0 tx=0).stop 2024-09-12T03:22:03.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.892+0000 7f55884f0640 1 -- 172.21.15.6:0/401136466 shutdown_connections 2024-09-12T03:22:03.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.892+0000 7f55884f0640 1 --2- 172.21.15.6:0/401136466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5580100aa0 0x7f558010ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.892+0000 7f55884f0640 1 --2- 172.21.15.6:0/401136466 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5580100100 0x7f5580100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.892+0000 7f55884f0640 1 --2- 172.21.15.6:0/401136466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5580106b80 0x7f5580106f60 secure :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5574009900 tx=0x7f557402a410 comp rx=0 tx=0).stop 2024-09-12T03:22:03.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.892+0000 7f55884f0640 1 -- 172.21.15.6:0/401136466 >> 172.21.15.6:0/401136466 conn(0x7f5580077230 msgr2=0x7f55800fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:03.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.893+0000 7f55884f0640 1 -- 172.21.15.6:0/401136466 shutdown_connections 2024-09-12T03:22:03.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.893+0000 7f55884f0640 1 -- 172.21.15.6:0/401136466 wait complete. 2024-09-12T03:22:03.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.893+0000 7f55884f0640 1 Processor -- start 2024-09-12T03:22:03.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.893+0000 7f55884f0640 1 -- start start 2024-09-12T03:22:03.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.894+0000 7f55884f0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5580100100 0x7f5580198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.894+0000 7f55884f0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5580100aa0 0x7f5580198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.894+0000 7f55884f0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f558019d010 0x7f558019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.894+0000 7f55884f0640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5580112050 con 0x7f558019d010 2024-09-12T03:22:03.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.894+0000 7f55884f0640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f5580111ed0 con 0x7f5580100aa0 2024-09-12T03:22:03.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.894+0000 7f55884f0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f55801121d0 con 0x7f5580100100 2024-09-12T03:22:03.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.894+0000 7f5586a8d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f558019d010 0x7f558019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.894+0000 7f5586a8d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f558019d010 0x7f558019d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51788/0 (socket says 172.21.15.6:51788) 2024-09-12T03:22:03.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.894+0000 7f5586a8d640 1 -- 172.21.15.6:0/171914170 learned_addr learned my addr 172.21.15.6:0/171914170 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:03.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.894+0000 7f5585a8b640 1 --2- 172.21.15.6:0/171914170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5580100aa0 0x7f5580198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.894+0000 7f5586a8d640 1 -- 172.21.15.6:0/171914170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5580100100 msgr2=0x7f5580198740 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:22:03.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.894+0000 7f5586a8d640 1 --2- 172.21.15.6:0/171914170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5580100100 0x7f5580198740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.894+0000 7f5586a8d640 1 -- 172.21.15.6:0/171914170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5580100aa0 msgr2=0x7f5580198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.894+0000 7f5586a8d640 1 --2- 172.21.15.6:0/171914170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5580100aa0 0x7f5580198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.894+0000 7f5586a8d640 1 -- 172.21.15.6:0/171914170 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558019dad0 con 0x7f558019d010 2024-09-12T03:22:03.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.894+0000 7f5585a8b640 1 --2- 172.21.15.6:0/171914170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5580100aa0 0x7f5580198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:03.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.895+0000 7f5586a8d640 1 --2- 172.21.15.6:0/171914170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f558019d010 0x7f558019d4c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f557800e9d0 tx=0x7f557800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:03.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.895+0000 7f55737fe640 1 -- 172.21.15.6:0/171914170 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557800cd80 con 0x7f558019d010 2024-09-12T03:22:03.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.895+0000 7f55884f0640 1 -- 172.21.15.6:0/171914170 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558019ddc0 con 0x7f558019d010 2024-09-12T03:22:03.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.895+0000 7f55884f0640 1 -- 172.21.15.6:0/171914170 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5580101210 con 0x7f558019d010 2024-09-12T03:22:03.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.895+0000 7f55737fe640 1 -- 172.21.15.6:0/171914170 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5578005020 con 0x7f558019d010 2024-09-12T03:22:03.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.895+0000 7f55737fe640 1 -- 172.21.15.6:0/171914170 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55780116d0 con 0x7f558019d010 2024-09-12T03:22:03.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.898+0000 7f558a8c9740 1 Processor -- start 2024-09-12T03:22:03.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.899+0000 7f55737fe640 1 -- 172.21.15.6:0/171914170 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f5578004a40 con 0x7f558019d010 2024-09-12T03:22:03.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.900+0000 7f558a8c9740 1 -- start start 2024-09-12T03:22:03.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.900+0000 7f558a8c9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76ef16060 0x55a76ef16440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.900+0000 7f558a8c9740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef16980 0x55a76ef20d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.900+0000 7f558a8c9740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a76ef21240 0x55a76ef23630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.900+0000 7f558a8c9740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a76ee7b620 con 0x55a76ef16980 2024-09-12T03:22:03.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.900+0000 7f558a8c9740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55a76ee22e50 con 0x55a76ef21240 2024-09-12T03:22:03.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.900+0000 7f558a8c9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a76ee775e0 con 0x55a76ef16060 2024-09-12T03:22:03.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.900+0000 7f558628c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef16980 0x55a76ef20d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.900+0000 7f55737fe640 1 --2- 172.21.15.6:0/171914170 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f556c0a2c90 0x7f556c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.900+0000 7f558628c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef16980 0x55a76ef20d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51800/0 (socket says 172.21.15.6:51800) 2024-09-12T03:22:03.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.900+0000 7f558628c640 1 -- 172.21.15.6:0/3846647154 learned_addr learned my addr 172.21.15.6:0/3846647154 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:03.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.900+0000 7f5585a8b640 1 --2- 172.21.15.6:0/3846647154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76ef16060 0x55a76ef16440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.901+0000 7f5585a8b640 1 --2- 172.21.15.6:0/3846647154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a76ef21240 0x55a76ef23630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.901+0000 7f5586a8d640 1 --2- 172.21.15.6:0/171914170 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f556c0a2c90 0x7f556c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.901+0000 7f55737fe640 1 -- 172.21.15.6:0/171914170 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f55780cf1c0 con 0x7f558019d010 2024-09-12T03:22:03.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.901+0000 7f558628c640 1 -- 172.21.15.6:0/3846647154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76ef16060 msgr2=0x55a76ef16440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:03.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.901+0000 7f558628c640 1 --2- 172.21.15.6:0/3846647154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76ef16060 0x55a76ef16440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.901+0000 7f558628c640 1 -- 172.21.15.6:0/3846647154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a76ef21240 msgr2=0x55a76ef23630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:03.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.901+0000 7f558628c640 1 --2- 172.21.15.6:0/3846647154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a76ef21240 0x55a76ef23630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.901+0000 7f558628c640 1 -- 172.21.15.6:0/3846647154 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a76ee13d40 con 0x55a76ef16980 2024-09-12T03:22:03.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.901+0000 7f5585a8b640 1 --2- 172.21.15.6:0/3846647154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76ef16060 0x55a76ef16440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:03.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.901+0000 7f5586a8d640 1 --2- 172.21.15.6:0/171914170 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f556c0a2c90 0x7f556c0a5150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f557800c640 tx=0x7f5578022e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:03.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.902+0000 7f558628c640 1 --2- 172.21.15.6:0/3846647154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef16980 0x55a76ef20d00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5574000c00 tx=0x7f5574009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d9216f98e27e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:03.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.903+0000 7f5570ff9640 1 -- 172.21.15.6:0/3846647154 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557403e070 con 0x55a76ef16980 2024-09-12T03:22:03.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.904+0000 7f5570ff9640 1 -- 172.21.15.6:0/3846647154 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5574004040 con 0x55a76ef16980 2024-09-12T03:22:03.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.904+0000 7f5570ff9640 1 -- 172.21.15.6:0/3846647154 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574035040 con 0x55a76ef16980 2024-09-12T03:22:03.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.904+0000 7f558a8c9740 1 -- 172.21.15.6:0/3846647154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef16980 msgr2=0x55a76ef20d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.904+0000 7f558a8c9740 1 --2- 172.21.15.6:0/3846647154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef16980 0x55a76ef20d00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5574000c00 tx=0x7f5574009520 comp rx=0 tx=0).stop 2024-09-12T03:22:03.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.905+0000 7f558a8c9740 1 -- 172.21.15.6:0/3846647154 shutdown_connections 2024-09-12T03:22:03.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.905+0000 7f558a8c9740 1 --2- 172.21.15.6:0/3846647154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a76ef21240 0x55a76ef23630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.905+0000 7f558a8c9740 1 --2- 172.21.15.6:0/3846647154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef16980 0x55a76ef20d00 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.905+0000 7f558a8c9740 1 --2- 172.21.15.6:0/3846647154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76ef16060 0x55a76ef16440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.905+0000 7f558a8c9740 1 -- 172.21.15.6:0/3846647154 >> 172.21.15.6:0/3846647154 conn(0x55a76ef125f0 msgr2=0x55a76ef129d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:03.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.905+0000 7f558a8c9740 1 -- 172.21.15.6:0/3846647154 shutdown_connections 2024-09-12T03:22:03.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.905+0000 7f558a8c9740 1 -- 172.21.15.6:0/3846647154 wait complete. 2024-09-12T03:22:03.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.905+0000 7f558a8c9740 1 -- start start 2024-09-12T03:22:03.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.905+0000 7f558a8c9740 1 Processor -- start 2024-09-12T03:22:03.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.906+0000 7f558a8c9740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a76ef16060 0x55a76ef1dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.906+0000 7f558a8c9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76ef16980 0x55a76ef1e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.906+0000 7f558a8c9740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef21240 0x55a76ef1ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.906+0000 7f5585a8b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef21240 0x55a76ef1ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.906+0000 7f5585a8b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef21240 0x55a76ef1ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51804/0 (socket says 172.21.15.6:51804) 2024-09-12T03:22:03.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.906+0000 7f5585a8b640 1 -- 172.21.15.6:0/217999077 learned_addr learned my addr 172.21.15.6:0/217999077 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:03.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.906+0000 7f558a8c9740 1 -- 172.21.15.6:0/217999077 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a76ef23e10 con 0x55a76ef21240 2024-09-12T03:22:03.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.906+0000 7f558a8c9740 1 -- 172.21.15.6:0/217999077 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55a76ee03f10 con 0x55a76ef16060 2024-09-12T03:22:03.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.906+0000 7f558a8c9740 1 -- 172.21.15.6:0/217999077 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a76ef23f90 con 0x55a76ef16980 2024-09-12T03:22:03.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.906+0000 7f558628c640 1 --2- 172.21.15.6:0/217999077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76ef16980 0x55a76ef1e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.906+0000 7f558628c640 1 -- 172.21.15.6:0/217999077 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a76ef16060 msgr2=0x55a76ef1dfe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:22:03.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.906+0000 7f558628c640 1 --2- 172.21.15.6:0/217999077 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a76ef16060 0x55a76ef1dfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.906+0000 7f558628c640 1 -- 172.21.15.6:0/217999077 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef21240 msgr2=0x55a76ef1ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:03.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.906+0000 7f558628c640 1 --2- 172.21.15.6:0/217999077 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef21240 0x55a76ef1ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.906+0000 7f558628c640 1 -- 172.21.15.6:0/217999077 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a76eb8a580 con 0x55a76ef16980 2024-09-12T03:22:03.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.906+0000 7f5585a8b640 1 --2- 172.21.15.6:0/217999077 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef21240 0x55a76ef1ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:03.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.907+0000 7f558a8c9740 10 client.0 _ll_register_callbacks cb 0x55a76ee27db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:03.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.907+0000 7f558a8c9740 1 -- 172.21.15.6:0/217999077 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5574004040 con 0x55a76ef16980 2024-09-12T03:22:03.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.907+0000 7f558628c640 1 --2- 172.21.15.6:0/217999077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76ef16980 0x55a76ef1e520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5574005110 tx=0x7f5574032ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a349e20402115e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:03.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.907+0000 7f5569ffb640 10 client.25075 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:22:03.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.907+0000 7f5569ffb640 1 -- 172.21.15.6:0/217999077 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557403e070 con 0x55a76ef16980 2024-09-12T03:22:03.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.907+0000 7f5569ffb640 1 -- 172.21.15.6:0/217999077 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f5574005310 con 0x55a76ef16980 2024-09-12T03:22:03.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.907+0000 7f5569ffb640 1 -- 172.21.15.6:0/217999077 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5574035040 con 0x55a76ef16980 2024-09-12T03:22:03.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.908+0000 7f558a8c9740 10 client.25075 fetch_fsmap learned FSMap version 25 2024-09-12T03:22:03.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.908+0000 7f558a8c9740 1 -- 172.21.15.6:0/217999077 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55a76ec54960 con 0x55a76ef16980 2024-09-12T03:22:03.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.908+0000 7f5569ffb640 1 -- 172.21.15.6:0/217999077 <== mon.1 v2:172.21.15.161:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f55740315b0 con 0x55a76ef16980 2024-09-12T03:22:03.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.908+0000 7f558a8c9740 10 client.25075 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:22:03.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.908+0000 7f558a8c9740 10 client.25075 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:22:03.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.908+0000 7f558a8c9740 20 client.25075 populate_metadata read hostname 'smithi006' 2024-09-12T03:22:03.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.908+0000 7f558a8c9740 1 --2- 172.21.15.6:0/217999077 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a76ef41820 0x55a76ef43c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.908+0000 7f558a8c9740 4 client.25075 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:22:03.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.908+0000 7f558a8c9740 1 -- 172.21.15.6:0/217999077 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a76ee88b90 con 0x55a76ef41820 2024-09-12T03:22:03.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.908+0000 7f5585a8b640 1 --2- 172.21.15.6:0/217999077 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a76ef41820 0x55a76ef43c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.909+0000 7f5569ffb640 10 client.25075 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:03.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.909+0000 7f5585a8b640 1 --2- 172.21.15.6:0/217999077 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a76ef41820 0x55a76ef43c10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69cf1f4412e08a6f server_cookie=1341635006b8fa87 in_seq=0 out_seq=0 2024-09-12T03:22:03.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.910+0000 7f5569ffb640 1 -- 172.21.15.6:0/217999077 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a76ee88b90 con 0x55a76ef41820 2024-09-12T03:22:03.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.910+0000 7f5569ffb640 10 client.25075 handle_command_reply: tid=0 2024-09-12T03:22:03.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.910+0000 7f558a8c9740 1 client.25075 shutdown 2024-09-12T03:22:03.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.910+0000 7f558a8c9740 1 -- 172.21.15.6:0/217999077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76ef16980 msgr2=0x55a76ef1e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.910+0000 7f558a8c9740 1 --2- 172.21.15.6:0/217999077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76ef16980 0x55a76ef1e520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5574005110 tx=0x7f5574032ee0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.910+0000 7f558a8c9740 1 -- 172.21.15.6:0/217999077 shutdown_connections 2024-09-12T03:22:03.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.910+0000 7f558a8c9740 1 --2- 172.21.15.6:0/217999077 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a76ef41820 0x55a76ef43c10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.910+0000 7f558a8c9740 1 --2- 172.21.15.6:0/217999077 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef21240 0x55a76ef1ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.910+0000 7f558a8c9740 1 --2- 172.21.15.6:0/217999077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76ef16980 0x55a76ef1e520 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.910+0000 7f5569ffb640 0 client.25075 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:03.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.910+0000 7f558a8c9740 1 --2- 172.21.15.6:0/217999077 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a76ef16060 0x55a76ef1dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.910+0000 7f558a8c9740 1 -- 172.21.15.6:0/217999077 >> 172.21.15.6:0/217999077 conn(0x55a76ef125f0 msgr2=0x55a76ef18320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:03.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.910+0000 7f558a8c9740 1 -- 172.21.15.6:0/217999077 shutdown_connections 2024-09-12T03:22:03.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.910+0000 7f558a8c9740 1 -- 172.21.15.6:0/217999077 wait complete. 2024-09-12T03:22:03.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.910+0000 7f558a8c9740 20 client.25075 trim_cache size 0 max 16384 2024-09-12T03:22:03.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.944+0000 7f558a8c9740 1 Processor -- start 2024-09-12T03:22:03.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.944+0000 7f558a8c9740 1 -- start start 2024-09-12T03:22:03.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.944+0000 7f558a8c9740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a76ef16060 0x55a76ef11640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.944+0000 7f558a8c9740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef16980 0x55a76ef494a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.944+0000 7f558a8c9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76ef21240 0x55a76ef499e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.944+0000 7f558a8c9740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5564010bb0 con 0x55a76ef16980 2024-09-12T03:22:03.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.944+0000 7f558a8c9740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f556400e480 con 0x55a76ef16060 2024-09-12T03:22:03.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.944+0000 7f558a8c9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f556400c2d0 con 0x55a76ef21240 2024-09-12T03:22:03.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.945+0000 7f558628c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef16980 0x55a76ef494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.945+0000 7f558628c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef16980 0x55a76ef494a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51814/0 (socket says 172.21.15.6:51814) 2024-09-12T03:22:03.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.945+0000 7f558628c640 1 -- 172.21.15.6:0/651963622 learned_addr learned my addr 172.21.15.6:0/651963622 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:03.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.945+0000 7f5585a8b640 1 --2- 172.21.15.6:0/651963622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76ef21240 0x55a76ef499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.945+0000 7f5585a8b640 1 --2- 172.21.15.6:0/651963622 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a76ef16060 0x55a76ef11640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.945+0000 7f558628c640 1 -- 172.21.15.6:0/651963622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76ef21240 msgr2=0x55a76ef499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:03.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.945+0000 7f558628c640 1 --2- 172.21.15.6:0/651963622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76ef21240 0x55a76ef499e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.945+0000 7f558628c640 1 -- 172.21.15.6:0/651963622 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a76ef16060 msgr2=0x55a76ef11640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:03.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.945+0000 7f558628c640 1 --2- 172.21.15.6:0/651963622 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a76ef16060 0x55a76ef11640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.945+0000 7f558628c640 1 -- 172.21.15.6:0/651963622 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a76ef12420 con 0x55a76ef16980 2024-09-12T03:22:03.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.945+0000 7f558628c640 1 --2- 172.21.15.6:0/651963622 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef16980 0x55a76ef494a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5574000c00 tx=0x7f5574047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63f13a2754ba332 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:03.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.945+0000 7f556bfff640 1 -- 172.21.15.6:0/651963622 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574004480 con 0x55a76ef16980 2024-09-12T03:22:03.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.945+0000 7f556bfff640 1 -- 172.21.15.6:0/651963622 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5574004040 con 0x55a76ef16980 2024-09-12T03:22:03.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.946+0000 7f556bfff640 1 -- 172.21.15.6:0/651963622 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574003c60 con 0x55a76ef16980 2024-09-12T03:22:03.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.946+0000 7f558a8c9740 1 -- 172.21.15.6:0/651963622 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef16980 msgr2=0x55a76ef494a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.946+0000 7f558a8c9740 1 --2- 172.21.15.6:0/651963622 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef16980 0x55a76ef494a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5574000c00 tx=0x7f5574047000 comp rx=0 tx=0).stop 2024-09-12T03:22:03.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.946+0000 7f558a8c9740 1 -- 172.21.15.6:0/651963622 shutdown_connections 2024-09-12T03:22:03.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.947+0000 7f558a8c9740 1 --2- 172.21.15.6:0/651963622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76ef21240 0x55a76ef499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.947+0000 7f558a8c9740 1 --2- 172.21.15.6:0/651963622 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef16980 0x55a76ef494a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.947+0000 7f558a8c9740 1 --2- 172.21.15.6:0/651963622 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a76ef16060 0x55a76ef11640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.947+0000 7f558a8c9740 1 -- 172.21.15.6:0/651963622 >> 172.21.15.6:0/651963622 conn(0x55a76ef125f0 msgr2=0x55a76ef10f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:03.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.947+0000 7f558a8c9740 1 -- 172.21.15.6:0/651963622 shutdown_connections 2024-09-12T03:22:03.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.947+0000 7f558a8c9740 1 -- 172.21.15.6:0/651963622 wait complete. 2024-09-12T03:22:03.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.947+0000 7f558a8c9740 1 -- start start 2024-09-12T03:22:03.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.948+0000 7f558a8c9740 1 Processor -- start 2024-09-12T03:22:03.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.948+0000 7f558a8c9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76ef16060 0x55a76ef3a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.948+0000 7f558a8c9740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef16980 0x55a76ef3a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.948+0000 7f558a8c9740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a76ef21240 0x55a76ef352c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.948+0000 7f558a8c9740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f556400c940 con 0x55a76ef16980 2024-09-12T03:22:03.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.948+0000 7f558a8c9740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f556400ddb0 con 0x55a76ef21240 2024-09-12T03:22:03.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.948+0000 7f558628c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef16980 0x55a76ef3a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.948+0000 7f558a8c9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f556400d010 con 0x55a76ef16060 2024-09-12T03:22:03.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.948+0000 7f558628c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef16980 0x55a76ef3a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51824/0 (socket says 172.21.15.6:51824) 2024-09-12T03:22:03.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.948+0000 7f558628c640 1 -- 172.21.15.6:0/2324170761 learned_addr learned my addr 172.21.15.6:0/2324170761 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:03.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.949+0000 7f558a8c9740 10 client.0 _ll_register_callbacks cb 0x7f5574031490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:03.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.949+0000 7f558628c640 1 -- 172.21.15.6:0/2324170761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76ef16060 msgr2=0x55a76ef3a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:03.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.949+0000 7f558628c640 1 --2- 172.21.15.6:0/2324170761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76ef16060 0x55a76ef3a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.949+0000 7f558628c640 1 -- 172.21.15.6:0/2324170761 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a76ef21240 msgr2=0x55a76ef352c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:22:03.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.949+0000 7f558628c640 1 --2- 172.21.15.6:0/2324170761 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a76ef21240 0x55a76ef352c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.949+0000 7f558628c640 1 -- 172.21.15.6:0/2324170761 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a76ef13280 con 0x55a76ef16980 2024-09-12T03:22:03.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.949+0000 7f558628c640 1 --2- 172.21.15.6:0/2324170761 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef16980 0x55a76ef3a850 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5574005cd0 tx=0x7f5574003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23df6765307d3a4a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:03.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.949+0000 7f5569ffb640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:22:03.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.949+0000 7f5569ffb640 1 -- 172.21.15.6:0/2324170761 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574005400 con 0x55a76ef16980 2024-09-12T03:22:03.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.949+0000 7f5569ffb640 1 -- 172.21.15.6:0/2324170761 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f55740055a0 con 0x55a76ef16980 2024-09-12T03:22:03.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.949+0000 7f558a8c9740 1 -- 172.21.15.6:0/2324170761 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5574004040 con 0x55a76ef16980 2024-09-12T03:22:03.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.950+0000 7f5569ffb640 1 -- 172.21.15.6:0/2324170761 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f55740039e0 con 0x55a76ef16980 2024-09-12T03:22:03.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.950+0000 7f558a8c9740 10 client.15480 fetch_fsmap learned FSMap version 25 2024-09-12T03:22:03.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.950+0000 7f558a8c9740 1 -- 172.21.15.6:0/2324170761 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a76efac910 con 0x55a76ef16980 2024-09-12T03:22:03.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.950+0000 7f5569ffb640 1 -- 172.21.15.6:0/2324170761 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5574004040 con 0x55a76ef16980 2024-09-12T03:22:03.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.950+0000 7f5569ffb640 1 -- 172.21.15.6:0/2324170761 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5574043450 con 0x55a76ef16980 2024-09-12T03:22:03.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.950+0000 7f558a8c9740 10 client.15480 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:22:03.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.950+0000 7f558a8c9740 10 client.15480 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:22:03.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.950+0000 7f558a8c9740 20 client.15480 populate_metadata read hostname 'smithi006' 2024-09-12T03:22:03.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.951+0000 7f558a8c9740 1 --2- 172.21.15.6:0/2324170761 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a76ef41820 0x55a76efaf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.951+0000 7f558a8c9740 4 client.15480 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2de2c24f", "release": true, "await": true}] 2024-09-12T03:22:03.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.951+0000 7f558a8c9740 1 -- 172.21.15.6:0/2324170761 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2de2c24f", "release": true, "await": true}) -- 0x55a76ef0d950 con 0x55a76ef41820 2024-09-12T03:22:03.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.951+0000 7f5585a8b640 1 --2- 172.21.15.6:0/2324170761 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a76ef41820 0x55a76efaf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.951+0000 7f5585a8b640 1 --2- 172.21.15.6:0/2324170761 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a76ef41820 0x55a76efaf080 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4ed8d32dbeeb085 server_cookie=3e3f7133f545182e in_seq=0 out_seq=0 2024-09-12T03:22:03.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.951+0000 7f5569ffb640 10 client.15480 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:03.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.953+0000 7f5569ffb640 1 -- 172.21.15.6:0/2324170761 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55a76ef0d950 con 0x55a76ef41820 2024-09-12T03:22:03.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.953+0000 7f5569ffb640 10 client.15480 handle_command_reply: tid=0 2024-09-12T03:22:03.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.953+0000 7f558a8c9740 1 client.15480 shutdown 2024-09-12T03:22:03.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.953+0000 7f558a8c9740 1 -- 172.21.15.6:0/2324170761 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef16980 msgr2=0x55a76ef3a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.954+0000 7f558a8c9740 1 --2- 172.21.15.6:0/2324170761 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef16980 0x55a76ef3a850 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5574005cd0 tx=0x7f5574003820 comp rx=0 tx=0).stop 2024-09-12T03:22:03.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.954+0000 7f558a8c9740 1 -- 172.21.15.6:0/2324170761 shutdown_connections 2024-09-12T03:22:03.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.954+0000 7f558a8c9740 1 --2- 172.21.15.6:0/2324170761 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a76ef41820 0x55a76efaf080 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.954+0000 7f5569ffb640 0 client.15480 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:03.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.954+0000 7f558a8c9740 1 --2- 172.21.15.6:0/2324170761 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a76ef21240 0x55a76ef352c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.954+0000 7f558a8c9740 1 --2- 172.21.15.6:0/2324170761 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a76ef16980 0x55a76ef3a850 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.954+0000 7f558a8c9740 1 --2- 172.21.15.6:0/2324170761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76ef16060 0x55a76ef3a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.954+0000 7f558a8c9740 1 -- 172.21.15.6:0/2324170761 >> 172.21.15.6:0/2324170761 conn(0x55a76ef125f0 msgr2=0x55a76ef3d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:03.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.954+0000 7f558a8c9740 1 -- 172.21.15.6:0/2324170761 shutdown_connections 2024-09-12T03:22:03.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.954+0000 7f558a8c9740 1 -- 172.21.15.6:0/2324170761 wait complete. 2024-09-12T03:22:03.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.955+0000 7f558a8c9740 20 client.15480 trim_cache size 0 max 16384 2024-09-12T03:22:03.956 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:22:03.957 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 25, 2024-09-12T03:22:03.957 INFO:teuthology.orchestra.run.smithi006.stdout: "leader": 24494, 2024-09-12T03:22:03.957 INFO:teuthology.orchestra.run.smithi006.stdout: "set_version": 4, 2024-09-12T03:22:03.957 INFO:teuthology.orchestra.run.smithi006.stdout: "sets": { 2024-09-12T03:22:03.957 INFO:teuthology.orchestra.run.smithi006.stdout: "2de2c24f": { 2024-09-12T03:22:03.957 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 4, 2024-09-12T03:22:03.957 INFO:teuthology.orchestra.run.smithi006.stdout: "age_ref": 0.0, 2024-09-12T03:22:03.957 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:22:03.957 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RELEASED", 2024-09-12T03:22:03.957 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:22:03.957 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:22:03.957 INFO:teuthology.orchestra.run.smithi006.stdout: "timeout": 90.0, 2024-09-12T03:22:03.958 INFO:teuthology.orchestra.run.smithi006.stdout: "expiration": 144.5, 2024-09-12T03:22:03.958 INFO:teuthology.orchestra.run.smithi006.stdout: "members": { 2024-09-12T03:22:03.958 INFO:teuthology.orchestra.run.smithi006.stdout: "file:/": { 2024-09-12T03:22:03.958 INFO:teuthology.orchestra.run.smithi006.stdout: "excluded": false, 2024-09-12T03:22:03.958 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:22:03.958 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RELEASED", 2024-09-12T03:22:03.958 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:22:03.958 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:22:03.958 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:22:03.958 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:22:03.958 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:22:03.958 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:22:03.958 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:22:03.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.956+0000 7f55884f0640 1 -- 172.21.15.6:0/171914170 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f556c0a2c90 msgr2=0x7f556c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.956+0000 7f55884f0640 1 --2- 172.21.15.6:0/171914170 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f556c0a2c90 0x7f556c0a5150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f557800c640 tx=0x7f5578022e30 comp rx=0 tx=0).stop 2024-09-12T03:22:03.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.956+0000 7f55884f0640 1 -- 172.21.15.6:0/171914170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f558019d010 msgr2=0x7f558019d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.956+0000 7f55884f0640 1 --2- 172.21.15.6:0/171914170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f558019d010 0x7f558019d4c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f557800e9d0 tx=0x7f557800ee90 comp rx=0 tx=0).stop 2024-09-12T03:22:03.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.956+0000 7f55884f0640 1 -- 172.21.15.6:0/171914170 shutdown_connections 2024-09-12T03:22:03.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.956+0000 7f55884f0640 1 --2- 172.21.15.6:0/171914170 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f556c0a2c90 0x7f556c0a5150 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.956+0000 7f55884f0640 1 --2- 172.21.15.6:0/171914170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f558019d010 0x7f558019d4c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.956+0000 7f55884f0640 1 --2- 172.21.15.6:0/171914170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5580100aa0 0x7f5580198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.956+0000 7f55884f0640 1 --2- 172.21.15.6:0/171914170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5580100100 0x7f5580198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.956+0000 7f55884f0640 1 -- 172.21.15.6:0/171914170 >> 172.21.15.6:0/171914170 conn(0x7f5580077230 msgr2=0x7f5580104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:03.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.956+0000 7f55884f0640 1 -- 172.21.15.6:0/171914170 shutdown_connections 2024-09-12T03:22:03.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:03.956+0000 7f55884f0640 1 -- 172.21.15.6:0/171914170 wait complete. 2024-09-12T03:22:04.664 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2de2c24f', total seconds elapsed: 28.1 2024-09-12T03:22:05.415 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:05 smithi161 ceph-mon[35485]: pgmap v364: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-12T03:22:05.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:05 smithi118 ceph-mon[35048]: pgmap v364: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-12T03:22:05.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:05 smithi006 ceph-mon[30991]: pgmap v364: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-12T03:22:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:07 smithi118 ceph-mon[35048]: pgmap v365: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:22:07.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:07 smithi006 ceph-mon[30991]: pgmap v365: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:22:07.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:07 smithi161 ceph-mon[35485]: pgmap v365: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:22:09.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:09 smithi118 ceph-mon[35048]: pgmap v366: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:22:09.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:09 smithi006 ceph-mon[30991]: pgmap v366: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:22:09.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:09 smithi161 ceph-mon[35485]: pgmap v366: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:22:11.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:11 smithi118 ceph-mon[35048]: pgmap v367: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:22:11.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:11 smithi006 ceph-mon[30991]: pgmap v367: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:22:11.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:11 smithi161 ceph-mon[35485]: pgmap v367: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:22:13.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:13 smithi006 ceph-mon[30991]: pgmap v368: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:22:13.434 INFO:tasks.workunit.client.0.smithi006.stderr:Updating files: 92% (11879/12903) Updating files: 93% (12000/12903) Updating files: 94% (12129/12903) Updating files: 95% (12258/12903) Updating files: 96% (12387/12903) Updating files: 97% (12516/12903) Updating files: 98% (12645/12903) Updating files: 99% (12774/12903) Updating files: 100% (12903/12903) Updating files: 100% (12903/12903), done. 2024-09-12T03:22:13.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:13 smithi118 ceph-mon[35048]: pgmap v368: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:22:13.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:13 smithi161 ceph-mon[35485]: pgmap v368: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:22:14.165 INFO:tasks.workunit.client.0.smithi006.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-12T03:22:14.165 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-09-12T03:22:14.165 INFO:tasks.workunit.client.0.smithi006.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-12T03:22:14.165 INFO:tasks.workunit.client.0.smithi006.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-12T03:22:14.165 INFO:tasks.workunit.client.0.smithi006.stderr:state without impacting any branches by switching back to a branch. 2024-09-12T03:22:14.166 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-09-12T03:22:14.166 INFO:tasks.workunit.client.0.smithi006.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-12T03:22:14.166 INFO:tasks.workunit.client.0.smithi006.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-12T03:22:14.166 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-09-12T03:22:14.166 INFO:tasks.workunit.client.0.smithi006.stderr: git switch -c 2024-09-12T03:22:14.166 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-09-12T03:22:14.166 INFO:tasks.workunit.client.0.smithi006.stderr:Or undo this operation with: 2024-09-12T03:22:14.166 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-09-12T03:22:14.166 INFO:tasks.workunit.client.0.smithi006.stderr: git switch - 2024-09-12T03:22:14.166 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-09-12T03:22:14.166 INFO:tasks.workunit.client.0.smithi006.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-12T03:22:14.166 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-09-12T03:22:14.166 INFO:tasks.workunit.client.0.smithi006.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-12T03:22:14.170 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-12T03:22:14.194 INFO:tasks.workunit.client.0.smithi006.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-12T03:22:14.196 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T03:22:14.196 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-12T03:22:14.928 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-12T03:22:15.011 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-12T03:22:15.054 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T03:22:15.055 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T03:22:15.055 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-12T03:22:15.096 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T03:22:15.099 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-09-12T03:22:15.099 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-12T03:22:15.157 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-09-12T03:22:15.158 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-09-12T03:22:15.158 DEBUG:teuthology.orchestra.run.smithi006:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-09-12T03:22:15.220 INFO:tasks.workunit.client.0.smithi006.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-09-12T03:22:15.221 INFO:tasks.workunit.client.0.smithi006.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-09-12T03:22:15.221 INFO:tasks.workunit.client.0.smithi006.stderr:+ trap -- retry EXIT 2024-09-12T03:22:15.221 INFO:tasks.workunit.client.0.smithi006.stderr:+ rm -rf ffsb 2024-09-12T03:22:15.222 INFO:tasks.workunit.client.0.smithi006.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-09-12T03:22:15.225 INFO:tasks.workunit.client.0.smithi006.stderr:Cloning into 'ffsb'... 2024-09-12T03:22:15.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:15 smithi118 ceph-mon[35048]: pgmap v369: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:22:15.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:15 smithi006 ceph-mon[30991]: pgmap v369: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:22:15.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:15 smithi161 ceph-mon[35485]: pgmap v369: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:22:16.165 INFO:tasks.workunit.client.0.smithi006.stderr:+ trap - EXIT 2024-09-12T03:22:16.165 INFO:tasks.workunit.client.0.smithi006.stderr:+ cd ffsb 2024-09-12T03:22:16.165 INFO:tasks.workunit.client.0.smithi006.stderr:+ ./configure 2024-09-12T03:22:16.279 INFO:tasks.workunit.client.0.smithi006.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-12T03:22:17.286 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether build environment is sane... yes 2024-09-12T03:22:17.294 INFO:tasks.workunit.client.0.smithi006.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-09-12T03:22:17.294 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gawk... gawk 2024-09-12T03:22:17.308 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether make sets $(MAKE)... yes 2024-09-12T03:22:17.336 INFO:tasks.workunit.client.0.smithi006.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-12T03:22:17.337 INFO:tasks.workunit.client.0.smithi006.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-12T03:22:17.346 INFO:tasks.workunit.client.0.smithi006.stdout:checking for style of include used by make... GNU 2024-09-12T03:22:17.350 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gcc... gcc 2024-09-12T03:22:17.400 INFO:tasks.workunit.client.0.smithi006.stdout:checking for C compiler default output file name... a.out 2024-09-12T03:22:17.401 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether the C compiler works... yes 2024-09-12T03:22:17.404 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether we are cross compiling... no 2024-09-12T03:22:17.439 INFO:tasks.workunit.client.0.smithi006.stdout:checking for suffix of executables... 2024-09-12T03:22:17.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:17 smithi118 ceph-mon[35048]: pgmap v370: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:22:17.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:17 smithi118 ceph-mon[35048]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:22:17.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:17 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:17.469 INFO:tasks.workunit.client.0.smithi006.stdout:checking for suffix of object files... o 2024-09-12T03:22:17.500 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether we are using the GNU C compiler... yes 2024-09-12T03:22:17.529 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether gcc accepts -g... yes 2024-09-12T03:22:17.611 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-12T03:22:17.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:17 smithi006 ceph-mon[30991]: pgmap v370: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:22:17.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:17 smithi006 ceph-mon[30991]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:22:17.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:17 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:17.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:17 smithi161 ceph-mon[35485]: pgmap v370: 97 pgs: 97 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:22:17.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:17 smithi161 ceph-mon[35485]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:22:17.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:17 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:17.709 INFO:tasks.workunit.client.0.smithi006.stdout:checking dependency style of gcc... gcc3 2024-09-12T03:22:17.778 INFO:tasks.workunit.client.0.smithi006.stdout:checking how to run the C preprocessor... gcc -E 2024-09-12T03:22:17.820 INFO:tasks.workunit.client.0.smithi006.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-12T03:22:17.822 INFO:tasks.workunit.client.0.smithi006.stdout:checking for egrep... /usr/bin/grep -E 2024-09-12T03:22:17.830 INFO:tasks.workunit.client.0.smithi006.stdout:checking for AIX... no 2024-09-12T03:22:17.849 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gcc... (cached) gcc 2024-09-12T03:22:17.857 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-12T03:22:17.858 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-12T03:22:17.858 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-12T03:22:17.858 INFO:tasks.workunit.client.0.smithi006.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-12T03:22:17.919 INFO:tasks.workunit.client.0.smithi006.stdout:checking for main in -lm... yes 2024-09-12T03:22:17.969 INFO:tasks.workunit.client.0.smithi006.stdout:checking for main in -lpthread... yes 2024-09-12T03:22:18.130 INFO:tasks.workunit.client.0.smithi006.stdout:checking for ANSI C header files... yes 2024-09-12T03:22:18.195 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-12T03:22:18.241 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/types.h... yes 2024-09-12T03:22:18.287 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/stat.h... yes 2024-09-12T03:22:18.333 INFO:tasks.workunit.client.0.smithi006.stdout:checking for stdlib.h... yes 2024-09-12T03:22:18.381 INFO:tasks.workunit.client.0.smithi006.stdout:checking for string.h... yes 2024-09-12T03:22:18.429 INFO:tasks.workunit.client.0.smithi006.stdout:checking for memory.h... yes 2024-09-12T03:22:18.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:18 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:22:18.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:18 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:22:18.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:18 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:18.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:18 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:18.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:18 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:22:18.475 INFO:tasks.workunit.client.0.smithi006.stdout:checking for strings.h... yes 2024-09-12T03:22:18.524 INFO:tasks.workunit.client.0.smithi006.stdout:checking for inttypes.h... yes 2024-09-12T03:22:18.572 INFO:tasks.workunit.client.0.smithi006.stdout:checking for stdint.h... yes 2024-09-12T03:22:18.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:18 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:22:18.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:18 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:22:18.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:18 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:18.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:18 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:18.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:18 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:22:18.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:18 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:22:18.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:18 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:22:18.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:18 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:18.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:18 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:18.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:18 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:22:18.621 INFO:tasks.workunit.client.0.smithi006.stdout:checking for unistd.h... yes 2024-09-12T03:22:18.685 INFO:tasks.workunit.client.0.smithi006.stdout:checking pthread.h usability... yes 2024-09-12T03:22:18.707 INFO:tasks.workunit.client.0.smithi006.stdout:checking pthread.h presence... yes 2024-09-12T03:22:18.707 INFO:tasks.workunit.client.0.smithi006.stdout:checking for pthread.h... yes 2024-09-12T03:22:18.758 INFO:tasks.workunit.client.0.smithi006.stdout:checking fcntl.h usability... yes 2024-09-12T03:22:18.778 INFO:tasks.workunit.client.0.smithi006.stdout:checking fcntl.h presence... yes 2024-09-12T03:22:18.779 INFO:tasks.workunit.client.0.smithi006.stdout:checking for fcntl.h... yes 2024-09-12T03:22:18.830 INFO:tasks.workunit.client.0.smithi006.stdout:checking limits.h usability... yes 2024-09-12T03:22:18.849 INFO:tasks.workunit.client.0.smithi006.stdout:checking limits.h presence... yes 2024-09-12T03:22:18.849 INFO:tasks.workunit.client.0.smithi006.stdout:checking for limits.h... yes 2024-09-12T03:22:18.863 INFO:tasks.workunit.client.0.smithi006.stdout:checking for stdint.h... (cached) yes 2024-09-12T03:22:18.915 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/time.h usability... yes 2024-09-12T03:22:18.936 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/time.h presence... yes 2024-09-12T03:22:18.937 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/time.h... yes 2024-09-12T03:22:18.950 INFO:tasks.workunit.client.0.smithi006.stdout:checking for unistd.h... (cached) yes 2024-09-12T03:22:18.993 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/vfs.h usability... yes 2024-09-12T03:22:19.013 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/vfs.h presence... yes 2024-09-12T03:22:19.013 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/vfs.h... yes 2024-09-12T03:22:19.059 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/limits.h usability... no 2024-09-12T03:22:19.079 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/limits.h presence... no 2024-09-12T03:22:19.080 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/limits.h... no 2024-09-12T03:22:19.113 INFO:tasks.workunit.client.0.smithi006.stdout:checking for an ANSI C-conforming const... yes 2024-09-12T03:22:19.146 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-12T03:22:19.147 INFO:tasks.workunit.client.0.smithi006.stdout:checking for function prototypes... yes 2024-09-12T03:22:19.174 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether setvbuf arguments are reversed... no 2024-09-12T03:22:19.247 INFO:tasks.workunit.client.0.smithi006.stdout:checking for system... yes 2024-09-12T03:22:19.309 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gettimeofday... yes 2024-09-12T03:22:19.371 INFO:tasks.workunit.client.0.smithi006.stdout:checking for mkdir... yes 2024-09-12T03:22:19.437 INFO:tasks.workunit.client.0.smithi006.stdout:checking for strerror... yes 2024-09-12T03:22:19.461 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:19 smithi118 ceph-mon[35048]: pgmap v371: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-12T03:22:19.503 INFO:tasks.workunit.client.0.smithi006.stdout:checking for open64... yes 2024-09-12T03:22:19.565 INFO:tasks.workunit.client.0.smithi006.stdout:checking for stat64... yes 2024-09-12T03:22:19.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:19 smithi006 ceph-mon[30991]: pgmap v371: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-12T03:22:19.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:19 smithi161 ceph-mon[35485]: pgmap v371: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-12T03:22:19.626 INFO:tasks.workunit.client.0.smithi006.stdout:checking for fseeko64... yes 2024-09-12T03:22:19.688 INFO:tasks.workunit.client.0.smithi006.stdout:checking for lrand48_r... yes 2024-09-12T03:22:19.751 INFO:tasks.workunit.client.0.smithi006.stdout:checking for srand48_r... yes 2024-09-12T03:22:19.774 INFO:tasks.workunit.client.0.smithi006.stdout:configure: creating ./config.status 2024-09-12T03:22:19.875 INFO:tasks.workunit.client.0.smithi006.stdout:config.status: creating Makefile 2024-09-12T03:22:19.889 INFO:tasks.workunit.client.0.smithi006.stdout:config.status: creating config.h 2024-09-12T03:22:19.906 INFO:tasks.workunit.client.0.smithi006.stdout:config.status: executing depfiles commands 2024-09-12T03:22:19.956 INFO:tasks.workunit.client.0.smithi006.stdout:config.status: executing default commands 2024-09-12T03:22:20.004 INFO:tasks.workunit.client.0.smithi006.stderr:+ make 2024-09-12T03:22:20.010 INFO:tasks.workunit.client.0.smithi006.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-09-12T03:22:20.012 INFO:tasks.workunit.client.0.smithi006.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-09-12T03:22:20.014 INFO:tasks.workunit.client.0.smithi006.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-09-12T03:22:20.014 INFO:tasks.workunit.client.0.smithi006.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-09-12T03:22:20.014 INFO:tasks.workunit.client.0.smithi006.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-09-12T03:22:20.014 INFO:tasks.workunit.client.0.smithi006.stderr: any GNU archive site. 2024-09-12T03:22:20.015 INFO:tasks.workunit.client.0.smithi006.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-09-12T03:22:20.016 INFO:tasks.workunit.client.0.smithi006.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-09-12T03:22:20.017 INFO:tasks.workunit.client.0.smithi006.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-09-12T03:22:20.017 INFO:tasks.workunit.client.0.smithi006.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-09-12T03:22:20.018 INFO:tasks.workunit.client.0.smithi006.stderr: You might want to install the `Automake' and `Perl' packages. 2024-09-12T03:22:20.018 INFO:tasks.workunit.client.0.smithi006.stderr: Grab them from any GNU archive site. 2024-09-12T03:22:20.071 INFO:tasks.workunit.client.0.smithi006.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-09-12T03:22:20.387 INFO:tasks.workunit.client.0.smithi006.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-09-12T03:22:20.387 INFO:tasks.workunit.client.0.smithi006.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-09-12T03:22:20.387 INFO:tasks.workunit.client.0.smithi006.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-09-12T03:22:20.388 INFO:tasks.workunit.client.0.smithi006.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-09-12T03:22:20.528 INFO:tasks.workunit.client.0.smithi006.stdout:/bin/sh ./config.status --recheck 2024-09-12T03:22:20.581 INFO:tasks.workunit.client.0.smithi006.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-09-12T03:22:20.683 INFO:tasks.workunit.client.0.smithi006.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-12T03:22:21.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:21 smithi118 ceph-mon[35048]: pgmap v372: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 165 KiB/s wr, 16 op/s 2024-09-12T03:22:21.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:21 smithi118 ceph-mon[35048]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-12T03:22:21.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:21 smithi006 ceph-mon[30991]: pgmap v372: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 165 KiB/s wr, 16 op/s 2024-09-12T03:22:21.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:21 smithi006 ceph-mon[30991]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-12T03:22:21.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:21 smithi161 ceph-mon[35485]: pgmap v372: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 165 KiB/s wr, 16 op/s 2024-09-12T03:22:21.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:21 smithi161 ceph-mon[35485]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-12T03:22:21.691 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether build environment is sane... yes 2024-09-12T03:22:21.696 INFO:tasks.workunit.client.0.smithi006.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-09-12T03:22:21.696 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gawk... gawk 2024-09-12T03:22:21.712 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether make sets $(MAKE)... yes 2024-09-12T03:22:21.747 INFO:tasks.workunit.client.0.smithi006.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-12T03:22:21.747 INFO:tasks.workunit.client.0.smithi006.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-12T03:22:21.759 INFO:tasks.workunit.client.0.smithi006.stdout:checking for style of include used by make... GNU 2024-09-12T03:22:21.764 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gcc... gcc 2024-09-12T03:22:21.852 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether the C compiler works... yes 2024-09-12T03:22:21.853 INFO:tasks.workunit.client.0.smithi006.stdout:checking for C compiler default output file name... a.out 2024-09-12T03:22:21.893 INFO:tasks.workunit.client.0.smithi006.stdout:checking for suffix of executables... 2024-09-12T03:22:21.933 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether we are cross compiling... no 2024-09-12T03:22:21.971 INFO:tasks.workunit.client.0.smithi006.stdout:checking for suffix of object files... o 2024-09-12T03:22:21.995 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether we are using the GNU C compiler... yes 2024-09-12T03:22:22.019 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether gcc accepts -g... yes 2024-09-12T03:22:22.062 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-12T03:22:22.163 INFO:tasks.workunit.client.0.smithi006.stdout:checking dependency style of gcc... gcc3 2024-09-12T03:22:22.218 INFO:tasks.workunit.client.0.smithi006.stdout:checking how to run the C preprocessor... gcc -E 2024-09-12T03:22:22.269 INFO:tasks.workunit.client.0.smithi006.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-12T03:22:22.270 INFO:tasks.workunit.client.0.smithi006.stdout:checking for egrep... /usr/bin/grep -E 2024-09-12T03:22:22.406 INFO:tasks.workunit.client.0.smithi006.stdout:checking for ANSI C header files... yes 2024-09-12T03:22:22.444 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/types.h... yes 2024-09-12T03:22:22.487 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/stat.h... yes 2024-09-12T03:22:22.531 INFO:tasks.workunit.client.0.smithi006.stdout:checking for stdlib.h... yes 2024-09-12T03:22:22.575 INFO:tasks.workunit.client.0.smithi006.stdout:checking for string.h... yes 2024-09-12T03:22:22.617 INFO:tasks.workunit.client.0.smithi006.stdout:checking for memory.h... yes 2024-09-12T03:22:22.657 INFO:tasks.workunit.client.0.smithi006.stdout:checking for strings.h... yes 2024-09-12T03:22:22.699 INFO:tasks.workunit.client.0.smithi006.stdout:checking for inttypes.h... yes 2024-09-12T03:22:22.740 INFO:tasks.workunit.client.0.smithi006.stdout:checking for stdint.h... yes 2024-09-12T03:22:22.782 INFO:tasks.workunit.client.0.smithi006.stdout:checking for unistd.h... yes 2024-09-12T03:22:22.827 INFO:tasks.workunit.client.0.smithi006.stdout:checking minix/config.h usability... no 2024-09-12T03:22:22.854 INFO:tasks.workunit.client.0.smithi006.stdout:checking minix/config.h presence... no 2024-09-12T03:22:22.854 INFO:tasks.workunit.client.0.smithi006.stdout:checking for minix/config.h... no 2024-09-12T03:22:22.889 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-09-12T03:22:22.889 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gcc... (cached) gcc 2024-09-12T03:22:22.924 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-12T03:22:22.924 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-12T03:22:22.924 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-12T03:22:22.925 INFO:tasks.workunit.client.0.smithi006.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-12T03:22:22.967 INFO:tasks.workunit.client.0.smithi006.stdout:checking for main in -lm... yes 2024-09-12T03:22:23.012 INFO:tasks.workunit.client.0.smithi006.stdout:checking for main in -lpthread... yes 2024-09-12T03:22:23.013 INFO:tasks.workunit.client.0.smithi006.stdout:checking for ANSI C header files... (cached) yes 2024-09-12T03:22:23.045 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-12T03:22:23.122 INFO:tasks.workunit.client.0.smithi006.stdout:checking pthread.h usability... yes 2024-09-12T03:22:23.140 INFO:tasks.workunit.client.0.smithi006.stdout:checking pthread.h presence... yes 2024-09-12T03:22:23.141 INFO:tasks.workunit.client.0.smithi006.stdout:checking for pthread.h... yes 2024-09-12T03:22:23.191 INFO:tasks.workunit.client.0.smithi006.stdout:checking fcntl.h usability... yes 2024-09-12T03:22:23.208 INFO:tasks.workunit.client.0.smithi006.stdout:checking fcntl.h presence... yes 2024-09-12T03:22:23.208 INFO:tasks.workunit.client.0.smithi006.stdout:checking for fcntl.h... yes 2024-09-12T03:22:23.256 INFO:tasks.workunit.client.0.smithi006.stdout:checking limits.h usability... yes 2024-09-12T03:22:23.274 INFO:tasks.workunit.client.0.smithi006.stdout:checking limits.h presence... yes 2024-09-12T03:22:23.274 INFO:tasks.workunit.client.0.smithi006.stdout:checking for limits.h... yes 2024-09-12T03:22:23.287 INFO:tasks.workunit.client.0.smithi006.stdout:checking for stdint.h... (cached) yes 2024-09-12T03:22:23.324 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/time.h usability... yes 2024-09-12T03:22:23.340 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/time.h presence... yes 2024-09-12T03:22:23.340 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/time.h... yes 2024-09-12T03:22:23.354 INFO:tasks.workunit.client.0.smithi006.stdout:checking for unistd.h... (cached) yes 2024-09-12T03:22:23.390 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/vfs.h usability... yes 2024-09-12T03:22:23.407 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/vfs.h presence... yes 2024-09-12T03:22:23.407 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/vfs.h... yes 2024-09-12T03:22:23.455 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/limits.h usability... no 2024-09-12T03:22:23.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:23 smithi118 ceph-mon[35048]: pgmap v373: 97 pgs: 97 active+clean; 2.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 197 KiB/s wr, 23 op/s 2024-09-12T03:22:23.484 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/limits.h presence... no 2024-09-12T03:22:23.484 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/limits.h... no 2024-09-12T03:22:23.515 INFO:tasks.workunit.client.0.smithi006.stdout:checking for an ANSI C-conforming const... yes 2024-09-12T03:22:23.548 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-12T03:22:23.609 INFO:tasks.workunit.client.0.smithi006.stdout:checking for system... yes 2024-09-12T03:22:23.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:23 smithi006 ceph-mon[30991]: pgmap v373: 97 pgs: 97 active+clean; 2.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 197 KiB/s wr, 23 op/s 2024-09-12T03:22:23.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:23 smithi161 ceph-mon[35485]: pgmap v373: 97 pgs: 97 active+clean; 2.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 197 KiB/s wr, 23 op/s 2024-09-12T03:22:23.669 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gettimeofday... yes 2024-09-12T03:22:23.727 INFO:tasks.workunit.client.0.smithi006.stdout:checking for mkdir... yes 2024-09-12T03:22:23.787 INFO:tasks.workunit.client.0.smithi006.stdout:checking for strerror... yes 2024-09-12T03:22:23.842 INFO:tasks.workunit.client.0.smithi006.stdout:checking for open64... yes 2024-09-12T03:22:23.897 INFO:tasks.workunit.client.0.smithi006.stdout:checking for stat64... yes 2024-09-12T03:22:23.952 INFO:tasks.workunit.client.0.smithi006.stdout:checking for fseeko64... yes 2024-09-12T03:22:24.008 INFO:tasks.workunit.client.0.smithi006.stdout:checking for lrand48_r... yes 2024-09-12T03:22:24.063 INFO:tasks.workunit.client.0.smithi006.stdout:checking for srand48_r... yes 2024-09-12T03:22:24.085 INFO:tasks.workunit.client.0.smithi006.stdout:configure: creating ./config.status 2024-09-12T03:22:24.147 INFO:tasks.workunit.client.0.smithi006.stdout: /bin/sh ./config.status 2024-09-12T03:22:24.193 INFO:tasks.workunit.client.0.smithi006.stdout:config.status: creating Makefile 2024-09-12T03:22:24.205 INFO:tasks.workunit.client.0.smithi006.stdout:config.status: creating config.h 2024-09-12T03:22:24.226 INFO:tasks.workunit.client.0.smithi006.stdout:config.status: executing depfiles commands 2024-09-12T03:22:24.237 INFO:tasks.workunit.client.0.smithi006.stdout:config.status: executing default commands 2024-09-12T03:22:24.261 INFO:tasks.workunit.client.0.smithi006.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-09-12T03:22:24.419 INFO:tasks.workunit.client.0.smithi006.stdout:rm -f stamp-h1 2024-09-12T03:22:24.421 INFO:tasks.workunit.client.0.smithi006.stdout:touch config.h.in 2024-09-12T03:22:24.423 INFO:tasks.workunit.client.0.smithi006.stdout:cd . && /bin/sh ./config.status config.h 2024-09-12T03:22:24.464 INFO:tasks.workunit.client.0.smithi006.stdout:config.status: creating config.h 2024-09-12T03:22:24.490 INFO:tasks.workunit.client.0.smithi006.stdout:make all-am 2024-09-12T03:22:24.496 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-12T03:22:24.496 INFO:tasks.workunit.client.0.smithi006.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-09-12T03:22:24.665 INFO:tasks.workunit.client.0.smithi006.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-09-12T03:22:24.668 INFO:tasks.workunit.client.0.smithi006.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-09-12T03:22:24.765 INFO:tasks.workunit.client.0.smithi006.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-09-12T03:22:24.767 INFO:tasks.workunit.client.0.smithi006.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-09-12T03:22:24.823 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:22:24.849 INFO:tasks.workunit.client.0.smithi006.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-09-12T03:22:24.852 INFO:tasks.workunit.client.0.smithi006.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-09-12T03:22:24.944 INFO:tasks.workunit.client.0.smithi006.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-09-12T03:22:24.946 INFO:tasks.workunit.client.0.smithi006.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-09-12T03:22:25.067 INFO:tasks.workunit.client.0.smithi006.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-09-12T03:22:25.070 INFO:tasks.workunit.client.0.smithi006.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-09-12T03:22:25.088 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:25.124 INFO:tasks.workunit.client.0.smithi006.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-09-12T03:22:25.127 INFO:tasks.workunit.client.0.smithi006.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-09-12T03:22:25.168 INFO:tasks.workunit.client.0.smithi006.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-09-12T03:22:25.172 INFO:tasks.workunit.client.0.smithi006.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-09-12T03:22:25.230 INFO:tasks.workunit.client.0.smithi006.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-09-12T03:22:25.233 INFO:tasks.workunit.client.0.smithi006.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-09-12T03:22:25.457 INFO:tasks.workunit.client.0.smithi006.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-09-12T03:22:25.460 INFO:tasks.workunit.client.0.smithi006.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-09-12T03:22:25.461 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:25 smithi118 ceph-mon[35048]: pgmap v374: 97 pgs: 97 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 442 KiB/s wr, 53 op/s 2024-09-12T03:22:25.529 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:25 smithi006 ceph-mon[30991]: pgmap v374: 97 pgs: 97 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 442 KiB/s wr, 53 op/s 2024-09-12T03:22:25.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:25 smithi161 ceph-mon[35485]: pgmap v374: 97 pgs: 97 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 442 KiB/s wr, 53 op/s 2024-09-12T03:22:25.629 INFO:tasks.workunit.client.0.smithi006.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-09-12T03:22:25.632 INFO:tasks.workunit.client.0.smithi006.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-09-12T03:22:25.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.619+0000 7f11519a7640 1 -- 172.21.15.6:0/1288886060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f114c103780 msgr2=0x7f114c107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.619+0000 7f11519a7640 1 --2- 172.21.15.6:0/1288886060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f114c103780 0x7f114c107bc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1140009930 tx=0x7f114002a420 comp rx=0 tx=0).stop 2024-09-12T03:22:25.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.619+0000 7f11519a7640 1 -- 172.21.15.6:0/1288886060 shutdown_connections 2024-09-12T03:22:25.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.619+0000 7f11519a7640 1 --2- 172.21.15.6:0/1288886060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f114c108300 0x7f114c10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.619+0000 7f11519a7640 1 --2- 172.21.15.6:0/1288886060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f114c103780 0x7f114c107bc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.619+0000 7f11519a7640 1 --2- 172.21.15.6:0/1288886060 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f114c102dd0 0x7f114c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.619+0000 7f11519a7640 1 -- 172.21.15.6:0/1288886060 >> 172.21.15.6:0/1288886060 conn(0x7f114c0fc7d0 msgr2=0x7f114c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:25.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.619+0000 7f11519a7640 1 -- 172.21.15.6:0/1288886060 shutdown_connections 2024-09-12T03:22:25.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.619+0000 7f11519a7640 1 -- 172.21.15.6:0/1288886060 wait complete. 2024-09-12T03:22:25.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.620+0000 7f11519a7640 1 Processor -- start 2024-09-12T03:22:25.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.620+0000 7f11519a7640 1 -- start start 2024-09-12T03:22:25.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.620+0000 7f11519a7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f114c102dd0 0x7f114c100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.620+0000 7f11519a7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f114c103780 0x7f114c100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.620+0000 7f11519a7640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f114c108300 0x7f114c1014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.620+0000 7f11519a7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f114c112050 con 0x7f114c102dd0 2024-09-12T03:22:25.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.620+0000 7f11519a7640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f114c111ed0 con 0x7f114c108300 2024-09-12T03:22:25.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.620+0000 7f11519a7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f114c1121d0 con 0x7f114c103780 2024-09-12T03:22:25.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.620+0000 7f114affd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f114c102dd0 0x7f114c100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.620+0000 7f114affd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f114c102dd0 0x7f114c100a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38828/0 (socket says 172.21.15.6:38828) 2024-09-12T03:22:25.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.620+0000 7f114affd640 1 -- 172.21.15.6:0/2290321508 learned_addr learned my addr 172.21.15.6:0/2290321508 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:25.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.620+0000 7f114a7fc640 1 --2- 172.21.15.6:0/2290321508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f114c103780 0x7f114c100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.620+0000 7f114b7fe640 1 --2- 172.21.15.6:0/2290321508 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f114c108300 0x7f114c1014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.621+0000 7f114affd640 1 -- 172.21.15.6:0/2290321508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f114c103780 msgr2=0x7f114c100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.621+0000 7f114affd640 1 --2- 172.21.15.6:0/2290321508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f114c103780 0x7f114c100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.621+0000 7f114affd640 1 -- 172.21.15.6:0/2290321508 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f114c108300 msgr2=0x7f114c1014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.621+0000 7f114affd640 1 --2- 172.21.15.6:0/2290321508 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f114c108300 0x7f114c1014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.621+0000 7f114affd640 1 -- 172.21.15.6:0/2290321508 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114c1a6a90 con 0x7f114c102dd0 2024-09-12T03:22:25.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.621+0000 7f114affd640 1 --2- 172.21.15.6:0/2290321508 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f114c102dd0 0x7f114c100a30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f113c00e9f0 tx=0x7f113c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:25.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.621+0000 7f11509a5640 1 -- 172.21.15.6:0/2290321508 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c00cd80 con 0x7f114c102dd0 2024-09-12T03:22:25.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.621+0000 7f11509a5640 1 -- 172.21.15.6:0/2290321508 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f113c010910 con 0x7f114c102dd0 2024-09-12T03:22:25.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.621+0000 7f11509a5640 1 -- 172.21.15.6:0/2290321508 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c018990 con 0x7f114c102dd0 2024-09-12T03:22:25.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.621+0000 7f11519a7640 1 -- 172.21.15.6:0/2290321508 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f114c1a6d80 con 0x7f114c102dd0 2024-09-12T03:22:25.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.621+0000 7f11519a7640 1 -- 172.21.15.6:0/2290321508 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f114c1a71e0 con 0x7f114c102dd0 2024-09-12T03:22:25.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.624+0000 7f11509a5640 1 -- 172.21.15.6:0/2290321508 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f113c010ac0 con 0x7f114c102dd0 2024-09-12T03:22:25.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.624+0000 7f1153d80740 1 Processor -- start 2024-09-12T03:22:25.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.624+0000 7f1153d80740 1 -- start start 2024-09-12T03:22:25.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.624+0000 7f1153d80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 0x559b0666b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.624+0000 7f1153d80740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b0666bdc0 0x559b06676140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.624+0000 7f1153d80740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b06676680 0x559b06678a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.624+0000 7f1153d80740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x559b065d12e0 con 0x559b06676680 2024-09-12T03:22:25.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.624+0000 7f1153d80740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x559b06578ce0 con 0x559b0666bdc0 2024-09-12T03:22:25.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.624+0000 7f1153d80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559b065cd2a0 con 0x559b0666b4a0 2024-09-12T03:22:25.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.624+0000 7f114a7fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b06676680 0x559b06678a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.624+0000 7f114affd640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b0666bdc0 0x559b06676140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.624+0000 7f114a7fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b06676680 0x559b06678a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38840/0 (socket says 172.21.15.6:38840) 2024-09-12T03:22:25.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.624+0000 7f114a7fc640 1 -- 172.21.15.6:0/1004484953 learned_addr learned my addr 172.21.15.6:0/1004484953 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:25.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.624+0000 7f114b7fe640 1 --2- 172.21.15.6:0/1004484953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 0x559b0666b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.625+0000 7f114a7fc640 1 -- 172.21.15.6:0/1004484953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 msgr2=0x559b0666b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.625+0000 7f114a7fc640 1 --2- 172.21.15.6:0/1004484953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 0x559b0666b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.625+0000 7f114a7fc640 1 -- 172.21.15.6:0/1004484953 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b0666bdc0 msgr2=0x559b06676140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.625+0000 7f114a7fc640 1 --2- 172.21.15.6:0/1004484953 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b0666bdc0 0x559b06676140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.625+0000 7f114a7fc640 1 -- 172.21.15.6:0/1004484953 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b0655d440 con 0x559b06676680 2024-09-12T03:22:25.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.625+0000 7f114affd640 1 --2- 172.21.15.6:0/1004484953 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b0666bdc0 0x559b06676140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:25.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.625+0000 7f11509a5640 1 --2- 172.21.15.6:0/2290321508 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f11340a2b80 0x7f11340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.625+0000 7f114b7fe640 1 --2- 172.21.15.6:0/2290321508 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f11340a2b80 0x7f11340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.625+0000 7f114a7fc640 1 --2- 172.21.15.6:0/1004484953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b06676680 0x559b06678a70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f114c0697d0 tx=0x7f1140005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b055440e9a7eabf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:25.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.625+0000 7f11509a5640 1 -- 172.21.15.6:0/2290321508 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f113c0cf460 con 0x7f114c102dd0 2024-09-12T03:22:25.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.625+0000 7f1131ffb640 1 -- 172.21.15.6:0/1004484953 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114003f070 con 0x559b06676680 2024-09-12T03:22:25.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.625+0000 7f1131ffb640 1 -- 172.21.15.6:0/1004484953 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1140004030 con 0x559b06676680 2024-09-12T03:22:25.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.625+0000 7f1131ffb640 1 -- 172.21.15.6:0/1004484953 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1140035070 con 0x559b06676680 2024-09-12T03:22:25.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.626+0000 7f1153d80740 1 -- 172.21.15.6:0/1004484953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b06676680 msgr2=0x559b06678a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.626+0000 7f1153d80740 1 --2- 172.21.15.6:0/1004484953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b06676680 0x559b06678a70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f114c0697d0 tx=0x7f1140005dc0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.626+0000 7f114b7fe640 1 --2- 172.21.15.6:0/2290321508 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f11340a2b80 0x7f11340a5040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1144005fd0 tx=0x7f11440096e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:25.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.626+0000 7f1153d80740 1 -- 172.21.15.6:0/1004484953 shutdown_connections 2024-09-12T03:22:25.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.626+0000 7f1153d80740 1 --2- 172.21.15.6:0/1004484953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b06676680 0x559b06678a70 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.626+0000 7f1153d80740 1 --2- 172.21.15.6:0/1004484953 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b0666bdc0 0x559b06676140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.626+0000 7f1153d80740 1 --2- 172.21.15.6:0/1004484953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 0x559b0666b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.626+0000 7f1153d80740 1 -- 172.21.15.6:0/1004484953 >> 172.21.15.6:0/1004484953 conn(0x559b06667a30 msgr2=0x559b06667e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:25.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.626+0000 7f1153d80740 1 -- 172.21.15.6:0/1004484953 shutdown_connections 2024-09-12T03:22:25.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.626+0000 7f1153d80740 1 -- 172.21.15.6:0/1004484953 wait complete. 2024-09-12T03:22:25.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.626+0000 7f1153d80740 1 -- start start 2024-09-12T03:22:25.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.627+0000 7f1153d80740 1 Processor -- start 2024-09-12T03:22:25.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.627+0000 7f1153d80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 0x559b06675f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.627+0000 7f1153d80740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b0666bdc0 0x559b066731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.627+0000 7f1153d80740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b06676680 0x559b06673720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.627+0000 7f1153d80740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x559b06679250 con 0x559b0666bdc0 2024-09-12T03:22:25.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.627+0000 7f1153d80740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x559b0658c050 con 0x559b06676680 2024-09-12T03:22:25.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.627+0000 7f1153d80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559b066793d0 con 0x559b0666b4a0 2024-09-12T03:22:25.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.627+0000 7f1153d80740 10 client.0 _ll_register_callbacks cb 0x559b065cc160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:25.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.627+0000 7f114a7fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b0666bdc0 0x559b066731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.627+0000 7f114a7fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b0666bdc0 0x559b066731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38846/0 (socket says 172.21.15.6:38846) 2024-09-12T03:22:25.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.627+0000 7f114a7fc640 1 -- 172.21.15.6:0/515330084 learned_addr learned my addr 172.21.15.6:0/515330084 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:25.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.627+0000 7f114affd640 1 --2- 172.21.15.6:0/515330084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 0x559b06675f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.627+0000 7f114b7fe640 1 --2- 172.21.15.6:0/515330084 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b06676680 0x559b06673720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.628+0000 7f114a7fc640 1 -- 172.21.15.6:0/515330084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 msgr2=0x559b06675f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.628+0000 7f114a7fc640 1 --2- 172.21.15.6:0/515330084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 0x559b06675f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.628+0000 7f114a7fc640 1 -- 172.21.15.6:0/515330084 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b06676680 msgr2=0x559b06673720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.628+0000 7f114a7fc640 1 --2- 172.21.15.6:0/515330084 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b06676680 0x559b06673720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.628+0000 7f114a7fc640 1 -- 172.21.15.6:0/515330084 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b063423c0 con 0x559b0666bdc0 2024-09-12T03:22:25.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.628+0000 7f114b7fe640 1 --2- 172.21.15.6:0/515330084 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b06676680 0x559b06673720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:25.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.628+0000 7f114a7fc640 1 --2- 172.21.15.6:0/515330084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b0666bdc0 0x559b066731e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f114002a460 tx=0x7f1140009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c652b377a5c162f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:25.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.628+0000 7f112affd640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:22:25.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.628+0000 7f112affd640 1 -- 172.21.15.6:0/515330084 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1140035070 con 0x559b0666bdc0 2024-09-12T03:22:25.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.628+0000 7f112affd640 1 -- 172.21.15.6:0/515330084 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f11400315b0 con 0x559b0666bdc0 2024-09-12T03:22:25.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.628+0000 7f1153d80740 1 -- 172.21.15.6:0/515330084 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1140004030 con 0x559b0666bdc0 2024-09-12T03:22:25.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.628+0000 7f112affd640 1 -- 172.21.15.6:0/515330084 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1140054070 con 0x559b0666bdc0 2024-09-12T03:22:25.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.628+0000 7f1153d80740 10 client.15492 fetch_fsmap learned FSMap version 26 2024-09-12T03:22:25.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.628+0000 7f1153d80740 1 -- 172.21.15.6:0/515330084 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559b06684bd0 con 0x559b0666bdc0 2024-09-12T03:22:25.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.629+0000 7f112affd640 1 -- 172.21.15.6:0/515330084 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1140004030 con 0x559b0666bdc0 2024-09-12T03:22:25.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.629+0000 7f112affd640 1 -- 172.21.15.6:0/515330084 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f114004b020 con 0x559b0666bdc0 2024-09-12T03:22:25.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.629+0000 7f1153d80740 10 client.15492 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:22:25.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.629+0000 7f1153d80740 10 client.15492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:22:25.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.629+0000 7f1153d80740 20 client.15492 populate_metadata read hostname 'smithi006' 2024-09-12T03:22:25.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.629+0000 7f1153d80740 1 --2- 172.21.15.6:0/515330084 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x559b06685f20 0x559b06688310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.629+0000 7f1153d80740 4 client.15492 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:22:25.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.629+0000 7f1153d80740 1 -- 172.21.15.6:0/515330084 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b065de850 con 0x559b06685f20 2024-09-12T03:22:25.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.629+0000 7f114affd640 1 --2- 172.21.15.6:0/515330084 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x559b06685f20 0x559b06688310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.629+0000 7f112affd640 10 client.15492 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:25.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.629+0000 7f114affd640 1 --2- 172.21.15.6:0/515330084 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x559b06685f20 0x559b06688310 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d32aefa2030c93ec server_cookie=38c0e6531da1d7c in_seq=0 out_seq=0 2024-09-12T03:22:25.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.630+0000 7f112affd640 1 -- 172.21.15.6:0/515330084 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b065de850 con 0x559b06685f20 2024-09-12T03:22:25.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.630+0000 7f112affd640 10 client.15492 handle_command_reply: tid=0 2024-09-12T03:22:25.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.630+0000 7f1153d80740 1 client.15492 shutdown 2024-09-12T03:22:25.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.630+0000 7f1153d80740 1 -- 172.21.15.6:0/515330084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b0666bdc0 msgr2=0x559b066731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.630+0000 7f1153d80740 1 --2- 172.21.15.6:0/515330084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b0666bdc0 0x559b066731e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f114002a460 tx=0x7f1140009520 comp rx=0 tx=0).stop 2024-09-12T03:22:25.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.630+0000 7f1153d80740 1 -- 172.21.15.6:0/515330084 shutdown_connections 2024-09-12T03:22:25.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.630+0000 7f1153d80740 1 --2- 172.21.15.6:0/515330084 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x559b06685f20 0x559b06688310 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.630+0000 7f1153d80740 1 --2- 172.21.15.6:0/515330084 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b06676680 0x559b06673720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.630+0000 7f1153d80740 1 --2- 172.21.15.6:0/515330084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b0666bdc0 0x559b066731e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.630+0000 7f1153d80740 1 --2- 172.21.15.6:0/515330084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 0x559b06675f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.630+0000 7f1153d80740 1 -- 172.21.15.6:0/515330084 >> 172.21.15.6:0/515330084 conn(0x559b06667a30 msgr2=0x559b0666d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:25.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.630+0000 7f112affd640 0 client.15492 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:25.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.631+0000 7f1153d80740 1 -- 172.21.15.6:0/515330084 shutdown_connections 2024-09-12T03:22:25.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.631+0000 7f1153d80740 1 -- 172.21.15.6:0/515330084 wait complete. 2024-09-12T03:22:25.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.631+0000 7f1153d80740 20 client.15492 trim_cache size 0 max 16384 2024-09-12T03:22:25.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.648+0000 7f1153d80740 1 Processor -- start 2024-09-12T03:22:25.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.648+0000 7f1153d80740 1 -- start start 2024-09-12T03:22:25.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.648+0000 7f1153d80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 0x559b06682510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.648+0000 7f1153d80740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b0666bdc0 0x559b0668d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.648+0000 7f1153d80740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b06676680 0x559b0668deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.649+0000 7f1153d80740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f114c1b1ef0 con 0x559b06676680 2024-09-12T03:22:25.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.649+0000 7f1153d80740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f114c1af7c0 con 0x559b0666bdc0 2024-09-12T03:22:25.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.649+0000 7f1153d80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f114c1ad630 con 0x559b0666b4a0 2024-09-12T03:22:25.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.649+0000 7f114b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b06676680 0x559b0668deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.649+0000 7f114b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b06676680 0x559b0668deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38852/0 (socket says 172.21.15.6:38852) 2024-09-12T03:22:25.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.649+0000 7f114b7fe640 1 -- 172.21.15.6:0/1243691602 learned_addr learned my addr 172.21.15.6:0/1243691602 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:25.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.649+0000 7f114affd640 1 --2- 172.21.15.6:0/1243691602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 0x559b06682510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.649+0000 7f114a7fc640 1 --2- 172.21.15.6:0/1243691602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b0666bdc0 0x559b0668d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.649+0000 7f114a7fc640 1 -- 172.21.15.6:0/1243691602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 msgr2=0x559b06682510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.649+0000 7f114a7fc640 1 --2- 172.21.15.6:0/1243691602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 0x559b06682510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.649+0000 7f114a7fc640 1 -- 172.21.15.6:0/1243691602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b06676680 msgr2=0x559b0668deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.649+0000 7f114a7fc640 1 --2- 172.21.15.6:0/1243691602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b06676680 0x559b0668deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.649+0000 7f114a7fc640 1 -- 172.21.15.6:0/1243691602 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b06667860 con 0x559b0666bdc0 2024-09-12T03:22:25.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.649+0000 7f114b7fe640 1 --2- 172.21.15.6:0/1243691602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b06676680 0x559b0668deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:25.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.649+0000 7f114affd640 1 --2- 172.21.15.6:0/1243691602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 0x559b06682510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:25.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.649+0000 7f114a7fc640 1 --2- 172.21.15.6:0/1243691602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b0666bdc0 0x559b0668d970 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f11400049a0 tx=0x7f1140000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbcbf45be703288a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:25.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.650+0000 7f11317fa640 1 -- 172.21.15.6:0/1243691602 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114005d020 con 0x559b0666bdc0 2024-09-12T03:22:25.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.650+0000 7f11317fa640 1 -- 172.21.15.6:0/1243691602 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1140004f00 con 0x559b0666bdc0 2024-09-12T03:22:25.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.650+0000 7f11317fa640 1 -- 172.21.15.6:0/1243691602 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1140035030 con 0x559b0666bdc0 2024-09-12T03:22:25.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.650+0000 7f1153d80740 1 -- 172.21.15.6:0/1243691602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b0666bdc0 msgr2=0x559b0668d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.650+0000 7f1153d80740 1 --2- 172.21.15.6:0/1243691602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b0666bdc0 0x559b0668d970 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f11400049a0 tx=0x7f1140000bc0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.650+0000 7f1153d80740 1 -- 172.21.15.6:0/1243691602 shutdown_connections 2024-09-12T03:22:25.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.650+0000 7f1153d80740 1 --2- 172.21.15.6:0/1243691602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b06676680 0x559b0668deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.650+0000 7f1153d80740 1 --2- 172.21.15.6:0/1243691602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b0666bdc0 0x559b0668d970 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.650+0000 7f1153d80740 1 --2- 172.21.15.6:0/1243691602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 0x559b06682510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.650+0000 7f1153d80740 1 -- 172.21.15.6:0/1243691602 >> 172.21.15.6:0/1243691602 conn(0x559b06667a30 msgr2=0x559b066722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:25.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.651+0000 7f1153d80740 1 -- 172.21.15.6:0/1243691602 shutdown_connections 2024-09-12T03:22:25.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.651+0000 7f1153d80740 1 -- 172.21.15.6:0/1243691602 wait complete. 2024-09-12T03:22:25.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.651+0000 7f1153d80740 1 -- start start 2024-09-12T03:22:25.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.651+0000 7f1153d80740 1 Processor -- start 2024-09-12T03:22:25.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.651+0000 7f1153d80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 0x559b0667beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.651+0000 7f1153d80740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b0666bdc0 0x559b0667c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.651+0000 7f1153d80740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b06676680 0x559b066f0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.651+0000 7f1153d80740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f114c1adc80 con 0x559b0666bdc0 2024-09-12T03:22:25.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.651+0000 7f1153d80740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f114c1af0f0 con 0x559b06676680 2024-09-12T03:22:25.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.651+0000 7f1153d80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f114c1ae350 con 0x559b0666b4a0 2024-09-12T03:22:25.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.652+0000 7f1153d80740 10 client.0 _ll_register_callbacks cb 0x7f113c0d2e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:25.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.652+0000 7f114a7fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b0666bdc0 0x559b0667c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.652+0000 7f114a7fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b0666bdc0 0x559b0667c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38866/0 (socket says 172.21.15.6:38866) 2024-09-12T03:22:25.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.652+0000 7f114a7fc640 1 -- 172.21.15.6:0/2285526016 learned_addr learned my addr 172.21.15.6:0/2285526016 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:25.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.652+0000 7f114affd640 1 --2- 172.21.15.6:0/2285526016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 0x559b0667beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.652+0000 7f114b7fe640 1 --2- 172.21.15.6:0/2285526016 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b06676680 0x559b066f0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.652+0000 7f114a7fc640 1 -- 172.21.15.6:0/2285526016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 msgr2=0x559b0667beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.652+0000 7f114a7fc640 1 --2- 172.21.15.6:0/2285526016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 0x559b0667beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.652+0000 7f114a7fc640 1 -- 172.21.15.6:0/2285526016 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b06676680 msgr2=0x559b066f0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.652+0000 7f114a7fc640 1 --2- 172.21.15.6:0/2285526016 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b06676680 0x559b066f0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.652+0000 7f114a7fc640 1 -- 172.21.15.6:0/2285526016 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b0666c3d0 con 0x559b0666bdc0 2024-09-12T03:22:25.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.652+0000 7f114affd640 1 --2- 172.21.15.6:0/2285526016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 0x559b0667beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:25.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.652+0000 7f114a7fc640 1 --2- 172.21.15.6:0/2285526016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b0666bdc0 0x559b0667c3f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1140033000 tx=0x7f1140009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd7e874a860933a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:25.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.652+0000 7f112affd640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:22:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.653+0000 7f112affd640 1 -- 172.21.15.6:0/2285526016 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1140035070 con 0x559b0666bdc0 2024-09-12T03:22:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.653+0000 7f112affd640 1 -- 172.21.15.6:0/2285526016 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f11400037f0 con 0x559b0666bdc0 2024-09-12T03:22:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.653+0000 7f1153d80740 1 -- 172.21.15.6:0/2285526016 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1140004f00 con 0x559b0666bdc0 2024-09-12T03:22:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.653+0000 7f112affd640 1 -- 172.21.15.6:0/2285526016 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1140068050 con 0x559b0666bdc0 2024-09-12T03:22:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.653+0000 7f1153d80740 10 client.15498 fetch_fsmap learned FSMap version 26 2024-09-12T03:22:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.653+0000 7f1153d80740 1 -- 172.21.15.6:0/2285526016 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559b066f8b30 con 0x559b0666bdc0 2024-09-12T03:22:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.653+0000 7f112affd640 1 -- 172.21.15.6:0/2285526016 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1140004f00 con 0x559b0666bdc0 2024-09-12T03:22:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.653+0000 7f112affd640 1 -- 172.21.15.6:0/2285526016 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f114004b070 con 0x559b0666bdc0 2024-09-12T03:22:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.653+0000 7f1153d80740 10 client.15498 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:22:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.653+0000 7f1153d80740 10 client.15498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:22:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.653+0000 7f1153d80740 20 client.15498 populate_metadata read hostname 'smithi006' 2024-09-12T03:22:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.653+0000 7f1153d80740 1 --2- 172.21.15.6:0/2285526016 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x559b066f92a0 0x559b066fb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.653+0000 7f1153d80740 4 client.15498 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:22:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.653+0000 7f1153d80740 1 -- 172.21.15.6:0/2285526016 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559b065d8d60 con 0x559b066f92a0 2024-09-12T03:22:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.654+0000 7f114affd640 1 --2- 172.21.15.6:0/2285526016 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x559b066f92a0 0x559b066fb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.654+0000 7f112affd640 10 client.15498 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.654+0000 7f114affd640 1 --2- 172.21.15.6:0/2285526016 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x559b066f92a0 0x559b066fb6c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b336c330d3cb37a server_cookie=64bb266162be6a9b in_seq=0 out_seq=0 2024-09-12T03:22:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.654+0000 7f112affd640 1 -- 172.21.15.6:0/2285526016 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559b065d8d60 con 0x559b066f92a0 2024-09-12T03:22:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.654+0000 7f112affd640 10 client.15498 handle_command_reply: tid=0 2024-09-12T03:22:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.654+0000 7f1153d80740 1 client.15498 shutdown 2024-09-12T03:22:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.654+0000 7f1153d80740 1 -- 172.21.15.6:0/2285526016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b0666bdc0 msgr2=0x559b0667c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.654+0000 7f1153d80740 1 --2- 172.21.15.6:0/2285526016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b0666bdc0 0x559b0667c3f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1140033000 tx=0x7f1140009590 comp rx=0 tx=0).stop 2024-09-12T03:22:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.654+0000 7f1153d80740 1 -- 172.21.15.6:0/2285526016 shutdown_connections 2024-09-12T03:22:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.654+0000 7f1153d80740 1 --2- 172.21.15.6:0/2285526016 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x559b066f92a0 0x559b066fb6c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.654+0000 7f1153d80740 1 --2- 172.21.15.6:0/2285526016 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b06676680 0x559b066f0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.654+0000 7f1153d80740 1 --2- 172.21.15.6:0/2285526016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b0666bdc0 0x559b0667c3f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.654+0000 7f1153d80740 1 --2- 172.21.15.6:0/2285526016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0666b4a0 0x559b0667beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.654+0000 7f1153d80740 1 -- 172.21.15.6:0/2285526016 >> 172.21.15.6:0/2285526016 conn(0x559b06667a30 msgr2=0x559b06674a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:25.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.654+0000 7f112affd640 0 client.15498 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:25.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.655+0000 7f1153d80740 1 -- 172.21.15.6:0/2285526016 shutdown_connections 2024-09-12T03:22:25.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.655+0000 7f1153d80740 1 -- 172.21.15.6:0/2285526016 wait complete. 2024-09-12T03:22:25.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.655+0000 7f1153d80740 20 client.15498 trim_cache size 0 max 16384 2024-09-12T03:22:25.672 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:22:25.672 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2024-09-12T03:22:25.672 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2024-09-12T03:22:25.672 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:22:25.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.655+0000 7f11519a7640 1 -- 172.21.15.6:0/2290321508 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f11340a2b80 msgr2=0x7f11340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.655+0000 7f11519a7640 1 --2- 172.21.15.6:0/2290321508 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f11340a2b80 0x7f11340a5040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1144005fd0 tx=0x7f11440096e0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.655+0000 7f11519a7640 1 -- 172.21.15.6:0/2290321508 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f114c102dd0 msgr2=0x7f114c100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.655+0000 7f11519a7640 1 --2- 172.21.15.6:0/2290321508 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f114c102dd0 0x7f114c100a30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f113c00e9f0 tx=0x7f113c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.656+0000 7f11519a7640 1 -- 172.21.15.6:0/2290321508 shutdown_connections 2024-09-12T03:22:25.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.656+0000 7f11519a7640 1 --2- 172.21.15.6:0/2290321508 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f11340a2b80 0x7f11340a5040 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.656+0000 7f11519a7640 1 --2- 172.21.15.6:0/2290321508 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f114c108300 0x7f114c1014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.656+0000 7f11519a7640 1 --2- 172.21.15.6:0/2290321508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f114c103780 0x7f114c100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.656+0000 7f11519a7640 1 --2- 172.21.15.6:0/2290321508 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f114c102dd0 0x7f114c100a30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.656+0000 7f11519a7640 1 -- 172.21.15.6:0/2290321508 >> 172.21.15.6:0/2290321508 conn(0x7f114c0fc7d0 msgr2=0x7f114c105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:25.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.656+0000 7f11519a7640 1 -- 172.21.15.6:0/2290321508 shutdown_connections 2024-09-12T03:22:25.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:25.656+0000 7f11519a7640 1 -- 172.21.15.6:0/2290321508 wait complete. 2024-09-12T03:22:25.806 INFO:tasks.workunit.client.0.smithi006.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-09-12T03:22:25.809 INFO:tasks.workunit.client.0.smithi006.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-09-12T03:22:25.881 INFO:tasks.workunit.client.0.smithi006.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-09-12T03:22:25.884 INFO:tasks.workunit.client.0.smithi006.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-09-12T03:22:25.958 INFO:tasks.workunit.client.0.smithi006.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-09-12T03:22:25.960 INFO:tasks.workunit.client.0.smithi006.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-09-12T03:22:26.067 INFO:tasks.workunit.client.0.smithi006.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-09-12T03:22:26.070 INFO:tasks.workunit.client.0.smithi006.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-09-12T03:22:26.098 INFO:tasks.workunit.client.0.smithi006.stderr:In file included from parser.h:22, 2024-09-12T03:22:26.098 INFO:tasks.workunit.client.0.smithi006.stderr: from parser.c:25: 2024-09-12T03:22:26.098 INFO:tasks.workunit.client.0.smithi006.stderr:parser.c: In function ‘init_filesys’: 2024-09-12T03:22:26.098 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T03:22:26.098 INFO:tasks.workunit.client.0.smithi006.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T03:22:26.098 INFO:tasks.workunit.client.0.smithi006.stderr: | ^ 2024-09-12T03:22:26.098 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T03:22:26.098 INFO:tasks.workunit.client.0.smithi006.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T03:22:26.098 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~ 2024-09-12T03:22:26.098 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T03:22:26.099 INFO:tasks.workunit.client.0.smithi006.stderr: 36 | container_of(ptr, type, member) 2024-09-12T03:22:26.099 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~ 2024-09-12T03:22:26.099 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-12T03:22:26.099 INFO:tasks.workunit.client.0.smithi006.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-12T03:22:26.099 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~ 2024-09-12T03:22:26.099 INFO:tasks.workunit.client.0.smithi006.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T03:22:26.099 INFO:tasks.workunit.client.0.smithi006.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-12T03:22:26.099 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T03:22:26.099 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T03:22:26.099 INFO:tasks.workunit.client.0.smithi006.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T03:22:26.099 INFO:tasks.workunit.client.0.smithi006.stderr: | ^ 2024-09-12T03:22:26.099 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T03:22:26.100 INFO:tasks.workunit.client.0.smithi006.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T03:22:26.100 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~ 2024-09-12T03:22:26.100 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T03:22:26.100 INFO:tasks.workunit.client.0.smithi006.stderr: 36 | container_of(ptr, type, member) 2024-09-12T03:22:26.100 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~ 2024-09-12T03:22:26.100 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-12T03:22:26.100 INFO:tasks.workunit.client.0.smithi006.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-12T03:22:26.100 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~ 2024-09-12T03:22:26.100 INFO:tasks.workunit.client.0.smithi006.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T03:22:26.100 INFO:tasks.workunit.client.0.smithi006.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-12T03:22:26.100 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T03:22:26.100 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T03:22:26.100 INFO:tasks.workunit.client.0.smithi006.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T03:22:26.101 INFO:tasks.workunit.client.0.smithi006.stderr: | ^ 2024-09-12T03:22:26.101 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T03:22:26.101 INFO:tasks.workunit.client.0.smithi006.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T03:22:26.101 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~ 2024-09-12T03:22:26.101 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T03:22:26.101 INFO:tasks.workunit.client.0.smithi006.stderr: 36 | container_of(ptr, type, member) 2024-09-12T03:22:26.101 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~ 2024-09-12T03:22:26.101 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-12T03:22:26.101 INFO:tasks.workunit.client.0.smithi006.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-12T03:22:26.101 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~ 2024-09-12T03:22:26.101 INFO:tasks.workunit.client.0.smithi006.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T03:22:26.101 INFO:tasks.workunit.client.0.smithi006.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-12T03:22:26.101 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T03:22:26.102 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T03:22:26.102 INFO:tasks.workunit.client.0.smithi006.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T03:22:26.102 INFO:tasks.workunit.client.0.smithi006.stderr: | ^ 2024-09-12T03:22:26.102 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T03:22:26.102 INFO:tasks.workunit.client.0.smithi006.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T03:22:26.102 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~ 2024-09-12T03:22:26.102 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T03:22:26.102 INFO:tasks.workunit.client.0.smithi006.stderr: 36 | container_of(ptr, type, member) 2024-09-12T03:22:26.102 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~ 2024-09-12T03:22:26.102 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-12T03:22:26.102 INFO:tasks.workunit.client.0.smithi006.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-12T03:22:26.102 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~ 2024-09-12T03:22:26.103 INFO:tasks.workunit.client.0.smithi006.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T03:22:26.103 INFO:tasks.workunit.client.0.smithi006.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-12T03:22:26.103 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T03:22:26.103 INFO:tasks.workunit.client.0.smithi006.stderr:parser.c: In function ‘init_tg_stats’: 2024-09-12T03:22:26.103 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T03:22:26.103 INFO:tasks.workunit.client.0.smithi006.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T03:22:26.103 INFO:tasks.workunit.client.0.smithi006.stderr: | ^ 2024-09-12T03:22:26.103 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T03:22:26.103 INFO:tasks.workunit.client.0.smithi006.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T03:22:26.103 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~ 2024-09-12T03:22:26.103 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T03:22:26.103 INFO:tasks.workunit.client.0.smithi006.stderr: 36 | container_of(ptr, type, member) 2024-09-12T03:22:26.104 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~ 2024-09-12T03:22:26.104 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-12T03:22:26.104 INFO:tasks.workunit.client.0.smithi006.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-12T03:22:26.104 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~ 2024-09-12T03:22:26.104 INFO:tasks.workunit.client.0.smithi006.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T03:22:26.104 INFO:tasks.workunit.client.0.smithi006.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-12T03:22:26.104 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T03:22:26.104 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T03:22:26.104 INFO:tasks.workunit.client.0.smithi006.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T03:22:26.104 INFO:tasks.workunit.client.0.smithi006.stderr: | ^ 2024-09-12T03:22:26.104 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T03:22:26.104 INFO:tasks.workunit.client.0.smithi006.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T03:22:26.104 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~ 2024-09-12T03:22:26.105 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T03:22:26.105 INFO:tasks.workunit.client.0.smithi006.stderr: 36 | container_of(ptr, type, member) 2024-09-12T03:22:26.105 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~ 2024-09-12T03:22:26.105 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-12T03:22:26.105 INFO:tasks.workunit.client.0.smithi006.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-12T03:22:26.105 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~ 2024-09-12T03:22:26.105 INFO:tasks.workunit.client.0.smithi006.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T03:22:26.105 INFO:tasks.workunit.client.0.smithi006.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-12T03:22:26.105 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T03:22:26.105 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T03:22:26.105 INFO:tasks.workunit.client.0.smithi006.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T03:22:26.105 INFO:tasks.workunit.client.0.smithi006.stderr: | ^ 2024-09-12T03:22:26.105 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T03:22:26.106 INFO:tasks.workunit.client.0.smithi006.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T03:22:26.106 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~ 2024-09-12T03:22:26.106 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T03:22:26.106 INFO:tasks.workunit.client.0.smithi006.stderr: 36 | container_of(ptr, type, member) 2024-09-12T03:22:26.106 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~ 2024-09-12T03:22:26.106 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-12T03:22:26.106 INFO:tasks.workunit.client.0.smithi006.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-12T03:22:26.106 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~ 2024-09-12T03:22:26.106 INFO:tasks.workunit.client.0.smithi006.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T03:22:26.106 INFO:tasks.workunit.client.0.smithi006.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-12T03:22:26.106 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T03:22:26.106 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T03:22:26.106 INFO:tasks.workunit.client.0.smithi006.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T03:22:26.106 INFO:tasks.workunit.client.0.smithi006.stderr: | ^ 2024-09-12T03:22:26.107 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T03:22:26.107 INFO:tasks.workunit.client.0.smithi006.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T03:22:26.107 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~ 2024-09-12T03:22:26.107 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T03:22:26.107 INFO:tasks.workunit.client.0.smithi006.stderr: 36 | container_of(ptr, type, member) 2024-09-12T03:22:26.107 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~ 2024-09-12T03:22:26.107 INFO:tasks.workunit.client.0.smithi006.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-12T03:22:26.107 INFO:tasks.workunit.client.0.smithi006.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-12T03:22:26.107 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~ 2024-09-12T03:22:26.107 INFO:tasks.workunit.client.0.smithi006.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T03:22:26.107 INFO:tasks.workunit.client.0.smithi006.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-12T03:22:26.107 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T03:22:26.149 INFO:tasks.workunit.client.0.smithi006.stderr:parser.c: In function ‘get_optstr’: 2024-09-12T03:22:26.149 INFO:tasks.workunit.client.0.smithi006.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-09-12T03:22:26.149 INFO:tasks.workunit.client.0.smithi006.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-09-12T03:22:26.149 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T03:22:26.149 INFO:tasks.workunit.client.0.smithi006.stderr:parser.c:200:14: note: source object declared here 2024-09-12T03:22:26.149 INFO:tasks.workunit.client.0.smithi006.stderr: 200 | char temp[BUFSIZE]; 2024-09-12T03:22:26.149 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~ 2024-09-12T03:22:26.157 INFO:tasks.workunit.client.0.smithi006.stderr:parser.c: In function ‘size64_convert’: 2024-09-12T03:22:26.158 INFO:tasks.workunit.client.0.smithi006.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-09-12T03:22:26.158 INFO:tasks.workunit.client.0.smithi006.stderr: 119 | memcpy(unit, "\0", 3); 2024-09-12T03:22:26.158 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-09-12T03:22:26.455 INFO:tasks.workunit.client.0.smithi006.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-09-12T03:22:26.458 INFO:tasks.workunit.client.0.smithi006.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-09-12T03:22:26.518 INFO:tasks.workunit.client.0.smithi006.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-09-12T03:22:26.521 INFO:tasks.workunit.client.0.smithi006.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-09-12T03:22:26.609 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:22:26.610 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:22:26.616 INFO:tasks.workunit.client.0.smithi006.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-09-12T03:22:26.618 INFO:tasks.workunit.client.0.smithi006.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-09-12T03:22:26.650 INFO:tasks.workunit.client.0.smithi006.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-09-12T03:22:26.653 INFO:tasks.workunit.client.0.smithi006.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-09-12T03:22:26.687 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-12T03:22:26.687 INFO:tasks.workunit.client.0.smithi006.stderr:+ cd .. 2024-09-12T03:22:26.687 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir tmp 2024-09-12T03:22:26.689 INFO:tasks.workunit.client.0.smithi006.stderr:+ cd tmp 2024-09-12T03:22:26.689 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in $mydir/*.ffsb 2024-09-12T03:22:26.689 INFO:tasks.workunit.client.0.smithi006.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-09-12T03:22:26.689 INFO:tasks.workunit.client.0.smithi006.stdout:FFSB version 6.0-RC2 started 2024-09-12T03:22:26.689 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-09-12T03:22:26.692 INFO:tasks.workunit.client.0.smithi006.stdout:benchmark time = 300 2024-09-12T03:22:26.692 INFO:tasks.workunit.client.0.smithi006.stdout:ThreadGroup 0 2024-09-12T03:22:26.692 INFO:tasks.workunit.client.0.smithi006.stdout:================ 2024-09-12T03:22:26.692 INFO:tasks.workunit.client.0.smithi006.stdout: num_threads = 32 2024-09-12T03:22:26.692 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-09-12T03:22:26.692 INFO:tasks.workunit.client.0.smithi006.stdout: read_random = off 2024-09-12T03:22:26.692 INFO:tasks.workunit.client.0.smithi006.stdout: read_size = 0 (0B) 2024-09-12T03:22:26.692 INFO:tasks.workunit.client.0.smithi006.stdout: read_blocksize = 0 (0B) 2024-09-12T03:22:26.693 INFO:tasks.workunit.client.0.smithi006.stdout: read_skip = off 2024-09-12T03:22:26.693 INFO:tasks.workunit.client.0.smithi006.stdout: read_skipsize = 0 (0B) 2024-09-12T03:22:26.693 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-09-12T03:22:26.693 INFO:tasks.workunit.client.0.smithi006.stdout: write_random = on 2024-09-12T03:22:26.693 INFO:tasks.workunit.client.0.smithi006.stdout: write_size = 5242880 (5MB) 2024-09-12T03:22:26.693 INFO:tasks.workunit.client.0.smithi006.stdout: fsync_file = 0 2024-09-12T03:22:26.693 INFO:tasks.workunit.client.0.smithi006.stdout: write_blocksize = 4096 (4KB) 2024-09-12T03:22:26.693 INFO:tasks.workunit.client.0.smithi006.stdout: wait time = 0 2024-09-12T03:22:26.693 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-09-12T03:22:26.693 INFO:tasks.workunit.client.0.smithi006.stdout: op weights 2024-09-12T03:22:26.693 INFO:tasks.workunit.client.0.smithi006.stdout: read = 0 (0.00%) 2024-09-12T03:22:26.693 INFO:tasks.workunit.client.0.smithi006.stdout: readall = 0 (0.00%) 2024-09-12T03:22:26.693 INFO:tasks.workunit.client.0.smithi006.stdout: write = 1 (100.00%) 2024-09-12T03:22:26.693 INFO:tasks.workunit.client.0.smithi006.stdout: create = 0 (0.00%) 2024-09-12T03:22:26.694 INFO:tasks.workunit.client.0.smithi006.stdout: append = 0 (0.00%) 2024-09-12T03:22:26.694 INFO:tasks.workunit.client.0.smithi006.stdout: delete = 0 (0.00%) 2024-09-12T03:22:26.694 INFO:tasks.workunit.client.0.smithi006.stdout: metaop = 0 (0.00%) 2024-09-12T03:22:26.694 INFO:tasks.workunit.client.0.smithi006.stdout: createdir = 0 (0.00%) 2024-09-12T03:22:26.694 INFO:tasks.workunit.client.0.smithi006.stdout: stat = 0 (0.00%) 2024-09-12T03:22:26.694 INFO:tasks.workunit.client.0.smithi006.stdout: writeall = 0 (0.00%) 2024-09-12T03:22:26.694 INFO:tasks.workunit.client.0.smithi006.stdout: writeall_fsync = 0 (0.00%) 2024-09-12T03:22:26.694 INFO:tasks.workunit.client.0.smithi006.stdout: open_close = 0 (0.00%) 2024-09-12T03:22:26.694 INFO:tasks.workunit.client.0.smithi006.stdout: write_fsync = 0 (0.00%) 2024-09-12T03:22:26.694 INFO:tasks.workunit.client.0.smithi006.stdout: create_fsync = 0 (0.00%) 2024-09-12T03:22:26.694 INFO:tasks.workunit.client.0.smithi006.stdout: append_fsync = 0 (0.00%) 2024-09-12T03:22:26.694 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-09-12T03:22:26.695 INFO:tasks.workunit.client.0.smithi006.stdout:FileSystem . 2024-09-12T03:22:26.695 INFO:tasks.workunit.client.0.smithi006.stdout:========== 2024-09-12T03:22:26.695 INFO:tasks.workunit.client.0.smithi006.stdout: num_dirs = 0 2024-09-12T03:22:26.695 INFO:tasks.workunit.client.0.smithi006.stdout: starting files = 128 2024-09-12T03:22:26.695 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-09-12T03:22:26.695 INFO:tasks.workunit.client.0.smithi006.stdout: min file size = 104857600 (100MB) 2024-09-12T03:22:26.695 INFO:tasks.workunit.client.0.smithi006.stdout: max file size = 104857600 (100MB) 2024-09-12T03:22:26.695 INFO:tasks.workunit.client.0.smithi006.stdout: directio = off 2024-09-12T03:22:26.695 INFO:tasks.workunit.client.0.smithi006.stdout: alignedio = on 2024-09-12T03:22:26.695 INFO:tasks.workunit.client.0.smithi006.stdout: bufferedio = off 2024-09-12T03:22:26.695 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-09-12T03:22:26.695 INFO:tasks.workunit.client.0.smithi006.stdout: aging is off 2024-09-12T03:22:26.696 INFO:tasks.workunit.client.0.smithi006.stdout: current utilization = 0.00% 2024-09-12T03:22:26.696 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-09-12T03:22:26.696 INFO:tasks.workunit.client.0.smithi006.stdout:checking existing fs: . 2024-09-12T03:22:26.696 INFO:tasks.workunit.client.0.smithi006.stdout:recreating new fileset 2024-09-12T03:22:26.696 INFO:tasks.workunit.client.0.smithi006.stderr:opendir: No such file or directory 2024-09-12T03:22:26.892 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:27.397 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:27 smithi006 ceph-mon[30991]: pgmap v375: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 481 KiB/s wr, 60 op/s 2024-09-12T03:22:27.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.403+0000 7f15cd655640 1 -- 172.21.15.6:0/1034193099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15c8102790 msgr2=0x7f15c81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.403+0000 7f15cd655640 1 --2- 172.21.15.6:0/1034193099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15c8102790 0x7f15c81067e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009900 tx=0x7f15bc02a410 comp rx=0 tx=0).stop 2024-09-12T03:22:27.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.404+0000 7f15cd655640 1 -- 172.21.15.6:0/1034193099 shutdown_connections 2024-09-12T03:22:27.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.404+0000 7f15cd655640 1 --2- 172.21.15.6:0/1034193099 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15c8106f10 0x7f15c810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.404+0000 7f15cd655640 1 --2- 172.21.15.6:0/1034193099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15c8102790 0x7f15c81067e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.404+0000 7f15cd655640 1 --2- 172.21.15.6:0/1034193099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15c8101de0 0x7f15c81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.404+0000 7f15cd655640 1 -- 172.21.15.6:0/1034193099 >> 172.21.15.6:0/1034193099 conn(0x7f15c80fd8c0 msgr2=0x7f15c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:27.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.404+0000 7f15cd655640 1 -- 172.21.15.6:0/1034193099 shutdown_connections 2024-09-12T03:22:27.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.404+0000 7f15cd655640 1 -- 172.21.15.6:0/1034193099 wait complete. 2024-09-12T03:22:27.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.404+0000 7f15cd655640 1 Processor -- start 2024-09-12T03:22:27.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.405+0000 7f15cd655640 1 -- start start 2024-09-12T03:22:27.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.405+0000 7f15cd655640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15c8101de0 0x7f15c8198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.405+0000 7f15cd655640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15c8102790 0x7f15c8198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.405+0000 7f15cd655640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15c8106f10 0x7f15c819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.405+0000 7f15cd655640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f15c8111f70 con 0x7f15c8102790 2024-09-12T03:22:27.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.405+0000 7f15cd655640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f15c8111df0 con 0x7f15c8106f10 2024-09-12T03:22:27.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.405+0000 7f15cd655640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f15c81120f0 con 0x7f15c8101de0 2024-09-12T03:22:27.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.405+0000 7f15c67fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15c8102790 0x7f15c8198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.405+0000 7f15c67fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15c8102790 0x7f15c8198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38882/0 (socket says 172.21.15.6:38882) 2024-09-12T03:22:27.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.405+0000 7f15c67fc640 1 -- 172.21.15.6:0/4206717975 learned_addr learned my addr 172.21.15.6:0/4206717975 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:27.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.405+0000 7f15c6ffd640 1 --2- 172.21.15.6:0/4206717975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15c8101de0 0x7f15c8198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.405+0000 7f15c77fe640 1 --2- 172.21.15.6:0/4206717975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15c8106f10 0x7f15c819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.405+0000 7f15c6ffd640 1 -- 172.21.15.6:0/4206717975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15c8106f10 msgr2=0x7f15c819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.405+0000 7f15c6ffd640 1 --2- 172.21.15.6:0/4206717975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15c8106f10 0x7f15c819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.405+0000 7f15c6ffd640 1 -- 172.21.15.6:0/4206717975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15c8102790 msgr2=0x7f15c8198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.405+0000 7f15c6ffd640 1 --2- 172.21.15.6:0/4206717975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15c8102790 0x7f15c8198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.405+0000 7f15c6ffd640 1 -- 172.21.15.6:0/4206717975 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c819d4c0 con 0x7f15c8101de0 2024-09-12T03:22:27.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.406+0000 7f15c6ffd640 1 --2- 172.21.15.6:0/4206717975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15c8101de0 0x7f15c8198510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f15b800e9f0 tx=0x7f15b800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:27.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.406+0000 7f15affff640 1 -- 172.21.15.6:0/4206717975 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b800ccc0 con 0x7f15c8101de0 2024-09-12T03:22:27.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.406+0000 7f15affff640 1 -- 172.21.15.6:0/4206717975 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15b8005020 con 0x7f15c8101de0 2024-09-12T03:22:27.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.406+0000 7f15cd655640 1 -- 172.21.15.6:0/4206717975 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c819d7b0 con 0x7f15c8101de0 2024-09-12T03:22:27.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.406+0000 7f15affff640 1 -- 172.21.15.6:0/4206717975 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8004a00 con 0x7f15c8101de0 2024-09-12T03:22:27.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.406+0000 7f15cd655640 1 -- 172.21.15.6:0/4206717975 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c81a5090 con 0x7f15c8101de0 2024-09-12T03:22:27.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.407+0000 7f15cd655640 1 -- 172.21.15.6:0/4206717975 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15c81021c0 con 0x7f15c8101de0 2024-09-12T03:22:27.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.408+0000 7f15affff640 1 -- 172.21.15.6:0/4206717975 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f15b8004ba0 con 0x7f15c8101de0 2024-09-12T03:22:27.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.409+0000 7f15affff640 1 --2- 172.21.15.6:0/4206717975 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f15b00a2b80 0x7f15b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.409+0000 7f15c67fc640 1 --2- 172.21.15.6:0/4206717975 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f15b00a2b80 0x7f15b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.409+0000 7f15affff640 1 -- 172.21.15.6:0/4206717975 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f15b80cf040 con 0x7f15c8101de0 2024-09-12T03:22:27.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.409+0000 7f15c67fc640 1 --2- 172.21.15.6:0/4206717975 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f15b00a2b80 0x7f15b00a5040 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f15bc031550 tx=0x7f15bc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:27.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.413+0000 7f15affff640 1 -- 172.21.15.6:0/4206717975 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f15b809bf70 con 0x7f15c8101de0 2024-09-12T03:22:27.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:27 smithi118 ceph-mon[35048]: pgmap v375: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 481 KiB/s wr, 60 op/s 2024-09-12T03:22:27.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:27 smithi161 ceph-mon[35485]: pgmap v375: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 481 KiB/s wr, 60 op/s 2024-09-12T03:22:27.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.627+0000 7f15cd655640 1 -- 172.21.15.6:0/4206717975 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15c810a150 con 0x7f15c8101de0 2024-09-12T03:22:27.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.628+0000 7f15affff640 1 -- 172.21.15.6:0/4206717975 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11911 (secure 0 0 0) 0x7f15b809b870 con 0x7f15c8101de0 2024-09-12T03:22:27.629 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:22:27.629 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":26,"btime":"2024-09-12T03:22:16:132848+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:22:16.064679+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24463,24464,24494]},"id":1}]} 2024-09-12T03:22:27.629 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 26 2024-09-12T03:22:27.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.630+0000 7f15cd655640 1 -- 172.21.15.6:0/4206717975 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f15b00a2b80 msgr2=0x7f15b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.630+0000 7f15cd655640 1 --2- 172.21.15.6:0/4206717975 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f15b00a2b80 0x7f15b00a5040 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f15bc031550 tx=0x7f15bc005a20 comp rx=0 tx=0).stop 2024-09-12T03:22:27.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.630+0000 7f15cd655640 1 -- 172.21.15.6:0/4206717975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15c8101de0 msgr2=0x7f15c8198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.630+0000 7f15cd655640 1 --2- 172.21.15.6:0/4206717975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15c8101de0 0x7f15c8198510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f15b800e9f0 tx=0x7f15b800edf0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.630+0000 7f15cd655640 1 -- 172.21.15.6:0/4206717975 shutdown_connections 2024-09-12T03:22:27.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.631+0000 7f15cd655640 1 --2- 172.21.15.6:0/4206717975 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f15b00a2b80 0x7f15b00a5040 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.631+0000 7f15cd655640 1 --2- 172.21.15.6:0/4206717975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15c8106f10 0x7f15c819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.631+0000 7f15cd655640 1 --2- 172.21.15.6:0/4206717975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15c8102790 0x7f15c8198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.631+0000 7f15cd655640 1 --2- 172.21.15.6:0/4206717975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15c8101de0 0x7f15c8198510 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.631+0000 7f15cd655640 1 -- 172.21.15.6:0/4206717975 >> 172.21.15.6:0/4206717975 conn(0x7f15c80fd8c0 msgr2=0x7f15c80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:27.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.631+0000 7f15cd655640 1 -- 172.21.15.6:0/4206717975 shutdown_connections 2024-09-12T03:22:27.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:27.631+0000 7f15cd655640 1 -- 172.21.15.6:0/4206717975 wait complete. 2024-09-12T03:22:28.376 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:22:28.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:28 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/4206717975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:22:28.513 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:28 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/4206717975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:22:28.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:28 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/4206717975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:22:28.830 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:29.227 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:29 smithi006 ceph-mon[30991]: pgmap v376: 97 pgs: 97 active+clean; 6.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 553 KiB/s wr, 65 op/s 2024-09-12T03:22:29.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.516+0000 7f0a2f4ee640 1 -- 172.21.15.6:0/774401507 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a28101df0 msgr2=0x7f0a281021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.516+0000 7f0a2f4ee640 1 --2- 172.21.15.6:0/774401507 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a28101df0 0x7f0a281021d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0a24009900 tx=0x7f0a2402a3f0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.516+0000 7f0a2f4ee640 1 -- 172.21.15.6:0/774401507 shutdown_connections 2024-09-12T03:22:29.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.516+0000 7f0a2f4ee640 1 --2- 172.21.15.6:0/774401507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a28106f20 0x7f0a2810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.516+0000 7f0a2f4ee640 1 --2- 172.21.15.6:0/774401507 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a281027a0 0x7f0a281067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.516+0000 7f0a2f4ee640 1 --2- 172.21.15.6:0/774401507 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a28101df0 0x7f0a281021d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.516+0000 7f0a2f4ee640 1 -- 172.21.15.6:0/774401507 >> 172.21.15.6:0/774401507 conn(0x7f0a280fd8b0 msgr2=0x7f0a280ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:29.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.516+0000 7f0a2f4ee640 1 -- 172.21.15.6:0/774401507 shutdown_connections 2024-09-12T03:22:29.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.516+0000 7f0a2f4ee640 1 -- 172.21.15.6:0/774401507 wait complete. 2024-09-12T03:22:29.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.517+0000 7f0a2f4ee640 1 Processor -- start 2024-09-12T03:22:29.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.517+0000 7f0a2f4ee640 1 -- start start 2024-09-12T03:22:29.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.517+0000 7f0a2f4ee640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a28101df0 0x7f0a281983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.517+0000 7f0a2f4ee640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a281027a0 0x7f0a281988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.517+0000 7f0a2f4ee640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a28106f20 0x7f0a2819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.517+0000 7f0a2f4ee640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0a28111f80 con 0x7f0a281027a0 2024-09-12T03:22:29.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.517+0000 7f0a2f4ee640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f0a28111e00 con 0x7f0a28101df0 2024-09-12T03:22:29.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.517+0000 7f0a2f4ee640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0a28112100 con 0x7f0a28106f20 2024-09-12T03:22:29.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.517+0000 7f0a2ca89640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a281027a0 0x7f0a281988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.517+0000 7f0a2ca89640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a281027a0 0x7f0a281988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44358/0 (socket says 172.21.15.6:44358) 2024-09-12T03:22:29.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.517+0000 7f0a2ca89640 1 -- 172.21.15.6:0/868885592 learned_addr learned my addr 172.21.15.6:0/868885592 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:29.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.517+0000 7f0a2da8b640 1 --2- 172.21.15.6:0/868885592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a28106f20 0x7f0a2819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.517+0000 7f0a2d28a640 1 --2- 172.21.15.6:0/868885592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a28101df0 0x7f0a281983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.517+0000 7f0a2ca89640 1 -- 172.21.15.6:0/868885592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a28106f20 msgr2=0x7f0a2819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.518+0000 7f0a2ca89640 1 --2- 172.21.15.6:0/868885592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a28106f20 0x7f0a2819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.518+0000 7f0a2ca89640 1 -- 172.21.15.6:0/868885592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a28101df0 msgr2=0x7f0a281983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.518+0000 7f0a2ca89640 1 --2- 172.21.15.6:0/868885592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a28101df0 0x7f0a281983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.518+0000 7f0a2ca89640 1 -- 172.21.15.6:0/868885592 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2819d350 con 0x7f0a281027a0 2024-09-12T03:22:29.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.518+0000 7f0a2d28a640 1 --2- 172.21.15.6:0/868885592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a28101df0 0x7f0a281983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:29.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.518+0000 7f0a2da8b640 1 --2- 172.21.15.6:0/868885592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a28106f20 0x7f0a2819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:29.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.518+0000 7f0a2ca89640 1 --2- 172.21.15.6:0/868885592 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a281027a0 0x7f0a281988e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800c920 tx=0x7f0a1800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:29.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.518+0000 7f0a1e7fc640 1 -- 172.21.15.6:0/868885592 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18007c70 con 0x7f0a281027a0 2024-09-12T03:22:29.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.518+0000 7f0a2f4ee640 1 -- 172.21.15.6:0/868885592 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2819d640 con 0x7f0a281027a0 2024-09-12T03:22:29.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.518+0000 7f0a1e7fc640 1 -- 172.21.15.6:0/868885592 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a18007e10 con 0x7f0a281027a0 2024-09-12T03:22:29.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.518+0000 7f0a1e7fc640 1 -- 172.21.15.6:0/868885592 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18018890 con 0x7f0a281027a0 2024-09-12T03:22:29.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.518+0000 7f0a2f4ee640 1 -- 172.21.15.6:0/868885592 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a281a4f20 con 0x7f0a281027a0 2024-09-12T03:22:29.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.521+0000 7f0a1e7fc640 1 -- 172.21.15.6:0/868885592 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0a18018a70 con 0x7f0a281027a0 2024-09-12T03:22:29.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.521+0000 7f0a318c7740 1 Processor -- start 2024-09-12T03:22:29.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.522+0000 7f0a318c7740 1 -- start start 2024-09-12T03:22:29.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.522+0000 7f0a318c7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eeced23b40 0x55eeced22090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.522+0000 7f0a318c7740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced225d0 0x55eeced22a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.522+0000 7f0a318c7740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced31780 0x55eeced33b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.522+0000 7f0a318c7740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55eecec882f0 con 0x55eeced225d0 2024-09-12T03:22:29.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.522+0000 7f0a318c7740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55eecec2fcf0 con 0x55eeced31780 2024-09-12T03:22:29.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.522+0000 7f0a318c7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eecec842b0 con 0x55eeced23b40 2024-09-12T03:22:29.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.522+0000 7f0a2d28a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced225d0 0x55eeced22a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.522+0000 7f0a2ca89640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced31780 0x55eeced33b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.522+0000 7f0a2da8b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eeced23b40 0x55eeced22090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.522+0000 7f0a2d28a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced225d0 0x55eeced22a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44364/0 (socket says 172.21.15.6:44364) 2024-09-12T03:22:29.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.522+0000 7f0a2d28a640 1 -- 172.21.15.6:0/3800513235 learned_addr learned my addr 172.21.15.6:0/3800513235 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:29.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.523+0000 7f0a1e7fc640 1 --2- 172.21.15.6:0/868885592 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0a140a2b80 0x7f0a140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.523+0000 7f0a2d28a640 1 -- 172.21.15.6:0/3800513235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eeced23b40 msgr2=0x55eeced22090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:29.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.523+0000 7f0a2d28a640 1 --2- 172.21.15.6:0/3800513235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eeced23b40 0x55eeced22090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.523+0000 7f0a2d28a640 1 -- 172.21.15.6:0/3800513235 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced31780 msgr2=0x55eeced33b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:29.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.523+0000 7f0a2d28a640 1 --2- 172.21.15.6:0/3800513235 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced31780 0x55eeced33b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.523+0000 7f0a2d28a640 1 -- 172.21.15.6:0/3800513235 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eecec143a0 con 0x55eeced225d0 2024-09-12T03:22:29.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.523+0000 7f0a2da8b640 1 --2- 172.21.15.6:0/868885592 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0a140a2b80 0x7f0a140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.523+0000 7f0a1e7fc640 1 -- 172.21.15.6:0/868885592 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f0a180cf210 con 0x7f0a281027a0 2024-09-12T03:22:29.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.523+0000 7f0a2d28a640 1 --2- 172.21.15.6:0/3800513235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced225d0 0x55eeced22a10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0a24031550 tx=0x7f0a24032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42254b7db46a5539 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:29.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.523+0000 7f0a137fe640 1 -- 172.21.15.6:0/3800513235 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2403f070 con 0x55eeced225d0 2024-09-12T03:22:29.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.523+0000 7f0a137fe640 1 -- 172.21.15.6:0/3800513235 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a240317b0 con 0x55eeced225d0 2024-09-12T03:22:29.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.523+0000 7f0a137fe640 1 -- 172.21.15.6:0/3800513235 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a24035070 con 0x55eeced225d0 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.524+0000 7f0a318c7740 1 -- 172.21.15.6:0/3800513235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced225d0 msgr2=0x55eeced22a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.524+0000 7f0a2da8b640 1 --2- 172.21.15.6:0/868885592 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0a140a2b80 0x7f0a140a5040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0a20005fd0 tx=0x7f0a20004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.524+0000 7f0a318c7740 1 --2- 172.21.15.6:0/3800513235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced225d0 0x55eeced22a10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0a24031550 tx=0x7f0a24032cd0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.524+0000 7f0a318c7740 1 -- 172.21.15.6:0/3800513235 shutdown_connections 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.524+0000 7f0a318c7740 1 --2- 172.21.15.6:0/3800513235 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced31780 0x55eeced33b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.524+0000 7f0a318c7740 1 --2- 172.21.15.6:0/3800513235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced225d0 0x55eeced22a10 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.524+0000 7f0a318c7740 1 --2- 172.21.15.6:0/3800513235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eeced23b40 0x55eeced22090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.524+0000 7f0a318c7740 1 -- 172.21.15.6:0/3800513235 >> 172.21.15.6:0/3800513235 conn(0x55eececeb310 msgr2=0x55eeced1ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.524+0000 7f0a318c7740 1 -- 172.21.15.6:0/3800513235 shutdown_connections 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.525+0000 7f0a318c7740 1 -- 172.21.15.6:0/3800513235 wait complete. 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.525+0000 7f0a318c7740 1 -- start start 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.525+0000 7f0a318c7740 1 Processor -- start 2024-09-12T03:22:29.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.525+0000 7f0a318c7740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced225d0 0x55eeced31110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.525+0000 7f0a318c7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eeced23b40 0x55eeced2e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.525+0000 7f0a318c7740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced31780 0x55eeced2e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.525+0000 7f0a318c7740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55eeced34470 con 0x55eeced31780 2024-09-12T03:22:29.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.525+0000 7f0a318c7740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55eecec43060 con 0x55eeced225d0 2024-09-12T03:22:29.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.525+0000 7f0a318c7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eeced345f0 con 0x55eeced23b40 2024-09-12T03:22:29.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.526+0000 7f0a318c7740 10 client.0 _ll_register_callbacks cb 0x55eece9968c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:29.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.526+0000 7f0a2d28a640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced225d0 0x55eeced31110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.526+0000 7f0a2ca89640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eeced23b40 0x55eeced2e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.526+0000 7f0a2d28a640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced225d0 0x55eeced31110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:35262/0 (socket says 172.21.15.6:35262) 2024-09-12T03:22:29.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.526+0000 7f0a2d28a640 1 -- 172.21.15.6:0/2896907931 learned_addr learned my addr 172.21.15.6:0/2896907931 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:29.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.526+0000 7f0a2da8b640 1 --2- 172.21.15.6:0/2896907931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced31780 0x55eeced2e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.526+0000 7f0a2d28a640 1 -- 172.21.15.6:0/2896907931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eeced23b40 msgr2=0x55eeced2e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:29.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.526+0000 7f0a2d28a640 1 --2- 172.21.15.6:0/2896907931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eeced23b40 0x55eeced2e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.526+0000 7f0a2d28a640 1 -- 172.21.15.6:0/2896907931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced31780 msgr2=0x55eeced2e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:29.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.526+0000 7f0a2d28a640 1 --2- 172.21.15.6:0/2896907931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced31780 0x55eeced2e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.526+0000 7f0a2d28a640 1 -- 172.21.15.6:0/2896907931 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eece9f93c0 con 0x55eeced225d0 2024-09-12T03:22:29.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.526+0000 7f0a2ca89640 1 --2- 172.21.15.6:0/2896907931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eeced23b40 0x55eeced2e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:29.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.526+0000 7f0a2d28a640 1 --2- 172.21.15.6:0/2896907931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced225d0 0x55eeced31110 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0a24000c00 tx=0x7f0a24032ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a67218967fa7193 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:29.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.526+0000 7f0a10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:22:29.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.526+0000 7f0a10ff9640 1 -- 172.21.15.6:0/2896907931 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a24035070 con 0x55eeced225d0 2024-09-12T03:22:29.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.526+0000 7f0a318c7740 1 -- 172.21.15.6:0/2896907931 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a240317b0 con 0x55eeced225d0 2024-09-12T03:22:29.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.526+0000 7f0a10ff9640 1 -- 172.21.15.6:0/2896907931 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f0a240046c0 con 0x55eeced225d0 2024-09-12T03:22:29.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.527+0000 7f0a10ff9640 1 -- 172.21.15.6:0/2896907931 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0a2403f070 con 0x55eeced225d0 2024-09-12T03:22:29.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.527+0000 7f0a318c7740 10 client.25085 fetch_fsmap learned FSMap version 26 2024-09-12T03:22:29.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.527+0000 7f0a318c7740 1 -- 172.21.15.6:0/2896907931 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eeced3fea0 con 0x55eeced225d0 2024-09-12T03:22:29.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.527+0000 7f0a10ff9640 1 -- 172.21.15.6:0/2896907931 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a240317b0 con 0x55eeced225d0 2024-09-12T03:22:29.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.527+0000 7f0a10ff9640 1 -- 172.21.15.6:0/2896907931 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0a2403b980 con 0x55eeced225d0 2024-09-12T03:22:29.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.527+0000 7f0a318c7740 10 client.25085 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:22:29.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.527+0000 7f0a318c7740 10 client.25085 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-12T03:22:29.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.527+0000 7f0a318c7740 20 client.25085 populate_metadata read hostname 'smithi006' 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.527+0000 7f0a318c7740 1 --2- 172.21.15.6:0/2896907931 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55eeced411f0 0x55eeced435e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.527+0000 7f0a318c7740 4 client.25085 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.527+0000 7f0a318c7740 1 -- 172.21.15.6:0/2896907931 --> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eecec95860 con 0x55eeced411f0 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.528+0000 7f0a2ca89640 1 --2- 172.21.15.6:0/2896907931 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55eeced411f0 0x55eeced435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.528+0000 7f0a2ca89640 1 --2- 172.21.15.6:0/2896907931 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55eeced411f0 0x55eeced435e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52567c1d8e845415 server_cookie=bb885b73fc399915 in_seq=0 out_seq=0 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.528+0000 7f0a10ff9640 10 client.25085 ms_handle_connect on v2:172.21.15.161:6838/3645555254 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.529+0000 7f0a10ff9640 1 -- 172.21.15.6:0/2896907931 <== mds.2 v2:172.21.15.161:6838/3645555254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eecec95860 con 0x55eeced411f0 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.529+0000 7f0a10ff9640 10 client.25085 handle_command_reply: tid=0 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.529+0000 7f0a318c7740 1 client.25085 shutdown 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.529+0000 7f0a318c7740 1 -- 172.21.15.6:0/2896907931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced225d0 msgr2=0x55eeced31110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.529+0000 7f0a318c7740 1 --2- 172.21.15.6:0/2896907931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced225d0 0x55eeced31110 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0a24000c00 tx=0x7f0a24032ec0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.529+0000 7f0a318c7740 1 -- 172.21.15.6:0/2896907931 shutdown_connections 2024-09-12T03:22:29.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.529+0000 7f0a318c7740 1 --2- 172.21.15.6:0/2896907931 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55eeced411f0 0x55eeced435e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.529+0000 7f0a318c7740 1 --2- 172.21.15.6:0/2896907931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced31780 0x55eeced2e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.529+0000 7f0a318c7740 1 --2- 172.21.15.6:0/2896907931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eeced23b40 0x55eeced2e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.529+0000 7f0a318c7740 1 --2- 172.21.15.6:0/2896907931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced225d0 0x55eeced31110 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.529+0000 7f0a318c7740 1 -- 172.21.15.6:0/2896907931 >> 172.21.15.6:0/2896907931 conn(0x55eececeb310 msgr2=0x55eeced24960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:29.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.529+0000 7f0a10ff9640 0 client.25085 ms_handle_reset on v2:172.21.15.161:6838/3645555254 2024-09-12T03:22:29.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.529+0000 7f0a318c7740 1 -- 172.21.15.6:0/2896907931 shutdown_connections 2024-09-12T03:22:29.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.530+0000 7f0a318c7740 1 -- 172.21.15.6:0/2896907931 wait complete. 2024-09-12T03:22:29.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.530+0000 7f0a318c7740 20 client.25085 trim_cache size 0 max 16384 2024-09-12T03:22:29.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.559+0000 7f0a318c7740 1 Processor -- start 2024-09-12T03:22:29.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.559+0000 7f0a318c7740 1 -- start start 2024-09-12T03:22:29.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.559+0000 7f0a318c7740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced225d0 0x55eeced3d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.559+0000 7f0a318c7740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced23b40 0x55eeced290b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.559+0000 7f0a318c7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eeced31780 0x55eeced2d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.559+0000 7f0a318c7740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0a281adb00 con 0x55eeced23b40 2024-09-12T03:22:29.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.559+0000 7f0a318c7740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f0a281ab3d0 con 0x55eeced225d0 2024-09-12T03:22:29.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.559+0000 7f0a318c7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0a281a9220 con 0x55eeced31780 2024-09-12T03:22:29.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.560+0000 7f0a2d28a640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced225d0 0x55eeced3d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.560+0000 7f0a2d28a640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced225d0 0x55eeced3d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:35276/0 (socket says 172.21.15.6:35276) 2024-09-12T03:22:29.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.560+0000 7f0a2d28a640 1 -- 172.21.15.6:0/131837116 learned_addr learned my addr 172.21.15.6:0/131837116 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:29.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.560+0000 7f0a2da8b640 1 --2- 172.21.15.6:0/131837116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eeced31780 0x55eeced2d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.560+0000 7f0a2d28a640 1 -- 172.21.15.6:0/131837116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eeced31780 msgr2=0x55eeced2d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:29.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.560+0000 7f0a2d28a640 1 --2- 172.21.15.6:0/131837116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eeced31780 0x55eeced2d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.560+0000 7f0a2ca89640 1 --2- 172.21.15.6:0/131837116 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced23b40 0x55eeced290b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.560+0000 7f0a2d28a640 1 -- 172.21.15.6:0/131837116 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced23b40 msgr2=0x55eeced290b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:29.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.560+0000 7f0a2d28a640 1 --2- 172.21.15.6:0/131837116 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced23b40 0x55eeced290b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.560+0000 7f0a2d28a640 1 -- 172.21.15.6:0/131837116 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeced23740 con 0x55eeced225d0 2024-09-12T03:22:29.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.560+0000 7f0a2ca89640 1 --2- 172.21.15.6:0/131837116 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced23b40 0x55eeced290b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:29.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.560+0000 7f0a2d28a640 1 --2- 172.21.15.6:0/131837116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced225d0 0x55eeced3d710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0a24031550 tx=0x7f0a24032fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=123366ac906647af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:29.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.560+0000 7f0a12ffd640 1 -- 172.21.15.6:0/131837116 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a24035070 con 0x55eeced225d0 2024-09-12T03:22:29.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.560+0000 7f0a12ffd640 1 -- 172.21.15.6:0/131837116 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a2404c070 con 0x55eeced225d0 2024-09-12T03:22:29.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.560+0000 7f0a12ffd640 1 -- 172.21.15.6:0/131837116 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a24004920 con 0x55eeced225d0 2024-09-12T03:22:29.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.561+0000 7f0a318c7740 1 -- 172.21.15.6:0/131837116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced225d0 msgr2=0x55eeced3d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.561+0000 7f0a318c7740 1 --2- 172.21.15.6:0/131837116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced225d0 0x55eeced3d710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0a24031550 tx=0x7f0a24032fb0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.561+0000 7f0a318c7740 1 -- 172.21.15.6:0/131837116 shutdown_connections 2024-09-12T03:22:29.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.561+0000 7f0a318c7740 1 --2- 172.21.15.6:0/131837116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eeced31780 0x55eeced2d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.561+0000 7f0a318c7740 1 --2- 172.21.15.6:0/131837116 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced23b40 0x55eeced290b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.562+0000 7f0a318c7740 1 --2- 172.21.15.6:0/131837116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced225d0 0x55eeced3d710 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.562+0000 7f0a318c7740 1 -- 172.21.15.6:0/131837116 >> 172.21.15.6:0/131837116 conn(0x55eececeb310 msgr2=0x55eeced2cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:29.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.562+0000 7f0a318c7740 1 -- 172.21.15.6:0/131837116 shutdown_connections 2024-09-12T03:22:29.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.562+0000 7f0a318c7740 1 -- 172.21.15.6:0/131837116 wait complete. 2024-09-12T03:22:29.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.562+0000 7f0a318c7740 1 -- start start 2024-09-12T03:22:29.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.562+0000 7f0a318c7740 1 Processor -- start 2024-09-12T03:22:29.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.563+0000 7f0a318c7740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced225d0 0x55eeced25670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.563+0000 7f0a318c7740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced23b40 0x55eeced25bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.563+0000 7f0a318c7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eeced31780 0x55eeced260f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.563+0000 7f0a318c7740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0a281a9890 con 0x55eeced225d0 2024-09-12T03:22:29.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.563+0000 7f0a318c7740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f0a281aad00 con 0x55eeced23b40 2024-09-12T03:22:29.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.563+0000 7f0a318c7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0a281a9f60 con 0x55eeced31780 2024-09-12T03:22:29.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.563+0000 7f0a318c7740 10 client.0 _ll_register_callbacks cb 0x7f0a180d2bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:29.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.563+0000 7f0a2ca89640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced23b40 0x55eeced25bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.563+0000 7f0a2ca89640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced23b40 0x55eeced25bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:35290/0 (socket says 172.21.15.6:35290) 2024-09-12T03:22:29.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.563+0000 7f0a2ca89640 1 -- 172.21.15.6:0/2150132205 learned_addr learned my addr 172.21.15.6:0/2150132205 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:29.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.563+0000 7f0a2da8b640 1 --2- 172.21.15.6:0/2150132205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eeced31780 0x55eeced260f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.563+0000 7f0a2ca89640 1 -- 172.21.15.6:0/2150132205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eeced31780 msgr2=0x55eeced260f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:29.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.563+0000 7f0a2ca89640 1 --2- 172.21.15.6:0/2150132205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eeced31780 0x55eeced260f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.563+0000 7f0a2ca89640 1 -- 172.21.15.6:0/2150132205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced225d0 msgr2=0x55eeced25670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:29.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.563+0000 7f0a2ca89640 1 --2- 172.21.15.6:0/2150132205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced225d0 0x55eeced25670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.563+0000 7f0a2ca89640 1 -- 172.21.15.6:0/2150132205 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeced38b10 con 0x55eeced23b40 2024-09-12T03:22:29.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.564+0000 7f0a2ca89640 1 --2- 172.21.15.6:0/2150132205 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced23b40 0x55eeced25bb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0a18007590 tx=0x7f0a180d2cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6081f85a67e2b2e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:29.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.564+0000 7f0a10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:22:29.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.564+0000 7f0a10ff9640 1 -- 172.21.15.6:0/2150132205 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1809b5e0 con 0x55eeced23b40 2024-09-12T03:22:29.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.564+0000 7f0a10ff9640 1 -- 172.21.15.6:0/2150132205 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f0a18018d60 con 0x55eeced23b40 2024-09-12T03:22:29.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.564+0000 7f0a318c7740 1 -- 172.21.15.6:0/2150132205 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a2404c070 con 0x55eeced23b40 2024-09-12T03:22:29.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.564+0000 7f0a10ff9640 1 -- 172.21.15.6:0/2150132205 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0a1809b400 con 0x55eeced23b40 2024-09-12T03:22:29.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.564+0000 7f0a318c7740 10 client.25091 fetch_fsmap learned FSMap version 26 2024-09-12T03:22:29.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.564+0000 7f0a318c7740 1 -- 172.21.15.6:0/2150132205 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eeced39de0 con 0x55eeced23b40 2024-09-12T03:22:29.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.564+0000 7f0a10ff9640 1 -- 172.21.15.6:0/2150132205 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a1800de20 con 0x55eeced23b40 2024-09-12T03:22:29.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.564+0000 7f0a10ff9640 1 -- 172.21.15.6:0/2150132205 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0a180a4280 con 0x55eeced23b40 2024-09-12T03:22:29.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.564+0000 7f0a318c7740 10 client.25091 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:22:29.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.564+0000 7f0a318c7740 10 client.25091 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-12T03:22:29.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.564+0000 7f0a318c7740 20 client.25091 populate_metadata read hostname 'smithi006' 2024-09-12T03:22:29.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.564+0000 7f0a318c7740 1 --2- 172.21.15.6:0/2150132205 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55eecedb03a0 0x55eecedb2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.564+0000 7f0a318c7740 4 client.25091 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:22:29.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.564+0000 7f0a318c7740 1 -- 172.21.15.6:0/2150132205 --> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f0a2404c070 con 0x55eecedb03a0 2024-09-12T03:22:29.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.565+0000 7f0a2d28a640 1 --2- 172.21.15.6:0/2150132205 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55eecedb03a0 0x55eecedb2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.566+0000 7f0a2d28a640 1 --2- 172.21.15.6:0/2150132205 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55eecedb03a0 0x55eecedb2790 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9f9efb147ea3bd4 server_cookie=ba828291328cff10 in_seq=0 out_seq=0 2024-09-12T03:22:29.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.566+0000 7f0a10ff9640 10 client.25091 ms_handle_connect on v2:172.21.15.161:6838/3645555254 2024-09-12T03:22:29.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.566+0000 7f0a10ff9640 1 -- 172.21.15.6:0/2150132205 <== mds.2 v2:172.21.15.161:6838/3645555254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f0a2404c070 con 0x55eecedb03a0 2024-09-12T03:22:29.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.566+0000 7f0a10ff9640 10 client.25091 handle_command_reply: tid=0 2024-09-12T03:22:29.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.566+0000 7f0a318c7740 1 client.25091 shutdown 2024-09-12T03:22:29.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.566+0000 7f0a318c7740 1 -- 172.21.15.6:0/2150132205 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced23b40 msgr2=0x55eeced25bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.566+0000 7f0a318c7740 1 --2- 172.21.15.6:0/2150132205 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced23b40 0x55eeced25bb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0a18007590 tx=0x7f0a180d2cf0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.566+0000 7f0a318c7740 1 -- 172.21.15.6:0/2150132205 shutdown_connections 2024-09-12T03:22:29.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.566+0000 7f0a318c7740 1 --2- 172.21.15.6:0/2150132205 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55eecedb03a0 0x55eecedb2790 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.566+0000 7f0a318c7740 1 --2- 172.21.15.6:0/2150132205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eeced31780 0x55eeced260f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.566+0000 7f0a318c7740 1 --2- 172.21.15.6:0/2150132205 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eeced23b40 0x55eeced25bb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.566+0000 7f0a318c7740 1 --2- 172.21.15.6:0/2150132205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eeced225d0 0x55eeced25670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.566+0000 7f0a318c7740 1 -- 172.21.15.6:0/2150132205 >> 172.21.15.6:0/2150132205 conn(0x55eececeb310 msgr2=0x55eeced2d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:29.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.566+0000 7f0a10ff9640 0 client.25091 ms_handle_reset on v2:172.21.15.161:6838/3645555254 2024-09-12T03:22:29.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.567+0000 7f0a318c7740 1 -- 172.21.15.6:0/2150132205 shutdown_connections 2024-09-12T03:22:29.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.567+0000 7f0a318c7740 1 -- 172.21.15.6:0/2150132205 wait complete. 2024-09-12T03:22:29.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.567+0000 7f0a318c7740 20 client.25091 trim_cache size 0 max 16384 2024-09-12T03:22:29.570 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-09-12T03:22:29.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.567+0000 7f0a2f4ee640 1 -- 172.21.15.6:0/868885592 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0a140a2b80 msgr2=0x7f0a140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.567+0000 7f0a2f4ee640 1 --2- 172.21.15.6:0/868885592 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0a140a2b80 0x7f0a140a5040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0a20005fd0 tx=0x7f0a20004a40 comp rx=0 tx=0).stop 2024-09-12T03:22:29.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.567+0000 7f0a2f4ee640 1 -- 172.21.15.6:0/868885592 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a281027a0 msgr2=0x7f0a281988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.567+0000 7f0a2f4ee640 1 --2- 172.21.15.6:0/868885592 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a281027a0 0x7f0a281988e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800c920 tx=0x7f0a1800cde0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.568+0000 7f0a2f4ee640 1 -- 172.21.15.6:0/868885592 shutdown_connections 2024-09-12T03:22:29.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.568+0000 7f0a2f4ee640 1 --2- 172.21.15.6:0/868885592 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f0a140a2b80 0x7f0a140a5040 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.568+0000 7f0a2f4ee640 1 --2- 172.21.15.6:0/868885592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a28106f20 0x7f0a2819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.568+0000 7f0a2f4ee640 1 --2- 172.21.15.6:0/868885592 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a281027a0 0x7f0a281988e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.568+0000 7f0a2f4ee640 1 --2- 172.21.15.6:0/868885592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a28101df0 0x7f0a281983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.568+0000 7f0a2f4ee640 1 -- 172.21.15.6:0/868885592 >> 172.21.15.6:0/868885592 conn(0x7f0a280fd8b0 msgr2=0x7f0a280fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:29.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.568+0000 7f0a2f4ee640 1 -- 172.21.15.6:0/868885592 shutdown_connections 2024-09-12T03:22:29.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:29.568+0000 7f0a2f4ee640 1 -- 172.21.15.6:0/868885592 wait complete. 2024-09-12T03:22:29.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:29 smithi161 ceph-mon[35485]: pgmap v376: 97 pgs: 97 active+clean; 6.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 553 KiB/s wr, 65 op/s 2024-09-12T03:22:29.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:29 smithi118 ceph-mon[35048]: pgmap v376: 97 pgs: 97 active+clean; 6.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 553 KiB/s wr, 65 op/s 2024-09-12T03:22:30.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:30 smithi006 ceph-mon[30991]: pgmap v377: 97 pgs: 97 active+clean; 10 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 917 KiB/s wr, 68 op/s 2024-09-12T03:22:30.521 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:22:30.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:30 smithi161 ceph-mon[35485]: pgmap v377: 97 pgs: 97 active+clean; 10 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 917 KiB/s wr, 68 op/s 2024-09-12T03:22:30.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:30 smithi118 ceph-mon[35048]: pgmap v377: 97 pgs: 97 active+clean; 10 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 917 KiB/s wr, 68 op/s 2024-09-12T03:22:30.874 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:31.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.667+0000 7f845a389640 1 -- 172.21.15.6:0/64330011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8454100e20 msgr2=0x7f8454101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:31.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.667+0000 7f845a389640 1 --2- 172.21.15.6:0/64330011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8454100e20 0x7f8454101200 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f843c009900 tx=0x7f843c02a480 comp rx=0 tx=0).stop 2024-09-12T03:22:31.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.667+0000 7f845a389640 1 -- 172.21.15.6:0/64330011 shutdown_connections 2024-09-12T03:22:31.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.667+0000 7f845a389640 1 --2- 172.21.15.6:0/64330011 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8454102ef0 0x7f845410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.667+0000 7f845a389640 1 --2- 172.21.15.6:0/64330011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8454102570 0x7f84541029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.667+0000 7f845a389640 1 --2- 172.21.15.6:0/64330011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8454100e20 0x7f8454101200 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.667+0000 7f845a389640 1 -- 172.21.15.6:0/64330011 >> 172.21.15.6:0/64330011 conn(0x7f84540fc7d0 msgr2=0x7f84540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:31.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.668+0000 7f845a389640 1 -- 172.21.15.6:0/64330011 shutdown_connections 2024-09-12T03:22:31.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.668+0000 7f845a389640 1 -- 172.21.15.6:0/64330011 wait complete. 2024-09-12T03:22:31.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.669+0000 7f845a389640 1 Processor -- start 2024-09-12T03:22:31.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.670+0000 7f845a389640 1 -- start start 2024-09-12T03:22:31.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.670+0000 7f845a389640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8454100e20 0x7f845419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.670+0000 7f845a389640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8454102570 0x7f845419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.671+0000 7f845a389640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8454102ef0 0x7f845419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.671+0000 7f845a389640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8454110fc0 con 0x7f8454102ef0 2024-09-12T03:22:31.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.671+0000 7f845a389640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f8454110e40 con 0x7f8454102570 2024-09-12T03:22:31.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.671+0000 7f845a389640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8454111140 con 0x7f8454100e20 2024-09-12T03:22:31.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.671+0000 7f8458926640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8454102ef0 0x7f845419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.671+0000 7f8458926640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8454102ef0 0x7f845419ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44404/0 (socket says 172.21.15.6:44404) 2024-09-12T03:22:31.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.671+0000 7f8458926640 1 -- 172.21.15.6:0/851049870 learned_addr learned my addr 172.21.15.6:0/851049870 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:31.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.672+0000 7f8458926640 1 -- 172.21.15.6:0/851049870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8454100e20 msgr2=0x7f845419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:31.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.672+0000 7f8458926640 1 --2- 172.21.15.6:0/851049870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8454100e20 0x7f845419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.672+0000 7f8458926640 1 -- 172.21.15.6:0/851049870 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8454102570 msgr2=0x7f845419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:31.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.672+0000 7f8458926640 1 --2- 172.21.15.6:0/851049870 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8454102570 0x7f845419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.672+0000 7f8458926640 1 -- 172.21.15.6:0/851049870 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845419f660 con 0x7f8454102ef0 2024-09-12T03:22:31.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.672+0000 7f8458926640 1 --2- 172.21.15.6:0/851049870 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8454102ef0 0x7f845419ef80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f844c00cad0 tx=0x7f844c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:31.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.673+0000 7f84517fa640 1 -- 172.21.15.6:0/851049870 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c00ae20 con 0x7f8454102ef0 2024-09-12T03:22:31.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.673+0000 7f845a389640 1 -- 172.21.15.6:0/851049870 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f845419f950 con 0x7f8454102ef0 2024-09-12T03:22:31.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.673+0000 7f845a389640 1 -- 172.21.15.6:0/851049870 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84541a7230 con 0x7f8454102ef0 2024-09-12T03:22:31.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.673+0000 7f84517fa640 1 -- 172.21.15.6:0/851049870 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f844c005020 con 0x7f8454102ef0 2024-09-12T03:22:31.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.673+0000 7f84517fa640 1 -- 172.21.15.6:0/851049870 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c00f6f0 con 0x7f8454102ef0 2024-09-12T03:22:31.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.677+0000 7f845c762740 1 Processor -- start 2024-09-12T03:22:31.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.677+0000 7f84517fa640 1 -- 172.21.15.6:0/851049870 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f844c004a40 con 0x7f8454102ef0 2024-09-12T03:22:31.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.677+0000 7f845c762740 1 -- start start 2024-09-12T03:22:31.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.678+0000 7f845c762740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8c45ef350 0x55e8c45ed8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.678+0000 7f845c762740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c45edde0 0x55e8c45ee220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.678+0000 7f845c762740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45f8b80 0x55e8c45faf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.678+0000 7f845c762740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e8c454b2f0 con 0x55e8c45f8b80 2024-09-12T03:22:31.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.678+0000 7f845c762740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55e8c44f2cf0 con 0x55e8c45ef350 2024-09-12T03:22:31.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.678+0000 7f845c762740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e8c45472b0 con 0x55e8c45edde0 2024-09-12T03:22:31.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.678+0000 7f84537fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45f8b80 0x55e8c45faf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.678+0000 7f84537fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45f8b80 0x55e8c45faf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44416/0 (socket says 172.21.15.6:44416) 2024-09-12T03:22:31.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.678+0000 7f84537fe640 1 -- 172.21.15.6:0/2388181149 learned_addr learned my addr 172.21.15.6:0/2388181149 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:31.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.679+0000 7f84517fa640 1 --2- 172.21.15.6:0/851049870 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f84400a2c00 0x7f84400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.679+0000 7f8458926640 1 --2- 172.21.15.6:0/851049870 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f84400a2c00 0x7f84400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.680+0000 7f84517fa640 1 -- 172.21.15.6:0/851049870 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f844c0cf2d0 con 0x7f8454102ef0 2024-09-12T03:22:31.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.680+0000 7f8453fff640 1 --2- 172.21.15.6:0/2388181149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c45edde0 0x55e8c45ee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.680+0000 7f84537fe640 1 -- 172.21.15.6:0/2388181149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c45edde0 msgr2=0x55e8c45ee220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:31.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.680+0000 7f84537fe640 1 --2- 172.21.15.6:0/2388181149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c45edde0 0x55e8c45ee220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.680+0000 7f84537fe640 1 -- 172.21.15.6:0/2388181149 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8c45ef350 msgr2=0x55e8c45ed8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:31.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.680+0000 7f84537fe640 1 --2- 172.21.15.6:0/2388181149 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8c45ef350 0x55e8c45ed8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.680+0000 7f84537fe640 1 -- 172.21.15.6:0/2388181149 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8c44d73a0 con 0x55e8c45f8b80 2024-09-12T03:22:31.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.681+0000 7f84537fe640 1 --2- 172.21.15.6:0/2388181149 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45f8b80 0x55e8c45faf70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f84440017d0 tx=0x7f8444001c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bed22cfab5d40e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:31.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.681+0000 7f8458926640 1 --2- 172.21.15.6:0/851049870 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f84400a2c00 0x7f84400a50c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f844c016000 tx=0x7f844c0c6690 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:31.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.681+0000 7f844a7fc640 1 -- 172.21.15.6:0/2388181149 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444006960 con 0x55e8c45f8b80 2024-09-12T03:22:31.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.681+0000 7f844a7fc640 1 -- 172.21.15.6:0/2388181149 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f844400c8d0 con 0x55e8c45f8b80 2024-09-12T03:22:31.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.681+0000 7f844a7fc640 1 -- 172.21.15.6:0/2388181149 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84440076c0 con 0x55e8c45f8b80 2024-09-12T03:22:31.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.682+0000 7f845c762740 1 -- 172.21.15.6:0/2388181149 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45f8b80 msgr2=0x55e8c45faf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:31.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.683+0000 7f845c762740 1 --2- 172.21.15.6:0/2388181149 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45f8b80 0x55e8c45faf70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f84440017d0 tx=0x7f8444001c90 comp rx=0 tx=0).stop 2024-09-12T03:22:31.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.683+0000 7f845c762740 1 -- 172.21.15.6:0/2388181149 shutdown_connections 2024-09-12T03:22:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.683+0000 7f845c762740 1 --2- 172.21.15.6:0/2388181149 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45f8b80 0x55e8c45faf70 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.683+0000 7f845c762740 1 --2- 172.21.15.6:0/2388181149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c45edde0 0x55e8c45ee220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.683+0000 7f845c762740 1 --2- 172.21.15.6:0/2388181149 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8c45ef350 0x55e8c45ed8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.683+0000 7f845c762740 1 -- 172.21.15.6:0/2388181149 >> 172.21.15.6:0/2388181149 conn(0x55e8c45ae310 msgr2=0x55e8c45ea470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.683+0000 7f845c762740 1 -- 172.21.15.6:0/2388181149 shutdown_connections 2024-09-12T03:22:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.683+0000 7f845c762740 1 -- 172.21.15.6:0/2388181149 wait complete. 2024-09-12T03:22:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.684+0000 7f845c762740 1 -- start start 2024-09-12T03:22:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.684+0000 7f845c762740 1 Processor -- start 2024-09-12T03:22:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.685+0000 7f845c762740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45edde0 0x55e8c45f85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.685+0000 7f845c762740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c45ef350 0x55e8c45f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.685+0000 7f845c762740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8c45f8b80 0x55e8c45f6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.685+0000 7f845c762740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e8c45fb880 con 0x55e8c45edde0 2024-09-12T03:22:31.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.685+0000 7f845c762740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55e8c4506060 con 0x55e8c45f8b80 2024-09-12T03:22:31.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.685+0000 7f845c762740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e8c45fba00 con 0x55e8c45ef350 2024-09-12T03:22:31.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.685+0000 7f845c762740 10 client.0 _ll_register_callbacks cb 0x55e8c42598c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:31.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.686+0000 7f8453fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c45ef350 0x55e8c45f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.686+0000 7f8453fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c45ef350 0x55e8c45f58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:38346/0 (socket says 172.21.15.6:38346) 2024-09-12T03:22:31.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.686+0000 7f8453fff640 1 -- 172.21.15.6:0/2231319294 learned_addr learned my addr 172.21.15.6:0/2231319294 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:31.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.686+0000 7f84537fe640 1 --2- 172.21.15.6:0/2231319294 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45edde0 0x55e8c45f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.686+0000 7f84537fe640 1 -- 172.21.15.6:0/2231319294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c45ef350 msgr2=0x55e8c45f58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:31.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.686+0000 7f84537fe640 1 --2- 172.21.15.6:0/2231319294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c45ef350 0x55e8c45f58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.686+0000 7f84537fe640 1 -- 172.21.15.6:0/2231319294 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8c45f8b80 msgr2=0x55e8c45f6050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:22:31.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.686+0000 7f84537fe640 1 --2- 172.21.15.6:0/2231319294 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8c45f8b80 0x55e8c45f6050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.686+0000 7f84537fe640 1 -- 172.21.15.6:0/2231319294 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8c42bc3c0 con 0x55e8c45edde0 2024-09-12T03:22:31.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.688+0000 7f84537fe640 1 --2- 172.21.15.6:0/2231319294 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45edde0 0x55e8c45f85e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8444007200 tx=0x7f8444001140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aee426f74df027a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:31.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.688+0000 7f84337fe640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:22:31.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.689+0000 7f84337fe640 1 -- 172.21.15.6:0/2231319294 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844400cd10 con 0x55e8c45edde0 2024-09-12T03:22:31.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.689+0000 7f84337fe640 1 -- 172.21.15.6:0/2231319294 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f8444004020 con 0x55e8c45edde0 2024-09-12T03:22:31.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.689+0000 7f845c762740 1 -- 172.21.15.6:0/2231319294 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f844400c8d0 con 0x55e8c45edde0 2024-09-12T03:22:31.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.689+0000 7f84337fe640 1 -- 172.21.15.6:0/2231319294 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8444029020 con 0x55e8c45edde0 2024-09-12T03:22:31.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.689+0000 7f845c762740 10 client.15528 fetch_fsmap learned FSMap version 26 2024-09-12T03:22:31.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.689+0000 7f845c762740 1 -- 172.21.15.6:0/2231319294 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e8c4618470 con 0x55e8c45edde0 2024-09-12T03:22:31.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.690+0000 7f84337fe640 1 -- 172.21.15.6:0/2231319294 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8444006e40 con 0x55e8c45edde0 2024-09-12T03:22:31.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.690+0000 7f84337fe640 1 -- 172.21.15.6:0/2231319294 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f84440035f0 con 0x55e8c45edde0 2024-09-12T03:22:31.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.690+0000 7f845c762740 10 client.15528 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:22:31.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.690+0000 7f845c762740 10 client.15528 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-12T03:22:31.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.690+0000 7f845c762740 20 client.15528 populate_metadata read hostname 'smithi006' 2024-09-12T03:22:31.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.690+0000 7f845c762740 1 --2- 172.21.15.6:0/2231319294 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55e8c4619820 0x55e8c461bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.690+0000 7f845c762740 4 client.15528 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:22:31.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.690+0000 7f845c762740 1 -- 172.21.15.6:0/2231319294 --> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f844400c8d0 con 0x55e8c4619820 2024-09-12T03:22:31.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.692+0000 7f8453fff640 1 --2- 172.21.15.6:0/2231319294 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55e8c4619820 0x55e8c461bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.694+0000 7f84337fe640 10 client.15528 ms_handle_connect on v2:172.21.15.118:6836/3879191727 2024-09-12T03:22:31.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.694+0000 7f8453fff640 1 --2- 172.21.15.6:0/2231319294 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55e8c4619820 0x55e8c461bc10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=227bfd40ea6fdcd0 server_cookie=861bea650d05c54d in_seq=0 out_seq=0 2024-09-12T03:22:31.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.696+0000 7f84337fe640 1 -- 172.21.15.6:0/2231319294 <== mds.1 v2:172.21.15.118:6836/3879191727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f844400c8d0 con 0x55e8c4619820 2024-09-12T03:22:31.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.696+0000 7f84337fe640 10 client.15528 handle_command_reply: tid=0 2024-09-12T03:22:31.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.696+0000 7f845c762740 1 client.15528 shutdown 2024-09-12T03:22:31.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.696+0000 7f845c762740 1 -- 172.21.15.6:0/2231319294 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45edde0 msgr2=0x55e8c45f85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:31.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.696+0000 7f845c762740 1 --2- 172.21.15.6:0/2231319294 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45edde0 0x55e8c45f85e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8444007200 tx=0x7f8444001140 comp rx=0 tx=0).stop 2024-09-12T03:22:31.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.697+0000 7f845c762740 1 -- 172.21.15.6:0/2231319294 shutdown_connections 2024-09-12T03:22:31.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.697+0000 7f845c762740 1 --2- 172.21.15.6:0/2231319294 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55e8c4619820 0x55e8c461bc10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.697+0000 7f845c762740 1 --2- 172.21.15.6:0/2231319294 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8c45f8b80 0x55e8c45f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.697+0000 7f845c762740 1 --2- 172.21.15.6:0/2231319294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c45ef350 0x55e8c45f58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.697+0000 7f845c762740 1 --2- 172.21.15.6:0/2231319294 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45edde0 0x55e8c45f85e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.697+0000 7f845c762740 1 -- 172.21.15.6:0/2231319294 >> 172.21.15.6:0/2231319294 conn(0x55e8c45ae310 msgr2=0x55e8c45f0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:31.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.697+0000 7f84337fe640 0 client.15528 ms_handle_reset on v2:172.21.15.118:6836/3879191727 2024-09-12T03:22:31.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.697+0000 7f845c762740 1 -- 172.21.15.6:0/2231319294 shutdown_connections 2024-09-12T03:22:31.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.697+0000 7f845c762740 1 -- 172.21.15.6:0/2231319294 wait complete. 2024-09-12T03:22:31.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.697+0000 7f845c762740 20 client.15528 trim_cache size 0 max 16384 2024-09-12T03:22:31.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.715+0000 7f845c762740 1 Processor -- start 2024-09-12T03:22:31.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.715+0000 7f845c762740 1 -- start start 2024-09-12T03:22:31.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.715+0000 7f845c762740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8c45edde0 0x55e8c45ea550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.715+0000 7f845c762740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45ef350 0x55e8c46213d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.715+0000 7f845c762740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c45f8b80 0x55e8c4621910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.715+0000 7f845c762740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8434010c40 con 0x55e8c45ef350 2024-09-12T03:22:31.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.715+0000 7f845c762740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f843400e510 con 0x55e8c45edde0 2024-09-12T03:22:31.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.715+0000 7f845c762740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f843400c360 con 0x55e8c45f8b80 2024-09-12T03:22:31.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.716+0000 7f8453fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45ef350 0x55e8c46213d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.716+0000 7f8453fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45ef350 0x55e8c46213d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44426/0 (socket says 172.21.15.6:44426) 2024-09-12T03:22:31.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.716+0000 7f8453fff640 1 -- 172.21.15.6:0/3760685246 learned_addr learned my addr 172.21.15.6:0/3760685246 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:31.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.716+0000 7f84537fe640 1 --2- 172.21.15.6:0/3760685246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c45f8b80 0x55e8c4621910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.716+0000 7f8453fff640 1 -- 172.21.15.6:0/3760685246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c45f8b80 msgr2=0x55e8c4621910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:31.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.716+0000 7f8453fff640 1 --2- 172.21.15.6:0/3760685246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c45f8b80 0x55e8c4621910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.716+0000 7f8453fff640 1 -- 172.21.15.6:0/3760685246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8c45edde0 msgr2=0x55e8c45ea550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:31.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.716+0000 7f8453fff640 1 --2- 172.21.15.6:0/3760685246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8c45edde0 0x55e8c45ea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.716+0000 7f8453fff640 1 -- 172.21.15.6:0/3760685246 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8c45f4890 con 0x55e8c45ef350 2024-09-12T03:22:31.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.716+0000 7f8453fff640 1 --2- 172.21.15.6:0/3760685246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45ef350 0x55e8c46213d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f843c009900 tx=0x7f843c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679d9a5019873da7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:31.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.717+0000 7f8449ffb640 1 -- 172.21.15.6:0/3760685246 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c035020 con 0x55e8c45ef350 2024-09-12T03:22:31.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.717+0000 7f8449ffb640 1 -- 172.21.15.6:0/3760685246 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f843c0046c0 con 0x55e8c45ef350 2024-09-12T03:22:31.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.717+0000 7f8449ffb640 1 -- 172.21.15.6:0/3760685246 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c003850 con 0x55e8c45ef350 2024-09-12T03:22:31.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.718+0000 7f845c762740 1 -- 172.21.15.6:0/3760685246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45ef350 msgr2=0x55e8c46213d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:31.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.718+0000 7f845c762740 1 --2- 172.21.15.6:0/3760685246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45ef350 0x55e8c46213d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f843c009900 tx=0x7f843c000bc0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.718+0000 7f845c762740 1 -- 172.21.15.6:0/3760685246 shutdown_connections 2024-09-12T03:22:31.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.718+0000 7f845c762740 1 --2- 172.21.15.6:0/3760685246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c45f8b80 0x55e8c4621910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.718+0000 7f845c762740 1 --2- 172.21.15.6:0/3760685246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45ef350 0x55e8c46213d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.718+0000 7f845c762740 1 --2- 172.21.15.6:0/3760685246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8c45edde0 0x55e8c45ea550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.718+0000 7f845c762740 1 -- 172.21.15.6:0/3760685246 >> 172.21.15.6:0/3760685246 conn(0x55e8c45ae310 msgr2=0x55e8c45e9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:31.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.718+0000 7f845c762740 1 -- 172.21.15.6:0/3760685246 shutdown_connections 2024-09-12T03:22:31.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.718+0000 7f845c762740 1 -- 172.21.15.6:0/3760685246 wait complete. 2024-09-12T03:22:31.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.719+0000 7f845c762740 1 -- start start 2024-09-12T03:22:31.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.719+0000 7f845c762740 1 Processor -- start 2024-09-12T03:22:31.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.719+0000 7f845c762740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8c45edde0 0x55e8c45fce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.720+0000 7f845c762740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45ef350 0x55e8c45fd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.720+0000 7f845c762740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c45f8b80 0x55e8c46833a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.720+0000 7f845c762740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f843400c9d0 con 0x55e8c45ef350 2024-09-12T03:22:31.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.720+0000 7f845c762740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f843400de40 con 0x55e8c45edde0 2024-09-12T03:22:31.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.720+0000 7f845c762740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f843400d0a0 con 0x55e8c45f8b80 2024-09-12T03:22:31.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.720+0000 7f845c762740 10 client.0 _ll_register_callbacks cb 0x7f8444001600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:31.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.720+0000 7f8453fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45ef350 0x55e8c45fd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.720+0000 7f84537fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8c45edde0 0x55e8c45fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.720+0000 7f8453fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45ef350 0x55e8c45fd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44430/0 (socket says 172.21.15.6:44430) 2024-09-12T03:22:31.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.720+0000 7f8453fff640 1 -- 172.21.15.6:0/2784460477 learned_addr learned my addr 172.21.15.6:0/2784460477 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:31.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.720+0000 7f84537fe640 1 --2- 172.21.15.6:0/2784460477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c45f8b80 0x55e8c46833a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.720+0000 7f8453fff640 1 -- 172.21.15.6:0/2784460477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c45f8b80 msgr2=0x55e8c46833a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:31.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.720+0000 7f8453fff640 1 --2- 172.21.15.6:0/2784460477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c45f8b80 0x55e8c46833a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.720+0000 7f8453fff640 1 -- 172.21.15.6:0/2784460477 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8c45edde0 msgr2=0x55e8c45fce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:31.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.720+0000 7f8453fff640 1 --2- 172.21.15.6:0/2784460477 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8c45edde0 0x55e8c45fce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.720+0000 7f8453fff640 1 -- 172.21.15.6:0/2784460477 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8c45f63c0 con 0x55e8c45ef350 2024-09-12T03:22:31.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.720+0000 7f84537fe640 1 --2- 172.21.15.6:0/2784460477 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8c45edde0 0x55e8c45fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:31.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.721+0000 7f8453fff640 1 --2- 172.21.15.6:0/2784460477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45ef350 0x55e8c45fd340 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f843c02a9a0 tx=0x7f843c0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61efd01486ce5021 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:31.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.721+0000 7f84337fe640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:22:31.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.721+0000 7f84337fe640 1 -- 172.21.15.6:0/2784460477 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c051070 con 0x55e8c45ef350 2024-09-12T03:22:31.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.721+0000 7f84337fe640 1 -- 172.21.15.6:0/2784460477 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f843c004d40 con 0x55e8c45ef350 2024-09-12T03:22:31.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.721+0000 7f845c762740 1 -- 172.21.15.6:0/2784460477 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f843c0046c0 con 0x55e8c45ef350 2024-09-12T03:22:31.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.721+0000 7f84337fe640 1 -- 172.21.15.6:0/2784460477 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f843c05d050 con 0x55e8c45ef350 2024-09-12T03:22:31.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.721+0000 7f845c762740 10 client.15534 fetch_fsmap learned FSMap version 26 2024-09-12T03:22:31.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.721+0000 7f845c762740 1 -- 172.21.15.6:0/2784460477 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e8c4684820 con 0x55e8c45ef350 2024-09-12T03:22:31.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.721+0000 7f84337fe640 1 -- 172.21.15.6:0/2784460477 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f843c0046c0 con 0x55e8c45ef350 2024-09-12T03:22:31.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.722+0000 7f84337fe640 1 -- 172.21.15.6:0/2784460477 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f843c03a040 con 0x55e8c45ef350 2024-09-12T03:22:31.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.722+0000 7f845c762740 10 client.15534 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:22:31.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.722+0000 7f845c762740 10 client.15534 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-12T03:22:31.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.722+0000 7f845c762740 20 client.15534 populate_metadata read hostname 'smithi006' 2024-09-12T03:22:31.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.722+0000 7f845c762740 1 --2- 172.21.15.6:0/2784460477 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55e8c4684f40 0x55e8c4687360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.722+0000 7f845c762740 4 client.15534 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:22:31.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.722+0000 7f845c762740 1 -- 172.21.15.6:0/2784460477 --> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e8c4558860 con 0x55e8c4684f40 2024-09-12T03:22:31.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.722+0000 7f84537fe640 1 --2- 172.21.15.6:0/2784460477 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55e8c4684f40 0x55e8c4687360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.723+0000 7f84537fe640 1 --2- 172.21.15.6:0/2784460477 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55e8c4684f40 0x55e8c4687360 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9aedb6f46d586ab server_cookie=9530c96117293d9d in_seq=0 out_seq=0 2024-09-12T03:22:31.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.723+0000 7f84337fe640 10 client.15534 ms_handle_connect on v2:172.21.15.118:6836/3879191727 2024-09-12T03:22:31.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.723+0000 7f84337fe640 1 -- 172.21.15.6:0/2784460477 <== mds.1 v2:172.21.15.118:6836/3879191727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e8c4558860 con 0x55e8c4684f40 2024-09-12T03:22:31.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.723+0000 7f84337fe640 10 client.15534 handle_command_reply: tid=0 2024-09-12T03:22:31.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.723+0000 7f845c762740 1 client.15534 shutdown 2024-09-12T03:22:31.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.724+0000 7f845c762740 1 -- 172.21.15.6:0/2784460477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45ef350 msgr2=0x55e8c45fd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:31.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.724+0000 7f845c762740 1 --2- 172.21.15.6:0/2784460477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45ef350 0x55e8c45fd340 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f843c02a9a0 tx=0x7f843c0050b0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.724+0000 7f845c762740 1 -- 172.21.15.6:0/2784460477 shutdown_connections 2024-09-12T03:22:31.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.724+0000 7f845c762740 1 --2- 172.21.15.6:0/2784460477 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55e8c4684f40 0x55e8c4687360 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.724+0000 7f845c762740 1 --2- 172.21.15.6:0/2784460477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c45f8b80 0x55e8c46833a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.724+0000 7f84337fe640 0 client.15534 ms_handle_reset on v2:172.21.15.118:6836/3879191727 2024-09-12T03:22:31.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.724+0000 7f845c762740 1 --2- 172.21.15.6:0/2784460477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8c45ef350 0x55e8c45fd340 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.724+0000 7f845c762740 1 --2- 172.21.15.6:0/2784460477 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8c45edde0 0x55e8c45fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.724+0000 7f845c762740 1 -- 172.21.15.6:0/2784460477 >> 172.21.15.6:0/2784460477 conn(0x55e8c45ae310 msgr2=0x55e8c45e04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:31.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.724+0000 7f845c762740 1 -- 172.21.15.6:0/2784460477 shutdown_connections 2024-09-12T03:22:31.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.724+0000 7f845c762740 1 -- 172.21.15.6:0/2784460477 wait complete. 2024-09-12T03:22:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.724+0000 7f845c762740 20 client.15534 trim_cache size 0 max 16384 2024-09-12T03:22:31.735 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-09-12T03:22:31.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.725+0000 7f845a389640 1 -- 172.21.15.6:0/851049870 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f84400a2c00 msgr2=0x7f84400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:31.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.725+0000 7f845a389640 1 --2- 172.21.15.6:0/851049870 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f84400a2c00 0x7f84400a50c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f844c016000 tx=0x7f844c0c6690 comp rx=0 tx=0).stop 2024-09-12T03:22:31.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.725+0000 7f845a389640 1 -- 172.21.15.6:0/851049870 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8454102ef0 msgr2=0x7f845419ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:31.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.725+0000 7f845a389640 1 --2- 172.21.15.6:0/851049870 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8454102ef0 0x7f845419ef80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f844c00cad0 tx=0x7f844c00cf90 comp rx=0 tx=0).stop 2024-09-12T03:22:31.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.725+0000 7f845a389640 1 -- 172.21.15.6:0/851049870 shutdown_connections 2024-09-12T03:22:31.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.725+0000 7f845a389640 1 --2- 172.21.15.6:0/851049870 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f84400a2c00 0x7f84400a50c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.725+0000 7f845a389640 1 --2- 172.21.15.6:0/851049870 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8454102ef0 0x7f845419ef80 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.725+0000 7f845a389640 1 --2- 172.21.15.6:0/851049870 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8454102570 0x7f845419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.725+0000 7f845a389640 1 --2- 172.21.15.6:0/851049870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8454100e20 0x7f845419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.725+0000 7f845a389640 1 -- 172.21.15.6:0/851049870 >> 172.21.15.6:0/851049870 conn(0x7f84540fc7d0 msgr2=0x7f845406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:31.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.726+0000 7f845a389640 1 -- 172.21.15.6:0/851049870 shutdown_connections 2024-09-12T03:22:31.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:31.726+0000 7f845a389640 1 -- 172.21.15.6:0/851049870 wait complete. 2024-09-12T03:22:32.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:32 smithi006 ceph-mon[30991]: pgmap v378: 97 pgs: 97 active+clean; 645 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 54 MiB/s wr, 131 op/s 2024-09-12T03:22:32.478 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:22:32.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:32 smithi118 ceph-mon[35048]: pgmap v378: 97 pgs: 97 active+clean; 645 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 54 MiB/s wr, 131 op/s 2024-09-12T03:22:32.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:32 smithi161 ceph-mon[35485]: pgmap v378: 97 pgs: 97 active+clean; 645 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 54 MiB/s wr, 131 op/s 2024-09-12T03:22:32.890 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:34.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.093+0000 7f7068d93640 1 -- 172.21.15.6:0/2112346835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7064109830 msgr2=0x7f7064109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.093+0000 7f7068d93640 1 --2- 172.21.15.6:0/2112346835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7064109830 0x7f7064109c90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7054009900 tx=0x7f705402a3f0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.093+0000 7f7068d93640 1 -- 172.21.15.6:0/2112346835 shutdown_connections 2024-09-12T03:22:34.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.093+0000 7f7068d93640 1 --2- 172.21.15.6:0/2112346835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f706411b140 0x7f706411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.093+0000 7f7068d93640 1 --2- 172.21.15.6:0/2112346835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7064109830 0x7f7064109c90 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.093+0000 7f7068d93640 1 --2- 172.21.15.6:0/2112346835 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7064109450 0x7f7064110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.093+0000 7f7068d93640 1 -- 172.21.15.6:0/2112346835 >> 172.21.15.6:0/2112346835 conn(0x7f706406e140 msgr2=0x7f706406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:34.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.093+0000 7f7068d93640 1 -- 172.21.15.6:0/2112346835 shutdown_connections 2024-09-12T03:22:34.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.093+0000 7f7068d93640 1 -- 172.21.15.6:0/2112346835 wait complete. 2024-09-12T03:22:34.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.095+0000 7f7068d93640 1 Processor -- start 2024-09-12T03:22:34.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.095+0000 7f7068d93640 1 -- start start 2024-09-12T03:22:34.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.096+0000 7f7068d93640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7064109450 0x7f70641a7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.096+0000 7f7068d93640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7064109830 0x7f70641a7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.096+0000 7f7068d93640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f706411b140 0x7f70641abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.096+0000 7f7068d93640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7064120df0 con 0x7f706411b140 2024-09-12T03:22:34.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.096+0000 7f7068d93640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f7064120c70 con 0x7f7064109450 2024-09-12T03:22:34.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.096+0000 7f7068d93640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7064120f70 con 0x7f7064109830 2024-09-12T03:22:34.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.096+0000 7f7062d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f706411b140 0x7f70641abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.096+0000 7f7062d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f706411b140 0x7f70641abb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44448/0 (socket says 172.21.15.6:44448) 2024-09-12T03:22:34.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.096+0000 7f7062d9d640 1 -- 172.21.15.6:0/3347060359 learned_addr learned my addr 172.21.15.6:0/3347060359 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:34.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.096+0000 7f7062d9d640 1 -- 172.21.15.6:0/3347060359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7064109830 msgr2=0x7f70641a7770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:22:34.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.096+0000 7f7062d9d640 1 --2- 172.21.15.6:0/3347060359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7064109830 0x7f70641a7770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.097+0000 7f7062d9d640 1 -- 172.21.15.6:0/3347060359 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7064109450 msgr2=0x7f70641a7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.097+0000 7f7062d9d640 1 --2- 172.21.15.6:0/3347060359 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7064109450 0x7f70641a7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.097+0000 7f7062d9d640 1 -- 172.21.15.6:0/3347060359 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70641ac1e0 con 0x7f706411b140 2024-09-12T03:22:34.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.097+0000 7f7062d9d640 1 --2- 172.21.15.6:0/3347060359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f706411b140 0x7f70641abb00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f705c00ebc0 tx=0x7f705c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:34.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.098+0000 7f70537fe640 1 -- 172.21.15.6:0/3347060359 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c004b20 con 0x7f706411b140 2024-09-12T03:22:34.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.098+0000 7f70537fe640 1 -- 172.21.15.6:0/3347060359 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f705c004cc0 con 0x7f706411b140 2024-09-12T03:22:34.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.098+0000 7f70537fe640 1 -- 172.21.15.6:0/3347060359 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c0107a0 con 0x7f706411b140 2024-09-12T03:22:34.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.098+0000 7f7068d93640 1 -- 172.21.15.6:0/3347060359 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70641ac4d0 con 0x7f706411b140 2024-09-12T03:22:34.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.098+0000 7f7068d93640 1 -- 172.21.15.6:0/3347060359 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70641b3db0 con 0x7f706411b140 2024-09-12T03:22:34.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.104+0000 7f706b16c740 1 Processor -- start 2024-09-12T03:22:34.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.105+0000 7f70537fe640 1 -- 172.21.15.6:0/3347060359 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f705c005020 con 0x7f706411b140 2024-09-12T03:22:34.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.106+0000 7f706b16c740 1 -- start start 2024-09-12T03:22:34.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.106+0000 7f70537fe640 1 --2- 172.21.15.6:0/3347060359 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f704c0a2d50 0x7f704c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.107+0000 7f706b16c740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556de10b5b40 0x556de10b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.107+0000 7f706b16c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556de10b45d0 0x556de10b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.107+0000 7f706b16c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10bf370 0x556de10c1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.107+0000 7f706b16c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x556de101a2f0 con 0x556de10bf370 2024-09-12T03:22:34.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.107+0000 7f706b16c740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x556de0fc1cf0 con 0x556de10b5b40 2024-09-12T03:22:34.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.107+0000 7f706b16c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556de10162b0 con 0x556de10b45d0 2024-09-12T03:22:34.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.107+0000 7f70537fe640 1 -- 172.21.15.6:0/3347060359 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f705c0cf6a0 con 0x7f706411b140 2024-09-12T03:22:34.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.107+0000 7f7061d9b640 1 --2- 172.21.15.6:0/3347060359 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f704c0a2d50 0x7f704c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.107+0000 7f7062d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10bf370 0x556de10c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.107+0000 7f706259c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556de10b5b40 0x556de10b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.107+0000 7f7062d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10bf370 0x556de10c1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44450/0 (socket says 172.21.15.6:44450) 2024-09-12T03:22:34.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.107+0000 7f7062d9d640 1 -- 172.21.15.6:0/257698285 learned_addr learned my addr 172.21.15.6:0/257698285 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:34.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.108+0000 7f7061d9b640 1 --2- 172.21.15.6:0/257698285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556de10b45d0 0x556de10b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.108+0000 7f706259c640 1 -- 172.21.15.6:0/257698285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556de10b45d0 msgr2=0x556de10b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:34.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.108+0000 7f706259c640 1 --2- 172.21.15.6:0/257698285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556de10b45d0 0x556de10b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.108+0000 7f706259c640 1 -- 172.21.15.6:0/257698285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10bf370 msgr2=0x556de10c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:34.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.108+0000 7f706259c640 1 --2- 172.21.15.6:0/257698285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10bf370 0x556de10c1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.108+0000 7f706259c640 1 -- 172.21.15.6:0/257698285 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556de0fa63a0 con 0x556de10b5b40 2024-09-12T03:22:34.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.108+0000 7f706259c640 1 --2- 172.21.15.6:0/257698285 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556de10b5b40 0x556de10b4090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f705800a9e0 tx=0x7f705800aea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c2460443fc8466a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:34.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.108+0000 7f7061d9b640 1 --2- 172.21.15.6:0/3347060359 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f704c0a2d50 0x7f704c0a5210 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f70540096c0 tx=0x7f7054005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:34.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.109+0000 7f7050ff9640 1 -- 172.21.15.6:0/257698285 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705800cdb0 con 0x556de10b5b40 2024-09-12T03:22:34.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.109+0000 7f7050ff9640 1 -- 172.21.15.6:0/257698285 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f705800f930 con 0x556de10b5b40 2024-09-12T03:22:34.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.109+0000 7f7050ff9640 1 -- 172.21.15.6:0/257698285 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705800ea10 con 0x556de10b5b40 2024-09-12T03:22:34.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.110+0000 7f706b16c740 1 -- 172.21.15.6:0/257698285 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556de10b5b40 msgr2=0x556de10b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.110+0000 7f706b16c740 1 --2- 172.21.15.6:0/257698285 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556de10b5b40 0x556de10b4090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f705800a9e0 tx=0x7f705800aea0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.110+0000 7f706b16c740 1 -- 172.21.15.6:0/257698285 shutdown_connections 2024-09-12T03:22:34.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.110+0000 7f706b16c740 1 --2- 172.21.15.6:0/257698285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10bf370 0x556de10c1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.110+0000 7f706b16c740 1 --2- 172.21.15.6:0/257698285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556de10b45d0 0x556de10b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.110+0000 7f706b16c740 1 --2- 172.21.15.6:0/257698285 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556de10b5b40 0x556de10b4090 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.110+0000 7f706b16c740 1 -- 172.21.15.6:0/257698285 >> 172.21.15.6:0/257698285 conn(0x556de107d310 msgr2=0x556de10b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:34.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.110+0000 7f706b16c740 1 -- 172.21.15.6:0/257698285 shutdown_connections 2024-09-12T03:22:34.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.110+0000 7f706b16c740 1 -- 172.21.15.6:0/257698285 wait complete. 2024-09-12T03:22:34.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.111+0000 7f706b16c740 1 -- start start 2024-09-12T03:22:34.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.111+0000 7f706b16c740 1 Processor -- start 2024-09-12T03:22:34.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.111+0000 7f706b16c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10b45d0 0x556de10becd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.111+0000 7f706b16c740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556de10b5b40 0x556de10bbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.111+0000 7f706b16c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556de10bf370 0x556de10bc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.111+0000 7f706b16c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x556de10c2070 con 0x556de10b45d0 2024-09-12T03:22:34.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.111+0000 7f706b16c740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x556de0fd5060 con 0x556de10b5b40 2024-09-12T03:22:34.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.111+0000 7f706b16c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556de10c21f0 con 0x556de10bf370 2024-09-12T03:22:34.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.112+0000 7f706b16c740 10 client.0 _ll_register_callbacks cb 0x556de0d288c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:34.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.112+0000 7f706259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10b45d0 0x556de10becd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.112+0000 7f706259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10b45d0 0x556de10becd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44454/0 (socket says 172.21.15.6:44454) 2024-09-12T03:22:34.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.112+0000 7f706259c640 1 -- 172.21.15.6:0/3959236822 learned_addr learned my addr 172.21.15.6:0/3959236822 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:34.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.112+0000 7f706259c640 1 -- 172.21.15.6:0/3959236822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556de10bf370 msgr2=0x556de10bc4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:22:34.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.112+0000 7f706259c640 1 --2- 172.21.15.6:0/3959236822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556de10bf370 0x556de10bc4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.112+0000 7f706259c640 1 -- 172.21.15.6:0/3959236822 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556de10b5b40 msgr2=0x556de10bbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:34.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.112+0000 7f706259c640 1 --2- 172.21.15.6:0/3959236822 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556de10b5b40 0x556de10bbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.112+0000 7f706259c640 1 -- 172.21.15.6:0/3959236822 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556de0d8b3c0 con 0x556de10b45d0 2024-09-12T03:22:34.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.113+0000 7f706259c640 1 --2- 172.21.15.6:0/3959236822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10b45d0 0x556de10becd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f705800f680 tx=0x7f705800f6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823b2451e8b0852f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:34.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.113+0000 7f7049ffb640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:22:34.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.113+0000 7f7049ffb640 1 -- 172.21.15.6:0/3959236822 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058028060 con 0x556de10b45d0 2024-09-12T03:22:34.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.113+0000 7f7049ffb640 1 -- 172.21.15.6:0/3959236822 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f705800fae0 con 0x556de10b45d0 2024-09-12T03:22:34.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.113+0000 7f706b16c740 1 -- 172.21.15.6:0/3959236822 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f705800f930 con 0x556de10b45d0 2024-09-12T03:22:34.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.114+0000 7f7049ffb640 1 -- 172.21.15.6:0/3959236822 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f705800a5d0 con 0x556de10b45d0 2024-09-12T03:22:34.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.114+0000 7f706b16c740 10 client.15546 fetch_fsmap learned FSMap version 26 2024-09-12T03:22:34.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.114+0000 7f706b16c740 1 -- 172.21.15.6:0/3959236822 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556de10c3e20 con 0x556de10b45d0 2024-09-12T03:22:34.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.114+0000 7f7049ffb640 1 -- 172.21.15.6:0/3959236822 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f705800f930 con 0x556de10b45d0 2024-09-12T03:22:34.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.114+0000 7f7049ffb640 1 -- 172.21.15.6:0/3959236822 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7058018780 con 0x556de10b45d0 2024-09-12T03:22:34.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.115+0000 7f706b16c740 10 client.15546 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:22:34.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.115+0000 7f706b16c740 10 client.15546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:22:34.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.115+0000 7f706b16c740 20 client.15546 populate_metadata read hostname 'smithi006' 2024-09-12T03:22:34.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.115+0000 7f706b16c740 1 --2- 172.21.15.6:0/3959236822 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x556de10c5280 0x556de10c7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.115+0000 7f706b16c740 4 client.15546 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:22:34.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.115+0000 7f706b16c740 1 -- 172.21.15.6:0/3959236822 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556de1027860 con 0x556de10c5280 2024-09-12T03:22:34.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.116+0000 7f7061d9b640 1 --2- 172.21.15.6:0/3959236822 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x556de10c5280 0x556de10c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.118+0000 7f7061d9b640 1 --2- 172.21.15.6:0/3959236822 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x556de10c5280 0x556de10c7720 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a684ca2815eb1ce server_cookie=6dabccb69d6c48fe in_seq=0 out_seq=0 2024-09-12T03:22:34.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.118+0000 7f7049ffb640 10 client.15546 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:34.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.119+0000 7f7049ffb640 1 -- 172.21.15.6:0/3959236822 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556de1027860 con 0x556de10c5280 2024-09-12T03:22:34.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.119+0000 7f7049ffb640 10 client.15546 handle_command_reply: tid=0 2024-09-12T03:22:34.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.120+0000 7f706b16c740 1 client.15546 shutdown 2024-09-12T03:22:34.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.120+0000 7f706b16c740 1 -- 172.21.15.6:0/3959236822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10b45d0 msgr2=0x556de10becd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.120+0000 7f706b16c740 1 --2- 172.21.15.6:0/3959236822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10b45d0 0x556de10becd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f705800f680 tx=0x7f705800f6b0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.120+0000 7f706b16c740 1 -- 172.21.15.6:0/3959236822 shutdown_connections 2024-09-12T03:22:34.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.120+0000 7f706b16c740 1 --2- 172.21.15.6:0/3959236822 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x556de10c5280 0x556de10c7720 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.120+0000 7f706b16c740 1 --2- 172.21.15.6:0/3959236822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556de10bf370 0x556de10bc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.120+0000 7f706b16c740 1 --2- 172.21.15.6:0/3959236822 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556de10b5b40 0x556de10bbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.120+0000 7f706b16c740 1 --2- 172.21.15.6:0/3959236822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10b45d0 0x556de10becd0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.120+0000 7f7049ffb640 0 client.15546 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:34.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.120+0000 7f706b16c740 1 -- 172.21.15.6:0/3959236822 >> 172.21.15.6:0/3959236822 conn(0x556de107d310 msgr2=0x556de10b6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:34.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.120+0000 7f706b16c740 1 -- 172.21.15.6:0/3959236822 shutdown_connections 2024-09-12T03:22:34.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.120+0000 7f706b16c740 1 -- 172.21.15.6:0/3959236822 wait complete. 2024-09-12T03:22:34.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.120+0000 7f706b16c740 20 client.15546 trim_cache size 0 max 16384 2024-09-12T03:22:34.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.149+0000 7f706b16c740 1 Processor -- start 2024-09-12T03:22:34.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.150+0000 7f706b16c740 1 -- start start 2024-09-12T03:22:34.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.150+0000 7f706b16c740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556de10b45d0 0x556de10bbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.150+0000 7f706b16c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556de10b5b40 0x556de10b91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.150+0000 7f706b16c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10bf370 0x556de10b9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.151+0000 7f706b16c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7044010c40 con 0x556de10bf370 2024-09-12T03:22:34.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.151+0000 7f706b16c740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f704400e510 con 0x556de10b45d0 2024-09-12T03:22:34.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.151+0000 7f706b16c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f704400c360 con 0x556de10b5b40 2024-09-12T03:22:34.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.151+0000 7f7062d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10bf370 0x556de10b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.151+0000 7f7062d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10bf370 0x556de10b9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44456/0 (socket says 172.21.15.6:44456) 2024-09-12T03:22:34.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.151+0000 7f7062d9d640 1 -- 172.21.15.6:0/4198920897 learned_addr learned my addr 172.21.15.6:0/4198920897 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:34.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.151+0000 7f7061d9b640 1 --2- 172.21.15.6:0/4198920897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556de10b5b40 0x556de10b91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.152+0000 7f7062d9d640 1 -- 172.21.15.6:0/4198920897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556de10b5b40 msgr2=0x556de10b91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:34.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.152+0000 7f7062d9d640 1 --2- 172.21.15.6:0/4198920897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556de10b5b40 0x556de10b91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.152+0000 7f7062d9d640 1 -- 172.21.15.6:0/4198920897 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556de10b45d0 msgr2=0x556de10bbbd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:22:34.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.152+0000 7f7062d9d640 1 --2- 172.21.15.6:0/4198920897 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556de10b45d0 0x556de10bbbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.152+0000 7f7062d9d640 1 -- 172.21.15.6:0/4198920897 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556de10bcfe0 con 0x556de10bf370 2024-09-12T03:22:34.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.153+0000 7f7062d9d640 1 --2- 172.21.15.6:0/4198920897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10bf370 0x556de10b9720 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f705c00c7d0 tx=0x7f705c00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4eb9f797ec18ca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:34.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.156+0000 7f704bfff640 1 -- 172.21.15.6:0/4198920897 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c0b7020 con 0x556de10bf370 2024-09-12T03:22:34.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.156+0000 7f704bfff640 1 -- 172.21.15.6:0/4198920897 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f705c09c5d0 con 0x556de10bf370 2024-09-12T03:22:34.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.156+0000 7f704bfff640 1 -- 172.21.15.6:0/4198920897 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c01fbf0 con 0x556de10bf370 2024-09-12T03:22:34.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.157+0000 7f706b16c740 1 -- 172.21.15.6:0/4198920897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10bf370 msgr2=0x556de10b9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.157+0000 7f706b16c740 1 --2- 172.21.15.6:0/4198920897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10bf370 0x556de10b9720 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f705c00c7d0 tx=0x7f705c00cf10 comp rx=0 tx=0).stop 2024-09-12T03:22:34.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.169+0000 7f706b16c740 1 -- 172.21.15.6:0/4198920897 shutdown_connections 2024-09-12T03:22:34.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.169+0000 7f706b16c740 1 --2- 172.21.15.6:0/4198920897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10bf370 0x556de10b9720 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.169+0000 7f706b16c740 1 --2- 172.21.15.6:0/4198920897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556de10b5b40 0x556de10b91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.169+0000 7f706b16c740 1 --2- 172.21.15.6:0/4198920897 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556de10b45d0 0x556de10bbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.169+0000 7f706b16c740 1 -- 172.21.15.6:0/4198920897 >> 172.21.15.6:0/4198920897 conn(0x556de107d310 msgr2=0x556de10bb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:34.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.170+0000 7f706b16c740 1 -- 172.21.15.6:0/4198920897 shutdown_connections 2024-09-12T03:22:34.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.170+0000 7f706b16c740 1 -- 172.21.15.6:0/4198920897 wait complete. 2024-09-12T03:22:34.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.170+0000 7f706b16c740 1 -- start start 2024-09-12T03:22:34.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.171+0000 7f706b16c740 1 Processor -- start 2024-09-12T03:22:34.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.171+0000 7f706b16c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556de10b45d0 0x556de10cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.171+0000 7f706b16c740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556de10b5b40 0x556de10cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.171+0000 7f706b16c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10bf370 0x556de10ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.171+0000 7f706b16c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f704400c9d0 con 0x556de10bf370 2024-09-12T03:22:34.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.171+0000 7f706b16c740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f704400de40 con 0x556de10b5b40 2024-09-12T03:22:34.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.171+0000 7f706b16c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f704400d0a0 con 0x556de10b45d0 2024-09-12T03:22:34.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.171+0000 7f706b16c740 10 client.0 _ll_register_callbacks cb 0x7f705c005b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:34.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.172+0000 7f7062d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10bf370 0x556de10ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.172+0000 7f7062d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10bf370 0x556de10ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44466/0 (socket says 172.21.15.6:44466) 2024-09-12T03:22:34.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.172+0000 7f7062d9d640 1 -- 172.21.15.6:0/4208865394 learned_addr learned my addr 172.21.15.6:0/4208865394 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:34.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.173+0000 7f7062d9d640 1 -- 172.21.15.6:0/4208865394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556de10b45d0 msgr2=0x556de10cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:34.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.173+0000 7f7062d9d640 1 --2- 172.21.15.6:0/4208865394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556de10b45d0 0x556de10cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.173+0000 7f7062d9d640 1 -- 172.21.15.6:0/4208865394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556de10b5b40 msgr2=0x556de10cbcf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:22:34.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.173+0000 7f7062d9d640 1 --2- 172.21.15.6:0/4208865394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556de10b5b40 0x556de10cbcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.173+0000 7f7062d9d640 1 -- 172.21.15.6:0/4208865394 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556de10cd3a0 con 0x556de10bf370 2024-09-12T03:22:34.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.173+0000 7f7062d9d640 1 --2- 172.21.15.6:0/4208865394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10bf370 0x556de10ccb50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f705c09bbd0 tx=0x7f705c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f04ee4f1d60035d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:34.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.173+0000 7f7049ffb640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:22:34.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.173+0000 7f7049ffb640 1 -- 172.21.15.6:0/4208865394 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c0b7070 con 0x556de10bf370 2024-09-12T03:22:34.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.173+0000 7f7049ffb640 1 -- 172.21.15.6:0/4208865394 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f705c09b320 con 0x556de10bf370 2024-09-12T03:22:34.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.174+0000 7f706b16c740 1 -- 172.21.15.6:0/4208865394 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f705c09c5d0 con 0x556de10bf370 2024-09-12T03:22:34.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.174+0000 7f7049ffb640 1 -- 172.21.15.6:0/4208865394 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f705c0a0280 con 0x556de10bf370 2024-09-12T03:22:34.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.174+0000 7f706b16c740 10 client.15552 fetch_fsmap learned FSMap version 26 2024-09-12T03:22:34.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.174+0000 7f706b16c740 1 -- 172.21.15.6:0/4208865394 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556de115bee0 con 0x556de10bf370 2024-09-12T03:22:34.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.174+0000 7f7049ffb640 1 -- 172.21.15.6:0/4208865394 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f705c09c5d0 con 0x556de10bf370 2024-09-12T03:22:34.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.175+0000 7f7049ffb640 1 -- 172.21.15.6:0/4208865394 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f705c09ad90 con 0x556de10bf370 2024-09-12T03:22:34.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.175+0000 7f706b16c740 10 client.15552 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:22:34.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.175+0000 7f706b16c740 10 client.15552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:22:34.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.175+0000 7f706b16c740 20 client.15552 populate_metadata read hostname 'smithi006' 2024-09-12T03:22:34.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.175+0000 7f706b16c740 1 --2- 172.21.15.6:0/4208865394 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x556de115c540 0x556de115e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.175+0000 7f706b16c740 4 client.15552 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:22:34.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.175+0000 7f706b16c740 1 -- 172.21.15.6:0/4208865394 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556de1021d70 con 0x556de115c540 2024-09-12T03:22:34.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.176+0000 7f706259c640 1 --2- 172.21.15.6:0/4208865394 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x556de115c540 0x556de115e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.178+0000 7f7049ffb640 10 client.15552 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:34.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.178+0000 7f706259c640 1 --2- 172.21.15.6:0/4208865394 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x556de115c540 0x556de115e9a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bb523b6b3a355f9 server_cookie=ef4a0e0c909c3cb3 in_seq=0 out_seq=0 2024-09-12T03:22:34.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.179+0000 7f7049ffb640 1 -- 172.21.15.6:0/4208865394 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556de1021d70 con 0x556de115c540 2024-09-12T03:22:34.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.179+0000 7f7049ffb640 10 client.15552 handle_command_reply: tid=0 2024-09-12T03:22:34.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.179+0000 7f706b16c740 1 client.15552 shutdown 2024-09-12T03:22:34.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.179+0000 7f706b16c740 1 -- 172.21.15.6:0/4208865394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10bf370 msgr2=0x556de10ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.179+0000 7f706b16c740 1 --2- 172.21.15.6:0/4208865394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10bf370 0x556de10ccb50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f705c09bbd0 tx=0x7f705c004a40 comp rx=0 tx=0).stop 2024-09-12T03:22:34.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.179+0000 7f706b16c740 1 -- 172.21.15.6:0/4208865394 shutdown_connections 2024-09-12T03:22:34.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.179+0000 7f706b16c740 1 --2- 172.21.15.6:0/4208865394 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x556de115c540 0x556de115e9a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.179+0000 7f706b16c740 1 --2- 172.21.15.6:0/4208865394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556de10bf370 0x556de10ccb50 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.179+0000 7f706b16c740 1 --2- 172.21.15.6:0/4208865394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556de10b5b40 0x556de10cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.179+0000 7f706b16c740 1 --2- 172.21.15.6:0/4208865394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556de10b45d0 0x556de10cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.179+0000 7f706b16c740 1 -- 172.21.15.6:0/4208865394 >> 172.21.15.6:0/4208865394 conn(0x556de107d310 msgr2=0x556de10bb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:34.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.179+0000 7f7049ffb640 0 client.15552 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:34.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.179+0000 7f706b16c740 1 -- 172.21.15.6:0/4208865394 shutdown_connections 2024-09-12T03:22:34.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.179+0000 7f706b16c740 1 -- 172.21.15.6:0/4208865394 wait complete. 2024-09-12T03:22:34.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.180+0000 7f706b16c740 20 client.15552 trim_cache size 0 max 16384 2024-09-12T03:22:34.181 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-09-12T03:22:34.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.181+0000 7f7068d93640 1 -- 172.21.15.6:0/3347060359 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f704c0a2d50 msgr2=0x7f704c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.181+0000 7f7068d93640 1 --2- 172.21.15.6:0/3347060359 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f704c0a2d50 0x7f704c0a5210 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f70540096c0 tx=0x7f7054005e40 comp rx=0 tx=0).stop 2024-09-12T03:22:34.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.181+0000 7f7068d93640 1 -- 172.21.15.6:0/3347060359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f706411b140 msgr2=0x7f70641abb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.181+0000 7f7068d93640 1 --2- 172.21.15.6:0/3347060359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f706411b140 0x7f70641abb00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f705c00ebc0 tx=0x7f705c00c2d0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.182+0000 7f7068d93640 1 -- 172.21.15.6:0/3347060359 shutdown_connections 2024-09-12T03:22:34.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.182+0000 7f7068d93640 1 --2- 172.21.15.6:0/3347060359 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f704c0a2d50 0x7f704c0a5210 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.182+0000 7f7068d93640 1 --2- 172.21.15.6:0/3347060359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f706411b140 0x7f70641abb00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.182+0000 7f7068d93640 1 --2- 172.21.15.6:0/3347060359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7064109830 0x7f70641a7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.182+0000 7f7068d93640 1 --2- 172.21.15.6:0/3347060359 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7064109450 0x7f70641a7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.182+0000 7f7068d93640 1 -- 172.21.15.6:0/3347060359 >> 172.21.15.6:0/3347060359 conn(0x7f706406e140 msgr2=0x7f706411bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:34.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.186+0000 7f7068d93640 1 -- 172.21.15.6:0/3347060359 shutdown_connections 2024-09-12T03:22:34.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:34.186+0000 7f7068d93640 1 -- 172.21.15.6:0/3347060359 wait complete. 2024-09-12T03:22:34.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:34 smithi118 ceph-mon[35048]: pgmap v379: 97 pgs: 97 active+clean; 2.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 187 MiB/s wr, 173 op/s 2024-09-12T03:22:34.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:34 smithi161 ceph-mon[35485]: pgmap v379: 97 pgs: 97 active+clean; 2.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 187 MiB/s wr, 173 op/s 2024-09-12T03:22:34.863 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:34 smithi006 ceph-mon[30991]: pgmap v379: 97 pgs: 97 active+clean; 2.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 187 MiB/s wr, 173 op/s 2024-09-12T03:22:35.909 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:22:35.910 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:22:36.318 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:36.835 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:36 smithi006 ceph-mon[30991]: pgmap v380: 97 pgs: 97 active+clean; 2.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 238 MiB/s wr, 157 op/s 2024-09-12T03:22:36.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:36 smithi161 ceph-mon[35485]: pgmap v380: 97 pgs: 97 active+clean; 2.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 238 MiB/s wr, 157 op/s 2024-09-12T03:22:36.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:36 smithi118 ceph-mon[35048]: pgmap v380: 97 pgs: 97 active+clean; 2.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 238 MiB/s wr, 157 op/s 2024-09-12T03:22:36.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.987+0000 7f990ff38640 1 -- 172.21.15.6:0/3965350746 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9908101440 msgr2=0x7f990810df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:36.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.987+0000 7f990ff38640 1 --2- 172.21.15.6:0/3965350746 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9908101440 0x7f990810df30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009900 tx=0x7f98f802a410 comp rx=0 tx=0).stop 2024-09-12T03:22:36.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.987+0000 7f990ff38640 1 -- 172.21.15.6:0/3965350746 shutdown_connections 2024-09-12T03:22:36.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.987+0000 7f990ff38640 1 --2- 172.21.15.6:0/3965350746 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f990810e500 0x7f9908110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.987+0000 7f990ff38640 1 --2- 172.21.15.6:0/3965350746 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9908101440 0x7f990810df30 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.987+0000 7f990ff38640 1 --2- 172.21.15.6:0/3965350746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9908100b20 0x7f9908100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.987+0000 7f990ff38640 1 -- 172.21.15.6:0/3965350746 >> 172.21.15.6:0/3965350746 conn(0x7f99080fc7d0 msgr2=0x7f99080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:36.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.987+0000 7f990ff38640 1 -- 172.21.15.6:0/3965350746 shutdown_connections 2024-09-12T03:22:36.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.987+0000 7f990ff38640 1 -- 172.21.15.6:0/3965350746 wait complete. 2024-09-12T03:22:36.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.987+0000 7f990ff38640 1 Processor -- start 2024-09-12T03:22:36.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.988+0000 7f990ff38640 1 -- start start 2024-09-12T03:22:36.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.988+0000 7f990ff38640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9908100b20 0x7f99081a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:36.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.988+0000 7f990ff38640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9908101440 0x7f99081a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:36.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.988+0000 7f990ff38640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f990810e500 0x7f990819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:36.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.988+0000 7f990ff38640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9908114250 con 0x7f990810e500 2024-09-12T03:22:36.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.988+0000 7f990ff38640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f99081140d0 con 0x7f9908101440 2024-09-12T03:22:36.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.988+0000 7f990ff38640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f99081143d0 con 0x7f9908100b20 2024-09-12T03:22:36.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.988+0000 7f990e4d5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f990810e500 0x7f990819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:36.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.988+0000 7f990e4d5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f990810e500 0x7f990819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44484/0 (socket says 172.21.15.6:44484) 2024-09-12T03:22:36.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.988+0000 7f990e4d5640 1 -- 172.21.15.6:0/54168304 learned_addr learned my addr 172.21.15.6:0/54168304 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:36.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.988+0000 7f990dcd4640 1 --2- 172.21.15.6:0/54168304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9908100b20 0x7f99081a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:36.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.988+0000 7f990d4d3640 1 --2- 172.21.15.6:0/54168304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9908101440 0x7f99081a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:36.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.988+0000 7f990e4d5640 1 -- 172.21.15.6:0/54168304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9908100b20 msgr2=0x7f99081a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:36.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.988+0000 7f990e4d5640 1 --2- 172.21.15.6:0/54168304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9908100b20 0x7f99081a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.988+0000 7f990e4d5640 1 -- 172.21.15.6:0/54168304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9908101440 msgr2=0x7f99081a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:36.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.988+0000 7f990e4d5640 1 --2- 172.21.15.6:0/54168304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9908101440 0x7f99081a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.988+0000 7f990e4d5640 1 -- 172.21.15.6:0/54168304 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990819ad30 con 0x7f990810e500 2024-09-12T03:22:36.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.989+0000 7f990e4d5640 1 --2- 172.21.15.6:0/54168304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f990810e500 0x7f990819a600 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f990000dac0 tx=0x7f990000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:36.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.989+0000 7f98feffd640 1 -- 172.21.15.6:0/54168304 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9900017070 con 0x7f990810e500 2024-09-12T03:22:36.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.989+0000 7f98feffd640 1 -- 172.21.15.6:0/54168304 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9900003e50 con 0x7f990810e500 2024-09-12T03:22:36.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.989+0000 7f98feffd640 1 -- 172.21.15.6:0/54168304 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99000106c0 con 0x7f990810e500 2024-09-12T03:22:36.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.989+0000 7f990ff38640 1 -- 172.21.15.6:0/54168304 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f990819b020 con 0x7f990810e500 2024-09-12T03:22:36.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.989+0000 7f990ff38640 1 -- 172.21.15.6:0/54168304 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99081a7310 con 0x7f990810e500 2024-09-12T03:22:36.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.992+0000 7f98feffd640 1 -- 172.21.15.6:0/54168304 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9900003970 con 0x7f990810e500 2024-09-12T03:22:36.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.992+0000 7f9912311740 1 Processor -- start 2024-09-12T03:22:36.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.992+0000 7f9912311740 1 -- start start 2024-09-12T03:22:36.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.993+0000 7f98feffd640 1 --2- 172.21.15.6:0/54168304 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f98f40a2c00 0x7f98f40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:36.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.993+0000 7f9912311740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd273db60 0x55ebd273df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:36.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.993+0000 7f9912311740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd273e480 0x55ebd2748800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:36.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.993+0000 7f9912311740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd2748d40 0x55ebd274b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:36.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.993+0000 7f9912311740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55ebd26a3280 con 0x55ebd2748d40 2024-09-12T03:22:36.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.993+0000 7f9912311740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55ebd264ab10 con 0x55ebd273e480 2024-09-12T03:22:36.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.993+0000 7f9912311740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ebd269f240 con 0x55ebd273db60 2024-09-12T03:22:36.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.993+0000 7f98feffd640 1 -- 172.21.15.6:0/54168304 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f99000ceb50 con 0x7f990810e500 2024-09-12T03:22:36.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.993+0000 7f990dcd4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd273db60 0x55ebd273df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:36.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.993+0000 7f990dcd4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd273db60 0x55ebd273df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:38428/0 (socket says 172.21.15.6:38428) 2024-09-12T03:22:36.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.993+0000 7f990dcd4640 1 -- 172.21.15.6:0/2869719018 learned_addr learned my addr 172.21.15.6:0/2869719018 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:36.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.993+0000 7f990d4d3640 1 --2- 172.21.15.6:0/2869719018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd273e480 0x55ebd2748800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:36.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.993+0000 7f990d4d3640 1 --2- 172.21.15.6:0/54168304 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f98f40a2c00 0x7f98f40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:36.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.993+0000 7f990dcd4640 1 -- 172.21.15.6:0/2869719018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd273e480 msgr2=0x55ebd2748800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:36.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.993+0000 7f990dcd4640 1 --2- 172.21.15.6:0/2869719018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd273e480 0x55ebd2748800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.993+0000 7f990dcd4640 1 -- 172.21.15.6:0/2869719018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd2748d40 msgr2=0x55ebd274b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:36.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.993+0000 7f990dcd4640 1 --2- 172.21.15.6:0/2869719018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd2748d40 0x55ebd274b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.994+0000 7f990dcd4640 1 -- 172.21.15.6:0/2869719018 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebd26571f0 con 0x55ebd273db60 2024-09-12T03:22:36.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.994+0000 7f990dcd4640 1 --2- 172.21.15.6:0/2869719018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd273db60 0x55ebd273df40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f990400bcd0 tx=0x7f99040107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=683e81fd1e97bd4e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:36.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.994+0000 7f990d4d3640 1 --2- 172.21.15.6:0/54168304 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f98f40a2c00 0x7f98f40a50c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f98f8004770 tx=0x7f98f80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:36.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.994+0000 7f98f3fff640 1 -- 172.21.15.6:0/2869719018 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904005220 con 0x55ebd273db60 2024-09-12T03:22:36.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.994+0000 7f98f3fff640 1 -- 172.21.15.6:0/2869719018 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9904004c80 con 0x55ebd273db60 2024-09-12T03:22:36.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.994+0000 7f98f3fff640 1 -- 172.21.15.6:0/2869719018 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99040117a0 con 0x55ebd273db60 2024-09-12T03:22:36.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.995+0000 7f9912311740 1 -- 172.21.15.6:0/2869719018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd273db60 msgr2=0x55ebd273df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:36.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.995+0000 7f9912311740 1 --2- 172.21.15.6:0/2869719018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd273db60 0x55ebd273df40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f990400bcd0 tx=0x7f99040107a0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.995+0000 7f9912311740 1 -- 172.21.15.6:0/2869719018 shutdown_connections 2024-09-12T03:22:37.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.995+0000 7f9912311740 1 --2- 172.21.15.6:0/2869719018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd2748d40 0x55ebd274b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.995+0000 7f9912311740 1 --2- 172.21.15.6:0/2869719018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd273e480 0x55ebd2748800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.995+0000 7f9912311740 1 --2- 172.21.15.6:0/2869719018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd273db60 0x55ebd273df40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.995+0000 7f9912311740 1 -- 172.21.15.6:0/2869719018 >> 172.21.15.6:0/2869719018 conn(0x55ebd273a0f0 msgr2=0x55ebd273a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:37.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.995+0000 7f9912311740 1 -- 172.21.15.6:0/2869719018 shutdown_connections 2024-09-12T03:22:37.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.995+0000 7f9912311740 1 -- 172.21.15.6:0/2869719018 wait complete. 2024-09-12T03:22:37.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.996+0000 7f9912311740 1 -- start start 2024-09-12T03:22:37.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.996+0000 7f9912311740 1 Processor -- start 2024-09-12T03:22:37.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.996+0000 7f9912311740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd273db60 0x55ebd2745c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.996+0000 7f9912311740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd273e480 0x55ebd2746140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.996+0000 7f9912311740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd2748d40 0x55ebd2746680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.996+0000 7f9912311740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55ebd274b910 con 0x55ebd273e480 2024-09-12T03:22:37.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.996+0000 7f9912311740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55ebd265df10 con 0x55ebd273db60 2024-09-12T03:22:37.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.996+0000 7f9912311740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ebd274ba90 con 0x55ebd2748d40 2024-09-12T03:22:37.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.996+0000 7f9912311740 10 client.0 _ll_register_callbacks cb 0x55ebd2626a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:37.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.997+0000 7f990d4d3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd273e480 0x55ebd2746140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.997+0000 7f990dcd4640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd273db60 0x55ebd2745c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.997+0000 7f990d4d3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd273e480 0x55ebd2746140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44502/0 (socket says 172.21.15.6:44502) 2024-09-12T03:22:37.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.997+0000 7f990d4d3640 1 -- 172.21.15.6:0/2236283694 learned_addr learned my addr 172.21.15.6:0/2236283694 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:37.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.997+0000 7f990e4d5640 1 --2- 172.21.15.6:0/2236283694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd2748d40 0x55ebd2746680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.997+0000 7f990dcd4640 1 -- 172.21.15.6:0/2236283694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd2748d40 msgr2=0x55ebd2746680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:37.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.997+0000 7f990dcd4640 1 --2- 172.21.15.6:0/2236283694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd2748d40 0x55ebd2746680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.997+0000 7f990dcd4640 1 -- 172.21.15.6:0/2236283694 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd273e480 msgr2=0x55ebd2746140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:37.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.997+0000 7f990dcd4640 1 --2- 172.21.15.6:0/2236283694 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd273e480 0x55ebd2746140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.997+0000 7f990dcd4640 1 -- 172.21.15.6:0/2236283694 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebd24146a0 con 0x55ebd273db60 2024-09-12T03:22:37.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.997+0000 7f990dcd4640 1 --2- 172.21.15.6:0/2236283694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd273db60 0x55ebd2745c00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9904011010 tx=0x7f99040048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5e8fe4a6f7cfc88 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:37.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.997+0000 7f98f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:22:37.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.997+0000 7f98f17fa640 1 -- 172.21.15.6:0/2236283694 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904004ae0 con 0x55ebd273db60 2024-09-12T03:22:37.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.997+0000 7f98f17fa640 1 -- 172.21.15.6:0/2236283694 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f9904005040 con 0x55ebd273db60 2024-09-12T03:22:37.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.997+0000 7f9912311740 1 -- 172.21.15.6:0/2236283694 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9904004c80 con 0x55ebd273db60 2024-09-12T03:22:37.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.998+0000 7f98f17fa640 1 -- 172.21.15.6:0/2236283694 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f990402d070 con 0x55ebd273db60 2024-09-12T03:22:37.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.998+0000 7f9912311740 10 client.25109 fetch_fsmap learned FSMap version 26 2024-09-12T03:22:37.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.998+0000 7f9912311740 1 -- 172.21.15.6:0/2236283694 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ebd2757520 con 0x55ebd273db60 2024-09-12T03:22:37.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.998+0000 7f98f17fa640 1 -- 172.21.15.6:0/2236283694 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9904004c80 con 0x55ebd273db60 2024-09-12T03:22:37.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.998+0000 7f98f17fa640 1 -- 172.21.15.6:0/2236283694 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9904026470 con 0x55ebd273db60 2024-09-12T03:22:37.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.998+0000 7f9912311740 10 client.25109 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:22:37.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.998+0000 7f9912311740 10 client.25109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:22:37.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.998+0000 7f9912311740 20 client.25109 populate_metadata read hostname 'smithi006' 2024-09-12T03:22:37.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.998+0000 7f9912311740 1 --2- 172.21.15.6:0/2236283694 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ebd27589e0 0x55ebd275add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.998+0000 7f9912311740 4 client.25109 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:22:37.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.998+0000 7f9912311740 1 -- 172.21.15.6:0/2236283694 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebd26b07f0 con 0x55ebd27589e0 2024-09-12T03:22:37.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.998+0000 7f990d4d3640 1 --2- 172.21.15.6:0/2236283694 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ebd27589e0 0x55ebd275add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.999+0000 7f98f17fa640 10 client.25109 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:37.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.999+0000 7f990d4d3640 1 --2- 172.21.15.6:0/2236283694 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ebd27589e0 0x55ebd275add0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68f68d0d81746250 server_cookie=5ae6eecf72f69a52 in_seq=0 out_seq=0 2024-09-12T03:22:37.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.999+0000 7f98f17fa640 1 -- 172.21.15.6:0/2236283694 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ebd26b07f0 con 0x55ebd27589e0 2024-09-12T03:22:37.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:36.999+0000 7f98f17fa640 10 client.25109 handle_command_reply: tid=0 2024-09-12T03:22:37.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.000+0000 7f9912311740 1 client.25109 shutdown 2024-09-12T03:22:37.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.000+0000 7f9912311740 1 -- 172.21.15.6:0/2236283694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd273db60 msgr2=0x55ebd2745c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.000+0000 7f9912311740 1 --2- 172.21.15.6:0/2236283694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd273db60 0x55ebd2745c00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9904011010 tx=0x7f99040048f0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.000+0000 7f9912311740 1 -- 172.21.15.6:0/2236283694 shutdown_connections 2024-09-12T03:22:37.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.000+0000 7f9912311740 1 --2- 172.21.15.6:0/2236283694 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ebd27589e0 0x55ebd275add0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.000+0000 7f9912311740 1 --2- 172.21.15.6:0/2236283694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd2748d40 0x55ebd2746680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.000+0000 7f9912311740 1 --2- 172.21.15.6:0/2236283694 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd273e480 0x55ebd2746140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.000+0000 7f98f17fa640 0 client.25109 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:37.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.000+0000 7f9912311740 1 --2- 172.21.15.6:0/2236283694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd273db60 0x55ebd2745c00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.000+0000 7f9912311740 1 -- 172.21.15.6:0/2236283694 >> 172.21.15.6:0/2236283694 conn(0x55ebd273a0f0 msgr2=0x55ebd273fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:37.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.000+0000 7f9912311740 1 -- 172.21.15.6:0/2236283694 shutdown_connections 2024-09-12T03:22:37.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.000+0000 7f9912311740 1 -- 172.21.15.6:0/2236283694 wait complete. 2024-09-12T03:22:37.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.000+0000 7f9912311740 20 client.25109 trim_cache size 0 max 16384 2024-09-12T03:22:37.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.018+0000 7f9912311740 1 Processor -- start 2024-09-12T03:22:37.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.018+0000 7f9912311740 1 -- start start 2024-09-12T03:22:37.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.018+0000 7f9912311740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd273db60 0x55ebd27453b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.018+0000 7f9912311740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd273e480 0x55ebd273fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.018+0000 7f9912311740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd2748d40 0x55ebd2760420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.018+0000 7f9912311740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f99081b1f50 con 0x55ebd273e480 2024-09-12T03:22:37.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.018+0000 7f9912311740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f99081af820 con 0x55ebd2748d40 2024-09-12T03:22:37.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.018+0000 7f9912311740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f99081ad690 con 0x55ebd273db60 2024-09-12T03:22:37.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.018+0000 7f990d4d3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd273e480 0x55ebd273fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.018+0000 7f990dcd4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd273db60 0x55ebd27453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.018+0000 7f990d4d3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd273e480 0x55ebd273fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44508/0 (socket says 172.21.15.6:44508) 2024-09-12T03:22:37.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.018+0000 7f990d4d3640 1 -- 172.21.15.6:0/1841581320 learned_addr learned my addr 172.21.15.6:0/1841581320 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:37.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.018+0000 7f990dcd4640 1 --2- 172.21.15.6:0/1841581320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd273db60 0x55ebd27453b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:38448/0 (socket says 172.21.15.6:38448) 2024-09-12T03:22:37.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.018+0000 7f990e4d5640 1 --2- 172.21.15.6:0/1841581320 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd2748d40 0x55ebd2760420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.018+0000 7f990dcd4640 1 -- 172.21.15.6:0/1841581320 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd2748d40 msgr2=0x55ebd2760420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:37.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.018+0000 7f990dcd4640 1 --2- 172.21.15.6:0/1841581320 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd2748d40 0x55ebd2760420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.018+0000 7f990dcd4640 1 -- 172.21.15.6:0/1841581320 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd273e480 msgr2=0x55ebd273fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:37.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.018+0000 7f990dcd4640 1 --2- 172.21.15.6:0/1841581320 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd273e480 0x55ebd273fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.018+0000 7f990dcd4640 1 -- 172.21.15.6:0/1841581320 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebd274ba70 con 0x55ebd273db60 2024-09-12T03:22:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.019+0000 7f990dcd4640 1 --2- 172.21.15.6:0/1841581320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd273db60 0x55ebd27453b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f99040056b0 tx=0x7f99040107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19c8f1eec7e39830 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.019+0000 7f98f37fe640 1 -- 172.21.15.6:0/1841581320 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990401f070 con 0x55ebd273db60 2024-09-12T03:22:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.019+0000 7f98f37fe640 1 -- 172.21.15.6:0/1841581320 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f990400b810 con 0x55ebd273db60 2024-09-12T03:22:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.019+0000 7f98f37fe640 1 -- 172.21.15.6:0/1841581320 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904004960 con 0x55ebd273db60 2024-09-12T03:22:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.020+0000 7f9912311740 1 -- 172.21.15.6:0/1841581320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd273db60 msgr2=0x55ebd27453b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.020+0000 7f9912311740 1 --2- 172.21.15.6:0/1841581320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd273db60 0x55ebd27453b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f99040056b0 tx=0x7f99040107a0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.020+0000 7f9912311740 1 -- 172.21.15.6:0/1841581320 shutdown_connections 2024-09-12T03:22:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.020+0000 7f9912311740 1 --2- 172.21.15.6:0/1841581320 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd2748d40 0x55ebd2760420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.020+0000 7f9912311740 1 --2- 172.21.15.6:0/1841581320 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd273e480 0x55ebd273fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.020+0000 7f9912311740 1 --2- 172.21.15.6:0/1841581320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd273db60 0x55ebd27453b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.020+0000 7f9912311740 1 -- 172.21.15.6:0/1841581320 >> 172.21.15.6:0/1841581320 conn(0x55ebd273a0f0 msgr2=0x55ebd273f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:37.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.020+0000 7f9912311740 1 -- 172.21.15.6:0/1841581320 shutdown_connections 2024-09-12T03:22:37.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.020+0000 7f9912311740 1 -- 172.21.15.6:0/1841581320 wait complete. 2024-09-12T03:22:37.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.021+0000 7f9912311740 1 -- start start 2024-09-12T03:22:37.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.021+0000 7f9912311740 1 Processor -- start 2024-09-12T03:22:37.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.021+0000 7f9912311740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd273db60 0x55ebd2751510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.021+0000 7f9912311740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd273e480 0x55ebd274c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.021+0000 7f9912311740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd2748d40 0x55ebd274cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.021+0000 7f9912311740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f99081adce0 con 0x55ebd273e480 2024-09-12T03:22:37.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.021+0000 7f9912311740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f99081af150 con 0x55ebd2748d40 2024-09-12T03:22:37.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.021+0000 7f9912311740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f99081ae3b0 con 0x55ebd273db60 2024-09-12T03:22:37.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.022+0000 7f9912311740 10 client.0 _ll_register_callbacks cb 0x7f98f8005dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:37.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.022+0000 7f990d4d3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd273e480 0x55ebd274c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.022+0000 7f990d4d3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd273e480 0x55ebd274c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44510/0 (socket says 172.21.15.6:44510) 2024-09-12T03:22:37.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.022+0000 7f990d4d3640 1 -- 172.21.15.6:0/816341482 learned_addr learned my addr 172.21.15.6:0/816341482 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:37.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.022+0000 7f990e4d5640 1 --2- 172.21.15.6:0/816341482 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd2748d40 0x55ebd274cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.022+0000 7f990dcd4640 1 --2- 172.21.15.6:0/816341482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd273db60 0x55ebd2751510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.022+0000 7f990e4d5640 1 -- 172.21.15.6:0/816341482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd273db60 msgr2=0x55ebd2751510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:37.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.022+0000 7f990e4d5640 1 --2- 172.21.15.6:0/816341482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd273db60 0x55ebd2751510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.022+0000 7f990e4d5640 1 -- 172.21.15.6:0/816341482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd273e480 msgr2=0x55ebd274c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:37.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.022+0000 7f990e4d5640 1 --2- 172.21.15.6:0/816341482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd273e480 0x55ebd274c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.022+0000 7f990e4d5640 1 -- 172.21.15.6:0/816341482 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebd273b280 con 0x55ebd2748d40 2024-09-12T03:22:37.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.022+0000 7f990d4d3640 1 --2- 172.21.15.6:0/816341482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd273e480 0x55ebd274c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:37.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.022+0000 7f990e4d5640 1 --2- 172.21.15.6:0/816341482 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd2748d40 0x55ebd274cdc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55ebd273b470 tx=0x7f990009adf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d99228c1af1e7dbe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:37.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.022+0000 7f98f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:22:37.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.022+0000 7f98f17fa640 1 -- 172.21.15.6:0/816341482 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9900020040 con 0x55ebd2748d40 2024-09-12T03:22:37.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.022+0000 7f98f17fa640 1 -- 172.21.15.6:0/816341482 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f99000041b0 con 0x55ebd2748d40 2024-09-12T03:22:37.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.022+0000 7f9912311740 1 -- 172.21.15.6:0/816341482 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f990400b810 con 0x55ebd2748d40 2024-09-12T03:22:37.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.023+0000 7f98f17fa640 1 -- 172.21.15.6:0/816341482 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f990009b490 con 0x55ebd2748d40 2024-09-12T03:22:37.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.023+0000 7f9912311740 10 client.25115 fetch_fsmap learned FSMap version 26 2024-09-12T03:22:37.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.023+0000 7f9912311740 1 -- 172.21.15.6:0/816341482 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ebd27c3300 con 0x55ebd2748d40 2024-09-12T03:22:37.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.023+0000 7f98f17fa640 1 -- 172.21.15.6:0/816341482 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f990400b810 con 0x55ebd2748d40 2024-09-12T03:22:37.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.023+0000 7f98f17fa640 1 -- 172.21.15.6:0/816341482 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f990007f3c0 con 0x55ebd2748d40 2024-09-12T03:22:37.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.023+0000 7f9912311740 10 client.25115 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:22:37.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.023+0000 7f9912311740 10 client.25115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:22:37.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.023+0000 7f9912311740 20 client.25115 populate_metadata read hostname 'smithi006' 2024-09-12T03:22:37.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.023+0000 7f9912311740 1 --2- 172.21.15.6:0/816341482 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ebd27c3ad0 0x55ebd27c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.023+0000 7f9912311740 4 client.25115 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:22:37.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.023+0000 7f9912311740 1 -- 172.21.15.6:0/816341482 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ebd26aad00 con 0x55ebd27c3ad0 2024-09-12T03:22:37.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.023+0000 7f990dcd4640 1 --2- 172.21.15.6:0/816341482 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ebd27c3ad0 0x55ebd27c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.024+0000 7f990dcd4640 1 --2- 172.21.15.6:0/816341482 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ebd27c3ad0 0x55ebd27c5ef0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57a01d2f43391eb6 server_cookie=c2a75596e52e1928 in_seq=0 out_seq=0 2024-09-12T03:22:37.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.024+0000 7f98f17fa640 10 client.25115 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:37.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.024+0000 7f98f17fa640 1 -- 172.21.15.6:0/816341482 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ebd26aad00 con 0x55ebd27c3ad0 2024-09-12T03:22:37.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.024+0000 7f98f17fa640 10 client.25115 handle_command_reply: tid=0 2024-09-12T03:22:37.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.024+0000 7f9912311740 1 client.25115 shutdown 2024-09-12T03:22:37.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.025+0000 7f9912311740 1 -- 172.21.15.6:0/816341482 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd2748d40 msgr2=0x55ebd274cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.025+0000 7f9912311740 1 --2- 172.21.15.6:0/816341482 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd2748d40 0x55ebd274cdc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55ebd273b470 tx=0x7f990009adf0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.025+0000 7f9912311740 1 -- 172.21.15.6:0/816341482 shutdown_connections 2024-09-12T03:22:37.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.025+0000 7f9912311740 1 --2- 172.21.15.6:0/816341482 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ebd27c3ad0 0x55ebd27c5ef0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.025+0000 7f9912311740 1 --2- 172.21.15.6:0/816341482 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ebd2748d40 0x55ebd274cdc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.025+0000 7f98f17fa640 0 client.25115 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:37.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.025+0000 7f9912311740 1 --2- 172.21.15.6:0/816341482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ebd273e480 0x55ebd274c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.025+0000 7f9912311740 1 --2- 172.21.15.6:0/816341482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebd273db60 0x55ebd2751510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.025+0000 7f9912311740 1 -- 172.21.15.6:0/816341482 >> 172.21.15.6:0/816341482 conn(0x55ebd273a0f0 msgr2=0x55ebd2747a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:37.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.025+0000 7f9912311740 1 -- 172.21.15.6:0/816341482 shutdown_connections 2024-09-12T03:22:37.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.025+0000 7f9912311740 1 -- 172.21.15.6:0/816341482 wait complete. 2024-09-12T03:22:37.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.025+0000 7f9912311740 20 client.25115 trim_cache size 0 max 16384 2024-09-12T03:22:37.030 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:22:37.030 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2024-09-12T03:22:37.030 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "08ddab5a-774f-4d21-aecd-e479d4c85235", 2024-09-12T03:22:37.030 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2024-09-12T03:22:37.030 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:22:37.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.026+0000 7f990ff38640 1 -- 172.21.15.6:0/54168304 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f98f40a2c00 msgr2=0x7f98f40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.026+0000 7f990ff38640 1 --2- 172.21.15.6:0/54168304 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f98f40a2c00 0x7f98f40a50c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f98f8004770 tx=0x7f98f80058d0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.026+0000 7f990ff38640 1 -- 172.21.15.6:0/54168304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f990810e500 msgr2=0x7f990819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.026+0000 7f990ff38640 1 --2- 172.21.15.6:0/54168304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f990810e500 0x7f990819a600 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f990000dac0 tx=0x7f990000df80 comp rx=0 tx=0).stop 2024-09-12T03:22:37.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.026+0000 7f990ff38640 1 -- 172.21.15.6:0/54168304 shutdown_connections 2024-09-12T03:22:37.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.026+0000 7f990ff38640 1 --2- 172.21.15.6:0/54168304 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f98f40a2c00 0x7f98f40a50c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.026+0000 7f990ff38640 1 --2- 172.21.15.6:0/54168304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f990810e500 0x7f990819a600 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.026+0000 7f990ff38640 1 --2- 172.21.15.6:0/54168304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9908101440 0x7f99081a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.026+0000 7f990ff38640 1 --2- 172.21.15.6:0/54168304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9908100b20 0x7f99081a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.026+0000 7f990ff38640 1 -- 172.21.15.6:0/54168304 >> 172.21.15.6:0/54168304 conn(0x7f99080fc7d0 msgr2=0x7f990810f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:37.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.026+0000 7f990ff38640 1 -- 172.21.15.6:0/54168304 shutdown_connections 2024-09-12T03:22:37.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:37.026+0000 7f990ff38640 1 -- 172.21.15.6:0/54168304 wait complete. 2024-09-12T03:22:37.781 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:37 smithi006 ceph-mon[30991]: scrub queued for path: / 2024-09-12T03:22:37.781 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:37 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:37.782 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:37 smithi006 ceph-mon[30991]: scrub summary: active paths [/] 2024-09-12T03:22:37.782 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:37 smithi006 ceph-mon[30991]: scrub summary: active paths [/] 2024-09-12T03:22:37.782 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:37 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:37.782 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:37 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:37.839 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:22:37.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:37 smithi161 ceph-mon[35485]: scrub queued for path: / 2024-09-12T03:22:37.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:37 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:37.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:37 smithi161 ceph-mon[35485]: scrub summary: active paths [/] 2024-09-12T03:22:37.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:37 smithi161 ceph-mon[35485]: scrub summary: active paths [/] 2024-09-12T03:22:37.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:37 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:37.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:37 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:37.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:37 smithi118 ceph-mon[35048]: scrub queued for path: / 2024-09-12T03:22:37.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:37 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:37.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:37 smithi118 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-12T03:22:37.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:37 smithi118 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-12T03:22:37.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:37 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:37.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:37 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:38.213 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:38.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:38 smithi118 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-12T03:22:38.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:38 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:38.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:38 smithi118 ceph-mon[35048]: pgmap v381: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 357 MiB/s wr, 210 op/s 2024-09-12T03:22:39.089 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:38 smithi006 ceph-mon[30991]: scrub summary: active paths [/] 2024-09-12T03:22:39.089 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:38 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.090 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:38 smithi006 ceph-mon[30991]: pgmap v381: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 357 MiB/s wr, 210 op/s 2024-09-12T03:22:39.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:38 smithi161 ceph-mon[35485]: scrub summary: active paths [/] 2024-09-12T03:22:39.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:38 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:38 smithi161 ceph-mon[35485]: pgmap v381: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 357 MiB/s wr, 210 op/s 2024-09-12T03:22:39.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.921+0000 7f07236ea640 1 -- 172.21.15.6:0/1406327878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f071c100e20 msgr2=0x7f071c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.921+0000 7f07236ea640 1 --2- 172.21.15.6:0/1406327878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f071c100e20 0x7f071c101200 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0718009ca0 tx=0x7f071802a950 comp rx=0 tx=0).stop 2024-09-12T03:22:39.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.921+0000 7f07236ea640 1 -- 172.21.15.6:0/1406327878 shutdown_connections 2024-09-12T03:22:39.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.921+0000 7f07236ea640 1 --2- 172.21.15.6:0/1406327878 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f071c102ef0 0x7f071c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.921+0000 7f07236ea640 1 --2- 172.21.15.6:0/1406327878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f071c102570 0x7f071c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.921+0000 7f07236ea640 1 --2- 172.21.15.6:0/1406327878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f071c100e20 0x7f071c101200 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.921+0000 7f07236ea640 1 -- 172.21.15.6:0/1406327878 >> 172.21.15.6:0/1406327878 conn(0x7f071c0fc7d0 msgr2=0x7f071c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:39.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.921+0000 7f07236ea640 1 -- 172.21.15.6:0/1406327878 shutdown_connections 2024-09-12T03:22:39.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.921+0000 7f07236ea640 1 -- 172.21.15.6:0/1406327878 wait complete. 2024-09-12T03:22:39.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.921+0000 7f07236ea640 1 Processor -- start 2024-09-12T03:22:39.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.921+0000 7f07236ea640 1 -- start start 2024-09-12T03:22:39.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.922+0000 7f07236ea640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f071c100e20 0x7f071c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.922+0000 7f07236ea640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f071c102570 0x7f071c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.922+0000 7f07236ea640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f071c102ef0 0x7f071c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.922+0000 7f07236ea640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f071c110f80 con 0x7f071c102ef0 2024-09-12T03:22:39.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.922+0000 7f07236ea640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f071c110e00 con 0x7f071c102570 2024-09-12T03:22:39.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.922+0000 7f07236ea640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f071c111100 con 0x7f071c100e20 2024-09-12T03:22:39.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.922+0000 7f0721c87640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f071c102ef0 0x7f071c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.922+0000 7f0721486640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f071c100e20 0x7f071c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.922+0000 7f0720c85640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f071c102570 0x7f071c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.922+0000 7f0721c87640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f071c102ef0 0x7f071c19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37074/0 (socket says 172.21.15.6:37074) 2024-09-12T03:22:39.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.922+0000 7f0721486640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f071c100e20 0x7f071c198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:41912/0 (socket says 172.21.15.6:41912) 2024-09-12T03:22:39.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.922+0000 7f0720c85640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f071c102570 0x7f071c1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:51274/0 (socket says 172.21.15.6:51274) 2024-09-12T03:22:39.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.922+0000 7f0721c87640 1 -- 172.21.15.6:0/121094110 learned_addr learned my addr 172.21.15.6:0/121094110 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:39.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.922+0000 7f0721c87640 1 -- 172.21.15.6:0/121094110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f071c100e20 msgr2=0x7f071c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.922+0000 7f0721c87640 1 --2- 172.21.15.6:0/121094110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f071c100e20 0x7f071c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.922+0000 7f0721c87640 1 -- 172.21.15.6:0/121094110 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f071c102570 msgr2=0x7f071c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.922+0000 7f0721c87640 1 --2- 172.21.15.6:0/121094110 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f071c102570 0x7f071c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.922+0000 7f0721c87640 1 -- 172.21.15.6:0/121094110 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071c19d420 con 0x7f071c102ef0 2024-09-12T03:22:39.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.922+0000 7f0720c85640 1 --2- 172.21.15.6:0/121094110 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f071c102570 0x7f071c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:39.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.922+0000 7f0721c87640 1 --2- 172.21.15.6:0/121094110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f071c102ef0 0x7f071c19cd40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f071400c970 tx=0x7f071400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:39.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.923+0000 7f07127fc640 1 -- 172.21.15.6:0/121094110 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714007bb0 con 0x7f071c102ef0 2024-09-12T03:22:39.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.923+0000 7f07236ea640 1 -- 172.21.15.6:0/121094110 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f071c19d710 con 0x7f071c102ef0 2024-09-12T03:22:39.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.923+0000 7f07127fc640 1 -- 172.21.15.6:0/121094110 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0714007d50 con 0x7f071c102ef0 2024-09-12T03:22:39.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.923+0000 7f07127fc640 1 -- 172.21.15.6:0/121094110 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714018800 con 0x7f071c102ef0 2024-09-12T03:22:39.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.923+0000 7f07236ea640 1 -- 172.21.15.6:0/121094110 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f071c1a4ff0 con 0x7f071c102ef0 2024-09-12T03:22:39.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.925+0000 7f07127fc640 1 -- 172.21.15.6:0/121094110 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f071400fa50 con 0x7f071c102ef0 2024-09-12T03:22:39.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.926+0000 7f07127fc640 1 --2- 172.21.15.6:0/121094110 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f07080a2b80 0x7f07080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.926+0000 7f0721486640 1 --2- 172.21.15.6:0/121094110 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f07080a2b80 0x7f07080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.926+0000 7f07127fc640 1 -- 172.21.15.6:0/121094110 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f07140cefe0 con 0x7f071c102ef0 2024-09-12T03:22:39.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.926+0000 7f0721486640 1 --2- 172.21.15.6:0/121094110 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f07080a2b80 0x7f07080a5040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0718004ad0 tx=0x7f0718004a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:39.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.926+0000 7f0725ac3740 1 Processor -- start 2024-09-12T03:22:39.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.927+0000 7f0725ac3740 1 -- start start 2024-09-12T03:22:39.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.927+0000 7f0725ac3740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832380 0x55ab56832760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.927+0000 7f0725ac3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56832ca0 0x55ab56841550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.927+0000 7f0725ac3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56841a90 0x55ab56843e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.927+0000 7f0725ac3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55ab567982e0 con 0x55ab56841a90 2024-09-12T03:22:39.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.927+0000 7f0725ac3740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55ab5673fce0 con 0x55ab56832380 2024-09-12T03:22:39.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.927+0000 7f0725ac3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ab567942a0 con 0x55ab56832ca0 2024-09-12T03:22:39.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.927+0000 7f0720c85640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56832ca0 0x55ab56841550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.927+0000 7f0721c87640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56841a90 0x55ab56843e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.927+0000 7f0720c85640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56832ca0 0x55ab56841550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:41918/0 (socket says 172.21.15.6:41918) 2024-09-12T03:22:39.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.927+0000 7f0720c85640 1 -- 172.21.15.6:0/4017019492 learned_addr learned my addr 172.21.15.6:0/4017019492 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:39.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.927+0000 7f0721486640 1 --2- 172.21.15.6:0/4017019492 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832380 0x55ab56832760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.927+0000 7f0720c85640 1 -- 172.21.15.6:0/4017019492 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832380 msgr2=0x55ab56832760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:39.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.927+0000 7f0720c85640 1 --2- 172.21.15.6:0/4017019492 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832380 0x55ab56832760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.927+0000 7f0720c85640 1 -- 172.21.15.6:0/4017019492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56841a90 msgr2=0x55ab56843e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:39.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.927+0000 7f0720c85640 1 --2- 172.21.15.6:0/4017019492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56841a90 0x55ab56843e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.927+0000 7f0720c85640 1 -- 172.21.15.6:0/4017019492 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab56724440 con 0x55ab56832ca0 2024-09-12T03:22:39.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.927+0000 7f0721c87640 1 --2- 172.21.15.6:0/4017019492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56841a90 0x55ab56843e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:39.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.928+0000 7f0721486640 1 --2- 172.21.15.6:0/4017019492 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832380 0x55ab56832760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:39.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.928+0000 7f0720c85640 1 --2- 172.21.15.6:0/4017019492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56832ca0 0x55ab56841550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f070c0114f0 tx=0x7f070c0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eda8e504200b612 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:39.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.928+0000 7f07077fe640 1 -- 172.21.15.6:0/4017019492 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c01e070 con 0x55ab56832ca0 2024-09-12T03:22:39.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.928+0000 7f07077fe640 1 -- 172.21.15.6:0/4017019492 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f070c0035f0 con 0x55ab56832ca0 2024-09-12T03:22:39.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.928+0000 7f07077fe640 1 -- 172.21.15.6:0/4017019492 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c00f820 con 0x55ab56832ca0 2024-09-12T03:22:39.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.929+0000 7f0725ac3740 1 -- 172.21.15.6:0/4017019492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56832ca0 msgr2=0x55ab56841550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.929+0000 7f0725ac3740 1 --2- 172.21.15.6:0/4017019492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56832ca0 0x55ab56841550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f070c0114f0 tx=0x7f070c0119b0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.929+0000 7f0725ac3740 1 -- 172.21.15.6:0/4017019492 shutdown_connections 2024-09-12T03:22:39.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.929+0000 7f0725ac3740 1 --2- 172.21.15.6:0/4017019492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56841a90 0x55ab56843e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.929+0000 7f0725ac3740 1 --2- 172.21.15.6:0/4017019492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56832ca0 0x55ab56841550 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.929+0000 7f0725ac3740 1 --2- 172.21.15.6:0/4017019492 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832380 0x55ab56832760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.929+0000 7f0725ac3740 1 -- 172.21.15.6:0/4017019492 >> 172.21.15.6:0/4017019492 conn(0x55ab5682ea30 msgr2=0x55ab5682ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:39.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.929+0000 7f0725ac3740 1 -- 172.21.15.6:0/4017019492 shutdown_connections 2024-09-12T03:22:39.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.929+0000 7f0725ac3740 1 -- 172.21.15.6:0/4017019492 wait complete. 2024-09-12T03:22:39.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.929+0000 7f0725ac3740 1 -- start start 2024-09-12T03:22:39.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.929+0000 7f0725ac3740 1 Processor -- start 2024-09-12T03:22:39.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.930+0000 7f0725ac3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56832380 0x55ab56841410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.930+0000 7f0725ac3740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832ca0 0x55ab5683e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.930+0000 7f0725ac3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56841a90 0x55ab5683ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.930+0000 7f0725ac3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55ab56844660 con 0x55ab56832380 2024-09-12T03:22:39.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.930+0000 7f0725ac3740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55ab56753050 con 0x55ab56832ca0 2024-09-12T03:22:39.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.930+0000 7f0725ac3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ab568447e0 con 0x55ab56841a90 2024-09-12T03:22:39.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.930+0000 7f0725ac3740 10 client.0 _ll_register_callbacks cb 0x55ab56793160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:39.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.930+0000 7f0721486640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56832380 0x55ab56841410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.930+0000 7f0721486640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56832380 0x55ab56841410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37094/0 (socket says 172.21.15.6:37094) 2024-09-12T03:22:39.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.930+0000 7f0721486640 1 -- 172.21.15.6:0/1123640304 learned_addr learned my addr 172.21.15.6:0/1123640304 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:39.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.930+0000 7f0721c87640 1 --2- 172.21.15.6:0/1123640304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56841a90 0x55ab5683ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.930+0000 7f0720c85640 1 --2- 172.21.15.6:0/1123640304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832ca0 0x55ab5683e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.930+0000 7f0721486640 1 -- 172.21.15.6:0/1123640304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56841a90 msgr2=0x55ab5683ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:39.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.930+0000 7f0721486640 1 --2- 172.21.15.6:0/1123640304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56841a90 0x55ab5683ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.930+0000 7f0721486640 1 -- 172.21.15.6:0/1123640304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832ca0 msgr2=0x55ab5683e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:39.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.930+0000 7f0721486640 1 --2- 172.21.15.6:0/1123640304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832ca0 0x55ab5683e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.930+0000 7f0721486640 1 -- 172.21.15.6:0/1123640304 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab565093c0 con 0x55ab56832380 2024-09-12T03:22:39.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.931+0000 7f0721486640 1 --2- 172.21.15.6:0/1123640304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56832380 0x55ab56841410 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0718035000 tx=0x7f07180095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a2650f6ea98578 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:39.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.931+0000 7f0704ff9640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:22:39.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.931+0000 7f0704ff9640 1 -- 172.21.15.6:0/1123640304 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0718034040 con 0x55ab56832380 2024-09-12T03:22:39.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.931+0000 7f0704ff9640 1 -- 172.21.15.6:0/1123640304 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f0718004040 con 0x55ab56832380 2024-09-12T03:22:39.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.931+0000 7f0725ac3740 1 -- 172.21.15.6:0/1123640304 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f070c0035f0 con 0x55ab56832380 2024-09-12T03:22:39.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.931+0000 7f0704ff9640 1 -- 172.21.15.6:0/1123640304 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0718004240 con 0x55ab56832380 2024-09-12T03:22:39.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.931+0000 7f0725ac3740 10 client.15579 fetch_fsmap learned FSMap version 26 2024-09-12T03:22:39.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.931+0000 7f0725ac3740 1 -- 172.21.15.6:0/1123640304 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ab56850090 con 0x55ab56832380 2024-09-12T03:22:39.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.931+0000 7f0704ff9640 1 -- 172.21.15.6:0/1123640304 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f070c0035f0 con 0x55ab56832380 2024-09-12T03:22:39.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.931+0000 7f0704ff9640 1 -- 172.21.15.6:0/1123640304 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0718050020 con 0x55ab56832380 2024-09-12T03:22:39.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.931+0000 7f0725ac3740 10 client.15579 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:22:39.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.931+0000 7f0725ac3740 10 client.15579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:22:39.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.931+0000 7f0725ac3740 20 client.15579 populate_metadata read hostname 'smithi006' 2024-09-12T03:22:39.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.931+0000 7f0725ac3740 1 --2- 172.21.15.6:0/1123640304 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ab56851550 0x55ab568539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.932+0000 7f0725ac3740 4 client.15579 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:22:39.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.932+0000 7f0725ac3740 1 -- 172.21.15.6:0/1123640304 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab567a5850 con 0x55ab56851550 2024-09-12T03:22:39.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.932+0000 7f0720c85640 1 --2- 172.21.15.6:0/1123640304 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ab56851550 0x55ab568539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.932+0000 7f0720c85640 1 --2- 172.21.15.6:0/1123640304 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ab56851550 0x55ab568539c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5411a609554beb0 server_cookie=2995d1c3afa8269 in_seq=0 out_seq=0 2024-09-12T03:22:39.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.932+0000 7f0704ff9640 10 client.15579 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:39.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.933+0000 7f0704ff9640 1 -- 172.21.15.6:0/1123640304 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab567a5850 con 0x55ab56851550 2024-09-12T03:22:39.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.934+0000 7f0704ff9640 10 client.15579 handle_command_reply: tid=0 2024-09-12T03:22:39.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.934+0000 7f0725ac3740 1 client.15579 shutdown 2024-09-12T03:22:39.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.934+0000 7f0725ac3740 1 -- 172.21.15.6:0/1123640304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56832380 msgr2=0x55ab56841410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.934+0000 7f0725ac3740 1 --2- 172.21.15.6:0/1123640304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56832380 0x55ab56841410 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0718035000 tx=0x7f07180095c0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.934+0000 7f0725ac3740 1 -- 172.21.15.6:0/1123640304 shutdown_connections 2024-09-12T03:22:39.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.934+0000 7f0725ac3740 1 --2- 172.21.15.6:0/1123640304 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ab56851550 0x55ab568539c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.934+0000 7f0725ac3740 1 --2- 172.21.15.6:0/1123640304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56841a90 0x55ab5683ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.934+0000 7f0725ac3740 1 --2- 172.21.15.6:0/1123640304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832ca0 0x55ab5683e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.934+0000 7f0725ac3740 1 --2- 172.21.15.6:0/1123640304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56832380 0x55ab56841410 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.934+0000 7f0704ff9640 0 client.15579 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:39.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.934+0000 7f0725ac3740 1 -- 172.21.15.6:0/1123640304 >> 172.21.15.6:0/1123640304 conn(0x55ab5682ea30 msgr2=0x55ab56834790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:39.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.934+0000 7f0725ac3740 1 -- 172.21.15.6:0/1123640304 shutdown_connections 2024-09-12T03:22:39.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.934+0000 7f0725ac3740 1 -- 172.21.15.6:0/1123640304 wait complete. 2024-09-12T03:22:39.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.934+0000 7f0725ac3740 20 client.15579 trim_cache size 0 max 16384 2024-09-12T03:22:39.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.953+0000 7f0725ac3740 1 Processor -- start 2024-09-12T03:22:39.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.953+0000 7f0725ac3740 1 -- start start 2024-09-12T03:22:39.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.953+0000 7f0725ac3740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832380 0x55ab5684d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.953+0000 7f0725ac3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56832ca0 0x55ab5682da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.954+0000 7f0725ac3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56841a90 0x55ab56859290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.954+0000 7f0725ac3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f071c1adaa0 con 0x55ab56841a90 2024-09-12T03:22:39.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.954+0000 7f0725ac3740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f071c1ab370 con 0x55ab56832380 2024-09-12T03:22:39.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.954+0000 7f0725ac3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f071c1a91e0 con 0x55ab56832ca0 2024-09-12T03:22:39.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.954+0000 7f0720c85640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56832ca0 0x55ab5682da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.954+0000 7f0720c85640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56832ca0 0x55ab5682da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:41948/0 (socket says 172.21.15.6:41948) 2024-09-12T03:22:39.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.954+0000 7f0720c85640 1 -- 172.21.15.6:0/1339141897 learned_addr learned my addr 172.21.15.6:0/1339141897 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:39.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.954+0000 7f0721486640 1 --2- 172.21.15.6:0/1339141897 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832380 0x55ab5684d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.954+0000 7f0721c87640 1 --2- 172.21.15.6:0/1339141897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56841a90 0x55ab56859290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.954+0000 7f0721486640 1 -- 172.21.15.6:0/1339141897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56832ca0 msgr2=0x55ab5682da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:39.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.954+0000 7f0721486640 1 --2- 172.21.15.6:0/1339141897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56832ca0 0x55ab5682da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.954+0000 7f0721486640 1 -- 172.21.15.6:0/1339141897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56841a90 msgr2=0x55ab56859290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:39.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.954+0000 7f0721486640 1 --2- 172.21.15.6:0/1339141897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56841a90 0x55ab56859290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.954+0000 7f0721486640 1 -- 172.21.15.6:0/1339141897 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab5683efb0 con 0x55ab56832380 2024-09-12T03:22:39.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.954+0000 7f0720c85640 1 --2- 172.21.15.6:0/1339141897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56832ca0 0x55ab5682da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:39.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.954+0000 7f0721486640 1 --2- 172.21.15.6:0/1339141897 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832380 0x55ab5684d900 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f071802ae70 tx=0x7f071802a950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c89142d64f37c93 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:39.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.954+0000 7f0706ffd640 1 -- 172.21.15.6:0/1339141897 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0718062070 con 0x55ab56832380 2024-09-12T03:22:39.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.955+0000 7f0706ffd640 1 -- 172.21.15.6:0/1339141897 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0718052e50 con 0x55ab56832380 2024-09-12T03:22:39.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.955+0000 7f0706ffd640 1 -- 172.21.15.6:0/1339141897 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0718034030 con 0x55ab56832380 2024-09-12T03:22:39.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.955+0000 7f0725ac3740 1 -- 172.21.15.6:0/1339141897 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832380 msgr2=0x55ab5684d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.955+0000 7f0725ac3740 1 --2- 172.21.15.6:0/1339141897 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832380 0x55ab5684d900 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f071802ae70 tx=0x7f071802a950 comp rx=0 tx=0).stop 2024-09-12T03:22:39.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.955+0000 7f0725ac3740 1 -- 172.21.15.6:0/1339141897 shutdown_connections 2024-09-12T03:22:39.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.955+0000 7f0725ac3740 1 --2- 172.21.15.6:0/1339141897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56841a90 0x55ab56859290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.955+0000 7f0725ac3740 1 --2- 172.21.15.6:0/1339141897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56832ca0 0x55ab5682da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.955+0000 7f0725ac3740 1 --2- 172.21.15.6:0/1339141897 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832380 0x55ab5684d900 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.955+0000 7f0725ac3740 1 -- 172.21.15.6:0/1339141897 >> 172.21.15.6:0/1339141897 conn(0x55ab5682ea30 msgr2=0x55ab5682d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:39.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.956+0000 7f0725ac3740 1 -- 172.21.15.6:0/1339141897 shutdown_connections 2024-09-12T03:22:39.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.956+0000 7f0725ac3740 1 -- 172.21.15.6:0/1339141897 wait complete. 2024-09-12T03:22:39.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.956+0000 7f0725ac3740 1 -- start start 2024-09-12T03:22:39.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.956+0000 7f0725ac3740 1 Processor -- start 2024-09-12T03:22:39.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.956+0000 7f0725ac3740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832380 0x55ab56845b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.956+0000 7f0725ac3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56832ca0 0x55ab56846080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.957+0000 7f0725ac3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56841a90 0x55ab568bb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.957+0000 7f0725ac3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f071c1a9830 con 0x55ab56832ca0 2024-09-12T03:22:39.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.957+0000 7f0725ac3740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f071c1aaca0 con 0x55ab56832380 2024-09-12T03:22:39.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.957+0000 7f0725ac3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f071c1a9f00 con 0x55ab56841a90 2024-09-12T03:22:39.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.957+0000 7f0725ac3740 10 client.0 _ll_register_callbacks cb 0x7f071802f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:39.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.957+0000 7f0721c87640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56841a90 0x55ab568bb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.957+0000 7f0721c87640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56841a90 0x55ab568bb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:41964/0 (socket says 172.21.15.6:41964) 2024-09-12T03:22:39.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.957+0000 7f0721c87640 1 -- 172.21.15.6:0/602878189 learned_addr learned my addr 172.21.15.6:0/602878189 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.957+0000 7f0721486640 1 --2- 172.21.15.6:0/602878189 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832380 0x55ab56845b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.957+0000 7f0720c85640 1 --2- 172.21.15.6:0/602878189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56832ca0 0x55ab56846080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.957+0000 7f0721c87640 1 -- 172.21.15.6:0/602878189 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832380 msgr2=0x55ab56845b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.957+0000 7f0721c87640 1 --2- 172.21.15.6:0/602878189 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832380 0x55ab56845b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.957+0000 7f0721c87640 1 -- 172.21.15.6:0/602878189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56832ca0 msgr2=0x55ab56846080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.957+0000 7f0721c87640 1 --2- 172.21.15.6:0/602878189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56832ca0 0x55ab56846080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.957+0000 7f0721c87640 1 -- 172.21.15.6:0/602878189 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab56846960 con 0x55ab56841a90 2024-09-12T03:22:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.957+0000 7f0721c87640 1 --2- 172.21.15.6:0/602878189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56841a90 0x55ab568bb260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0714097cb0 tx=0x7f071400a570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb99bd75866745cc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.957+0000 7f0704ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:22:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.958+0000 7f0704ff9640 1 -- 172.21.15.6:0/602878189 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07140b8070 con 0x55ab56841a90 2024-09-12T03:22:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.958+0000 7f0704ff9640 1 -- 172.21.15.6:0/602878189 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f071409cb80 con 0x55ab56841a90 2024-09-12T03:22:39.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.958+0000 7f0725ac3740 1 -- 172.21.15.6:0/602878189 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0718052e50 con 0x55ab56841a90 2024-09-12T03:22:39.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.958+0000 7f0704ff9640 1 -- 172.21.15.6:0/602878189 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f071409cd80 con 0x55ab56841a90 2024-09-12T03:22:39.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.958+0000 7f0725ac3740 10 client.25177 fetch_fsmap learned FSMap version 26 2024-09-12T03:22:39.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.958+0000 7f0725ac3740 1 -- 172.21.15.6:0/602878189 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ab568bc8c0 con 0x55ab56841a90 2024-09-12T03:22:39.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.958+0000 7f0704ff9640 1 -- 172.21.15.6:0/602878189 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0718052e50 con 0x55ab56841a90 2024-09-12T03:22:39.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.959+0000 7f0704ff9640 1 -- 172.21.15.6:0/602878189 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0714020030 con 0x55ab56841a90 2024-09-12T03:22:39.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.959+0000 7f0725ac3740 10 client.25177 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:22:39.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.959+0000 7f0725ac3740 10 client.25177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:22:39.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.959+0000 7f0725ac3740 20 client.25177 populate_metadata read hostname 'smithi006' 2024-09-12T03:22:39.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.959+0000 7f0725ac3740 1 --2- 172.21.15.6:0/602878189 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ab568bd010 0x55ab568bf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.959+0000 7f0725ac3740 4 client.25177 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:22:39.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.959+0000 7f0725ac3740 1 -- 172.21.15.6:0/602878189 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ab5679fd60 con 0x55ab568bd010 2024-09-12T03:22:39.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.959+0000 7f0721486640 1 --2- 172.21.15.6:0/602878189 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ab568bd010 0x55ab568bf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.960+0000 7f0704ff9640 10 client.25177 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:39.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.960+0000 7f0721486640 1 --2- 172.21.15.6:0/602878189 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ab568bd010 0x55ab568bf430 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d9a738877c69bc0 server_cookie=7fe553bfd22740ef in_seq=0 out_seq=0 2024-09-12T03:22:39.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.961+0000 7f0704ff9640 1 -- 172.21.15.6:0/602878189 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ab5679fd60 con 0x55ab568bd010 2024-09-12T03:22:39.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.961+0000 7f0704ff9640 10 client.25177 handle_command_reply: tid=0 2024-09-12T03:22:39.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.961+0000 7f0725ac3740 1 client.25177 shutdown 2024-09-12T03:22:39.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.961+0000 7f0725ac3740 1 -- 172.21.15.6:0/602878189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56841a90 msgr2=0x55ab568bb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.962+0000 7f0725ac3740 1 --2- 172.21.15.6:0/602878189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56841a90 0x55ab568bb260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0714097cb0 tx=0x7f071400a570 comp rx=0 tx=0).stop 2024-09-12T03:22:39.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.962+0000 7f0725ac3740 1 -- 172.21.15.6:0/602878189 shutdown_connections 2024-09-12T03:22:39.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.962+0000 7f0725ac3740 1 --2- 172.21.15.6:0/602878189 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ab568bd010 0x55ab568bf430 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.962+0000 7f0725ac3740 1 --2- 172.21.15.6:0/602878189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab56841a90 0x55ab568bb260 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.962+0000 7f0704ff9640 0 client.25177 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:39.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.962+0000 7f0725ac3740 1 --2- 172.21.15.6:0/602878189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ab56832ca0 0x55ab56846080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.963+0000 7f0725ac3740 1 --2- 172.21.15.6:0/602878189 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ab56832380 0x55ab56845b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.963+0000 7f0725ac3740 1 -- 172.21.15.6:0/602878189 >> 172.21.15.6:0/602878189 conn(0x55ab5682ea30 msgr2=0x55ab5682db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:39.965 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:22:39.965 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:22:39.965 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2024-09-12T03:22:39.965 INFO:teuthology.orchestra.run.smithi006.stdout: "08ddab5a-774f-4d21-aecd-e479d4c85235": { 2024-09-12T03:22:39.965 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-09-12T03:22:39.965 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "08ddab5a-774f-4d21-aecd-e479d4c85235", 2024-09-12T03:22:39.965 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2024-09-12T03:22:39.965 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:22:39.966 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:22:39.966 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:22:39.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.963+0000 7f0725ac3740 1 -- 172.21.15.6:0/602878189 shutdown_connections 2024-09-12T03:22:39.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.963+0000 7f0725ac3740 1 -- 172.21.15.6:0/602878189 wait complete. 2024-09-12T03:22:39.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.963+0000 7f0725ac3740 20 client.25177 trim_cache size 0 max 16384 2024-09-12T03:22:39.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.964+0000 7f07236ea640 1 -- 172.21.15.6:0/121094110 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f07080a2b80 msgr2=0x7f07080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.964+0000 7f07236ea640 1 --2- 172.21.15.6:0/121094110 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f07080a2b80 0x7f07080a5040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0718004ad0 tx=0x7f0718004a60 comp rx=0 tx=0).stop 2024-09-12T03:22:39.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.964+0000 7f07236ea640 1 -- 172.21.15.6:0/121094110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f071c102ef0 msgr2=0x7f071c19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.964+0000 7f07236ea640 1 --2- 172.21.15.6:0/121094110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f071c102ef0 0x7f071c19cd40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f071400c970 tx=0x7f071400ce30 comp rx=0 tx=0).stop 2024-09-12T03:22:39.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.965+0000 7f07236ea640 1 -- 172.21.15.6:0/121094110 shutdown_connections 2024-09-12T03:22:39.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.965+0000 7f07236ea640 1 --2- 172.21.15.6:0/121094110 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f07080a2b80 0x7f07080a5040 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.965+0000 7f07236ea640 1 --2- 172.21.15.6:0/121094110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f071c102ef0 0x7f071c19cd40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.965+0000 7f07236ea640 1 --2- 172.21.15.6:0/121094110 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f071c102570 0x7f071c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.965+0000 7f07236ea640 1 --2- 172.21.15.6:0/121094110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f071c100e20 0x7f071c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.965+0000 7f07236ea640 1 -- 172.21.15.6:0/121094110 >> 172.21.15.6:0/121094110 conn(0x7f071c0fc7d0 msgr2=0x7f071c105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:39.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.965+0000 7f07236ea640 1 -- 172.21.15.6:0/121094110 shutdown_connections 2024-09-12T03:22:39.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:39.965+0000 7f07236ea640 1 -- 172.21.15.6:0/121094110 wait complete. 2024-09-12T03:22:40.831 INFO:tasks.cephfs.filesystem:scrub status for tag:08ddab5a-774f-4d21-aecd-e479d4c85235 - {'path': '/', 'tag': '08ddab5a-774f-4d21-aecd-e479d4c85235', 'options': 'recursive,force'} 2024-09-12T03:22:40.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:40 smithi118 ceph-mon[35048]: pgmap v382: 97 pgs: 97 active+clean; 5.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 497 MiB/s wr, 245 op/s 2024-09-12T03:22:41.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:40 smithi006 ceph-mon[30991]: pgmap v382: 97 pgs: 97 active+clean; 5.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 497 MiB/s wr, 245 op/s 2024-09-12T03:22:41.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:40 smithi161 ceph-mon[35485]: pgmap v382: 97 pgs: 97 active+clean; 5.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 497 MiB/s wr, 245 op/s 2024-09-12T03:22:42.572 INFO:tasks.workunit.client.0.smithi006.stdout:fs setup took 15 secs 2024-09-12T03:22:43.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:42 smithi006 ceph-mon[30991]: pgmap v383: 97 pgs: 97 active+clean; 7.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 607 MiB/s wr, 280 op/s 2024-09-12T03:22:43.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:42 smithi161 ceph-mon[35485]: pgmap v383: 97 pgs: 97 active+clean; 7.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 607 MiB/s wr, 280 op/s 2024-09-12T03:22:43.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:42 smithi118 ceph-mon[35048]: pgmap v383: 97 pgs: 97 active+clean; 7.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 607 MiB/s wr, 280 op/s 2024-09-12T03:22:45.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:44 smithi006 ceph-mon[30991]: pgmap v384: 97 pgs: 97 active+clean; 9.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 711 MiB/s wr, 286 op/s 2024-09-12T03:22:45.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:44 smithi161 ceph-mon[35485]: pgmap v384: 97 pgs: 97 active+clean; 9.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 711 MiB/s wr, 286 op/s 2024-09-12T03:22:45.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:44 smithi118 ceph-mon[35048]: pgmap v384: 97 pgs: 97 active+clean; 9.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 711 MiB/s wr, 286 op/s 2024-09-12T03:22:46.191 INFO:tasks.workunit.client.0.smithi006.stdout:Syncing()...3 sec 2024-09-12T03:22:46.191 INFO:tasks.workunit.client.0.smithi006.stdout:Starting Actual Benchmark At: Thu Sep 12 03:22:46 2024 2024-09-12T03:22:46.191 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-09-12T03:22:47.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:46 smithi006 ceph-mon[30991]: pgmap v385: 97 pgs: 97 active+clean; 9.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 648 MiB/s wr, 254 op/s 2024-09-12T03:22:47.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:46 smithi006 ceph-mon[30991]: scrub summary: idle 2024-09-12T03:22:47.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:46 smithi006 ceph-mon[30991]: scrub summary: idle 2024-09-12T03:22:47.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:46 smithi006 ceph-mon[30991]: scrub summary: idle 2024-09-12T03:22:47.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:46 smithi161 ceph-mon[35485]: pgmap v385: 97 pgs: 97 active+clean; 9.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 648 MiB/s wr, 254 op/s 2024-09-12T03:22:47.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:46 smithi161 ceph-mon[35485]: scrub summary: idle 2024-09-12T03:22:47.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:46 smithi161 ceph-mon[35485]: scrub summary: idle 2024-09-12T03:22:47.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:46 smithi161 ceph-mon[35485]: scrub summary: idle 2024-09-12T03:22:47.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:46 smithi118 ceph-mon[35048]: pgmap v385: 97 pgs: 97 active+clean; 9.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 648 MiB/s wr, 254 op/s 2024-09-12T03:22:47.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:46 smithi118 ceph-mon[35048]: scrub summary: idle 2024-09-12T03:22:47.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:46 smithi118 ceph-mon[35048]: scrub summary: idle 2024-09-12T03:22:47.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:46 smithi118 ceph-mon[35048]: scrub summary: idle 2024-09-12T03:22:49.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:48 smithi006 ceph-mon[30991]: pgmap v386: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 715 MiB/s wr, 277 op/s 2024-09-12T03:22:49.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:48 smithi161 ceph-mon[35485]: pgmap v386: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 715 MiB/s wr, 277 op/s 2024-09-12T03:22:49.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:48 smithi118 ceph-mon[35048]: pgmap v386: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 715 MiB/s wr, 277 op/s 2024-09-12T03:22:50.166 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.4 2024-09-12T03:22:50.166 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:22:50.776 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:51.304 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:50 smithi006 ceph-mon[30991]: pgmap v387: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 720 MiB/s wr, 2.24k op/s 2024-09-12T03:22:51.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:50 smithi161 ceph-mon[35485]: pgmap v387: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 720 MiB/s wr, 2.24k op/s 2024-09-12T03:22:51.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:50 smithi118 ceph-mon[35048]: pgmap v387: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 720 MiB/s wr, 2.24k op/s 2024-09-12T03:22:52.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.461+0000 7fe5985e2640 1 -- 172.21.15.6:0/3481460713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe590114ab0 msgr2=0x7fe590116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:52.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.461+0000 7fe5985e2640 1 --2- 172.21.15.6:0/3481460713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe590114ab0 0x7fe590116ea0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe588009900 tx=0x7fe58802a4a0 comp rx=0 tx=0).stop 2024-09-12T03:22:52.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.461+0000 7fe5985e2640 1 -- 172.21.15.6:0/3481460713 shutdown_connections 2024-09-12T03:22:52.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.461+0000 7fe5985e2640 1 --2- 172.21.15.6:0/3481460713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe590114ab0 0x7fe590116ea0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:52.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.461+0000 7fe5985e2640 1 --2- 172.21.15.6:0/3481460713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe590109b10 0x7fe590114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:52.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.461+0000 7fe5985e2640 1 --2- 172.21.15.6:0/3481460713 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5901091f0 0x7fe5901095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:52.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.461+0000 7fe5985e2640 1 -- 172.21.15.6:0/3481460713 >> 172.21.15.6:0/3481460713 conn(0x7fe59006e140 msgr2=0x7fe59006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:52.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.462+0000 7fe5985e2640 1 -- 172.21.15.6:0/3481460713 shutdown_connections 2024-09-12T03:22:52.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.462+0000 7fe5985e2640 1 -- 172.21.15.6:0/3481460713 wait complete. 2024-09-12T03:22:52.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.463+0000 7fe5985e2640 1 Processor -- start 2024-09-12T03:22:52.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.463+0000 7fe5985e2640 1 -- start start 2024-09-12T03:22:52.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.463+0000 7fe5985e2640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5901091f0 0x7fe5901144d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:52.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.463+0000 7fe5985e2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe590109b10 0x7fe590111580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:52.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.463+0000 7fe5985e2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe590114ab0 0x7fe590111b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:52.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.463+0000 7fe5985e2640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe59011c9d0 con 0x7fe590109b10 2024-09-12T03:22:52.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.463+0000 7fe5985e2640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe59011c850 con 0x7fe5901091f0 2024-09-12T03:22:52.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.463+0000 7fe5985e2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe59011cb50 con 0x7fe590114ab0 2024-09-12T03:22:52.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.464+0000 7fe59637e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5901091f0 0x7fe5901144d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:52.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.464+0000 7fe59637e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5901091f0 0x7fe5901144d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:53568/0 (socket says 172.21.15.6:53568) 2024-09-12T03:22:52.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.464+0000 7fe59637e640 1 -- 172.21.15.6:0/1001753566 learned_addr learned my addr 172.21.15.6:0/1001753566 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:52.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.464+0000 7fe596b7f640 1 --2- 172.21.15.6:0/1001753566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe590114ab0 0x7fe590111b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:52.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.465+0000 7fe595b7d640 1 --2- 172.21.15.6:0/1001753566 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe590109b10 0x7fe590111580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:52.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.465+0000 7fe59637e640 1 -- 172.21.15.6:0/1001753566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe590114ab0 msgr2=0x7fe590111b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:52.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.465+0000 7fe59637e640 1 --2- 172.21.15.6:0/1001753566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe590114ab0 0x7fe590111b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:52.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.465+0000 7fe59637e640 1 -- 172.21.15.6:0/1001753566 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe590109b10 msgr2=0x7fe590111580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:52.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.465+0000 7fe59637e640 1 --2- 172.21.15.6:0/1001753566 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe590109b10 0x7fe590111580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:52.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.465+0000 7fe59637e640 1 -- 172.21.15.6:0/1001753566 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5901a8ef0 con 0x7fe5901091f0 2024-09-12T03:22:52.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.466+0000 7fe59637e640 1 --2- 172.21.15.6:0/1001753566 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5901091f0 0x7fe5901144d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe58400ebc0 tx=0x7fe58400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:52.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.466+0000 7fe58f7fe640 1 -- 172.21.15.6:0/1001753566 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584010630 con 0x7fe5901091f0 2024-09-12T03:22:52.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.466+0000 7fe5985e2640 1 -- 172.21.15.6:0/1001753566 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5901a91e0 con 0x7fe5901091f0 2024-09-12T03:22:52.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.467+0000 7fe5985e2640 1 -- 172.21.15.6:0/1001753566 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5901a9770 con 0x7fe5901091f0 2024-09-12T03:22:52.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.467+0000 7fe58f7fe640 1 -- 172.21.15.6:0/1001753566 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe584010cb0 con 0x7fe5901091f0 2024-09-12T03:22:52.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.467+0000 7fe58f7fe640 1 -- 172.21.15.6:0/1001753566 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584018c40 con 0x7fe5901091f0 2024-09-12T03:22:52.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.469+0000 7fe5985e2640 1 -- 172.21.15.6:0/1001753566 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe574005290 con 0x7fe5901091f0 2024-09-12T03:22:52.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.471+0000 7fe58f7fe640 1 -- 172.21.15.6:0/1001753566 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe584018420 con 0x7fe5901091f0 2024-09-12T03:22:52.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.472+0000 7fe58f7fe640 1 --2- 172.21.15.6:0/1001753566 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5800a2cc0 0x7fe5800a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:52.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.472+0000 7fe58f7fe640 1 -- 172.21.15.6:0/1001753566 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe5840cf6c0 con 0x7fe5901091f0 2024-09-12T03:22:52.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.473+0000 7fe595b7d640 1 --2- 172.21.15.6:0/1001753566 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5800a2cc0 0x7fe5800a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:52.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.475+0000 7fe595b7d640 1 --2- 172.21.15.6:0/1001753566 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5800a2cc0 0x7fe5800a5180 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe590112d40 tx=0x7fe57c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:52.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.490+0000 7fe58f7fe640 1 -- 172.21.15.6:0/1001753566 <== mon.2 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe58409c5f0 con 0x7fe5901091f0 2024-09-12T03:22:52.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.948+0000 7fe5985e2640 1 -- 172.21.15.6:0/1001753566 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe574005580 con 0x7fe5901091f0 2024-09-12T03:22:52.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.949+0000 7fe58f7fe640 1 -- 172.21.15.6:0/1001753566 <== mon.2 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11911 (secure 0 0 0) 0x7fe58409bef0 con 0x7fe5901091f0 2024-09-12T03:22:52.950 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:22:52.950 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":26,"btime":"2024-09-12T03:22:16:132848+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:22:16.064679+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24463,24464,24494]},"id":1}]} 2024-09-12T03:22:52.950 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 26 2024-09-12T03:22:52.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.954+0000 7fe5985e2640 1 -- 172.21.15.6:0/1001753566 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5800a2cc0 msgr2=0x7fe5800a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:52.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.954+0000 7fe5985e2640 1 --2- 172.21.15.6:0/1001753566 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5800a2cc0 0x7fe5800a5180 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe590112d40 tx=0x7fe57c008040 comp rx=0 tx=0).stop 2024-09-12T03:22:52.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.954+0000 7fe5985e2640 1 -- 172.21.15.6:0/1001753566 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5901091f0 msgr2=0x7fe5901144d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:52.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.954+0000 7fe5985e2640 1 --2- 172.21.15.6:0/1001753566 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5901091f0 0x7fe5901144d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe58400ebc0 tx=0x7fe58400c6a0 comp rx=0 tx=0).stop 2024-09-12T03:22:52.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.955+0000 7fe5985e2640 1 -- 172.21.15.6:0/1001753566 shutdown_connections 2024-09-12T03:22:52.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.955+0000 7fe5985e2640 1 --2- 172.21.15.6:0/1001753566 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5800a2cc0 0x7fe5800a5180 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:52.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.955+0000 7fe5985e2640 1 --2- 172.21.15.6:0/1001753566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe590114ab0 0x7fe590111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:52.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.955+0000 7fe5985e2640 1 --2- 172.21.15.6:0/1001753566 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe590109b10 0x7fe590111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:52.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.955+0000 7fe5985e2640 1 --2- 172.21.15.6:0/1001753566 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5901091f0 0x7fe5901144d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:52.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.955+0000 7fe5985e2640 1 -- 172.21.15.6:0/1001753566 >> 172.21.15.6:0/1001753566 conn(0x7fe59006e140 msgr2=0x7fe5900707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:52.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.956+0000 7fe5985e2640 1 -- 172.21.15.6:0/1001753566 shutdown_connections 2024-09-12T03:22:52.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:52.956+0000 7fe5985e2640 1 -- 172.21.15.6:0/1001753566 wait complete. 2024-09-12T03:22:53.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:53 smithi161 ceph-mon[35485]: pgmap v388: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 612 MiB/s wr, 6.58k op/s 2024-09-12T03:22:53.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:53 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1001753566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:22:53.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:53 smithi006 ceph-mon[30991]: pgmap v388: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 612 MiB/s wr, 6.58k op/s 2024-09-12T03:22:53.364 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:53 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1001753566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:22:53.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:53 smithi118 ceph-mon[35048]: pgmap v388: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 612 MiB/s wr, 6.58k op/s 2024-09-12T03:22:53.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:53 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1001753566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:22:55.290 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:55 smithi006 ceph-mon[30991]: pgmap v389: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 547 MiB/s wr, 12.81k op/s 2024-09-12T03:22:55.291 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0' 2024-09-12T03:22:55.292 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0 2024-09-12T03:22:55.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:55 smithi161 ceph-mon[35485]: pgmap v389: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 547 MiB/s wr, 12.81k op/s 2024-09-12T03:22:55.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:55 smithi118 ceph-mon[35048]: pgmap v389: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 547 MiB/s wr, 12.81k op/s 2024-09-12T03:22:55.828 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:57.343 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:57 smithi006 ceph-mon[30991]: pgmap v390: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 417 MiB/s wr, 16.08k op/s 2024-09-12T03:22:57.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:57 smithi161 ceph-mon[35485]: pgmap v390: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 417 MiB/s wr, 16.08k op/s 2024-09-12T03:22:57.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:57 smithi118 ceph-mon[35048]: pgmap v390: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 417 MiB/s wr, 16.08k op/s 2024-09-12T03:22:57.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.514+0000 7f7b7ae8c640 1 -- 172.21.15.6:0/2494395007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b7410d690 msgr2=0x7f7b7410db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:57.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.515+0000 7f7b737fe640 1 -- 172.21.15.6:0/2494395007 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64004230 con 0x7f7b7410d690 2024-09-12T03:22:57.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.515+0000 7f7b7ae8c640 1 --2- 172.21.15.6:0/2494395007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b7410d690 0x7f7b7410db40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009900 tx=0x7f7b6402a410 comp rx=0 tx=0).stop 2024-09-12T03:22:57.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.516+0000 7f7b7ae8c640 1 -- 172.21.15.6:0/2494395007 shutdown_connections 2024-09-12T03:22:57.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.516+0000 7f7b7ae8c640 1 --2- 172.21.15.6:0/2494395007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b7410d690 0x7f7b7410db40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.516+0000 7f7b7ae8c640 1 --2- 172.21.15.6:0/2494395007 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b74073ca0 0x7f7b74109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.516+0000 7f7b7ae8c640 1 --2- 172.21.15.6:0/2494395007 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b74073830 0x7f7b74109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.516+0000 7f7b7ae8c640 1 -- 172.21.15.6:0/2494395007 >> 172.21.15.6:0/2494395007 conn(0x7f7b7406e140 msgr2=0x7f7b7406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:57.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.518+0000 7f7b7ae8c640 1 -- 172.21.15.6:0/2494395007 shutdown_connections 2024-09-12T03:22:57.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.519+0000 7f7b7ae8c640 1 -- 172.21.15.6:0/2494395007 wait complete. 2024-09-12T03:22:57.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.520+0000 7f7b7ae8c640 1 Processor -- start 2024-09-12T03:22:57.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.520+0000 7f7b7ae8c640 1 -- start start 2024-09-12T03:22:57.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.520+0000 7f7b7ae8c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b74073830 0x7f7b7411eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.520+0000 7f7b7ae8c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b74073ca0 0x7f7b7411efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.521+0000 7f7b7ae8c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b7410d690 0x7f7b74119c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.521+0000 7f7b7ae8c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74116610 con 0x7f7b7410d690 2024-09-12T03:22:57.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.521+0000 7f7b7ae8c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74116490 con 0x7f7b74073830 2024-09-12T03:22:57.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.521+0000 7f7b7ae8c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74116790 con 0x7f7b74073ca0 2024-09-12T03:22:57.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.523+0000 7f7b79429640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b7410d690 0x7f7b74119c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.523+0000 7f7b79429640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b7410d690 0x7f7b74119c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32962/0 (socket says 172.21.15.6:32962) 2024-09-12T03:22:57.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.523+0000 7f7b73fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b74073ca0 0x7f7b7411efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.523+0000 7f7b79429640 1 -- 172.21.15.6:0/1362060025 learned_addr learned my addr 172.21.15.6:0/1362060025 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:57.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.523+0000 7f7b78c28640 1 --2- 172.21.15.6:0/1362060025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b74073830 0x7f7b7411eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.524+0000 7f7b79429640 1 -- 172.21.15.6:0/1362060025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b74073ca0 msgr2=0x7f7b7411efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:57.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.524+0000 7f7b79429640 1 --2- 172.21.15.6:0/1362060025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b74073ca0 0x7f7b7411efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.524+0000 7f7b79429640 1 -- 172.21.15.6:0/1362060025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b74073830 msgr2=0x7f7b7411eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:57.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.524+0000 7f7b79429640 1 --2- 172.21.15.6:0/1362060025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b74073830 0x7f7b7411eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.524+0000 7f7b79429640 1 -- 172.21.15.6:0/1362060025 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7411a4d0 con 0x7f7b7410d690 2024-09-12T03:22:57.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.557+0000 7f7b79429640 1 --2- 172.21.15.6:0/1362060025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b7410d690 0x7f7b74119c10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7b6402aa00 tx=0x7f7b64005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:57.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.557+0000 7f7b71ffb640 1 -- 172.21.15.6:0/1362060025 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64040030 con 0x7f7b7410d690 2024-09-12T03:22:57.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.558+0000 7f7b7ae8c640 1 -- 172.21.15.6:0/1362060025 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7411a760 con 0x7f7b7410d690 2024-09-12T03:22:57.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.558+0000 7f7b7ae8c640 1 -- 172.21.15.6:0/1362060025 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b74124990 con 0x7f7b7410d690 2024-09-12T03:22:57.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.558+0000 7f7b71ffb640 1 -- 172.21.15.6:0/1362060025 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b6403c920 con 0x7f7b7410d690 2024-09-12T03:22:57.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.560+0000 7f7b71ffb640 1 -- 172.21.15.6:0/1362060025 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6404a060 con 0x7f7b7410d690 2024-09-12T03:22:57.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.563+0000 7f7b7d265740 1 Processor -- start 2024-09-12T03:22:57.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.566+0000 7f7b71ffb640 1 -- 172.21.15.6:0/1362060025 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7b64043030 con 0x7f7b7410d690 2024-09-12T03:22:57.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.566+0000 7f7b7d265740 1 -- start start 2024-09-12T03:22:57.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.568+0000 7f7b7d265740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56029beea760 0x56029beeab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.568+0000 7f7b7d265740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029beeb080 0x56029bef5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.568+0000 7f7b7d265740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029bef5940 0x56029bef7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.568+0000 7f7b7d265740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56029be502a0 con 0x56029beea760 2024-09-12T03:22:57.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.568+0000 7f7b7d265740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x56029bdf7b10 con 0x56029bef5940 2024-09-12T03:22:57.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.568+0000 7f7b7d265740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56029be4c260 con 0x56029beeb080 2024-09-12T03:22:57.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.568+0000 7f7b71ffb640 1 --2- 172.21.15.6:0/1362060025 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7b5c0a2c40 0x7f7b5c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.569+0000 7f7b71ffb640 1 -- 172.21.15.6:0/1362060025 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7b640ef720 con 0x7f7b7410d690 2024-09-12T03:22:57.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.570+0000 7f7b73fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56029beea760 0x56029beeab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.571+0000 7f7b73fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56029beea760 0x56029beeab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32964/0 (socket says 172.21.15.6:32964) 2024-09-12T03:22:57.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.571+0000 7f7b73fff640 1 -- 172.21.15.6:0/1611344966 learned_addr learned my addr 172.21.15.6:0/1611344966 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:57.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.571+0000 7f7b79429640 1 --2- 172.21.15.6:0/1362060025 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7b5c0a2c40 0x7f7b5c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.571+0000 7f7b73fff640 1 --2- 172.21.15.6:0/1611344966 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029bef5940 0x56029bef7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.571+0000 7f7b78c28640 1 --2- 172.21.15.6:0/1611344966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029beeb080 0x56029bef5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.573+0000 7f7b79429640 1 --2- 172.21.15.6:0/1362060025 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7b5c0a2c40 0x7f7b5c0a5100 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7b6402a450 tx=0x7f7b64005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:57.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.573+0000 7f7b73fff640 1 -- 172.21.15.6:0/1611344966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029beeb080 msgr2=0x56029bef5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:57.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.573+0000 7f7b73fff640 1 --2- 172.21.15.6:0/1611344966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029beeb080 0x56029bef5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.573+0000 7f7b73fff640 1 -- 172.21.15.6:0/1611344966 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029bef5940 msgr2=0x56029bef7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:57.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.573+0000 7f7b73fff640 1 --2- 172.21.15.6:0/1611344966 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029bef5940 0x56029bef7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.574+0000 7f7b73fff640 1 -- 172.21.15.6:0/1611344966 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56029bbc1b90 con 0x56029beea760 2024-09-12T03:22:57.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.576+0000 7f7b73fff640 1 --2- 172.21.15.6:0/1611344966 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56029beea760 0x56029beeab40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c00bd30 tx=0x7f7b6c009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb0b0382940ef80 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:57.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.577+0000 7f7b5affd640 1 -- 172.21.15.6:0/1611344966 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c014040 con 0x56029beea760 2024-09-12T03:22:57.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.578+0000 7f7b5affd640 1 -- 172.21.15.6:0/1611344966 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b6c008d50 con 0x56029beea760 2024-09-12T03:22:57.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.578+0000 7f7b5affd640 1 -- 172.21.15.6:0/1611344966 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c0195e0 con 0x56029beea760 2024-09-12T03:22:57.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.580+0000 7f7b7d265740 1 -- 172.21.15.6:0/1611344966 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56029beea760 msgr2=0x56029beeab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:57.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.580+0000 7f7b7d265740 1 --2- 172.21.15.6:0/1611344966 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56029beea760 0x56029beeab40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c00bd30 tx=0x7f7b6c009b70 comp rx=0 tx=0).stop 2024-09-12T03:22:57.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.580+0000 7f7b7d265740 1 -- 172.21.15.6:0/1611344966 shutdown_connections 2024-09-12T03:22:57.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.580+0000 7f7b7d265740 1 --2- 172.21.15.6:0/1611344966 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029bef5940 0x56029bef7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.580+0000 7f7b7d265740 1 --2- 172.21.15.6:0/1611344966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029beeb080 0x56029bef5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.580+0000 7f7b7d265740 1 --2- 172.21.15.6:0/1611344966 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56029beea760 0x56029beeab40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.580+0000 7f7b7d265740 1 -- 172.21.15.6:0/1611344966 >> 172.21.15.6:0/1611344966 conn(0x56029bee6cf0 msgr2=0x56029bee70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:57.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.581+0000 7f7b7d265740 1 -- 172.21.15.6:0/1611344966 shutdown_connections 2024-09-12T03:22:57.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.581+0000 7f7b7d265740 1 -- 172.21.15.6:0/1611344966 wait complete. 2024-09-12T03:22:57.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.581+0000 7f7b7d265740 1 -- start start 2024-09-12T03:22:57.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.582+0000 7f7b7d265740 1 Processor -- start 2024-09-12T03:22:57.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.583+0000 7f7b7d265740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56029beea760 0x56029bef2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.583+0000 7f7b7d265740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029beeb080 0x56029bef2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.583+0000 7f7b7d265740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029bef5940 0x56029bef3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.583+0000 7f7b7d265740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56029bef8510 con 0x56029beea760 2024-09-12T03:22:57.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.583+0000 7f7b7d265740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x56029bdd86e0 con 0x56029bef5940 2024-09-12T03:22:57.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.583+0000 7f7b73fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029bef5940 0x56029bef3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.584+0000 7f7b73fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029bef5940 0x56029bef3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:53616/0 (socket says 172.21.15.6:53616) 2024-09-12T03:22:57.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.584+0000 7f7b73fff640 1 -- 172.21.15.6:0/2007808939 learned_addr learned my addr 172.21.15.6:0/2007808939 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:57.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.584+0000 7f7b7d265740 1 -- 172.21.15.6:0/2007808939 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56029bef8690 con 0x56029beeb080 2024-09-12T03:22:57.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.584+0000 7f7b7d265740 10 client.0 _ll_register_callbacks cb 0x56029bdfca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:57.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.584+0000 7f7b78c28640 1 --2- 172.21.15.6:0/2007808939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029beeb080 0x56029bef2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.584+0000 7f7b73fff640 1 -- 172.21.15.6:0/2007808939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029beeb080 msgr2=0x56029bef2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:57.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.584+0000 7f7b73fff640 1 --2- 172.21.15.6:0/2007808939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029beeb080 0x56029bef2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.584+0000 7f7b73fff640 1 -- 172.21.15.6:0/2007808939 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56029beea760 msgr2=0x56029bef2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:57.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.584+0000 7f7b73fff640 1 --2- 172.21.15.6:0/2007808939 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56029beea760 0x56029bef2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.584+0000 7f7b73fff640 1 -- 172.21.15.6:0/2007808939 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56029bf01a90 con 0x56029bef5940 2024-09-12T03:22:57.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.585+0000 7f7b73fff640 1 --2- 172.21.15.6:0/2007808939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029bef5940 0x56029bef3290 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0097a0 tx=0x7f7b6c008f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ab1eea1848d6548 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:57.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.585+0000 7f7b53fff640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:22:57.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.585+0000 7f7b53fff640 1 -- 172.21.15.6:0/2007808939 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c027020 con 0x56029bef5940 2024-09-12T03:22:57.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.585+0000 7f7b7d265740 1 -- 172.21.15.6:0/2007808939 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b6c008d50 con 0x56029bef5940 2024-09-12T03:22:57.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.586+0000 7f7b53fff640 1 -- 172.21.15.6:0/2007808939 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7b6c008b50 con 0x56029bef5940 2024-09-12T03:22:57.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.586+0000 7f7b53fff640 1 -- 172.21.15.6:0/2007808939 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7b6c01a5e0 con 0x56029bef5940 2024-09-12T03:22:57.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.586+0000 7f7b7d265740 10 client.25145 fetch_fsmap learned FSMap version 26 2024-09-12T03:22:57.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.586+0000 7f7b7d265740 1 -- 172.21.15.6:0/2007808939 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56029bf043e0 con 0x56029bef5940 2024-09-12T03:22:57.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.586+0000 7f7b53fff640 1 -- 172.21.15.6:0/2007808939 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b6c008d50 con 0x56029bef5940 2024-09-12T03:22:57.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.587+0000 7f7b53fff640 1 -- 172.21.15.6:0/2007808939 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7b6c01bce0 con 0x56029bef5940 2024-09-12T03:22:57.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.587+0000 7f7b7d265740 10 client.25145 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:22:57.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.587+0000 7f7b7d265740 10 client.25145 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:22:57.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.587+0000 7f7b7d265740 20 client.25145 populate_metadata read hostname 'smithi006' 2024-09-12T03:22:57.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.587+0000 7f7b7d265740 1 --2- 172.21.15.6:0/2007808939 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56029bf05bb0 0x56029bf08010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.587+0000 7f7b7d265740 4 client.25145 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:22:57.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.587+0000 7f7b7d265740 1 -- 172.21.15.6:0/2007808939 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56029be5d9d0 con 0x56029bf05bb0 2024-09-12T03:22:57.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.588+0000 7f7b78c28640 1 --2- 172.21.15.6:0/2007808939 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56029bf05bb0 0x56029bf08010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.590+0000 7f7b78c28640 1 --2- 172.21.15.6:0/2007808939 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56029bf05bb0 0x56029bf08010 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f423bc9f5bddf12b server_cookie=6e57b340c6158b92 in_seq=0 out_seq=0 2024-09-12T03:22:57.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.590+0000 7f7b53fff640 10 client.25145 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:57.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.592+0000 7f7b53fff640 1 -- 172.21.15.6:0/2007808939 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56029be5d9d0 con 0x56029bf05bb0 2024-09-12T03:22:57.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.592+0000 7f7b53fff640 10 client.25145 handle_command_reply: tid=0 2024-09-12T03:22:57.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.592+0000 7f7b7d265740 1 client.25145 shutdown 2024-09-12T03:22:57.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.592+0000 7f7b7d265740 1 -- 172.21.15.6:0/2007808939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029bef5940 msgr2=0x56029bef3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:57.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.593+0000 7f7b7d265740 1 --2- 172.21.15.6:0/2007808939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029bef5940 0x56029bef3290 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0097a0 tx=0x7f7b6c008f00 comp rx=0 tx=0).stop 2024-09-12T03:22:57.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.593+0000 7f7b7d265740 1 -- 172.21.15.6:0/2007808939 shutdown_connections 2024-09-12T03:22:57.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.593+0000 7f7b7d265740 1 --2- 172.21.15.6:0/2007808939 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56029bf05bb0 0x56029bf08010 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.593+0000 7f7b7d265740 1 --2- 172.21.15.6:0/2007808939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029bef5940 0x56029bef3290 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.593+0000 7f7b7d265740 1 --2- 172.21.15.6:0/2007808939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029beeb080 0x56029bef2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.593+0000 7f7b7d265740 1 --2- 172.21.15.6:0/2007808939 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56029beea760 0x56029bef2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.593+0000 7f7b7d265740 1 -- 172.21.15.6:0/2007808939 >> 172.21.15.6:0/2007808939 conn(0x56029bee6cf0 msgr2=0x56029bee5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:57.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.593+0000 7f7b53fff640 0 client.25145 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:57.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.593+0000 7f7b7d265740 1 -- 172.21.15.6:0/2007808939 shutdown_connections 2024-09-12T03:22:57.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.593+0000 7f7b7d265740 1 -- 172.21.15.6:0/2007808939 wait complete. 2024-09-12T03:22:57.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.593+0000 7f7b7d265740 20 client.25145 trim_cache size 0 max 16384 2024-09-12T03:22:57.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.634+0000 7f7b7d265740 1 Processor -- start 2024-09-12T03:22:57.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.634+0000 7f7b7d265740 1 -- start start 2024-09-12T03:22:57.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.634+0000 7f7b7d265740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029beea760 0x56029bef30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.635+0000 7f7b7d265740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029beeb080 0x56029bf0d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.635+0000 7f7b7d265740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56029bef5940 0x56029bf0de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.635+0000 7f7b73fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029beea760 0x56029bef30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.635+0000 7f7b73fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029beea760 0x56029bef30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:53624/0 (socket says 172.21.15.6:53624) 2024-09-12T03:22:57.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.635+0000 7f7b73fff640 1 -- 172.21.15.6:0/4278532280 learned_addr learned my addr 172.21.15.6:0/4278532280 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:57.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.635+0000 7f7b7d265740 1 -- 172.21.15.6:0/4278532280 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7b60010cf0 con 0x56029bef5940 2024-09-12T03:22:57.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.635+0000 7f7b78c28640 1 --2- 172.21.15.6:0/4278532280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029beeb080 0x56029bf0d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.635+0000 7f7b7d265740 1 -- 172.21.15.6:0/4278532280 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6000e5c0 con 0x56029beea760 2024-09-12T03:22:57.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.635+0000 7f7b7d265740 1 -- 172.21.15.6:0/4278532280 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6000c410 con 0x56029beeb080 2024-09-12T03:22:57.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.635+0000 7f7b73fff640 1 -- 172.21.15.6:0/4278532280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029beeb080 msgr2=0x56029bf0d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:57.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.635+0000 7f7b73fff640 1 --2- 172.21.15.6:0/4278532280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029beeb080 0x56029bf0d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.635+0000 7f7b73fff640 1 -- 172.21.15.6:0/4278532280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56029bef5940 msgr2=0x56029bf0de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:57.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.635+0000 7f7b73fff640 1 --2- 172.21.15.6:0/4278532280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56029bef5940 0x56029bf0de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.635+0000 7f7b73fff640 1 -- 172.21.15.6:0/4278532280 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56029beeaeb0 con 0x56029beea760 2024-09-12T03:22:57.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.636+0000 7f7b73fff640 1 --2- 172.21.15.6:0/4278532280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029beea760 0x56029bef30e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c02ea10 tx=0x7f7b6c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=502d81b276872df0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:57.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.638+0000 7f7b5a7fc640 1 -- 172.21.15.6:0/4278532280 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c027040 con 0x56029beea760 2024-09-12T03:22:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.639+0000 7f7b5a7fc640 1 -- 172.21.15.6:0/4278532280 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b6c007000 con 0x56029beea760 2024-09-12T03:22:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.640+0000 7f7b7d265740 1 -- 172.21.15.6:0/4278532280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029beea760 msgr2=0x56029bef30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.640+0000 7f7b7d265740 1 --2- 172.21.15.6:0/4278532280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029beea760 0x56029bef30e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c02ea10 tx=0x7f7b6c004a40 comp rx=0 tx=0).stop 2024-09-12T03:22:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.640+0000 7f7b7d265740 1 -- 172.21.15.6:0/4278532280 shutdown_connections 2024-09-12T03:22:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.640+0000 7f7b7d265740 1 --2- 172.21.15.6:0/4278532280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56029bef5940 0x56029bf0de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.640+0000 7f7b7d265740 1 --2- 172.21.15.6:0/4278532280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029beeb080 0x56029bf0d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.640+0000 7f7b7d265740 1 --2- 172.21.15.6:0/4278532280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029beea760 0x56029bef30e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.640+0000 7f7b7d265740 1 -- 172.21.15.6:0/4278532280 >> 172.21.15.6:0/4278532280 conn(0x56029bee6cf0 msgr2=0x56029bef4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.640+0000 7f7b7d265740 1 -- 172.21.15.6:0/4278532280 shutdown_connections 2024-09-12T03:22:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.640+0000 7f7b7d265740 1 -- 172.21.15.6:0/4278532280 wait complete. 2024-09-12T03:22:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.641+0000 7f7b7d265740 1 -- start start 2024-09-12T03:22:57.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.641+0000 7f7b7d265740 1 Processor -- start 2024-09-12T03:22:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.641+0000 7f7b7d265740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029beeb080 0x56029bef9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.641+0000 7f7b7d265740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56029bef5940 0x56029bef95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.641+0000 7f7b7d265740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029bf68920 0x56029bf68da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.641+0000 7f7b7d265740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6000ca80 con 0x56029bef5940 2024-09-12T03:22:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.641+0000 7f7b7d265740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6000def0 con 0x56029beeb080 2024-09-12T03:22:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.641+0000 7f7b7d265740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6000d150 con 0x56029bf68920 2024-09-12T03:22:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.641+0000 7f7b7d265740 10 client.0 _ll_register_callbacks cb 0x7f7b6c0096d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.642+0000 7f7b73fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029bf68920 0x56029bf68da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.642+0000 7f7b73fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029bf68920 0x56029bf68da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:56318/0 (socket says 172.21.15.6:56318) 2024-09-12T03:22:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.642+0000 7f7b73fff640 1 -- 172.21.15.6:0/4142128384 learned_addr learned my addr 172.21.15.6:0/4142128384 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:22:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.642+0000 7f7b78c28640 1 --2- 172.21.15.6:0/4142128384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56029bef5940 0x56029bef95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.643+0000 7f7b73fff640 1 --2- 172.21.15.6:0/4142128384 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029beeb080 0x56029bef9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.643+0000 7f7b73fff640 1 -- 172.21.15.6:0/4142128384 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029beeb080 msgr2=0x56029bef9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:57.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.643+0000 7f7b73fff640 1 --2- 172.21.15.6:0/4142128384 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029beeb080 0x56029bef9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.643+0000 7f7b73fff640 1 -- 172.21.15.6:0/4142128384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56029bef5940 msgr2=0x56029bef95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:57.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.643+0000 7f7b73fff640 1 --2- 172.21.15.6:0/4142128384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56029bef5940 0x56029bef95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.643+0000 7f7b73fff640 1 -- 172.21.15.6:0/4142128384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56029bf69390 con 0x56029bf68920 2024-09-12T03:22:57.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.643+0000 7f7b73fff640 1 --2- 172.21.15.6:0/4142128384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029bf68920 0x56029bf68da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c01ae90 tx=0x7f7b6c01ae20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e857ce1817dce3b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:57.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.644+0000 7f7b53fff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:22:57.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.646+0000 7f7b53fff640 1 -- 172.21.15.6:0/4142128384 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c027030 con 0x56029bf68920 2024-09-12T03:22:57.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.646+0000 7f7b53fff640 1 -- 172.21.15.6:0/4142128384 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7b6c02e810 con 0x56029bf68920 2024-09-12T03:22:57.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.646+0000 7f7b7d265740 1 -- 172.21.15.6:0/4142128384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56029be57d20 con 0x56029bf68920 2024-09-12T03:22:57.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.649+0000 7f7b53fff640 1 -- 172.21.15.6:0/4142128384 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7b6c03c040 con 0x56029bf68920 2024-09-12T03:22:57.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.649+0000 7f7b7d265740 10 client.25201 fetch_fsmap learned FSMap version 26 2024-09-12T03:22:57.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.649+0000 7f7b7d265740 1 -- 172.21.15.6:0/4142128384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56029bf70780 con 0x56029bf68920 2024-09-12T03:22:57.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.650+0000 7f7b53fff640 1 -- 172.21.15.6:0/4142128384 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56029be57d20 con 0x56029bf68920 2024-09-12T03:22:57.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.651+0000 7f7b53fff640 1 -- 172.21.15.6:0/4142128384 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7b6c037640 con 0x56029bf68920 2024-09-12T03:22:57.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.651+0000 7f7b7d265740 10 client.25201 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:22:57.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.651+0000 7f7b7d265740 10 client.25201 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:22:57.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.651+0000 7f7b7d265740 20 client.25201 populate_metadata read hostname 'smithi006' 2024-09-12T03:22:57.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.652+0000 7f7b7d265740 1 --2- 172.21.15.6:0/4142128384 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56029bf70c60 0x56029bf73050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.652+0000 7f7b7d265740 4 client.25201 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}] 2024-09-12T03:22:57.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.652+0000 7f7b7d265740 1 -- 172.21.15.6:0/4142128384 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}) -- 0x56029bdd3ec0 con 0x56029bf70c60 2024-09-12T03:22:57.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.652+0000 7f7b78c28640 1 --2- 172.21.15.6:0/4142128384 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56029bf70c60 0x56029bf73050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.653+0000 7f7b78c28640 1 --2- 172.21.15.6:0/4142128384 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56029bf70c60 0x56029bf73050 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d06cd66df133ac14 server_cookie=4d0d149e775666f3 in_seq=0 out_seq=0 2024-09-12T03:22:57.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:57.653+0000 7f7b53fff640 10 client.25201 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.566+0000 7f7b53fff640 1 -- 172.21.15.6:0/4142128384 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56029bdd3ec0 con 0x56029bf70c60 2024-09-12T03:22:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.566+0000 7f7b53fff640 10 client.25201 handle_command_reply: tid=0 2024-09-12T03:22:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.566+0000 7f7b7d265740 1 client.25201 shutdown 2024-09-12T03:22:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.567+0000 7f7b7d265740 1 -- 172.21.15.6:0/4142128384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029bf68920 msgr2=0x56029bf68da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.567+0000 7f7b7d265740 1 --2- 172.21.15.6:0/4142128384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029bf68920 0x56029bf68da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c01ae90 tx=0x7f7b6c01ae20 comp rx=0 tx=0).stop 2024-09-12T03:22:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.567+0000 7f7b7d265740 1 -- 172.21.15.6:0/4142128384 shutdown_connections 2024-09-12T03:22:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.567+0000 7f7b7d265740 1 --2- 172.21.15.6:0/4142128384 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56029bf70c60 0x56029bf73050 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.567+0000 7f7b7d265740 1 --2- 172.21.15.6:0/4142128384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56029bf68920 0x56029bf68da0 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c01ae90 tx=0x7f7b6c01ae20 comp rx=0 tx=0).stop 2024-09-12T03:22:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.567+0000 7f7b7d265740 1 --2- 172.21.15.6:0/4142128384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56029bef5940 0x56029bef95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.567+0000 7f7b7d265740 1 --2- 172.21.15.6:0/4142128384 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56029beeb080 0x56029bef9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.567+0000 7f7b7d265740 1 -- 172.21.15.6:0/4142128384 >> 172.21.15.6:0/4142128384 conn(0x56029bee6cf0 msgr2=0x56029beec1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:58.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.567+0000 7f7b53fff640 0 client.25201 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:22:58.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.567+0000 7f7b7d265740 1 -- 172.21.15.6:0/4142128384 shutdown_connections 2024-09-12T03:22:58.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.568+0000 7f7b7d265740 1 -- 172.21.15.6:0/4142128384 wait complete. 2024-09-12T03:22:58.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.568+0000 7f7b7d265740 20 client.25201 trim_cache size 0 max 16384 2024-09-12T03:22:58.569 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:22:58.569 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 26, 2024-09-12T03:22:58.569 INFO:teuthology.orchestra.run.smithi006.stdout: "leader": 24494, 2024-09-12T03:22:58.569 INFO:teuthology.orchestra.run.smithi006.stdout: "set_version": 6, 2024-09-12T03:22:58.569 INFO:teuthology.orchestra.run.smithi006.stdout: "sets": { 2024-09-12T03:22:58.569 INFO:teuthology.orchestra.run.smithi006.stdout: "4008eb8d": { 2024-09-12T03:22:58.570 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 6, 2024-09-12T03:22:58.570 INFO:teuthology.orchestra.run.smithi006.stdout: "age_ref": 0.0, 2024-09-12T03:22:58.570 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:22:58.570 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "QUIESCED", 2024-09-12T03:22:58.570 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:22:58.570 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:22:58.570 INFO:teuthology.orchestra.run.smithi006.stdout: "timeout": 90.0, 2024-09-12T03:22:58.570 INFO:teuthology.orchestra.run.smithi006.stdout: "expiration": 133.4, 2024-09-12T03:22:58.570 INFO:teuthology.orchestra.run.smithi006.stdout: "members": { 2024-09-12T03:22:58.570 INFO:teuthology.orchestra.run.smithi006.stdout: "file:/": { 2024-09-12T03:22:58.570 INFO:teuthology.orchestra.run.smithi006.stdout: "excluded": false, 2024-09-12T03:22:58.570 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:22:58.570 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "QUIESCED", 2024-09-12T03:22:58.570 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:22:58.570 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:22:58.571 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:22:58.571 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:22:58.571 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:22:58.571 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:22:58.571 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:22:58.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.570+0000 7f7b7ae8c640 1 -- 172.21.15.6:0/1362060025 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7b5c0a2c40 msgr2=0x7f7b5c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:58.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.570+0000 7f7b7ae8c640 1 --2- 172.21.15.6:0/1362060025 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7b5c0a2c40 0x7f7b5c0a5100 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7b6402a450 tx=0x7f7b64005fd0 comp rx=0 tx=0).stop 2024-09-12T03:22:58.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.570+0000 7f7b7ae8c640 1 -- 172.21.15.6:0/1362060025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b7410d690 msgr2=0x7f7b74119c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:58.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.570+0000 7f7b7ae8c640 1 --2- 172.21.15.6:0/1362060025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b7410d690 0x7f7b74119c10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7b6402aa00 tx=0x7f7b64005320 comp rx=0 tx=0).stop 2024-09-12T03:22:58.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.570+0000 7f7b7ae8c640 1 -- 172.21.15.6:0/1362060025 shutdown_connections 2024-09-12T03:22:58.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.570+0000 7f7b7ae8c640 1 --2- 172.21.15.6:0/1362060025 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7b5c0a2c40 0x7f7b5c0a5100 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:58.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.570+0000 7f7b7ae8c640 1 --2- 172.21.15.6:0/1362060025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b7410d690 0x7f7b74119c10 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:58.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.570+0000 7f7b7ae8c640 1 --2- 172.21.15.6:0/1362060025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b74073ca0 0x7f7b7411efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:58.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.570+0000 7f7b7ae8c640 1 --2- 172.21.15.6:0/1362060025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b74073830 0x7f7b7411eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:58.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.570+0000 7f7b7ae8c640 1 -- 172.21.15.6:0/1362060025 >> 172.21.15.6:0/1362060025 conn(0x7f7b7406e140 msgr2=0x7f7b7410c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:58.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.570+0000 7f7b7ae8c640 1 -- 172.21.15.6:0/1362060025 shutdown_connections 2024-09-12T03:22:58.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:22:58.570+0000 7f7b7ae8c640 1 -- 172.21.15.6:0/1362060025 wait complete. 2024-09-12T03:22:59.245 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:22:59 smithi006 ceph-mon[30991]: pgmap v391: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 384 MiB/s wr, 20.68k op/s 2024-09-12T03:22:59.273 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4008eb8d', quiesce took 9.1 seconds. Will release after: 4.300000000000001 2024-09-12T03:22:59.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:22:59 smithi161 ceph-mon[35485]: pgmap v391: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 384 MiB/s wr, 20.68k op/s 2024-09-12T03:22:59.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:22:59 smithi118 ceph-mon[35048]: pgmap v391: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 384 MiB/s wr, 20.68k op/s 2024-09-12T03:23:01.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:01 smithi006 ceph-mon[30991]: pgmap v392: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 319 MiB/s wr, 26.44k op/s 2024-09-12T03:23:01.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:01 smithi161 ceph-mon[35485]: pgmap v392: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 319 MiB/s wr, 26.44k op/s 2024-09-12T03:23:01.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:01 smithi118 ceph-mon[35048]: pgmap v392: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 319 MiB/s wr, 26.44k op/s 2024-09-12T03:23:03.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:03 smithi006 ceph-mon[30991]: pgmap v393: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 220 MiB/s wr, 26.96k op/s 2024-09-12T03:23:03.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:03 smithi161 ceph-mon[35485]: pgmap v393: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 220 MiB/s wr, 26.96k op/s 2024-09-12T03:23:03.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:03 smithi118 ceph-mon[35048]: pgmap v393: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 220 MiB/s wr, 26.96k op/s 2024-09-12T03:23:03.574 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:23:04.084 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:04.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.643+0000 7f5e11610640 1 -- 172.21.15.6:0/759855222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e0c102ef0 msgr2=0x7f5e0c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:04.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.643+0000 7f5e11610640 1 --2- 172.21.15.6:0/759855222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e0c102ef0 0x7f5e0c10da40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5e04009ca0 tx=0x7f5e0402a950 comp rx=0 tx=0).stop 2024-09-12T03:23:04.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.643+0000 7f5e11610640 1 -- 172.21.15.6:0/759855222 shutdown_connections 2024-09-12T03:23:04.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.643+0000 7f5e11610640 1 --2- 172.21.15.6:0/759855222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e0c102ef0 0x7f5e0c10da40 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:04.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.643+0000 7f5e11610640 1 --2- 172.21.15.6:0/759855222 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e0c102570 0x7f5e0c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:04.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.643+0000 7f5e11610640 1 --2- 172.21.15.6:0/759855222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e0c100e20 0x7f5e0c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:04.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.643+0000 7f5e11610640 1 -- 172.21.15.6:0/759855222 >> 172.21.15.6:0/759855222 conn(0x7f5e0c0fc7d0 msgr2=0x7f5e0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:04.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.643+0000 7f5e11610640 1 -- 172.21.15.6:0/759855222 shutdown_connections 2024-09-12T03:23:04.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.643+0000 7f5e11610640 1 -- 172.21.15.6:0/759855222 wait complete. 2024-09-12T03:23:04.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.644+0000 7f5e11610640 1 Processor -- start 2024-09-12T03:23:04.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.644+0000 7f5e11610640 1 -- start start 2024-09-12T03:23:04.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.644+0000 7f5e11610640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e0c100e20 0x7f5e0c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:04.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.644+0000 7f5e11610640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e0c102570 0x7f5e0c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:04.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.644+0000 7f5e11610640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e0c102ef0 0x7f5e0c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:04.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.644+0000 7f5e11610640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c110f80 con 0x7f5e0c102570 2024-09-12T03:23:04.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.644+0000 7f5e11610640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c110e00 con 0x7f5e0c100e20 2024-09-12T03:23:04.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.644+0000 7f5e11610640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c111100 con 0x7f5e0c102ef0 2024-09-12T03:23:04.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.644+0000 7f5e0a7fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e0c102570 0x7f5e0c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:04.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.644+0000 7f5e0a7fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e0c102570 0x7f5e0c1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49882/0 (socket says 172.21.15.6:49882) 2024-09-12T03:23:04.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.644+0000 7f5e0a7fc640 1 -- 172.21.15.6:0/2330627892 learned_addr learned my addr 172.21.15.6:0/2330627892 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:04.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.645+0000 7f5e0b7fe640 1 --2- 172.21.15.6:0/2330627892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e0c102ef0 0x7f5e0c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:04.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.645+0000 7f5e0affd640 1 --2- 172.21.15.6:0/2330627892 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e0c100e20 0x7f5e0c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:04.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.645+0000 7f5e0a7fc640 1 -- 172.21.15.6:0/2330627892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e0c102ef0 msgr2=0x7f5e0c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:04.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.645+0000 7f5e0a7fc640 1 --2- 172.21.15.6:0/2330627892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e0c102ef0 0x7f5e0c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:04.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.645+0000 7f5e0a7fc640 1 -- 172.21.15.6:0/2330627892 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e0c100e20 msgr2=0x7f5e0c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:04.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.645+0000 7f5e0a7fc640 1 --2- 172.21.15.6:0/2330627892 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e0c100e20 0x7f5e0c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:04.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.645+0000 7f5e0a7fc640 1 -- 172.21.15.6:0/2330627892 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c19d420 con 0x7f5e0c102570 2024-09-12T03:23:04.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.645+0000 7f5e0a7fc640 1 --2- 172.21.15.6:0/2330627892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e0c102570 0x7f5e0c1989b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000d950 tx=0x7f5e0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:04.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.645+0000 7f5df3fff640 1 -- 172.21.15.6:0/2330627892 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00017030 con 0x7f5e0c102570 2024-09-12T03:23:04.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.645+0000 7f5e11610640 1 -- 172.21.15.6:0/2330627892 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0c19d710 con 0x7f5e0c102570 2024-09-12T03:23:04.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.645+0000 7f5df3fff640 1 -- 172.21.15.6:0/2330627892 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e00004020 con 0x7f5e0c102570 2024-09-12T03:23:04.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.645+0000 7f5df3fff640 1 -- 172.21.15.6:0/2330627892 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e000047a0 con 0x7f5e0c102570 2024-09-12T03:23:04.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.645+0000 7f5e11610640 1 -- 172.21.15.6:0/2330627892 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0c1a4ff0 con 0x7f5e0c102570 2024-09-12T03:23:04.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.646+0000 7f5e11610640 1 -- 172.21.15.6:0/2330627892 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e0c101200 con 0x7f5e0c102570 2024-09-12T03:23:04.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.647+0000 7f5df3fff640 1 -- 172.21.15.6:0/2330627892 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5e000041d0 con 0x7f5e0c102570 2024-09-12T03:23:04.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.648+0000 7f5df3fff640 1 --2- 172.21.15.6:0/2330627892 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5df40a2b80 0x7f5df40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:04.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.648+0000 7f5df3fff640 1 -- 172.21.15.6:0/2330627892 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f5e000ce940 con 0x7f5e0c102570 2024-09-12T03:23:04.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.648+0000 7f5e0affd640 1 --2- 172.21.15.6:0/2330627892 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5df40a2b80 0x7f5df40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:04.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.649+0000 7f5e0affd640 1 --2- 172.21.15.6:0/2330627892 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5df40a2b80 0x7f5df40a5040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00bbe0 tx=0x7f5dfc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:04.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.653+0000 7f5df3fff640 1 -- 172.21.15.6:0/2330627892 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e0009b8d0 con 0x7f5e0c102570 2024-09-12T03:23:04.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.864+0000 7f5e11610640 1 -- 172.21.15.6:0/2330627892 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e0c10a0a0 con 0x7f5e0c102570 2024-09-12T03:23:04.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.865+0000 7f5df3fff640 1 -- 172.21.15.6:0/2330627892 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11911 (secure 0 0 0) 0x7f5e0009b1d0 con 0x7f5e0c102570 2024-09-12T03:23:04.866 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:23:04.866 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":26,"btime":"2024-09-12T03:22:16:132848+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:22:16.064679+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24463,24464,24494]},"id":1}]} 2024-09-12T03:23:04.866 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 26 2024-09-12T03:23:04.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.867+0000 7f5e11610640 1 -- 172.21.15.6:0/2330627892 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5df40a2b80 msgr2=0x7f5df40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:04.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.867+0000 7f5e11610640 1 --2- 172.21.15.6:0/2330627892 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5df40a2b80 0x7f5df40a5040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00bbe0 tx=0x7f5dfc00b390 comp rx=0 tx=0).stop 2024-09-12T03:23:04.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.867+0000 7f5e11610640 1 -- 172.21.15.6:0/2330627892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e0c102570 msgr2=0x7f5e0c1989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:04.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.867+0000 7f5e11610640 1 --2- 172.21.15.6:0/2330627892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e0c102570 0x7f5e0c1989b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000d950 tx=0x7f5e0000de10 comp rx=0 tx=0).stop 2024-09-12T03:23:04.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.867+0000 7f5e11610640 1 -- 172.21.15.6:0/2330627892 shutdown_connections 2024-09-12T03:23:04.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.867+0000 7f5e11610640 1 --2- 172.21.15.6:0/2330627892 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5df40a2b80 0x7f5df40a5040 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:04.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.867+0000 7f5e11610640 1 --2- 172.21.15.6:0/2330627892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e0c102ef0 0x7f5e0c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:04.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.867+0000 7f5e11610640 1 --2- 172.21.15.6:0/2330627892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e0c102570 0x7f5e0c1989b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:04.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.867+0000 7f5e11610640 1 --2- 172.21.15.6:0/2330627892 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e0c100e20 0x7f5e0c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:04.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.867+0000 7f5e11610640 1 -- 172.21.15.6:0/2330627892 >> 172.21.15.6:0/2330627892 conn(0x7f5e0c0fc7d0 msgr2=0x7f5e0c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:04.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.868+0000 7f5e11610640 1 -- 172.21.15.6:0/2330627892 shutdown_connections 2024-09-12T03:23:04.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:04.868+0000 7f5e11610640 1 -- 172.21.15.6:0/2330627892 wait complete. 2024-09-12T03:23:05.252 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:05 smithi006 ceph-mon[30991]: pgmap v394: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 193 MiB/s wr, 23.17k op/s 2024-09-12T03:23:05.252 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:05 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2330627892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:05.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:05 smithi161 ceph-mon[35485]: pgmap v394: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 193 MiB/s wr, 23.17k op/s 2024-09-12T03:23:05.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:05 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2330627892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:05.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:05 smithi118 ceph-mon[35048]: pgmap v394: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 193 MiB/s wr, 23.17k op/s 2024-09-12T03:23:05.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:05 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2330627892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:05.541 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 4008eb8d --release --await' 2024-09-12T03:23:05.541 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 4008eb8d --release --await 2024-09-12T03:23:05.797 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:07.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.639+0000 7f8f8b808640 1 -- 172.21.15.6:0/3153540394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f8411b140 msgr2=0x7f8f8411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.639+0000 7f8f8b808640 1 --2- 172.21.15.6:0/3153540394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f8411b140 0x7f8f8411d530 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c01acb0 tx=0x7f8f7c03b920 comp rx=0 tx=0).stop 2024-09-12T03:23:07.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.639+0000 7f8f8b808640 1 -- 172.21.15.6:0/3153540394 shutdown_connections 2024-09-12T03:23:07.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.639+0000 7f8f8b808640 1 --2- 172.21.15.6:0/3153540394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f8411b140 0x7f8f8411d530 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.639+0000 7f8f8b808640 1 --2- 172.21.15.6:0/3153540394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f84109830 0x7f8f84109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.639+0000 7f8f8b808640 1 --2- 172.21.15.6:0/3153540394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f84109450 0x7f8f84110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.639+0000 7f8f8b808640 1 -- 172.21.15.6:0/3153540394 >> 172.21.15.6:0/3153540394 conn(0x7f8f8406e140 msgr2=0x7f8f8406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:07.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.640+0000 7f8f8b808640 1 -- 172.21.15.6:0/3153540394 shutdown_connections 2024-09-12T03:23:07.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.640+0000 7f8f8b808640 1 -- 172.21.15.6:0/3153540394 wait complete. 2024-09-12T03:23:07.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.641+0000 7f8f8b808640 1 Processor -- start 2024-09-12T03:23:07.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.641+0000 7f8f8b808640 1 -- start start 2024-09-12T03:23:07.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.641+0000 7f8f8b808640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f84109450 0x7f8f8411a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.641+0000 7f8f8b808640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f84109830 0x7f8f84116880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.641+0000 7f8f8b808640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f8411b140 0x7f8f84116dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.641+0000 7f8f8b808640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8f841132d0 con 0x7f8f84109830 2024-09-12T03:23:07.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.641+0000 7f8f8b808640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f8f84113150 con 0x7f8f84109450 2024-09-12T03:23:07.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.641+0000 7f8f8b808640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8f84113450 con 0x7f8f8411b140 2024-09-12T03:23:07.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.641+0000 7f8f895a4640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f84109450 0x7f8f8411a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.641+0000 7f8f89da5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f8411b140 0x7f8f84116dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.641+0000 7f8f895a4640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f84109450 0x7f8f8411a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:50214/0 (socket says 172.21.15.6:50214) 2024-09-12T03:23:07.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.641+0000 7f8f895a4640 1 -- 172.21.15.6:0/325909606 learned_addr learned my addr 172.21.15.6:0/325909606 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:07.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.642+0000 7f8f88da3640 1 --2- 172.21.15.6:0/325909606 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f84109830 0x7f8f84116880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.642+0000 7f8f89da5640 1 -- 172.21.15.6:0/325909606 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f84109450 msgr2=0x7f8f8411a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.642+0000 7f8f89da5640 1 --2- 172.21.15.6:0/325909606 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f84109450 0x7f8f8411a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.642+0000 7f8f89da5640 1 -- 172.21.15.6:0/325909606 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f84109830 msgr2=0x7f8f84116880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.642+0000 7f8f89da5640 1 --2- 172.21.15.6:0/325909606 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f84109830 0x7f8f84116880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.642+0000 7f8f89da5640 1 -- 172.21.15.6:0/325909606 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f841174f0 con 0x7f8f8411b140 2024-09-12T03:23:07.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.643+0000 7f8f89da5640 1 --2- 172.21.15.6:0/325909606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f8411b140 0x7f8f84116dc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c01ac00 tx=0x7f8f7c040a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:07.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.643+0000 7f8f7a7fc640 1 -- 172.21.15.6:0/325909606 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c045070 con 0x7f8f8411b140 2024-09-12T03:23:07.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.643+0000 7f8f8b808640 1 -- 172.21.15.6:0/325909606 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f841ab170 con 0x7f8f8411b140 2024-09-12T03:23:07.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.643+0000 7f8f8b808640 1 -- 172.21.15.6:0/325909606 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f841ab510 con 0x7f8f8411b140 2024-09-12T03:23:07.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.643+0000 7f8f7a7fc640 1 -- 172.21.15.6:0/325909606 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f7c040b50 con 0x7f8f8411b140 2024-09-12T03:23:07.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.644+0000 7f8f7a7fc640 1 -- 172.21.15.6:0/325909606 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c04a690 con 0x7f8f8411b140 2024-09-12T03:23:07.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.647+0000 7f8f7a7fc640 1 -- 172.21.15.6:0/325909606 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8f7c048020 con 0x7f8f8411b140 2024-09-12T03:23:07.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.648+0000 7f8f8dbe1740 1 Processor -- start 2024-09-12T03:23:07.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.648+0000 7f8f8dbe1740 1 -- start start 2024-09-12T03:23:07.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.648+0000 7f8f8dbe1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b66060 0x55a274b66440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.648+0000 7f8f8dbe1740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a274b66980 0x55a274b70d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.648+0000 7f8f8dbe1740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a274b71240 0x55a274b73630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.648+0000 7f8f8dbe1740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a274acb620 con 0x55a274b66980 2024-09-12T03:23:07.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.648+0000 7f8f8dbe1740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55a274a72e50 con 0x55a274b71240 2024-09-12T03:23:07.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.648+0000 7f8f8dbe1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a274ac75e0 con 0x55a274b66060 2024-09-12T03:23:07.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.648+0000 7f8f7a7fc640 1 --2- 172.21.15.6:0/325909606 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8f700a2cc0 0x7f8f700a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.648+0000 7f8f88da3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b66060 0x55a274b66440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.648+0000 7f8f88da3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b66060 0x55a274b66440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:45320/0 (socket says 172.21.15.6:45320) 2024-09-12T03:23:07.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.648+0000 7f8f88da3640 1 -- 172.21.15.6:0/1028416275 learned_addr learned my addr 172.21.15.6:0/1028416275 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:07.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.649+0000 7f8f895a4640 1 --2- 172.21.15.6:0/1028416275 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a274b66980 0x55a274b70d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.649+0000 7f8f89da5640 1 --2- 172.21.15.6:0/325909606 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8f700a2cc0 0x7f8f700a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.649+0000 7f8f7a7fc640 1 -- 172.21.15.6:0/325909606 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f8f7c101e60 con 0x7f8f8411b140 2024-09-12T03:23:07.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.649+0000 7f8f88da3640 1 -- 172.21.15.6:0/1028416275 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a274b71240 msgr2=0x55a274b73630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:07.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.649+0000 7f8f88da3640 1 --2- 172.21.15.6:0/1028416275 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a274b71240 0x55a274b73630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.649+0000 7f8f88da3640 1 -- 172.21.15.6:0/1028416275 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a274b66980 msgr2=0x55a274b70d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:07.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.649+0000 7f8f88da3640 1 --2- 172.21.15.6:0/1028416275 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a274b66980 0x55a274b70d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.649+0000 7f8f88da3640 1 -- 172.21.15.6:0/1028416275 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a274a63d40 con 0x55a274b66060 2024-09-12T03:23:07.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.649+0000 7f8f895a4640 1 --2- 172.21.15.6:0/1028416275 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a274b66980 0x55a274b70d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:07.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.649+0000 7f8f89da5640 1 --2- 172.21.15.6:0/325909606 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8f700a2cc0 0x7f8f700a5180 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c004af0 tx=0x7f8f7c03b920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:07.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.649+0000 7f8f88da3640 1 --2- 172.21.15.6:0/1028416275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b66060 0x55a274b66440 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009560 tx=0x7f8f74012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aba2d98df2694a4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:07.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.650+0000 7f8f6f7fe640 1 -- 172.21.15.6:0/1028416275 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7401b440 con 0x55a274b66060 2024-09-12T03:23:07.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.650+0000 7f8f6f7fe640 1 -- 172.21.15.6:0/1028416275 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f7400f910 con 0x55a274b66060 2024-09-12T03:23:07.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.650+0000 7f8f6f7fe640 1 -- 172.21.15.6:0/1028416275 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7401aae0 con 0x55a274b66060 2024-09-12T03:23:07.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.651+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/1028416275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b66060 msgr2=0x55a274b66440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.651+0000 7f8f8dbe1740 1 --2- 172.21.15.6:0/1028416275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b66060 0x55a274b66440 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009560 tx=0x7f8f74012820 comp rx=0 tx=0).stop 2024-09-12T03:23:07.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.652+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/1028416275 shutdown_connections 2024-09-12T03:23:07.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.652+0000 7f8f8dbe1740 1 --2- 172.21.15.6:0/1028416275 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a274b71240 0x55a274b73630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.652+0000 7f8f8dbe1740 1 --2- 172.21.15.6:0/1028416275 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a274b66980 0x55a274b70d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.652+0000 7f8f8dbe1740 1 --2- 172.21.15.6:0/1028416275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b66060 0x55a274b66440 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.652+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/1028416275 >> 172.21.15.6:0/1028416275 conn(0x55a274b625f0 msgr2=0x55a274b629d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:07.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.652+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/1028416275 shutdown_connections 2024-09-12T03:23:07.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.653+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/1028416275 wait complete. 2024-09-12T03:23:07.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.653+0000 7f8f8dbe1740 1 -- start start 2024-09-12T03:23:07.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.653+0000 7f8f8dbe1740 1 Processor -- start 2024-09-12T03:23:07.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.658+0000 7f8f8dbe1740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a274b66060 0x55a274b6e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.658+0000 7f8f8dbe1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b66980 0x55a274b6e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.658+0000 7f8f8dbe1740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a274b71240 0x55a274b6ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.658+0000 7f8f8dbe1740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a274b73e10 con 0x55a274b66060 2024-09-12T03:23:07.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.658+0000 7f8f8dbe1740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55a274a53f10 con 0x55a274b71240 2024-09-12T03:23:07.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.658+0000 7f8f8dbe1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a274b73f90 con 0x55a274b66980 2024-09-12T03:23:07.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.658+0000 7f8f88da3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a274b66060 0x55a274b6e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.658+0000 7f8f88da3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a274b66060 0x55a274b6e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49916/0 (socket says 172.21.15.6:49916) 2024-09-12T03:23:07.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.658+0000 7f8f88da3640 1 -- 172.21.15.6:0/46263244 learned_addr learned my addr 172.21.15.6:0/46263244 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:07.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.659+0000 7f8f895a4640 1 --2- 172.21.15.6:0/46263244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b66980 0x55a274b6e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.659+0000 7f8f88da3640 1 --2- 172.21.15.6:0/46263244 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a274b71240 0x55a274b6ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.659+0000 7f8f8dbe1740 10 client.0 _ll_register_callbacks cb 0x55a274a77db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:07.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.660+0000 7f8f88da3640 1 -- 172.21.15.6:0/46263244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b66980 msgr2=0x55a274b6e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:07.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.660+0000 7f8f88da3640 1 --2- 172.21.15.6:0/46263244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b66980 0x55a274b6e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.660+0000 7f8f88da3640 1 -- 172.21.15.6:0/46263244 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a274b66060 msgr2=0x55a274b6e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:07.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.660+0000 7f8f88da3640 1 --2- 172.21.15.6:0/46263244 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a274b66060 0x55a274b6e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.660+0000 7f8f88da3640 1 -- 172.21.15.6:0/46263244 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2747da580 con 0x55a274b71240 2024-09-12T03:23:07.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.660+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/46263244 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f7400f910 con 0x55a274b71240 2024-09-12T03:23:07.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.660+0000 7f8f895a4640 1 --2- 172.21.15.6:0/46263244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b66980 0x55a274b6e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:07.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.660+0000 7f8f88da3640 1 --2- 172.21.15.6:0/46263244 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a274b71240 0x55a274b6ec60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8f7401a7c0 tx=0x7f8f74009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97be3fe8b7b51b8c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:07.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.660+0000 7f8f6cff9640 10 client.25166 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:23:07.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.661+0000 7f8f6cff9640 1 -- 172.21.15.6:0/46263244 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7400f430 con 0x55a274b71240 2024-09-12T03:23:07.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.661+0000 7f8f6cff9640 1 -- 172.21.15.6:0/46263244 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f8f7400f5d0 con 0x55a274b71240 2024-09-12T03:23:07.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.661+0000 7f8f6cff9640 1 -- 172.21.15.6:0/46263244 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8f74004030 con 0x55a274b71240 2024-09-12T03:23:07.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.661+0000 7f8f8dbe1740 10 client.25166 fetch_fsmap learned FSMap version 26 2024-09-12T03:23:07.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.661+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/46263244 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55a2748a4960 con 0x55a274b71240 2024-09-12T03:23:07.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.662+0000 7f8f6cff9640 1 -- 172.21.15.6:0/46263244 <== mon.2 v2:172.21.15.118:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8f74023030 con 0x55a274b71240 2024-09-12T03:23:07.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.662+0000 7f8f8dbe1740 10 client.25166 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:23:07.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.662+0000 7f8f8dbe1740 10 client.25166 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:23:07.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.662+0000 7f8f8dbe1740 20 client.25166 populate_metadata read hostname 'smithi006' 2024-09-12T03:23:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.662+0000 7f8f8dbe1740 1 --2- 172.21.15.6:0/46263244 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a274b81250 0x55a274b836b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.662+0000 7f8f8dbe1740 4 client.25166 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:23:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.662+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/46263244 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a274ad8b90 con 0x55a274b81250 2024-09-12T03:23:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.663+0000 7f8f895a4640 1 --2- 172.21.15.6:0/46263244 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a274b81250 0x55a274b836b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.666+0000 7f8f895a4640 1 --2- 172.21.15.6:0/46263244 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a274b81250 0x55a274b836b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52dd727f486d2 server_cookie=42e63242a243d9f1 in_seq=0 out_seq=0 2024-09-12T03:23:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.666+0000 7f8f6cff9640 10 client.25166 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.668+0000 7f8f6cff9640 1 -- 172.21.15.6:0/46263244 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a274ad8b90 con 0x55a274b81250 2024-09-12T03:23:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.668+0000 7f8f6cff9640 10 client.25166 handle_command_reply: tid=0 2024-09-12T03:23:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.668+0000 7f8f8dbe1740 1 client.25166 shutdown 2024-09-12T03:23:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.669+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/46263244 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a274b71240 msgr2=0x55a274b6ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.669+0000 7f8f8dbe1740 1 --2- 172.21.15.6:0/46263244 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a274b71240 0x55a274b6ec60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8f7401a7c0 tx=0x7f8f74009590 comp rx=0 tx=0).stop 2024-09-12T03:23:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.669+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/46263244 shutdown_connections 2024-09-12T03:23:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.669+0000 7f8f8dbe1740 1 --2- 172.21.15.6:0/46263244 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a274b81250 0x55a274b836b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.669+0000 7f8f8dbe1740 1 --2- 172.21.15.6:0/46263244 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a274b71240 0x55a274b6ec60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.669+0000 7f8f8dbe1740 1 --2- 172.21.15.6:0/46263244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b66980 0x55a274b6e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.669+0000 7f8f8dbe1740 1 --2- 172.21.15.6:0/46263244 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a274b66060 0x55a274b6e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.669+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/46263244 >> 172.21.15.6:0/46263244 conn(0x55a274b625f0 msgr2=0x55a274b68320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.669+0000 7f8f6cff9640 0 client.25166 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.670+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/46263244 shutdown_connections 2024-09-12T03:23:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.670+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/46263244 wait complete. 2024-09-12T03:23:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.670+0000 7f8f8dbe1740 20 client.25166 trim_cache size 0 max 16384 2024-09-12T03:23:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.714+0000 7f8f8dbe1740 1 Processor -- start 2024-09-12T03:23:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.714+0000 7f8f8dbe1740 1 -- start start 2024-09-12T03:23:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.714+0000 7f8f8dbe1740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a274b66060 0x55a274b68ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.715+0000 7f8f8dbe1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b66980 0x55a274b88fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.715+0000 7f8f8dbe1740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a274b71240 0x55a274b89500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.715+0000 7f8f895a4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b66980 0x55a274b88fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.715+0000 7f8f895a4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b66980 0x55a274b88fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:45332/0 (socket says 172.21.15.6:45332) 2024-09-12T03:23:07.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.715+0000 7f8f895a4640 1 -- 172.21.15.6:0/3886801366 learned_addr learned my addr 172.21.15.6:0/3886801366 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:07.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.716+0000 7f8f88da3640 1 --2- 172.21.15.6:0/3886801366 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a274b66060 0x55a274b68ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.715+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/3886801366 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8f68010c20 con 0x55a274b71240 2024-09-12T03:23:07.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.716+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/3886801366 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6800e4f0 con 0x55a274b66060 2024-09-12T03:23:07.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.716+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/3886801366 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6800c340 con 0x55a274b66980 2024-09-12T03:23:07.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.716+0000 7f8f895a4640 1 -- 172.21.15.6:0/3886801366 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a274b66060 msgr2=0x55a274b68ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:07.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.716+0000 7f8f895a4640 1 --2- 172.21.15.6:0/3886801366 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a274b66060 0x55a274b68ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.716+0000 7f8f895a4640 1 -- 172.21.15.6:0/3886801366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a274b71240 msgr2=0x55a274b89500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:07.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.716+0000 7f8f895a4640 1 --2- 172.21.15.6:0/3886801366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a274b71240 0x55a274b89500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.716+0000 7f8f895a4640 1 -- 172.21.15.6:0/3886801366 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a274b636a0 con 0x55a274b66980 2024-09-12T03:23:07.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.717+0000 7f8f895a4640 1 --2- 172.21.15.6:0/3886801366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b66980 0x55a274b88fc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55a274b6da70 tx=0x7f8f80008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0d8ff0d89405495 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.718+0000 7f8f6effd640 1 -- 172.21.15.6:0/3886801366 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80022070 con 0x55a274b66980 2024-09-12T03:23:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.718+0000 7f8f6effd640 1 -- 172.21.15.6:0/3886801366 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f8001e7b0 con 0x55a274b66980 2024-09-12T03:23:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.718+0000 7f8f6effd640 1 -- 172.21.15.6:0/3886801366 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80016b30 con 0x55a274b66980 2024-09-12T03:23:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.719+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/3886801366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b66980 msgr2=0x55a274b88fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.719+0000 7f8f8dbe1740 1 --2- 172.21.15.6:0/3886801366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b66980 0x55a274b88fc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55a274b6da70 tx=0x7f8f80008040 comp rx=0 tx=0).stop 2024-09-12T03:23:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.719+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/3886801366 shutdown_connections 2024-09-12T03:23:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.719+0000 7f8f8dbe1740 1 --2- 172.21.15.6:0/3886801366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a274b71240 0x55a274b89500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.719+0000 7f8f8dbe1740 1 --2- 172.21.15.6:0/3886801366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b66980 0x55a274b88fc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.719+0000 7f8f8dbe1740 1 --2- 172.21.15.6:0/3886801366 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a274b66060 0x55a274b68ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.719+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/3886801366 >> 172.21.15.6:0/3886801366 conn(0x55a274b625f0 msgr2=0x55a274b6cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.720+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/3886801366 shutdown_connections 2024-09-12T03:23:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.720+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/3886801366 wait complete. 2024-09-12T03:23:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.721+0000 7f8f8dbe1740 1 -- start start 2024-09-12T03:23:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.721+0000 7f8f8dbe1740 1 Processor -- start 2024-09-12T03:23:07.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.722+0000 7f8f8dbe1740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a274b66060 0x55a274b750b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.722+0000 7f8f8dbe1740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a274b66980 0x55a274b755f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.722+0000 7f8f8dbe1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b71240 0x55a274b75b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.722+0000 7f8f8dbe1740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6800c9b0 con 0x55a274b66060 2024-09-12T03:23:07.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.722+0000 7f8f8dbe1740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6800de20 con 0x55a274b66980 2024-09-12T03:23:07.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.722+0000 7f8f8dbe1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6800d080 con 0x55a274b71240 2024-09-12T03:23:07.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.722+0000 7f8f8dbe1740 10 client.0 _ll_register_callbacks cb 0x7f8f7400fea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:07.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.722+0000 7f8f88da3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b71240 0x55a274b75b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.722+0000 7f8f88da3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b71240 0x55a274b75b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:45344/0 (socket says 172.21.15.6:45344) 2024-09-12T03:23:07.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.722+0000 7f8f88da3640 1 -- 172.21.15.6:0/1149685265 learned_addr learned my addr 172.21.15.6:0/1149685265 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:07.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.723+0000 7f8f88da3640 1 -- 172.21.15.6:0/1149685265 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a274b66980 msgr2=0x55a274b755f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:07.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.723+0000 7f8f88da3640 1 --2- 172.21.15.6:0/1149685265 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a274b66980 0x55a274b755f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.723+0000 7f8f88da3640 1 -- 172.21.15.6:0/1149685265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a274b66060 msgr2=0x55a274b750b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:07.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.723+0000 7f8f88da3640 1 --2- 172.21.15.6:0/1149685265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a274b66060 0x55a274b750b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.724+0000 7f8f88da3640 1 -- 172.21.15.6:0/1149685265 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a274b65e90 con 0x55a274b71240 2024-09-12T03:23:07.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.725+0000 7f8f88da3640 1 --2- 172.21.15.6:0/1149685265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b71240 0x55a274b75b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8f7401a770 tx=0x7f8f7400fdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c18c48c6bbd66a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:07.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.725+0000 7f8f6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:23:07.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.725+0000 7f8f6cff9640 1 -- 172.21.15.6:0/1149685265 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74012d20 con 0x55a274b71240 2024-09-12T03:23:07.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.725+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/1149685265 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f8001e7b0 con 0x55a274b71240 2024-09-12T03:23:07.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.725+0000 7f8f6cff9640 1 -- 172.21.15.6:0/1149685265 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f8f7401aa00 con 0x55a274b71240 2024-09-12T03:23:07.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.725+0000 7f8f6cff9640 1 -- 172.21.15.6:0/1149685265 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8f7400f660 con 0x55a274b71240 2024-09-12T03:23:07.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.726+0000 7f8f8dbe1740 10 client.25225 fetch_fsmap learned FSMap version 26 2024-09-12T03:23:07.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.726+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/1149685265 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a274bec1c0 con 0x55a274b71240 2024-09-12T03:23:07.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.726+0000 7f8f6cff9640 1 -- 172.21.15.6:0/1149685265 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f8001e7b0 con 0x55a274b71240 2024-09-12T03:23:07.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.727+0000 7f8f6cff9640 1 -- 172.21.15.6:0/1149685265 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8f7403b070 con 0x55a274b71240 2024-09-12T03:23:07.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.727+0000 7f8f8dbe1740 10 client.25225 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:23:07.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.727+0000 7f8f8dbe1740 10 client.25225 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:23:07.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.727+0000 7f8f8dbe1740 20 client.25225 populate_metadata read hostname 'smithi006' 2024-09-12T03:23:07.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.727+0000 7f8f8dbe1740 1 --2- 172.21.15.6:0/1149685265 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a274bec9a0 0x55a274beee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.727+0000 7f8f8dbe1740 4 client.25225 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4008eb8d", "release": true, "await": true}] 2024-09-12T03:23:07.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.727+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/1149685265 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4008eb8d", "release": true, "await": true}) -- 0x55a274ad30a0 con 0x55a274bec9a0 2024-09-12T03:23:07.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.727+0000 7f8f895a4640 1 --2- 172.21.15.6:0/1149685265 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a274bec9a0 0x55a274beee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.728+0000 7f8f895a4640 1 --2- 172.21.15.6:0/1149685265 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a274bec9a0 0x55a274beee00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da924a898a7158ef server_cookie=fb9544fe1b780144 in_seq=0 out_seq=0 2024-09-12T03:23:07.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.728+0000 7f8f6cff9640 10 client.25225 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:07.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.778+0000 7f8f6cff9640 1 -- 172.21.15.6:0/1149685265 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55a274ad30a0 con 0x55a274bec9a0 2024-09-12T03:23:07.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.779+0000 7f8f6cff9640 10 client.25225 handle_command_reply: tid=0 2024-09-12T03:23:07.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.779+0000 7f8f8dbe1740 1 client.25225 shutdown 2024-09-12T03:23:07.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.779+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/1149685265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b71240 msgr2=0x55a274b75b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.779+0000 7f8f8dbe1740 1 --2- 172.21.15.6:0/1149685265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b71240 0x55a274b75b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8f7401a770 tx=0x7f8f7400fdd0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.038 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:23:07.038 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 26, 2024-09-12T03:23:07.038 INFO:teuthology.orchestra.run.smithi006.stdout: "leader": 24494, 2024-09-12T03:23:07.038 INFO:teuthology.orchestra.run.smithi006.stdout: "set_version": 8, 2024-09-12T03:23:07.038 INFO:teuthology.orchestra.run.smithi006.stdout: "sets": { 2024-09-12T03:23:07.038 INFO:teuthology.orchestra.run.smithi006.stdout: "4008eb8d": { 2024-09-12T03:23:07.038 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 8, 2024-09-12T03:23:07.038 INFO:teuthology.orchestra.run.smithi006.stdout: "age_ref": 0.0, 2024-09-12T03:23:07.038 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:23:07.038 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RELEASED", 2024-09-12T03:23:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:23:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:23:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: "timeout": 90.0, 2024-09-12T03:23:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: "expiration": 133.4, 2024-09-12T03:23:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: "members": { 2024-09-12T03:23:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: "file:/": { 2024-09-12T03:23:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: "excluded": false, 2024-09-12T03:23:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:23:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RELEASED", 2024-09-12T03:23:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:23:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:23:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:23:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:23:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:23:07.040 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:23:07.040 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:23:07.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.779+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/1149685265 shutdown_connections 2024-09-12T03:23:07.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.779+0000 7f8f8dbe1740 1 --2- 172.21.15.6:0/1149685265 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a274bec9a0 0x55a274beee00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.779+0000 7f8f8dbe1740 1 --2- 172.21.15.6:0/1149685265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a274b71240 0x55a274b75b30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.779+0000 7f8f8dbe1740 1 --2- 172.21.15.6:0/1149685265 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a274b66980 0x55a274b755f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.779+0000 7f8f8dbe1740 1 --2- 172.21.15.6:0/1149685265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a274b66060 0x55a274b750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.779+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/1149685265 >> 172.21.15.6:0/1149685265 conn(0x55a274b625f0 msgr2=0x55a274b6e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:07.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.779+0000 7f8f6cff9640 0 client.25225 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:07.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.780+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/1149685265 shutdown_connections 2024-09-12T03:23:07.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.780+0000 7f8f8dbe1740 1 -- 172.21.15.6:0/1149685265 wait complete. 2024-09-12T03:23:07.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.780+0000 7f8f8dbe1740 20 client.25225 trim_cache size 0 max 16384 2024-09-12T03:23:07.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.782+0000 7f8f8b808640 1 -- 172.21.15.6:0/325909606 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8f700a2cc0 msgr2=0x7f8f700a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.782+0000 7f8f8b808640 1 --2- 172.21.15.6:0/325909606 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8f700a2cc0 0x7f8f700a5180 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c004af0 tx=0x7f8f7c03b920 comp rx=0 tx=0).stop 2024-09-12T03:23:07.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.782+0000 7f8f8b808640 1 -- 172.21.15.6:0/325909606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f8411b140 msgr2=0x7f8f84116dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.782+0000 7f8f8b808640 1 --2- 172.21.15.6:0/325909606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f8411b140 0x7f8f84116dc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c01ac00 tx=0x7f8f7c040a60 comp rx=0 tx=0).stop 2024-09-12T03:23:07.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.785+0000 7f8f8b808640 1 -- 172.21.15.6:0/325909606 shutdown_connections 2024-09-12T03:23:07.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.786+0000 7f8f8b808640 1 --2- 172.21.15.6:0/325909606 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8f700a2cc0 0x7f8f700a5180 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.786+0000 7f8f8b808640 1 --2- 172.21.15.6:0/325909606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f8411b140 0x7f8f84116dc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.786+0000 7f8f8b808640 1 --2- 172.21.15.6:0/325909606 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f84109830 0x7f8f84116880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.786+0000 7f8f8b808640 1 --2- 172.21.15.6:0/325909606 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f84109450 0x7f8f8411a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.786+0000 7f8f8b808640 1 -- 172.21.15.6:0/325909606 >> 172.21.15.6:0/325909606 conn(0x7f8f8406e140 msgr2=0x7f8f84070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:07.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.786+0000 7f8f8b808640 1 -- 172.21.15.6:0/325909606 shutdown_connections 2024-09-12T03:23:07.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:06.786+0000 7f8f8b808640 1 -- 172.21.15.6:0/325909606 wait complete. 2024-09-12T03:23:07.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:07 smithi006 ceph-mon[30991]: pgmap v395: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 151 MiB/s wr, 17.24k op/s 2024-09-12T03:23:07.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:07 smithi161 ceph-mon[35485]: pgmap v395: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 151 MiB/s wr, 17.24k op/s 2024-09-12T03:23:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:07 smithi118 ceph-mon[35048]: pgmap v395: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 151 MiB/s wr, 17.24k op/s 2024-09-12T03:23:08.883 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4008eb8d', total seconds elapsed: 18.7 2024-09-12T03:23:09.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:09 smithi006 ceph-mon[30991]: pgmap v396: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 132 MiB/s wr, 14.66k op/s 2024-09-12T03:23:09.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:09 smithi161 ceph-mon[35485]: pgmap v396: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 132 MiB/s wr, 14.66k op/s 2024-09-12T03:23:09.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:09 smithi118 ceph-mon[35048]: pgmap v396: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 132 MiB/s wr, 14.66k op/s 2024-09-12T03:23:10.832 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:23:11.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:11 smithi006 ceph-mon[30991]: pgmap v397: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 134 MiB/s wr, 14.03k op/s 2024-09-12T03:23:11.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:11 smithi161 ceph-mon[35485]: pgmap v397: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 134 MiB/s wr, 14.03k op/s 2024-09-12T03:23:11.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:11 smithi118 ceph-mon[35048]: pgmap v397: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 134 MiB/s wr, 14.03k op/s 2024-09-12T03:23:11.654 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:12.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.550+0000 7fccfe540640 1 -- 172.21.15.6:0/2918452587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf00942e0 msgr2=0x7fccf00946c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:12.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.550+0000 7fccfe540640 1 --2- 172.21.15.6:0/2918452587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf00942e0 0x7fccf00946c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fccf8067960 tx=0x7fccf8096e10 comp rx=0 tx=0).stop 2024-09-12T03:23:12.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.550+0000 7fccfe540640 1 -- 172.21.15.6:0/2918452587 shutdown_connections 2024-09-12T03:23:12.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.550+0000 7fccfe540640 1 --2- 172.21.15.6:0/2918452587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fccf0098fb0 0x7fccf00a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.550+0000 7fccfe540640 1 --2- 172.21.15.6:0/2918452587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccf0094c90 0x7fccf0098880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.550+0000 7fccfe540640 1 --2- 172.21.15.6:0/2918452587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf00942e0 0x7fccf00946c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.550+0000 7fccfe540640 1 -- 172.21.15.6:0/2918452587 >> 172.21.15.6:0/2918452587 conn(0x7fccf0009930 msgr2=0x7fccf0009d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:12.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.552+0000 7fccfe540640 1 -- 172.21.15.6:0/2918452587 shutdown_connections 2024-09-12T03:23:12.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.552+0000 7fccfe540640 1 -- 172.21.15.6:0/2918452587 wait complete. 2024-09-12T03:23:12.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.552+0000 7fccfe540640 1 Processor -- start 2024-09-12T03:23:12.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.553+0000 7fccfe540640 1 -- start start 2024-09-12T03:23:12.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.553+0000 7fccfe540640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf00942e0 0x7fccf00a2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.553+0000 7fccfe540640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccf0094c90 0x7fccf00a2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.553+0000 7fccfe540640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fccf0098fb0 0x7fccf00a2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.553+0000 7fccfe540640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fccf00b6a00 con 0x7fccf00942e0 2024-09-12T03:23:12.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.553+0000 7fccfe540640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fccf00b6880 con 0x7fccf0098fb0 2024-09-12T03:23:12.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.553+0000 7fccfe540640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fccf00b6b80 con 0x7fccf0094c90 2024-09-12T03:23:12.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.553+0000 7fccfdd3f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fccf0098fb0 0x7fccf00a2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:12.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.553+0000 7fccfdd3f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fccf0098fb0 0x7fccf00a2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:43112/0 (socket says 172.21.15.6:43112) 2024-09-12T03:23:12.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.553+0000 7fccfdd3f640 1 -- 172.21.15.6:0/3817004674 learned_addr learned my addr 172.21.15.6:0/3817004674 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:12.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.554+0000 7fccfcd3d640 1 --2- 172.21.15.6:0/3817004674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccf0094c90 0x7fccf00a2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:12.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.554+0000 7fccfd53e640 1 --2- 172.21.15.6:0/3817004674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf00942e0 0x7fccf00a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:12.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.554+0000 7fccfdd3f640 1 -- 172.21.15.6:0/3817004674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccf0094c90 msgr2=0x7fccf00a2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:12.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.554+0000 7fccfdd3f640 1 --2- 172.21.15.6:0/3817004674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccf0094c90 0x7fccf00a2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.554+0000 7fccfdd3f640 1 -- 172.21.15.6:0/3817004674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf00942e0 msgr2=0x7fccf00a2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:12.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.554+0000 7fccfdd3f640 1 --2- 172.21.15.6:0/3817004674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf00942e0 0x7fccf00a2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.554+0000 7fccfdd3f640 1 -- 172.21.15.6:0/3817004674 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf0141110 con 0x7fccf0098fb0 2024-09-12T03:23:12.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.555+0000 7fccfdd3f640 1 --2- 172.21.15.6:0/3817004674 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fccf0098fb0 0x7fccf00a2cd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fccf400b3f0 tx=0x7fccf400b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:12.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.556+0000 7fccee7fc640 1 -- 172.21.15.6:0/3817004674 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4015020 con 0x7fccf0098fb0 2024-09-12T03:23:12.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.556+0000 7fccfe540640 1 -- 172.21.15.6:0/3817004674 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf0141400 con 0x7fccf0098fb0 2024-09-12T03:23:12.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.556+0000 7fccfe540640 1 -- 172.21.15.6:0/3817004674 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf0141940 con 0x7fccf0098fb0 2024-09-12T03:23:12.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.556+0000 7fccee7fc640 1 -- 172.21.15.6:0/3817004674 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccf4011d10 con 0x7fccf0098fb0 2024-09-12T03:23:12.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.557+0000 7fccee7fc640 1 -- 172.21.15.6:0/3817004674 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf400f2e0 con 0x7fccf0098fb0 2024-09-12T03:23:12.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.562+0000 7fcd01b7b740 1 Processor -- start 2024-09-12T03:23:12.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.563+0000 7fcd01b7b740 1 -- start start 2024-09-12T03:23:12.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.563+0000 7fcd01b7b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400ab670 0x5647400aba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.563+0000 7fcd01b7b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647400abf90 0x5647400b6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.563+0000 7fcd01b7b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647400b6850 0x5647400b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.563+0000 7fcd01b7b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5647400112e0 con 0x5647400ab670 2024-09-12T03:23:12.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.563+0000 7fcd01b7b740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x56473ffb8ce0 con 0x5647400b6850 2024-09-12T03:23:12.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.563+0000 7fcd01b7b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56474000d2a0 con 0x5647400abf90 2024-09-12T03:23:12.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.567+0000 7fccee7fc640 1 -- 172.21.15.6:0/3817004674 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fccf4018020 con 0x7fccf0098fb0 2024-09-12T03:23:12.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.567+0000 7fccfcd3d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647400b6850 0x5647400b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:12.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.567+0000 7fccfcd3d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400ab670 0x5647400aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:12.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.567+0000 7fccfcd3d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400ab670 0x5647400aba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34910/0 (socket says 172.21.15.6:34910) 2024-09-12T03:23:12.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.568+0000 7fccfcd3d640 1 -- 172.21.15.6:0/1684773729 learned_addr learned my addr 172.21.15.6:0/1684773729 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:12.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.568+0000 7fccee7fc640 1 --2- 172.21.15.6:0/3817004674 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcce40a2d90 0x7fcce40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.569+0000 7fccee7fc640 1 -- 172.21.15.6:0/3817004674 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fccf40ce860 con 0x7fccf0098fb0 2024-09-12T03:23:12.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.569+0000 7fccfdd3f640 1 --2- 172.21.15.6:0/3817004674 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcce40a2d90 0x7fcce40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:12.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.570+0000 7fccfdd3f640 1 --2- 172.21.15.6:0/3817004674 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcce40a2d90 0x7fcce40a5250 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fccf4006e80 tx=0x7fccf400bf60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:12.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.571+0000 7fccfcd3d640 1 -- 172.21.15.6:0/1684773729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647400abf90 msgr2=0x5647400b6310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:23:12.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.571+0000 7fccfcd3d640 1 --2- 172.21.15.6:0/1684773729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647400abf90 0x5647400b6310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.571+0000 7fccfcd3d640 1 -- 172.21.15.6:0/1684773729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400ab670 msgr2=0x5647400aba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:12.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.571+0000 7fccfcd3d640 1 --2- 172.21.15.6:0/1684773729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400ab670 0x5647400aba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.571+0000 7fccfcd3d640 1 -- 172.21.15.6:0/1684773729 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56473ff9d440 con 0x5647400b6850 2024-09-12T03:23:12.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.572+0000 7fccfcd3d640 1 --2- 172.21.15.6:0/1684773729 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647400b6850 0x5647400b8c40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcce800cfd0 tx=0x7fcce8006ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ad3fc5e3b6db92d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:12.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.572+0000 7fcce37fe640 1 -- 172.21.15.6:0/1684773729 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce801d070 con 0x5647400b6850 2024-09-12T03:23:12.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.572+0000 7fcce37fe640 1 -- 172.21.15.6:0/1684773729 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce8018b20 con 0x5647400b6850 2024-09-12T03:23:12.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.573+0000 7fcd01b7b740 1 -- 172.21.15.6:0/1684773729 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647400b6850 msgr2=0x5647400b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:12.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.573+0000 7fcd01b7b740 1 --2- 172.21.15.6:0/1684773729 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647400b6850 0x5647400b8c40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcce800cfd0 tx=0x7fcce8006ee0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.574+0000 7fcd01b7b740 1 -- 172.21.15.6:0/1684773729 shutdown_connections 2024-09-12T03:23:12.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.574+0000 7fcd01b7b740 1 --2- 172.21.15.6:0/1684773729 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647400b6850 0x5647400b8c40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.574+0000 7fcd01b7b740 1 --2- 172.21.15.6:0/1684773729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647400abf90 0x5647400b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.574+0000 7fcd01b7b740 1 --2- 172.21.15.6:0/1684773729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400ab670 0x5647400aba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.574+0000 7fcd01b7b740 1 -- 172.21.15.6:0/1684773729 >> 172.21.15.6:0/1684773729 conn(0x5647400a7c00 msgr2=0x5647400a7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:12.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.576+0000 7fcd01b7b740 1 -- 172.21.15.6:0/1684773729 shutdown_connections 2024-09-12T03:23:12.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.582+0000 7fcd01b7b740 1 -- 172.21.15.6:0/1684773729 wait complete. 2024-09-12T03:23:12.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.582+0000 7fcd01b7b740 1 -- start start 2024-09-12T03:23:12.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.583+0000 7fcd01b7b740 1 Processor -- start 2024-09-12T03:23:12.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.583+0000 7fcd01b7b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400ab670 0x5647400b6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.583+0000 7fcd01b7b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647400abf90 0x5647400b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.583+0000 7fcd01b7b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647400b6850 0x5647400b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.583+0000 7fcd01b7b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5647400b94c0 con 0x5647400ab670 2024-09-12T03:23:12.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.583+0000 7fcd01b7b740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x56473ffcc050 con 0x5647400abf90 2024-09-12T03:23:12.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.583+0000 7fcd01b7b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5647400b9640 con 0x5647400b6850 2024-09-12T03:23:12.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.583+0000 7fcd01b7b740 10 client.0 _ll_register_callbacks cb 0x56474000c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:12.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.584+0000 7fccfcd3d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647400b6850 0x5647400b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:12.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.584+0000 7fccfcd3d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647400b6850 0x5647400b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:55880/0 (socket says 172.21.15.6:55880) 2024-09-12T03:23:12.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.584+0000 7fccfcd3d640 1 -- 172.21.15.6:0/3277757335 learned_addr learned my addr 172.21.15.6:0/3277757335 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:12.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.586+0000 7fccfcd3d640 1 --2- 172.21.15.6:0/3277757335 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400ab670 0x5647400b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:12.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.586+0000 7fccfcd3d640 1 -- 172.21.15.6:0/3277757335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647400abf90 msgr2=0x5647400b3320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:23:12.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.586+0000 7fccfcd3d640 1 --2- 172.21.15.6:0/3277757335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647400abf90 0x5647400b3320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.586+0000 7fccfcd3d640 1 -- 172.21.15.6:0/3277757335 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400ab670 msgr2=0x5647400b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:12.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.586+0000 7fccfcd3d640 1 --2- 172.21.15.6:0/3277757335 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400ab670 0x5647400b6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.586+0000 7fccfcd3d640 1 -- 172.21.15.6:0/3277757335 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56473fd823c0 con 0x5647400b6850 2024-09-12T03:23:12.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.588+0000 7fccfcd3d640 1 --2- 172.21.15.6:0/3277757335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647400b6850 0x5647400b3860 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcce8006e10 tx=0x7fcce8017720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f33eaea36395ff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:12.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.588+0000 7fcce0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:23:12.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.589+0000 7fcce0ff9640 1 -- 172.21.15.6:0/3277757335 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce801d040 con 0x5647400b6850 2024-09-12T03:23:12.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.589+0000 7fcd01b7b740 1 -- 172.21.15.6:0/3277757335 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce8018b20 con 0x5647400b6850 2024-09-12T03:23:12.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.589+0000 7fcce0ff9640 1 -- 172.21.15.6:0/3277757335 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fcce8018440 con 0x5647400b6850 2024-09-12T03:23:12.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.589+0000 7fcce0ff9640 1 -- 172.21.15.6:0/3277757335 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcce80186a0 con 0x5647400b6850 2024-09-12T03:23:12.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.590+0000 7fcd01b7b740 10 client.25231 fetch_fsmap learned FSMap version 26 2024-09-12T03:23:12.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.591+0000 7fcd01b7b740 1 -- 172.21.15.6:0/3277757335 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5647400c4e50 con 0x5647400b6850 2024-09-12T03:23:12.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.592+0000 7fcce0ff9640 1 -- 172.21.15.6:0/3277757335 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce8018b20 con 0x5647400b6850 2024-09-12T03:23:12.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.592+0000 7fcce0ff9640 1 -- 172.21.15.6:0/3277757335 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcce8012070 con 0x5647400b6850 2024-09-12T03:23:12.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.592+0000 7fcd01b7b740 10 client.25231 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:23:12.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.592+0000 7fcd01b7b740 10 client.25231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:23:12.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.592+0000 7fcd01b7b740 20 client.25231 populate_metadata read hostname 'smithi006' 2024-09-12T03:23:12.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.592+0000 7fcd01b7b740 1 --2- 172.21.15.6:0/3277757335 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5647400c6620 0x5647400c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.592+0000 7fcd01b7b740 4 client.25231 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:23:12.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.592+0000 7fcd01b7b740 1 -- 172.21.15.6:0/3277757335 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56474001e850 con 0x5647400c6620 2024-09-12T03:23:12.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.593+0000 7fccfd53e640 1 --2- 172.21.15.6:0/3277757335 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5647400c6620 0x5647400c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:12.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.600+0000 7fccfd53e640 1 --2- 172.21.15.6:0/3277757335 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5647400c6620 0x5647400c8aa0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4fb818143fc1ee4 server_cookie=c1efec805563346b in_seq=0 out_seq=0 2024-09-12T03:23:12.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.600+0000 7fcce0ff9640 10 client.25231 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:12.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.603+0000 7fcce0ff9640 1 -- 172.21.15.6:0/3277757335 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56474001e850 con 0x5647400c6620 2024-09-12T03:23:12.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.604+0000 7fcce0ff9640 10 client.25231 handle_command_reply: tid=0 2024-09-12T03:23:12.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.604+0000 7fcd01b7b740 1 client.25231 shutdown 2024-09-12T03:23:12.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.607+0000 7fcd01b7b740 1 -- 172.21.15.6:0/3277757335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647400b6850 msgr2=0x5647400b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:12.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.607+0000 7fcd01b7b740 1 --2- 172.21.15.6:0/3277757335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647400b6850 0x5647400b3860 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcce8006e10 tx=0x7fcce8017720 comp rx=0 tx=0).stop 2024-09-12T03:23:12.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.608+0000 7fcd01b7b740 1 -- 172.21.15.6:0/3277757335 shutdown_connections 2024-09-12T03:23:12.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.608+0000 7fcd01b7b740 1 --2- 172.21.15.6:0/3277757335 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5647400c6620 0x5647400c8aa0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.608+0000 7fcd01b7b740 1 --2- 172.21.15.6:0/3277757335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647400b6850 0x5647400b3860 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.608+0000 7fcd01b7b740 1 --2- 172.21.15.6:0/3277757335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647400abf90 0x5647400b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.608+0000 7fcd01b7b740 1 --2- 172.21.15.6:0/3277757335 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400ab670 0x5647400b6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.608+0000 7fcd01b7b740 1 -- 172.21.15.6:0/3277757335 >> 172.21.15.6:0/3277757335 conn(0x5647400a7c00 msgr2=0x5647400ad930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:12.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.608+0000 7fcce0ff9640 0 client.25231 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:12.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.608+0000 7fcd01b7b740 1 -- 172.21.15.6:0/3277757335 shutdown_connections 2024-09-12T03:23:12.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.609+0000 7fcd01b7b740 1 -- 172.21.15.6:0/3277757335 wait complete. 2024-09-12T03:23:12.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.609+0000 7fcd01b7b740 20 client.25231 trim_cache size 0 max 16384 2024-09-12T03:23:12.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.651+0000 7fcd01b7b740 1 Processor -- start 2024-09-12T03:23:12.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.660+0000 7fcd01b7b740 1 -- start start 2024-09-12T03:23:12.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.661+0000 7fcd01b7b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647400ab670 0x5647400ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.661+0000 7fcd01b7b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400abf90 0x5647400ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.661+0000 7fcd01b7b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647400b6850 0x5647400a6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.661+0000 7fcd01b7b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc010ce0 con 0x5647400abf90 2024-09-12T03:23:12.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.661+0000 7fcd01b7b740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc00e5b0 con 0x5647400b6850 2024-09-12T03:23:12.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.661+0000 7fcd01b7b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc00c400 con 0x5647400ab670 2024-09-12T03:23:12.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.662+0000 7fccfd53e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400abf90 0x5647400ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:12.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.662+0000 7fccfd53e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400abf90 0x5647400ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34922/0 (socket says 172.21.15.6:34922) 2024-09-12T03:23:12.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.662+0000 7fccfd53e640 1 -- 172.21.15.6:0/4033296240 learned_addr learned my addr 172.21.15.6:0/4033296240 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:12.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.665+0000 7fccfd53e640 1 -- 172.21.15.6:0/4033296240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647400ab670 msgr2=0x5647400ce370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:23:12.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.665+0000 7fccfd53e640 1 --2- 172.21.15.6:0/4033296240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647400ab670 0x5647400ce370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.665+0000 7fccfd53e640 1 -- 172.21.15.6:0/4033296240 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647400b6850 msgr2=0x5647400a6500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:23:12.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.665+0000 7fccfd53e640 1 --2- 172.21.15.6:0/4033296240 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647400b6850 0x5647400a6500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.665+0000 7fccfd53e640 1 -- 172.21.15.6:0/4033296240 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647400a7a30 con 0x5647400abf90 2024-09-12T03:23:12.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.667+0000 7fccfd53e640 1 --2- 172.21.15.6:0/4033296240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400abf90 0x5647400ce8b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fccf8051500 tx=0x7fccf8068cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe932635631a433b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:12.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.672+0000 7fcce2ffd640 1 -- 172.21.15.6:0/4033296240 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf80ad020 con 0x5647400abf90 2024-09-12T03:23:12.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.672+0000 7fcce2ffd640 1 -- 172.21.15.6:0/4033296240 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccf809b040 con 0x5647400abf90 2024-09-12T03:23:12.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.672+0000 7fcce2ffd640 1 -- 172.21.15.6:0/4033296240 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf809f410 con 0x5647400abf90 2024-09-12T03:23:12.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.673+0000 7fcd01b7b740 1 -- 172.21.15.6:0/4033296240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400abf90 msgr2=0x5647400ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:12.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.673+0000 7fcd01b7b740 1 --2- 172.21.15.6:0/4033296240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400abf90 0x5647400ce8b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fccf8051500 tx=0x7fccf8068cb0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.676+0000 7fcd01b7b740 1 -- 172.21.15.6:0/4033296240 shutdown_connections 2024-09-12T03:23:12.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.676+0000 7fcd01b7b740 1 --2- 172.21.15.6:0/4033296240 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647400b6850 0x5647400a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.677+0000 7fcd01b7b740 1 --2- 172.21.15.6:0/4033296240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400abf90 0x5647400ce8b0 secure :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fccf8051500 tx=0x7fccf8068cb0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.677+0000 7fcd01b7b740 1 --2- 172.21.15.6:0/4033296240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647400ab670 0x5647400ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.677+0000 7fcd01b7b740 1 -- 172.21.15.6:0/4033296240 >> 172.21.15.6:0/4033296240 conn(0x5647400a7c00 msgr2=0x5647400b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:12.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.680+0000 7fcd01b7b740 1 -- 172.21.15.6:0/4033296240 shutdown_connections 2024-09-12T03:23:12.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.680+0000 7fcd01b7b740 1 -- 172.21.15.6:0/4033296240 wait complete. 2024-09-12T03:23:12.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.681+0000 7fcd01b7b740 1 -- start start 2024-09-12T03:23:12.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.681+0000 7fcd01b7b740 1 Processor -- start 2024-09-12T03:23:12.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.682+0000 7fcd01b7b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400ab670 0x5647400af950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.682+0000 7fcd01b7b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647400b6850 0x5647400afe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.683+0000 7fcd01b7b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647400b0530 0x5647400bd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.683+0000 7fcd01b7b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc00ca70 con 0x5647400ab670 2024-09-12T03:23:12.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.683+0000 7fcd01b7b740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc00dee0 con 0x5647400b6850 2024-09-12T03:23:12.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.683+0000 7fccfd53e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647400b6850 0x5647400afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:12.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.683+0000 7fccfd53e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647400b6850 0x5647400afe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:43142/0 (socket says 172.21.15.6:43142) 2024-09-12T03:23:12.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.683+0000 7fccfd53e640 1 -- 172.21.15.6:0/3694356476 learned_addr learned my addr 172.21.15.6:0/3694356476 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:12.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.684+0000 7fccfcd3d640 1 --2- 172.21.15.6:0/3694356476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647400b0530 0x5647400bd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:12.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.684+0000 7fcd01b7b740 1 -- 172.21.15.6:0/3694356476 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc00d140 con 0x5647400b0530 2024-09-12T03:23:12.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.684+0000 7fcd01b7b740 10 client.0 _ll_register_callbacks cb 0x7fcce801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:12.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.685+0000 7fccfd53e640 1 -- 172.21.15.6:0/3694356476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647400b0530 msgr2=0x5647400bd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:12.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.685+0000 7fccfd53e640 1 --2- 172.21.15.6:0/3694356476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647400b0530 0x5647400bd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.685+0000 7fccfd53e640 1 -- 172.21.15.6:0/3694356476 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400ab670 msgr2=0x5647400af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:12.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.685+0000 7fccfd53e640 1 --2- 172.21.15.6:0/3694356476 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400ab670 0x5647400af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.685+0000 7fccfd53e640 1 -- 172.21.15.6:0/3694356476 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647400be1a0 con 0x5647400b6850 2024-09-12T03:23:12.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.685+0000 7fcd01b7b740 1 -- 172.21.15.6:0/3694356476 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccf809b040 con 0x5647400b6850 2024-09-12T03:23:12.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.685+0000 7fccfd53e640 1 --2- 172.21.15.6:0/3694356476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647400b6850 0x5647400afe90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fccf8065360 tx=0x7fccf8067a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73f166598b29218b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:12.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.686+0000 7fcce0ff9640 10 client.25178 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:23:12.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.687+0000 7fcce0ff9640 1 -- 172.21.15.6:0/3694356476 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf80c1050 con 0x5647400b6850 2024-09-12T03:23:12.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.688+0000 7fcce0ff9640 1 -- 172.21.15.6:0/3694356476 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fccf80a8850 con 0x5647400b6850 2024-09-12T03:23:12.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.688+0000 7fcce0ff9640 1 -- 172.21.15.6:0/3694356476 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fccf80b7070 con 0x5647400b6850 2024-09-12T03:23:12.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.688+0000 7fcd01b7b740 10 client.25178 fetch_fsmap learned FSMap version 26 2024-09-12T03:23:12.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.688+0000 7fcd01b7b740 1 -- 172.21.15.6:0/3694356476 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5647400bf350 con 0x5647400b6850 2024-09-12T03:23:12.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.691+0000 7fcce0ff9640 1 -- 172.21.15.6:0/3694356476 <== mon.2 v2:172.21.15.118:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fccf80a8a50 con 0x5647400b6850 2024-09-12T03:23:12.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.692+0000 7fcd01b7b740 10 client.25178 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:23:12.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.692+0000 7fcd01b7b740 10 client.25178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:23:12.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.692+0000 7fcd01b7b740 20 client.25178 populate_metadata read hostname 'smithi006' 2024-09-12T03:23:12.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.692+0000 7fcd01b7b740 1 --2- 172.21.15.6:0/3694356476 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x564740131390 0x5647401337b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.692+0000 7fcd01b7b740 4 client.25178 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:23:12.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.692+0000 7fcd01b7b740 1 -- 172.21.15.6:0/3694356476 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fccf809b040 con 0x564740131390 2024-09-12T03:23:12.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.695+0000 7fccfcd3d640 1 --2- 172.21.15.6:0/3694356476 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x564740131390 0x5647401337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:12.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.697+0000 7fccfcd3d640 1 --2- 172.21.15.6:0/3694356476 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x564740131390 0x5647401337b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e813da453e95cc5 server_cookie=5ce615061745cd45 in_seq=0 out_seq=0 2024-09-12T03:23:12.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.697+0000 7fcce0ff9640 10 client.25178 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:12.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.699+0000 7fcce0ff9640 1 -- 172.21.15.6:0/3694356476 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fccf809b040 con 0x564740131390 2024-09-12T03:23:12.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.699+0000 7fcce0ff9640 10 client.25178 handle_command_reply: tid=0 2024-09-12T03:23:12.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.699+0000 7fcd01b7b740 1 client.25178 shutdown 2024-09-12T03:23:12.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.702+0000 7fcd01b7b740 1 -- 172.21.15.6:0/3694356476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647400b6850 msgr2=0x5647400afe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:12.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.703+0000 7fcd01b7b740 1 --2- 172.21.15.6:0/3694356476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647400b6850 0x5647400afe90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fccf8065360 tx=0x7fccf8067a20 comp rx=0 tx=0).stop 2024-09-12T03:23:12.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.703+0000 7fcd01b7b740 1 -- 172.21.15.6:0/3694356476 shutdown_connections 2024-09-12T03:23:12.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.703+0000 7fcd01b7b740 1 --2- 172.21.15.6:0/3694356476 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x564740131390 0x5647401337b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.703+0000 7fcd01b7b740 1 --2- 172.21.15.6:0/3694356476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647400b0530 0x5647400bd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.703+0000 7fcd01b7b740 1 --2- 172.21.15.6:0/3694356476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647400b6850 0x5647400afe90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.703+0000 7fcd01b7b740 1 --2- 172.21.15.6:0/3694356476 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647400ab670 0x5647400af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.703+0000 7fcd01b7b740 1 -- 172.21.15.6:0/3694356476 >> 172.21.15.6:0/3694356476 conn(0x5647400a7c00 msgr2=0x5647400a6500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:12.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.703+0000 7fcce0ff9640 0 client.25178 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:12.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.704+0000 7fcd01b7b740 1 -- 172.21.15.6:0/3694356476 shutdown_connections 2024-09-12T03:23:12.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.704+0000 7fcd01b7b740 1 -- 172.21.15.6:0/3694356476 wait complete. 2024-09-12T03:23:12.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.704+0000 7fcd01b7b740 20 client.25178 trim_cache size 0 max 16384 2024-09-12T03:23:12.709 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:23:12.709 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2024-09-12T03:23:12.709 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2024-09-12T03:23:12.709 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:23:12.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.709+0000 7fccfe540640 1 -- 172.21.15.6:0/3817004674 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcce40a2d90 msgr2=0x7fcce40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:12.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.709+0000 7fccfe540640 1 --2- 172.21.15.6:0/3817004674 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcce40a2d90 0x7fcce40a5250 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fccf4006e80 tx=0x7fccf400bf60 comp rx=0 tx=0).stop 2024-09-12T03:23:12.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.710+0000 7fccfe540640 1 -- 172.21.15.6:0/3817004674 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fccf0098fb0 msgr2=0x7fccf00a2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:12.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.710+0000 7fccfe540640 1 --2- 172.21.15.6:0/3817004674 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fccf0098fb0 0x7fccf00a2cd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fccf400b3f0 tx=0x7fccf400b8b0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.710+0000 7fccfe540640 1 -- 172.21.15.6:0/3817004674 shutdown_connections 2024-09-12T03:23:12.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.710+0000 7fccfe540640 1 --2- 172.21.15.6:0/3817004674 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcce40a2d90 0x7fcce40a5250 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.710+0000 7fccfe540640 1 --2- 172.21.15.6:0/3817004674 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fccf0098fb0 0x7fccf00a2cd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.710+0000 7fccfe540640 1 --2- 172.21.15.6:0/3817004674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccf0094c90 0x7fccf00a2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.710+0000 7fccfe540640 1 --2- 172.21.15.6:0/3817004674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf00942e0 0x7fccf00a2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.711+0000 7fccfe540640 1 -- 172.21.15.6:0/3817004674 >> 172.21.15.6:0/3817004674 conn(0x7fccf0009930 msgr2=0x7fccf0092450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:12.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.713+0000 7fccfe540640 1 -- 172.21.15.6:0/3817004674 shutdown_connections 2024-09-12T03:23:12.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:12.713+0000 7fccfe540640 1 -- 172.21.15.6:0/3817004674 wait complete. 2024-09-12T03:23:13.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:13 smithi006 ceph-mon[30991]: pgmap v398: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 114 MiB/s wr, 11.47k op/s 2024-09-12T03:23:13.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:13 smithi161 ceph-mon[35485]: pgmap v398: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 114 MiB/s wr, 11.47k op/s 2024-09-12T03:23:13.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:13 smithi118 ceph-mon[35048]: pgmap v398: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 114 MiB/s wr, 11.47k op/s 2024-09-12T03:23:14.377 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:23:14.378 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:23:15.005 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:15.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:15 smithi118 ceph-mon[35048]: pgmap v399: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 157 MiB/s wr, 14.70k op/s 2024-09-12T03:23:15.554 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:15 smithi006 ceph-mon[30991]: pgmap v399: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 157 MiB/s wr, 14.70k op/s 2024-09-12T03:23:15.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:15 smithi161 ceph-mon[35485]: pgmap v399: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 157 MiB/s wr, 14.70k op/s 2024-09-12T03:23:16.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.077+0000 7fd9bee41640 1 -- 172.21.15.6:0/1149343300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9b8109450 msgr2=0x7fd9b8110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:16.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.077+0000 7fd9bee41640 1 --2- 172.21.15.6:0/1149343300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9b8109450 0x7fd9b8110250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009860 tx=0x7fd9ac02a380 comp rx=0 tx=0).stop 2024-09-12T03:23:16.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.077+0000 7fd9bee41640 1 -- 172.21.15.6:0/1149343300 shutdown_connections 2024-09-12T03:23:16.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.077+0000 7fd9bee41640 1 --2- 172.21.15.6:0/1149343300 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9b811b140 0x7fd9b811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.077+0000 7fd9bee41640 1 --2- 172.21.15.6:0/1149343300 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9b8109830 0x7fd9b8109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.077+0000 7fd9bee41640 1 --2- 172.21.15.6:0/1149343300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9b8109450 0x7fd9b8110250 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.077+0000 7fd9bee41640 1 -- 172.21.15.6:0/1149343300 >> 172.21.15.6:0/1149343300 conn(0x7fd9b806e140 msgr2=0x7fd9b806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:16.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.077+0000 7fd9bee41640 1 -- 172.21.15.6:0/1149343300 shutdown_connections 2024-09-12T03:23:16.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.078+0000 7fd9bee41640 1 -- 172.21.15.6:0/1149343300 wait complete. 2024-09-12T03:23:16.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.078+0000 7fd9bee41640 1 Processor -- start 2024-09-12T03:23:16.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.079+0000 7fd9bee41640 1 -- start start 2024-09-12T03:23:16.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.081+0000 7fd9bee41640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9b8109450 0x7fd9b80793d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.081+0000 7fd9bee41640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9b8109830 0x7fd9b8079910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.081+0000 7fd9bee41640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9b811b140 0x7fd9b80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.081+0000 7fd9bee41640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b8120df0 con 0x7fd9b8109830 2024-09-12T03:23:16.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.081+0000 7fd9bee41640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b8120c70 con 0x7fd9b811b140 2024-09-12T03:23:16.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.081+0000 7fd9bee41640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b8120f70 con 0x7fd9b8109450 2024-09-12T03:23:16.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.083+0000 7fd9b7fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9b8109830 0x7fd9b8079910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.083+0000 7fd9b7fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9b8109830 0x7fd9b8079910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34938/0 (socket says 172.21.15.6:34938) 2024-09-12T03:23:16.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.083+0000 7fd9b7fff640 1 -- 172.21.15.6:0/2535929780 learned_addr learned my addr 172.21.15.6:0/2535929780 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:16.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.083+0000 7fd9bd3de640 1 --2- 172.21.15.6:0/2535929780 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9b811b140 0x7fd9b80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.084+0000 7fd9b7fff640 1 -- 172.21.15.6:0/2535929780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9b8109450 msgr2=0x7fd9b80793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:23:16.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.084+0000 7fd9b7fff640 1 --2- 172.21.15.6:0/2535929780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9b8109450 0x7fd9b80793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.084+0000 7fd9b7fff640 1 -- 172.21.15.6:0/2535929780 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9b811b140 msgr2=0x7fd9b80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:16.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.084+0000 7fd9b7fff640 1 --2- 172.21.15.6:0/2535929780 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9b811b140 0x7fd9b80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.084+0000 7fd9b7fff640 1 -- 172.21.15.6:0/2535929780 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b8074db0 con 0x7fd9b8109830 2024-09-12T03:23:16.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.085+0000 7fd9b7fff640 1 --2- 172.21.15.6:0/2535929780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9b8109830 0x7fd9b8079910 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000ed80 tx=0x7fd9b000c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:16.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.086+0000 7fd9b5ffb640 1 -- 172.21.15.6:0/2535929780 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b001c070 con 0x7fd9b8109830 2024-09-12T03:23:16.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.086+0000 7fd9b5ffb640 1 -- 172.21.15.6:0/2535929780 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9b0017b20 con 0x7fd9b8109830 2024-09-12T03:23:16.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.086+0000 7fd9b5ffb640 1 -- 172.21.15.6:0/2535929780 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b000fca0 con 0x7fd9b8109830 2024-09-12T03:23:16.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.086+0000 7fd9bee41640 1 -- 172.21.15.6:0/2535929780 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b80750a0 con 0x7fd9b8109830 2024-09-12T03:23:16.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.087+0000 7fd9bee41640 1 -- 172.21.15.6:0/2535929780 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b80812d0 con 0x7fd9b8109830 2024-09-12T03:23:16.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.089+0000 7fd99f7fe640 1 -- 172.21.15.6:0/2535929780 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9b8075300 con 0x7fd9b8109830 2024-09-12T03:23:16.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.093+0000 7fd9b5ffb640 1 -- 172.21.15.6:0/2535929780 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd9b001f070 con 0x7fd9b8109830 2024-09-12T03:23:16.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.094+0000 7fd9b5ffb640 1 --2- 172.21.15.6:0/2535929780 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd9a00a2d90 0x7fd9a00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.094+0000 7fd9b5ffb640 1 -- 172.21.15.6:0/2535929780 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd9b00cf370 con 0x7fd9b8109830 2024-09-12T03:23:16.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.097+0000 7fd9bcbdd640 1 --2- 172.21.15.6:0/2535929780 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd9a00a2d90 0x7fd9a00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.099+0000 7fd9b5ffb640 1 -- 172.21.15.6:0/2535929780 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd9b0026080 con 0x7fd9b8109830 2024-09-12T03:23:16.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.109+0000 7fd9bcbdd640 1 --2- 172.21.15.6:0/2535929780 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd9a00a2d90 0x7fd9a00a5250 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0095f0 tx=0x7fd9ac005760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:16.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.561+0000 7fd99f7fe640 1 -- 172.21.15.6:0/2535929780 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9b8109c90 con 0x7fd9b8109830 2024-09-12T03:23:16.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.574+0000 7fd9b5ffb640 1 -- 172.21.15.6:0/2535929780 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11911 (secure 0 0 0) 0x7fd9b009c320 con 0x7fd9b8109830 2024-09-12T03:23:16.578 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:23:16.578 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":26,"btime":"2024-09-12T03:22:16:132848+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:22:16.064679+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24463,24464,24494]},"id":1}]} 2024-09-12T03:23:16.578 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 26 2024-09-12T03:23:16.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.578+0000 7fd99f7fe640 1 -- 172.21.15.6:0/2535929780 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd9a00a2d90 msgr2=0x7fd9a00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:16.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.578+0000 7fd99f7fe640 1 --2- 172.21.15.6:0/2535929780 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd9a00a2d90 0x7fd9a00a5250 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0095f0 tx=0x7fd9ac005760 comp rx=0 tx=0).stop 2024-09-12T03:23:16.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.578+0000 7fd99f7fe640 1 -- 172.21.15.6:0/2535929780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9b8109830 msgr2=0x7fd9b8079910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:16.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.578+0000 7fd99f7fe640 1 --2- 172.21.15.6:0/2535929780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9b8109830 0x7fd9b8079910 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000ed80 tx=0x7fd9b000c6e0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.579+0000 7fd99f7fe640 1 -- 172.21.15.6:0/2535929780 shutdown_connections 2024-09-12T03:23:16.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.579+0000 7fd99f7fe640 1 --2- 172.21.15.6:0/2535929780 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd9a00a2d90 0x7fd9a00a5250 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.579+0000 7fd99f7fe640 1 --2- 172.21.15.6:0/2535929780 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9b811b140 0x7fd9b80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.579+0000 7fd99f7fe640 1 --2- 172.21.15.6:0/2535929780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9b8109830 0x7fd9b8079910 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.579+0000 7fd99f7fe640 1 --2- 172.21.15.6:0/2535929780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9b8109450 0x7fd9b80793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.579+0000 7fd99f7fe640 1 -- 172.21.15.6:0/2535929780 >> 172.21.15.6:0/2535929780 conn(0x7fd9b806e140 msgr2=0x7fd9b811bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:16.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.579+0000 7fd99f7fe640 1 -- 172.21.15.6:0/2535929780 shutdown_connections 2024-09-12T03:23:16.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:16.579+0000 7fd99f7fe640 1 -- 172.21.15.6:0/2535929780 wait complete. 2024-09-12T03:23:17.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:17 smithi118 ceph-mon[35048]: pgmap v400: 97 pgs: 97 active+clean; 13 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 184 MiB/s wr, 16.63k op/s 2024-09-12T03:23:17.485 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:17 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2535929780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:17.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:17 smithi006 ceph-mon[30991]: pgmap v400: 97 pgs: 97 active+clean; 13 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 184 MiB/s wr, 16.63k op/s 2024-09-12T03:23:17.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:17 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2535929780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:17.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:17 smithi161 ceph-mon[35485]: pgmap v400: 97 pgs: 97 active+clean; 13 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 184 MiB/s wr, 16.63k op/s 2024-09-12T03:23:17.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:17 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2535929780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:17.936 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:23:18.176 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:18 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:18.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:18 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:18.546 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:18.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:18 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:19 smithi006 ceph-mon[30991]: pgmap v401: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 255 MiB/s wr, 21.77k op/s 2024-09-12T03:23:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:23:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:23:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:23:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:23:19.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:19.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:19.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:23:19.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:19 smithi118 ceph-mon[35048]: pgmap v401: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 255 MiB/s wr, 21.77k op/s 2024-09-12T03:23:19.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:23:19.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:23:19.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:23:19.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:23:19.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:19.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:19.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:23:19.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.441+0000 7f44b0df2640 1 -- 172.21.15.6:0/701967892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44ac110190 msgr2=0x7f44ac06e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:19.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.441+0000 7f44b0df2640 1 --2- 172.21.15.6:0/701967892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44ac110190 0x7f44ac06e4f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f44a4009ca0 tx=0x7f44a402a8f0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.443+0000 7f44b0df2640 1 -- 172.21.15.6:0/701967892 shutdown_connections 2024-09-12T03:23:19.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.443+0000 7f44b0df2640 1 --2- 172.21.15.6:0/701967892 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44ac11b140 0x7f44ac11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.443+0000 7f44b0df2640 1 --2- 172.21.15.6:0/701967892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44ac06ea30 0x7f44ac06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.443+0000 7f44b0df2640 1 --2- 172.21.15.6:0/701967892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44ac110190 0x7f44ac06e4f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.443+0000 7f44b0df2640 1 -- 172.21.15.6:0/701967892 >> 172.21.15.6:0/701967892 conn(0x7f44ac073180 msgr2=0x7f44ac073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:19.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.443+0000 7f44b0df2640 1 -- 172.21.15.6:0/701967892 shutdown_connections 2024-09-12T03:23:19.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.443+0000 7f44b0df2640 1 -- 172.21.15.6:0/701967892 wait complete. 2024-09-12T03:23:19.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.444+0000 7f44b0df2640 1 Processor -- start 2024-09-12T03:23:19.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.444+0000 7f44b0df2640 1 -- start start 2024-09-12T03:23:19.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.444+0000 7f44b0df2640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44ac06ea30 0x7f44ac11a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.445+0000 7f44b0df2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44ac110190 0x7f44ac11ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.445+0000 7f44b0df2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44ac11b140 0x7f44ac1138c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.445+0000 7f44ab7fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44ac06ea30 0x7f44ac11a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.445+0000 7f44ab7fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44ac06ea30 0x7f44ac11a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:43336/0 (socket says 172.21.15.6:43336) 2024-09-12T03:23:19.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.445+0000 7f44ab7fe640 1 -- 172.21.15.6:0/965557966 learned_addr learned my addr 172.21.15.6:0/965557966 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:19.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.445+0000 7f44b0df2640 1 -- 172.21.15.6:0/965557966 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac120cc0 con 0x7f44ac110190 2024-09-12T03:23:19.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.445+0000 7f44b0df2640 1 -- 172.21.15.6:0/965557966 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac120b40 con 0x7f44ac06ea30 2024-09-12T03:23:19.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.445+0000 7f44b0df2640 1 -- 172.21.15.6:0/965557966 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac120e40 con 0x7f44ac11b140 2024-09-12T03:23:19.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.445+0000 7f44abfff640 1 --2- 172.21.15.6:0/965557966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44ac11b140 0x7f44ac1138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.449+0000 7f44ab7fe640 1 -- 172.21.15.6:0/965557966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44ac11b140 msgr2=0x7f44ac1138c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:19.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.449+0000 7f44ab7fe640 1 --2- 172.21.15.6:0/965557966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44ac11b140 0x7f44ac1138c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.449+0000 7f44ab7fe640 1 -- 172.21.15.6:0/965557966 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44ac110190 msgr2=0x7f44ac11ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:19.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.449+0000 7f44ab7fe640 1 --2- 172.21.15.6:0/965557966 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44ac110190 0x7f44ac11ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.449+0000 7f44ab7fe640 1 -- 172.21.15.6:0/965557966 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac114180 con 0x7f44ac06ea30 2024-09-12T03:23:19.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.449+0000 7f44abfff640 1 --2- 172.21.15.6:0/965557966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44ac11b140 0x7f44ac1138c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:19.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.450+0000 7f44ab7fe640 1 --2- 172.21.15.6:0/965557966 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44ac06ea30 0x7f44ac11a650 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f44a402f570 tx=0x7f44a4003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:19.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.451+0000 7f44a8ff9640 1 -- 172.21.15.6:0/965557966 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a4031930 con 0x7f44ac06ea30 2024-09-12T03:23:19.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.451+0000 7f44a8ff9640 1 -- 172.21.15.6:0/965557966 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44a402d070 con 0x7f44ac06ea30 2024-09-12T03:23:19.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.451+0000 7f44b0df2640 1 -- 172.21.15.6:0/965557966 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ac114410 con 0x7f44ac06ea30 2024-09-12T03:23:19.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.451+0000 7f44a8ff9640 1 -- 172.21.15.6:0/965557966 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a402fd40 con 0x7f44ac06ea30 2024-09-12T03:23:19.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.451+0000 7f44b0df2640 1 -- 172.21.15.6:0/965557966 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ac1183a0 con 0x7f44ac06ea30 2024-09-12T03:23:19.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.458+0000 7f44b442d740 1 Processor -- start 2024-09-12T03:23:19.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.458+0000 7f44b442d740 1 -- start start 2024-09-12T03:23:19.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.458+0000 7f44b442d740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563185e82d20 0x563185e81270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.458+0000 7f44b442d740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e817b0 0x563185e81bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.458+0000 7f44b442d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e8c550 0x563185e8e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.458+0000 7f44b442d740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x563185de72f0 con 0x563185e817b0 2024-09-12T03:23:19.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.458+0000 7f44b442d740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x563185d8ecf0 con 0x563185e82d20 2024-09-12T03:23:19.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.458+0000 7f44b442d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563185de32b0 con 0x563185e8c550 2024-09-12T03:23:19.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.459+0000 7f44a8ff9640 1 -- 172.21.15.6:0/965557966 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f44a40305f0 con 0x7f44ac06ea30 2024-09-12T03:23:19.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.460+0000 7f44a8ff9640 1 --2- 172.21.15.6:0/965557966 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f44980a2c40 0x7f44980a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.460+0000 7f44a8ff9640 1 -- 172.21.15.6:0/965557966 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f44a40f1990 con 0x7f44ac06ea30 2024-09-12T03:23:19.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.461+0000 7f44ab7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e817b0 0x563185e81bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.461+0000 7f44ab7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e817b0 0x563185e81bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48468/0 (socket says 172.21.15.6:48468) 2024-09-12T03:23:19.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.461+0000 7f44ab7fe640 1 -- 172.21.15.6:0/1390384370 learned_addr learned my addr 172.21.15.6:0/1390384370 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:19.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.461+0000 7f44abfff640 1 --2- 172.21.15.6:0/1390384370 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563185e82d20 0x563185e81270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.462+0000 7f44abfff640 1 --2- 172.21.15.6:0/965557966 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f44980a2c40 0x7f44980a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.462+0000 7f44ab7fe640 1 -- 172.21.15.6:0/1390384370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e8c550 msgr2=0x563185e8e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:19.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.462+0000 7f44ab7fe640 1 --2- 172.21.15.6:0/1390384370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e8c550 0x563185e8e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.462+0000 7f44ab7fe640 1 -- 172.21.15.6:0/1390384370 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563185e82d20 msgr2=0x563185e81270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:19.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.462+0000 7f44ab7fe640 1 --2- 172.21.15.6:0/1390384370 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563185e82d20 0x563185e81270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.465+0000 7f44ab7fe640 1 -- 172.21.15.6:0/1390384370 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563185d733a0 con 0x563185e817b0 2024-09-12T03:23:19.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.466+0000 7f44ab7fe640 1 --2- 172.21.15.6:0/1390384370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e817b0 0x563185e81bf0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f44a40ba620 tx=0x7f44a40be050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6792a0ee78be42ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:19.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.466+0000 7f44a1ffb640 1 -- 172.21.15.6:0/1390384370 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a40be8c0 con 0x563185e817b0 2024-09-12T03:23:19.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.466+0000 7f44a1ffb640 1 -- 172.21.15.6:0/1390384370 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44a4036030 con 0x563185e817b0 2024-09-12T03:23:19.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.467+0000 7f44b442d740 1 -- 172.21.15.6:0/1390384370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e817b0 msgr2=0x563185e81bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:19.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.467+0000 7f44b442d740 1 --2- 172.21.15.6:0/1390384370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e817b0 0x563185e81bf0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f44a40ba620 tx=0x7f44a40be050 comp rx=0 tx=0).stop 2024-09-12T03:23:19.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.468+0000 7f44abfff640 1 --2- 172.21.15.6:0/965557966 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f44980a2c40 0x7f44980a5100 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f449c009fd0 tx=0x7f449c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:19.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.468+0000 7f44b442d740 1 -- 172.21.15.6:0/1390384370 shutdown_connections 2024-09-12T03:23:19.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.468+0000 7f44b442d740 1 --2- 172.21.15.6:0/1390384370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e8c550 0x563185e8e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.468+0000 7f44b442d740 1 --2- 172.21.15.6:0/1390384370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e817b0 0x563185e81bf0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.468+0000 7f44b442d740 1 --2- 172.21.15.6:0/1390384370 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563185e82d20 0x563185e81270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.468+0000 7f44b442d740 1 -- 172.21.15.6:0/1390384370 >> 172.21.15.6:0/1390384370 conn(0x563185e4a310 msgr2=0x563185e7de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:19.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.468+0000 7f44b442d740 1 -- 172.21.15.6:0/1390384370 shutdown_connections 2024-09-12T03:23:19.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.468+0000 7f44b442d740 1 -- 172.21.15.6:0/1390384370 wait complete. 2024-09-12T03:23:19.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.469+0000 7f44b442d740 1 -- start start 2024-09-12T03:23:19.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.469+0000 7f44b442d740 1 Processor -- start 2024-09-12T03:23:19.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.469+0000 7f44b442d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e817b0 0x563185e8bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.469+0000 7f44b442d740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563185e82d20 0x563185e89220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.469+0000 7f44b442d740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e8c550 0x563185e89830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.470+0000 7f44b442d740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x563185e8f010 con 0x563185e8c550 2024-09-12T03:23:19.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.470+0000 7f44b442d740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x563185da2060 con 0x563185e82d20 2024-09-12T03:23:19.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.470+0000 7f44b442d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563185e8f190 con 0x563185e817b0 2024-09-12T03:23:19.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.470+0000 7f44b442d740 10 client.0 _ll_register_callbacks cb 0x563185af58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:19.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.470+0000 7f44ab7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e817b0 0x563185e8bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.470+0000 7f44ab7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e817b0 0x563185e8bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:58932/0 (socket says 172.21.15.6:58932) 2024-09-12T03:23:19.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.470+0000 7f44ab7fe640 1 -- 172.21.15.6:0/616795985 learned_addr learned my addr 172.21.15.6:0/616795985 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:19.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.470+0000 7f44aaffd640 1 --2- 172.21.15.6:0/616795985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563185e82d20 0x563185e89220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.471+0000 7f44abfff640 1 --2- 172.21.15.6:0/616795985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e8c550 0x563185e89830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.472+0000 7f44ab7fe640 1 -- 172.21.15.6:0/616795985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563185e82d20 msgr2=0x563185e89220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:19.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.472+0000 7f44ab7fe640 1 --2- 172.21.15.6:0/616795985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563185e82d20 0x563185e89220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.472+0000 7f44ab7fe640 1 -- 172.21.15.6:0/616795985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e8c550 msgr2=0x563185e89830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:19.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.472+0000 7f44ab7fe640 1 --2- 172.21.15.6:0/616795985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e8c550 0x563185e89830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.472+0000 7f44ab7fe640 1 -- 172.21.15.6:0/616795985 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563185b583c0 con 0x563185e817b0 2024-09-12T03:23:19.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.472+0000 7f44abfff640 1 --2- 172.21.15.6:0/616795985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e8c550 0x563185e89830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:19.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.472+0000 7f44ab7fe640 1 --2- 172.21.15.6:0/616795985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e817b0 0x563185e8bf70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f44a40c6900 tx=0x7f44a40bab00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffee955c11695ae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:19.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.472+0000 7f448affd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:23:19.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.473+0000 7f448affd640 1 -- 172.21.15.6:0/616795985 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a40d7060 con 0x563185e817b0 2024-09-12T03:23:19.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.473+0000 7f448affd640 1 -- 172.21.15.6:0/616795985 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f44a40de0b0 con 0x563185e817b0 2024-09-12T03:23:19.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.473+0000 7f44b442d740 1 -- 172.21.15.6:0/616795985 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563185df4860 con 0x563185e817b0 2024-09-12T03:23:19.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.473+0000 7f448affd640 1 -- 172.21.15.6:0/616795985 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f44a40de2b0 con 0x563185e817b0 2024-09-12T03:23:19.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.477+0000 7f44b442d740 10 client.25243 fetch_fsmap learned FSMap version 26 2024-09-12T03:23:19.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.477+0000 7f44b442d740 1 -- 172.21.15.6:0/616795985 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563185e8f010 con 0x563185e817b0 2024-09-12T03:23:19.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.481+0000 7f448affd640 1 -- 172.21.15.6:0/616795985 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563185df4860 con 0x563185e817b0 2024-09-12T03:23:19.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.481+0000 7f448affd640 1 -- 172.21.15.6:0/616795985 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f44a4004b40 con 0x563185e817b0 2024-09-12T03:23:19.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.481+0000 7f44b442d740 10 client.25243 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:23:19.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.481+0000 7f44b442d740 10 client.25243 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-12T03:23:19.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.481+0000 7f44b442d740 20 client.25243 populate_metadata read hostname 'smithi006' 2024-09-12T03:23:19.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.482+0000 7f44b442d740 1 --2- 172.21.15.6:0/616795985 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x563185e9c030 0x563185e9e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.482+0000 7f44b442d740 4 client.25243 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:23:19.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.482+0000 7f44b442d740 1 -- 172.21.15.6:0/616795985 --> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563185deed70 con 0x563185e9c030 2024-09-12T03:23:19.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.482+0000 7f44aaffd640 1 --2- 172.21.15.6:0/616795985 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x563185e9c030 0x563185e9e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.483+0000 7f44aaffd640 1 --2- 172.21.15.6:0/616795985 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x563185e9c030 0x563185e9e490 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=243110b791c13cc server_cookie=d93ed278f1e0314e in_seq=0 out_seq=0 2024-09-12T03:23:19.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.483+0000 7f448affd640 10 client.25243 ms_handle_connect on v2:172.21.15.161:6838/3645555254 2024-09-12T03:23:19.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.484+0000 7f448affd640 1 -- 172.21.15.6:0/616795985 <== mds.2 v2:172.21.15.161:6838/3645555254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563185deed70 con 0x563185e9c030 2024-09-12T03:23:19.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.484+0000 7f448affd640 10 client.25243 handle_command_reply: tid=0 2024-09-12T03:23:19.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.484+0000 7f44b442d740 1 client.25243 shutdown 2024-09-12T03:23:19.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.485+0000 7f44b442d740 1 -- 172.21.15.6:0/616795985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e817b0 msgr2=0x563185e8bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:19.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.485+0000 7f44b442d740 1 --2- 172.21.15.6:0/616795985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e817b0 0x563185e8bf70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f44a40c6900 tx=0x7f44a40bab00 comp rx=0 tx=0).stop 2024-09-12T03:23:19.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.488+0000 7f44b442d740 1 -- 172.21.15.6:0/616795985 shutdown_connections 2024-09-12T03:23:19.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.488+0000 7f44b442d740 1 --2- 172.21.15.6:0/616795985 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x563185e9c030 0x563185e9e490 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.488+0000 7f44b442d740 1 --2- 172.21.15.6:0/616795985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e8c550 0x563185e89830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.488+0000 7f44b442d740 1 --2- 172.21.15.6:0/616795985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563185e82d20 0x563185e89220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.488+0000 7f44b442d740 1 --2- 172.21.15.6:0/616795985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e817b0 0x563185e8bf70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.488+0000 7f44b442d740 1 -- 172.21.15.6:0/616795985 >> 172.21.15.6:0/616795985 conn(0x563185e4a310 msgr2=0x563185e83b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:19.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.489+0000 7f448affd640 0 client.25243 ms_handle_reset on v2:172.21.15.161:6838/3645555254 2024-09-12T03:23:19.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.489+0000 7f44b442d740 1 -- 172.21.15.6:0/616795985 shutdown_connections 2024-09-12T03:23:19.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.489+0000 7f44b442d740 1 -- 172.21.15.6:0/616795985 wait complete. 2024-09-12T03:23:19.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.489+0000 7f44b442d740 20 client.25243 trim_cache size 0 max 16384 2024-09-12T03:23:19.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.528+0000 7f44b442d740 1 Processor -- start 2024-09-12T03:23:19.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.528+0000 7f44b442d740 1 -- start start 2024-09-12T03:23:19.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.529+0000 7f44b442d740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563185e817b0 0x563185e891f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.529+0000 7f44b442d740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e82d20 0x563185e98310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.529+0000 7f44b442d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e8c550 0x563185ea3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.529+0000 7f44b442d740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f448c010ce0 con 0x563185e82d20 2024-09-12T03:23:19.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.529+0000 7f44b442d740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f448c00e5b0 con 0x563185e817b0 2024-09-12T03:23:19.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.529+0000 7f44b442d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f448c00c400 con 0x563185e8c550 2024-09-12T03:23:19.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.530+0000 7f44abfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e8c550 0x563185ea3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.530+0000 7f44abfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e8c550 0x563185ea3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:58936/0 (socket says 172.21.15.6:58936) 2024-09-12T03:23:19.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.530+0000 7f44abfff640 1 -- 172.21.15.6:0/373833923 learned_addr learned my addr 172.21.15.6:0/373833923 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:19.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.530+0000 7f44ab7fe640 1 --2- 172.21.15.6:0/373833923 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563185e817b0 0x563185e891f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.530+0000 7f44abfff640 1 -- 172.21.15.6:0/373833923 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563185e817b0 msgr2=0x563185e891f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:19.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.530+0000 7f44abfff640 1 --2- 172.21.15.6:0/373833923 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563185e817b0 0x563185e891f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.530+0000 7f44abfff640 1 -- 172.21.15.6:0/373833923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e82d20 msgr2=0x563185e98310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:19.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.530+0000 7f44abfff640 1 --2- 172.21.15.6:0/373833923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e82d20 0x563185e98310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.530+0000 7f44abfff640 1 -- 172.21.15.6:0/373833923 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563185e7eb40 con 0x563185e8c550 2024-09-12T03:23:19.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.531+0000 7f44abfff640 1 --2- 172.21.15.6:0/373833923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e8c550 0x563185ea3d60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f449c009f30 tx=0x7f449c0097b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b50d1eee0ca181b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:19.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.532+0000 7f44a17fa640 1 -- 172.21.15.6:0/373833923 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c027070 con 0x563185e8c550 2024-09-12T03:23:19.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.532+0000 7f44a17fa640 1 -- 172.21.15.6:0/373833923 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f449c01c040 con 0x563185e8c550 2024-09-12T03:23:19.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.532+0000 7f44a17fa640 1 -- 172.21.15.6:0/373833923 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c00f040 con 0x563185e8c550 2024-09-12T03:23:19.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.533+0000 7f44b442d740 1 -- 172.21.15.6:0/373833923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e8c550 msgr2=0x563185ea3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:19.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.533+0000 7f44b442d740 1 --2- 172.21.15.6:0/373833923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e8c550 0x563185ea3d60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f449c009f30 tx=0x7f449c0097b0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.533+0000 7f44b442d740 1 -- 172.21.15.6:0/373833923 shutdown_connections 2024-09-12T03:23:19.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.533+0000 7f44b442d740 1 --2- 172.21.15.6:0/373833923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e8c550 0x563185ea3d60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.533+0000 7f44b442d740 1 --2- 172.21.15.6:0/373833923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e82d20 0x563185e98310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.533+0000 7f44b442d740 1 --2- 172.21.15.6:0/373833923 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563185e817b0 0x563185e891f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.533+0000 7f44b442d740 1 -- 172.21.15.6:0/373833923 >> 172.21.15.6:0/373833923 conn(0x563185e4a310 msgr2=0x563185e88ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:19.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.533+0000 7f44b442d740 1 -- 172.21.15.6:0/373833923 shutdown_connections 2024-09-12T03:23:19.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.533+0000 7f44b442d740 1 -- 172.21.15.6:0/373833923 wait complete. 2024-09-12T03:23:19.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.534+0000 7f44b442d740 1 -- start start 2024-09-12T03:23:19.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.534+0000 7f44b442d740 1 Processor -- start 2024-09-12T03:23:19.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.535+0000 7f44b442d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e817b0 0x563185e94b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.536+0000 7f44b442d740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e82d20 0x563185e85fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.536+0000 7f44ab7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e817b0 0x563185e94b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.536+0000 7f44ab7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e817b0 0x563185e94b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:58948/0 (socket says 172.21.15.6:58948) 2024-09-12T03:23:19.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.536+0000 7f44ab7fe640 1 -- 172.21.15.6:0/76021663 learned_addr learned my addr 172.21.15.6:0/76021663 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:19.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.536+0000 7f44b442d740 1 --2- 172.21.15.6:0/76021663 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563185e8c550 0x563185e864e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.536+0000 7f44b442d740 1 -- 172.21.15.6:0/76021663 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f448c00ca70 con 0x563185e82d20 2024-09-12T03:23:19.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.536+0000 7f44b442d740 1 -- 172.21.15.6:0/76021663 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f448c00dee0 con 0x563185e8c550 2024-09-12T03:23:19.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.536+0000 7f44b442d740 1 -- 172.21.15.6:0/76021663 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f448c00d140 con 0x563185e817b0 2024-09-12T03:23:19.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.537+0000 7f44b442d740 10 client.0 _ll_register_callbacks cb 0x7f44a40c2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:19.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.537+0000 7f44aaffd640 1 --2- 172.21.15.6:0/76021663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e82d20 0x563185e85fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.538+0000 7f44abfff640 1 --2- 172.21.15.6:0/76021663 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563185e8c550 0x563185e864e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.538+0000 7f44aaffd640 1 -- 172.21.15.6:0/76021663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e817b0 msgr2=0x563185e94b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:19.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.538+0000 7f44aaffd640 1 --2- 172.21.15.6:0/76021663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e817b0 0x563185e94b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.538+0000 7f44aaffd640 1 -- 172.21.15.6:0/76021663 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563185e8c550 msgr2=0x563185e864e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:19.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.538+0000 7f44aaffd640 1 --2- 172.21.15.6:0/76021663 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563185e8c550 0x563185e864e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.538+0000 7f44aaffd640 1 -- 172.21.15.6:0/76021663 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563185e86cc0 con 0x563185e82d20 2024-09-12T03:23:19.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.538+0000 7f44abfff640 1 --2- 172.21.15.6:0/76021663 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563185e8c550 0x563185e864e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:19.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.538+0000 7f44aaffd640 1 --2- 172.21.15.6:0/76021663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e82d20 0x563185e85fa0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x563185e8a860 tx=0x7f44940143b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b0541c94033fb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:19.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.538+0000 7f448affd640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:23:19.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.539+0000 7f448affd640 1 -- 172.21.15.6:0/76021663 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494023060 con 0x563185e82d20 2024-09-12T03:23:19.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.539+0000 7f44b442d740 1 -- 172.21.15.6:0/76021663 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f449c01c040 con 0x563185e82d20 2024-09-12T03:23:19.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.539+0000 7f448affd640 1 -- 172.21.15.6:0/76021663 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f449400f040 con 0x563185e82d20 2024-09-12T03:23:19.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.539+0000 7f448affd640 1 -- 172.21.15.6:0/76021663 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4494014cc0 con 0x563185e82d20 2024-09-12T03:23:19.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.541+0000 7f44b442d740 10 client.15648 fetch_fsmap learned FSMap version 26 2024-09-12T03:23:19.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.541+0000 7f44b442d740 1 -- 172.21.15.6:0/76021663 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563185f06da0 con 0x563185e82d20 2024-09-12T03:23:19.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.542+0000 7f448affd640 1 -- 172.21.15.6:0/76021663 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f449c01c040 con 0x563185e82d20 2024-09-12T03:23:19.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.542+0000 7f448affd640 1 -- 172.21.15.6:0/76021663 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f449401d070 con 0x563185e82d20 2024-09-12T03:23:19.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.543+0000 7f44b442d740 10 client.15648 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:23:19.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.543+0000 7f44b442d740 10 client.15648 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-12T03:23:19.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.543+0000 7f44b442d740 20 client.15648 populate_metadata read hostname 'smithi006' 2024-09-12T03:23:19.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.543+0000 7f44b442d740 1 --2- 172.21.15.6:0/76021663 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x563185f07420 0x563185f09810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.543+0000 7f44b442d740 4 client.15648 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:23:19.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.543+0000 7f44b442d740 1 -- 172.21.15.6:0/76021663 --> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563185d6b170 con 0x563185f07420 2024-09-12T03:23:19.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.546+0000 7f44ab7fe640 1 --2- 172.21.15.6:0/76021663 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x563185f07420 0x563185f09810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.547+0000 7f448affd640 10 client.15648 ms_handle_connect on v2:172.21.15.161:6838/3645555254 2024-09-12T03:23:19.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.547+0000 7f44ab7fe640 1 --2- 172.21.15.6:0/76021663 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x563185f07420 0x563185f09810 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cdf00dd85e7d2dd0 server_cookie=f7e5efd3d2349fb3 in_seq=0 out_seq=0 2024-09-12T03:23:19.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.548+0000 7f448affd640 1 -- 172.21.15.6:0/76021663 <== mds.2 v2:172.21.15.161:6838/3645555254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563185d6b170 con 0x563185f07420 2024-09-12T03:23:19.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.548+0000 7f448affd640 10 client.15648 handle_command_reply: tid=0 2024-09-12T03:23:19.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.548+0000 7f44b442d740 1 client.15648 shutdown 2024-09-12T03:23:19.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.551+0000 7f44b442d740 1 -- 172.21.15.6:0/76021663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e82d20 msgr2=0x563185e85fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:19.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.551+0000 7f44b442d740 1 --2- 172.21.15.6:0/76021663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e82d20 0x563185e85fa0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x563185e8a860 tx=0x7f44940143b0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.552+0000 7f44b442d740 1 -- 172.21.15.6:0/76021663 shutdown_connections 2024-09-12T03:23:19.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.552+0000 7f44b442d740 1 --2- 172.21.15.6:0/76021663 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x563185f07420 0x563185f09810 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.552+0000 7f44b442d740 1 --2- 172.21.15.6:0/76021663 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563185e8c550 0x563185e864e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.552+0000 7f44b442d740 1 --2- 172.21.15.6:0/76021663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563185e82d20 0x563185e85fa0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.552+0000 7f44b442d740 1 --2- 172.21.15.6:0/76021663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563185e817b0 0x563185e94b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.552+0000 7f44b442d740 1 -- 172.21.15.6:0/76021663 >> 172.21.15.6:0/76021663 conn(0x563185e4a310 msgr2=0x563185e7c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:19.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.552+0000 7f448affd640 0 client.15648 ms_handle_reset on v2:172.21.15.161:6838/3645555254 2024-09-12T03:23:19.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.552+0000 7f44b442d740 1 -- 172.21.15.6:0/76021663 shutdown_connections 2024-09-12T03:23:19.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.552+0000 7f44b442d740 1 -- 172.21.15.6:0/76021663 wait complete. 2024-09-12T03:23:19.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.552+0000 7f44b442d740 20 client.15648 trim_cache size 0 max 16384 2024-09-12T03:23:19.557 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-09-12T03:23:19.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.556+0000 7f44b0df2640 1 -- 172.21.15.6:0/965557966 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f44980a2c40 msgr2=0x7f44980a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:19.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.556+0000 7f44b0df2640 1 --2- 172.21.15.6:0/965557966 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f44980a2c40 0x7f44980a5100 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f449c009fd0 tx=0x7f449c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.556+0000 7f44b0df2640 1 -- 172.21.15.6:0/965557966 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44ac06ea30 msgr2=0x7f44ac11a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:19.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.556+0000 7f44b0df2640 1 --2- 172.21.15.6:0/965557966 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44ac06ea30 0x7f44ac11a650 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f44a402f570 tx=0x7f44a4003ba0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.556+0000 7f44b0df2640 1 -- 172.21.15.6:0/965557966 shutdown_connections 2024-09-12T03:23:19.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.556+0000 7f44b0df2640 1 --2- 172.21.15.6:0/965557966 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f44980a2c40 0x7f44980a5100 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.556+0000 7f44b0df2640 1 --2- 172.21.15.6:0/965557966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44ac11b140 0x7f44ac1138c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.556+0000 7f44b0df2640 1 --2- 172.21.15.6:0/965557966 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44ac110190 0x7f44ac11ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.556+0000 7f44b0df2640 1 --2- 172.21.15.6:0/965557966 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44ac06ea30 0x7f44ac11a650 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.556+0000 7f44b0df2640 1 -- 172.21.15.6:0/965557966 >> 172.21.15.6:0/965557966 conn(0x7f44ac073180 msgr2=0x7f44ac11b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:19.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.558+0000 7f44b0df2640 1 -- 172.21.15.6:0/965557966 shutdown_connections 2024-09-12T03:23:19.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:19.558+0000 7f44b0df2640 1 -- 172.21.15.6:0/965557966 wait complete. 2024-09-12T03:23:19.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:19 smithi161 ceph-mon[35485]: pgmap v401: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 255 MiB/s wr, 21.77k op/s 2024-09-12T03:23:19.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:23:19.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:23:19.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:23:19.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:23:19.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:19.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:19.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:23:21.133 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:23:21.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:21 smithi006 ceph-mon[30991]: pgmap v402: 97 pgs: 97 active+clean; 13 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 749 KiB/s rd, 326 MiB/s wr, 26.46k op/s 2024-09-12T03:23:21.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:21 smithi118 ceph-mon[35048]: pgmap v402: 97 pgs: 97 active+clean; 13 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 749 KiB/s rd, 326 MiB/s wr, 26.46k op/s 2024-09-12T03:23:21.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:21 smithi161 ceph-mon[35485]: pgmap v402: 97 pgs: 97 active+clean; 13 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 749 KiB/s rd, 326 MiB/s wr, 26.46k op/s 2024-09-12T03:23:21.705 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:22.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.706+0000 7f4047236640 1 -- 172.21.15.6:0/2176372410 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40401011e0 msgr2=0x7f4040105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.706+0000 7f4047236640 1 --2- 172.21.15.6:0/2176372410 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40401011e0 0x7f4040105980 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f402c009860 tx=0x7f402c02a360 comp rx=0 tx=0).stop 2024-09-12T03:23:22.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.710+0000 7f4047236640 1 -- 172.21.15.6:0/2176372410 shutdown_connections 2024-09-12T03:23:22.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.710+0000 7f4047236640 1 --2- 172.21.15.6:0/2176372410 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4040105ec0 0x7f4040106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.710+0000 7f4047236640 1 --2- 172.21.15.6:0/2176372410 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40401011e0 0x7f4040105980 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.710+0000 7f4047236640 1 --2- 172.21.15.6:0/2176372410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40401008c0 0x7f4040100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.710+0000 7f4047236640 1 -- 172.21.15.6:0/2176372410 >> 172.21.15.6:0/2176372410 conn(0x7f40400fc7d0 msgr2=0x7f40400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:22.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.711+0000 7f4047236640 1 -- 172.21.15.6:0/2176372410 shutdown_connections 2024-09-12T03:23:22.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.711+0000 7f4047236640 1 -- 172.21.15.6:0/2176372410 wait complete. 2024-09-12T03:23:22.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.712+0000 7f4047236640 1 Processor -- start 2024-09-12T03:23:22.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.714+0000 7f4047236640 1 -- start start 2024-09-12T03:23:22.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.715+0000 7f4047236640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40401008c0 0x7f4040072bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.715+0000 7f4047236640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40401011e0 0x7f4040073100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.715+0000 7f4047236640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4040105ec0 0x7f4040109fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.715+0000 7f4047236640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4040118720 con 0x7f4040105ec0 2024-09-12T03:23:22.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.715+0000 7f4047236640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f40401185a0 con 0x7f40401011e0 2024-09-12T03:23:22.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.715+0000 7f4047236640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f40401188a0 con 0x7f40401008c0 2024-09-12T03:23:22.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.716+0000 7f40457d3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4040105ec0 0x7f4040109fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.716+0000 7f40457d3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4040105ec0 0x7f4040109fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48502/0 (socket says 172.21.15.6:48502) 2024-09-12T03:23:22.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.716+0000 7f40457d3640 1 -- 172.21.15.6:0/1065058367 learned_addr learned my addr 172.21.15.6:0/1065058367 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:22.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.716+0000 7f40457d3640 1 -- 172.21.15.6:0/1065058367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40401008c0 msgr2=0x7f4040072bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.716+0000 7f40457d3640 1 --2- 172.21.15.6:0/1065058367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40401008c0 0x7f4040072bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.716+0000 7f40457d3640 1 -- 172.21.15.6:0/1065058367 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40401011e0 msgr2=0x7f4040073100 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:23:22.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.716+0000 7f40457d3640 1 --2- 172.21.15.6:0/1065058367 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40401011e0 0x7f4040073100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.717+0000 7f40457d3640 1 -- 172.21.15.6:0/1065058367 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404010a6e0 con 0x7f4040105ec0 2024-09-12T03:23:22.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.717+0000 7f40457d3640 1 --2- 172.21.15.6:0/1065058367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4040105ec0 0x7f4040109fb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f403800eaa0 tx=0x7f403800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:22.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.718+0000 7f403dffb640 1 -- 172.21.15.6:0/1065058367 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403800cdb0 con 0x7f4040105ec0 2024-09-12T03:23:22.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.718+0000 7f4047236640 1 -- 172.21.15.6:0/1065058367 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404010a9d0 con 0x7f4040105ec0 2024-09-12T03:23:22.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.718+0000 7f4047236640 1 -- 172.21.15.6:0/1065058367 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4040112280 con 0x7f4040105ec0 2024-09-12T03:23:22.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.719+0000 7f403dffb640 1 -- 172.21.15.6:0/1065058367 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4038005020 con 0x7f4040105ec0 2024-09-12T03:23:22.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.719+0000 7f403dffb640 1 -- 172.21.15.6:0/1065058367 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40380116b0 con 0x7f4040105ec0 2024-09-12T03:23:22.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.723+0000 7f403dffb640 1 -- 172.21.15.6:0/1065058367 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4038004a40 con 0x7f4040105ec0 2024-09-12T03:23:22.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.725+0000 7f404960f740 1 Processor -- start 2024-09-12T03:23:22.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.725+0000 7f404960f740 1 -- start start 2024-09-12T03:23:22.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.726+0000 7f404960f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c41b112b40 0x55c41b111090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.726+0000 7f404960f740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c41b1115d0 0x55c41b111a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.726+0000 7f404960f740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b11c370 0x55c41b11e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.726+0000 7f404960f740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55c41b0772f0 con 0x55c41b1115d0 2024-09-12T03:23:22.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.726+0000 7f404960f740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55c41b01ecf0 con 0x55c41b11c370 2024-09-12T03:23:22.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.726+0000 7f403ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c41b112b40 0x55c41b111090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.726+0000 7f404960f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c41b0732b0 con 0x55c41b112b40 2024-09-12T03:23:22.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.726+0000 7f403ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c41b112b40 0x55c41b111090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:58976/0 (socket says 172.21.15.6:58976) 2024-09-12T03:23:22.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.726+0000 7f403ffff640 1 -- 172.21.15.6:0/1618068385 learned_addr learned my addr 172.21.15.6:0/1618068385 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:22.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.727+0000 7f403ffff640 1 --2- 172.21.15.6:0/1618068385 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b11c370 0x55c41b11e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.727+0000 7f403dffb640 1 --2- 172.21.15.6:0/1065058367 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f40300a2cc0 0x7f40300a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.727+0000 7f403ffff640 1 -- 172.21.15.6:0/1618068385 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b11c370 msgr2=0x55c41b11e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:22.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.727+0000 7f403ffff640 1 --2- 172.21.15.6:0/1618068385 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b11c370 0x55c41b11e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.727+0000 7f403ffff640 1 -- 172.21.15.6:0/1618068385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c41b1115d0 msgr2=0x55c41b111a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:23:22.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.727+0000 7f403ffff640 1 --2- 172.21.15.6:0/1618068385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c41b1115d0 0x55c41b111a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.727+0000 7f403ffff640 1 -- 172.21.15.6:0/1618068385 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c41b0033a0 con 0x55c41b112b40 2024-09-12T03:23:22.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.728+0000 7f403dffb640 1 -- 172.21.15.6:0/1065058367 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f40380cf0f0 con 0x7f4040105ec0 2024-09-12T03:23:22.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.728+0000 7f403ffff640 1 --2- 172.21.15.6:0/1618068385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c41b112b40 0x55c41b111090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f402c0095f0 tx=0x7f402c005c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5827b119cb1ae08a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:22.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.728+0000 7f40457d3640 1 --2- 172.21.15.6:0/1065058367 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f40300a2cc0 0x7f40300a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.729+0000 7f402affd640 1 -- 172.21.15.6:0/1618068385 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c037030 con 0x55c41b112b40 2024-09-12T03:23:22.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.729+0000 7f402affd640 1 -- 172.21.15.6:0/1618068385 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f402c030e30 con 0x55c41b112b40 2024-09-12T03:23:22.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.729+0000 7f402affd640 1 -- 172.21.15.6:0/1618068385 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c0053a0 con 0x55c41b112b40 2024-09-12T03:23:22.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.730+0000 7f404960f740 1 -- 172.21.15.6:0/1618068385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c41b112b40 msgr2=0x55c41b111090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.730+0000 7f404960f740 1 --2- 172.21.15.6:0/1618068385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c41b112b40 0x55c41b111090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f402c0095f0 tx=0x7f402c005c70 comp rx=0 tx=0).stop 2024-09-12T03:23:22.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.730+0000 7f40457d3640 1 --2- 172.21.15.6:0/1065058367 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f40300a2cc0 0x7f40300a5180 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4038000f80 tx=0x7f4038009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:22.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.738+0000 7f404960f740 1 -- 172.21.15.6:0/1618068385 shutdown_connections 2024-09-12T03:23:22.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.738+0000 7f404960f740 1 --2- 172.21.15.6:0/1618068385 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b11c370 0x55c41b11e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.738+0000 7f404960f740 1 --2- 172.21.15.6:0/1618068385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c41b1115d0 0x55c41b111a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.738+0000 7f404960f740 1 --2- 172.21.15.6:0/1618068385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c41b112b40 0x55c41b111090 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.738+0000 7f404960f740 1 -- 172.21.15.6:0/1618068385 >> 172.21.15.6:0/1618068385 conn(0x55c41b0da310 msgr2=0x55c41b10dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:22.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.739+0000 7f404960f740 1 -- 172.21.15.6:0/1618068385 shutdown_connections 2024-09-12T03:23:22.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.741+0000 7f404960f740 1 -- 172.21.15.6:0/1618068385 wait complete. 2024-09-12T03:23:22.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.742+0000 7f404960f740 1 -- start start 2024-09-12T03:23:22.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.742+0000 7f404960f740 1 Processor -- start 2024-09-12T03:23:22.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.743+0000 7f404960f740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b1115d0 0x55c41b11bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.743+0000 7f404960f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c41b112b40 0x55c41b119050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.743+0000 7f404960f740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c41b11c370 0x55c41b1197e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.743+0000 7f404960f740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55c41b11f010 con 0x55c41b11c370 2024-09-12T03:23:22.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.743+0000 7f404960f740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55c41b032060 con 0x55c41b1115d0 2024-09-12T03:23:22.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.743+0000 7f404960f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c41b11f190 con 0x55c41b112b40 2024-09-12T03:23:22.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.744+0000 7f404960f740 10 client.0 _ll_register_callbacks cb 0x55c41ad858c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:22.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.744+0000 7f403ffff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b1115d0 0x55c41b11bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.744+0000 7f403ffff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b1115d0 0x55c41b11bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:43428/0 (socket says 172.21.15.6:43428) 2024-09-12T03:23:22.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.744+0000 7f403ffff640 1 -- 172.21.15.6:0/2393813141 learned_addr learned my addr 172.21.15.6:0/2393813141 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:22.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.744+0000 7f403ffff640 1 --2- 172.21.15.6:0/2393813141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c41b11c370 0x55c41b1197e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.746+0000 7f403ffff640 1 -- 172.21.15.6:0/2393813141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c41b112b40 msgr2=0x55c41b119050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:23:22.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.746+0000 7f403ffff640 1 --2- 172.21.15.6:0/2393813141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c41b112b40 0x55c41b119050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.746+0000 7f403ffff640 1 -- 172.21.15.6:0/2393813141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c41b11c370 msgr2=0x55c41b1197e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:22.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.746+0000 7f403ffff640 1 --2- 172.21.15.6:0/2393813141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c41b11c370 0x55c41b1197e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.746+0000 7f403ffff640 1 -- 172.21.15.6:0/2393813141 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c41ade83c0 con 0x55c41b1115d0 2024-09-12T03:23:22.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.746+0000 7f403ffff640 1 --2- 172.21.15.6:0/2393813141 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b1115d0 0x55c41b11bda0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f402c005080 tx=0x7f402c0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd5da533689e8ef1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:22.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.746+0000 7f401ffff640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:23:22.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.749+0000 7f401ffff640 1 -- 172.21.15.6:0/2393813141 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c037040 con 0x55c41b1115d0 2024-09-12T03:23:22.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.749+0000 7f401ffff640 1 -- 172.21.15.6:0/2393813141 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f402c030980 con 0x55c41b1115d0 2024-09-12T03:23:22.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.749+0000 7f404960f740 1 -- 172.21.15.6:0/2393813141 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f402c030e30 con 0x55c41b1115d0 2024-09-12T03:23:22.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.750+0000 7f401ffff640 1 -- 172.21.15.6:0/2393813141 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f402c054050 con 0x55c41b1115d0 2024-09-12T03:23:22.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.751+0000 7f404960f740 10 client.25208 fetch_fsmap learned FSMap version 26 2024-09-12T03:23:22.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.751+0000 7f404960f740 1 -- 172.21.15.6:0/2393813141 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c41b12ab50 con 0x55c41b1115d0 2024-09-12T03:23:22.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.753+0000 7f401ffff640 1 -- 172.21.15.6:0/2393813141 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f402c004ba0 con 0x55c41b1115d0 2024-09-12T03:23:22.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.753+0000 7f401ffff640 1 -- 172.21.15.6:0/2393813141 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f402c04c040 con 0x55c41b1115d0 2024-09-12T03:23:22.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.753+0000 7f404960f740 10 client.25208 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:23:22.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.754+0000 7f404960f740 10 client.25208 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-12T03:23:22.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.754+0000 7f404960f740 20 client.25208 populate_metadata read hostname 'smithi006' 2024-09-12T03:23:22.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.754+0000 7f404960f740 1 --2- 172.21.15.6:0/2393813141 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55c41b12bea0 0x55c41b12e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.754+0000 7f404960f740 4 client.25208 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:23:22.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.754+0000 7f404960f740 1 -- 172.21.15.6:0/2393813141 --> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f402c030e30 con 0x55c41b12bea0 2024-09-12T03:23:22.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.757+0000 7f4044fd2640 1 --2- 172.21.15.6:0/2393813141 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55c41b12bea0 0x55c41b12e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.759+0000 7f401ffff640 10 client.25208 ms_handle_connect on v2:172.21.15.118:6836/3879191727 2024-09-12T03:23:22.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.759+0000 7f4044fd2640 1 --2- 172.21.15.6:0/2393813141 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55c41b12bea0 0x55c41b12e290 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4623cca71813ebd1 server_cookie=f0757a101f6f4169 in_seq=0 out_seq=0 2024-09-12T03:23:22.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.761+0000 7f401ffff640 1 -- 172.21.15.6:0/2393813141 <== mds.1 v2:172.21.15.118:6836/3879191727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f402c030e30 con 0x55c41b12bea0 2024-09-12T03:23:22.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.761+0000 7f401ffff640 10 client.25208 handle_command_reply: tid=0 2024-09-12T03:23:22.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.761+0000 7f404960f740 1 client.25208 shutdown 2024-09-12T03:23:22.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.763+0000 7f404960f740 1 -- 172.21.15.6:0/2393813141 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b1115d0 msgr2=0x55c41b11bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.763+0000 7f404960f740 1 --2- 172.21.15.6:0/2393813141 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b1115d0 0x55c41b11bda0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f402c005080 tx=0x7f402c0045e0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.766+0000 7f404960f740 1 -- 172.21.15.6:0/2393813141 shutdown_connections 2024-09-12T03:23:22.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.766+0000 7f404960f740 1 --2- 172.21.15.6:0/2393813141 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55c41b12bea0 0x55c41b12e290 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.766+0000 7f404960f740 1 --2- 172.21.15.6:0/2393813141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c41b11c370 0x55c41b1197e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.766+0000 7f404960f740 1 --2- 172.21.15.6:0/2393813141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c41b112b40 0x55c41b119050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.766+0000 7f404960f740 1 --2- 172.21.15.6:0/2393813141 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b1115d0 0x55c41b11bda0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.766+0000 7f404960f740 1 -- 172.21.15.6:0/2393813141 >> 172.21.15.6:0/2393813141 conn(0x55c41b0da310 msgr2=0x55c41b113930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:22.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.766+0000 7f401ffff640 0 client.25208 ms_handle_reset on v2:172.21.15.118:6836/3879191727 2024-09-12T03:23:22.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.766+0000 7f404960f740 1 -- 172.21.15.6:0/2393813141 shutdown_connections 2024-09-12T03:23:22.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.766+0000 7f404960f740 1 -- 172.21.15.6:0/2393813141 wait complete. 2024-09-12T03:23:22.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.768+0000 7f404960f740 20 client.25208 trim_cache size 0 max 16384 2024-09-12T03:23:22.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.796+0000 7f404960f740 1 Processor -- start 2024-09-12T03:23:22.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.796+0000 7f404960f740 1 -- start start 2024-09-12T03:23:22.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.797+0000 7f404960f740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c41b1115d0 0x55c41b10dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.797+0000 7f404960f740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b112b40 0x55c41b10cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.797+0000 7f403ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c41b1115d0 0x55c41b10dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.797+0000 7f403ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c41b1115d0 0x55c41b10dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48508/0 (socket says 172.21.15.6:48508) 2024-09-12T03:23:22.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.798+0000 7f403ffff640 1 -- 172.21.15.6:0/742878318 learned_addr learned my addr 172.21.15.6:0/742878318 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:22.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.798+0000 7f404960f740 1 --2- 172.21.15.6:0/742878318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c41b11c370 0x55c41b114120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.798+0000 7f4044fd2640 1 --2- 172.21.15.6:0/742878318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b112b40 0x55c41b10cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.798+0000 7f404960f740 1 -- 172.21.15.6:0/742878318 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4024010cf0 con 0x55c41b1115d0 2024-09-12T03:23:22.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.798+0000 7f404960f740 1 -- 172.21.15.6:0/742878318 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f402400e5c0 con 0x55c41b112b40 2024-09-12T03:23:22.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.799+0000 7f404960f740 1 -- 172.21.15.6:0/742878318 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f402400c410 con 0x55c41b11c370 2024-09-12T03:23:22.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.799+0000 7f403ffff640 1 --2- 172.21.15.6:0/742878318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c41b11c370 0x55c41b114120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.799+0000 7f4044fd2640 1 -- 172.21.15.6:0/742878318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c41b11c370 msgr2=0x55c41b114120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:22.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.799+0000 7f4044fd2640 1 --2- 172.21.15.6:0/742878318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c41b11c370 0x55c41b114120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.799+0000 7f4044fd2640 1 -- 172.21.15.6:0/742878318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c41b1115d0 msgr2=0x55c41b10dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:22.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.799+0000 7f4044fd2640 1 --2- 172.21.15.6:0/742878318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c41b1115d0 0x55c41b10dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.799+0000 7f4044fd2640 1 -- 172.21.15.6:0/742878318 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c41b11ecf0 con 0x55c41b112b40 2024-09-12T03:23:22.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.800+0000 7f4044fd2640 1 --2- 172.21.15.6:0/742878318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b112b40 0x55c41b10cbf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55c41b11a600 tx=0x7f4034005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df9164560662ba52 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:22.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.800+0000 7f402a7fc640 1 -- 172.21.15.6:0/742878318 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034009cf0 con 0x55c41b112b40 2024-09-12T03:23:22.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.800+0000 7f402a7fc640 1 -- 172.21.15.6:0/742878318 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4034005500 con 0x55c41b112b40 2024-09-12T03:23:22.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.800+0000 7f402a7fc640 1 -- 172.21.15.6:0/742878318 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40340172d0 con 0x55c41b112b40 2024-09-12T03:23:22.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.801+0000 7f404960f740 1 -- 172.21.15.6:0/742878318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b112b40 msgr2=0x55c41b10cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.801+0000 7f404960f740 1 --2- 172.21.15.6:0/742878318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b112b40 0x55c41b10cbf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55c41b11a600 tx=0x7f4034005850 comp rx=0 tx=0).stop 2024-09-12T03:23:22.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.801+0000 7f404960f740 1 -- 172.21.15.6:0/742878318 shutdown_connections 2024-09-12T03:23:22.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.801+0000 7f404960f740 1 --2- 172.21.15.6:0/742878318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c41b11c370 0x55c41b114120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.801+0000 7f404960f740 1 --2- 172.21.15.6:0/742878318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b112b40 0x55c41b10cbf0 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55c41b11a600 tx=0x7f4034005850 comp rx=0 tx=0).stop 2024-09-12T03:23:22.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.801+0000 7f404960f740 1 --2- 172.21.15.6:0/742878318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c41b1115d0 0x55c41b10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.801+0000 7f404960f740 1 -- 172.21.15.6:0/742878318 >> 172.21.15.6:0/742878318 conn(0x55c41b0da310 msgr2=0x55c41b10c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:22.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.802+0000 7f404960f740 1 -- 172.21.15.6:0/742878318 shutdown_connections 2024-09-12T03:23:22.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.802+0000 7f404960f740 1 -- 172.21.15.6:0/742878318 wait complete. 2024-09-12T03:23:22.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.803+0000 7f404960f740 1 -- start start 2024-09-12T03:23:22.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.803+0000 7f404960f740 1 Processor -- start 2024-09-12T03:23:22.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.804+0000 7f404960f740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c41b1115d0 0x55c41b131a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.805+0000 7f404960f740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b11c370 0x55c41b12ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.805+0000 7f404960f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c41b12f210 0x55c41b12f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.805+0000 7f404960f740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f402400ca80 con 0x55c41b1115d0 2024-09-12T03:23:22.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.805+0000 7f404960f740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f402400def0 con 0x55c41b11c370 2024-09-12T03:23:22.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.805+0000 7f404960f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f402400d150 con 0x55c41b12f210 2024-09-12T03:23:22.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.805+0000 7f404960f740 10 client.0 _ll_register_callbacks cb 0x7f4034005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:22.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.805+0000 7f4044fd2640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b11c370 0x55c41b12ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.805+0000 7f4044fd2640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b11c370 0x55c41b12ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:43448/0 (socket says 172.21.15.6:43448) 2024-09-12T03:23:22.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.805+0000 7f4044fd2640 1 -- 172.21.15.6:0/37914951 learned_addr learned my addr 172.21.15.6:0/37914951 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:22.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.806+0000 7f4044fd2640 1 -- 172.21.15.6:0/37914951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c41b12f210 msgr2=0x55c41b12f670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:23:22.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.806+0000 7f4044fd2640 1 --2- 172.21.15.6:0/37914951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c41b12f210 0x55c41b12f670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.806+0000 7f4044fd2640 1 -- 172.21.15.6:0/37914951 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c41b1115d0 msgr2=0x55c41b131a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:22.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.806+0000 7f4044fd2640 1 --2- 172.21.15.6:0/37914951 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c41b1115d0 0x55c41b131a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.806+0000 7f4044fd2640 1 -- 172.21.15.6:0/37914951 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c41b195910 con 0x55c41b11c370 2024-09-12T03:23:22.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.810+0000 7f4044fd2640 1 --2- 172.21.15.6:0/37914951 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b11c370 0x55c41b12ecd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4034009a50 tx=0x7f4034009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c625195bc2b682af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:22.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.810+0000 7f401ffff640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:23:22.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.811+0000 7f401ffff640 1 -- 172.21.15.6:0/37914951 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034009cf0 con 0x55c41b11c370 2024-09-12T03:23:22.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.812+0000 7f401ffff640 1 -- 172.21.15.6:0/37914951 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f40340047a0 con 0x55c41b11c370 2024-09-12T03:23:22.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.812+0000 7f404960f740 1 -- 172.21.15.6:0/37914951 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4034005500 con 0x55c41b11c370 2024-09-12T03:23:22.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.812+0000 7f401ffff640 1 -- 172.21.15.6:0/37914951 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4034030050 con 0x55c41b11c370 2024-09-12T03:23:22.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.814+0000 7f404960f740 10 client.25214 fetch_fsmap learned FSMap version 26 2024-09-12T03:23:22.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.814+0000 7f404960f740 1 -- 172.21.15.6:0/37914951 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c41b196c00 con 0x55c41b11c370 2024-09-12T03:23:22.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.816+0000 7f401ffff640 1 -- 172.21.15.6:0/37914951 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4034005500 con 0x55c41b11c370 2024-09-12T03:23:22.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.816+0000 7f401ffff640 1 -- 172.21.15.6:0/37914951 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f40340206e0 con 0x55c41b11c370 2024-09-12T03:23:22.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.817+0000 7f404960f740 10 client.25214 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:23:22.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.817+0000 7f404960f740 10 client.25214 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-12T03:23:22.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.817+0000 7f404960f740 20 client.25214 populate_metadata read hostname 'smithi006' 2024-09-12T03:23:22.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.817+0000 7f404960f740 1 --2- 172.21.15.6:0/37914951 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55c41b1971d0 0x55c41b1995f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.817+0000 7f404960f740 4 client.25214 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:23:22.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.817+0000 7f404960f740 1 -- 172.21.15.6:0/37914951 --> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c41b084860 con 0x55c41b1971d0 2024-09-12T03:23:22.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.820+0000 7f403ffff640 1 --2- 172.21.15.6:0/37914951 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55c41b1971d0 0x55c41b1995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.822+0000 7f403ffff640 1 --2- 172.21.15.6:0/37914951 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55c41b1971d0 0x55c41b1995f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eda6880269c05e21 server_cookie=caa9674744d69d2 in_seq=0 out_seq=0 2024-09-12T03:23:22.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.822+0000 7f401ffff640 10 client.25214 ms_handle_connect on v2:172.21.15.118:6836/3879191727 2024-09-12T03:23:22.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.822+0000 7f401ffff640 1 -- 172.21.15.6:0/37914951 <== mds.1 v2:172.21.15.118:6836/3879191727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c41b084860 con 0x55c41b1971d0 2024-09-12T03:23:22.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.822+0000 7f401ffff640 10 client.25214 handle_command_reply: tid=0 2024-09-12T03:23:22.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.822+0000 7f404960f740 1 client.25214 shutdown 2024-09-12T03:23:22.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.823+0000 7f404960f740 1 -- 172.21.15.6:0/37914951 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b11c370 msgr2=0x55c41b12ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.823+0000 7f404960f740 1 --2- 172.21.15.6:0/37914951 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b11c370 0x55c41b12ecd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4034009a50 tx=0x7f4034009780 comp rx=0 tx=0).stop 2024-09-12T03:23:22.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.823+0000 7f404960f740 1 -- 172.21.15.6:0/37914951 shutdown_connections 2024-09-12T03:23:22.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.823+0000 7f404960f740 1 --2- 172.21.15.6:0/37914951 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x55c41b1971d0 0x55c41b1995f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.823+0000 7f401ffff640 0 client.25214 ms_handle_reset on v2:172.21.15.118:6836/3879191727 2024-09-12T03:23:22.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.823+0000 7f404960f740 1 --2- 172.21.15.6:0/37914951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c41b12f210 0x55c41b12f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.823+0000 7f404960f740 1 --2- 172.21.15.6:0/37914951 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c41b11c370 0x55c41b12ecd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.823+0000 7f404960f740 1 --2- 172.21.15.6:0/37914951 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c41b1115d0 0x55c41b131a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.823+0000 7f404960f740 1 -- 172.21.15.6:0/37914951 >> 172.21.15.6:0/37914951 conn(0x55c41b0da310 msgr2=0x55c41b127ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:22.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.823+0000 7f404960f740 1 -- 172.21.15.6:0/37914951 shutdown_connections 2024-09-12T03:23:22.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.823+0000 7f404960f740 1 -- 172.21.15.6:0/37914951 wait complete. 2024-09-12T03:23:22.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.824+0000 7f404960f740 20 client.25214 trim_cache size 0 max 16384 2024-09-12T03:23:22.825 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-09-12T03:23:22.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.826+0000 7f4047236640 1 -- 172.21.15.6:0/1065058367 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f40300a2cc0 msgr2=0x7f40300a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.826+0000 7f4047236640 1 --2- 172.21.15.6:0/1065058367 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f40300a2cc0 0x7f40300a5180 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4038000f80 tx=0x7f4038009270 comp rx=0 tx=0).stop 2024-09-12T03:23:22.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.826+0000 7f4047236640 1 -- 172.21.15.6:0/1065058367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4040105ec0 msgr2=0x7f4040109fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.826+0000 7f4047236640 1 --2- 172.21.15.6:0/1065058367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4040105ec0 0x7f4040109fb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f403800eaa0 tx=0x7f403800ef60 comp rx=0 tx=0).stop 2024-09-12T03:23:22.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.826+0000 7f4047236640 1 -- 172.21.15.6:0/1065058367 shutdown_connections 2024-09-12T03:23:22.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.826+0000 7f4047236640 1 --2- 172.21.15.6:0/1065058367 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f40300a2cc0 0x7f40300a5180 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.826+0000 7f4047236640 1 --2- 172.21.15.6:0/1065058367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4040105ec0 0x7f4040109fb0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.826+0000 7f4047236640 1 --2- 172.21.15.6:0/1065058367 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40401011e0 0x7f4040073100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.826+0000 7f4047236640 1 --2- 172.21.15.6:0/1065058367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40401008c0 0x7f4040072bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.826+0000 7f4047236640 1 -- 172.21.15.6:0/1065058367 >> 172.21.15.6:0/1065058367 conn(0x7f40400fc7d0 msgr2=0x7f4040103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:22.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.828+0000 7f4047236640 1 -- 172.21.15.6:0/1065058367 shutdown_connections 2024-09-12T03:23:22.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:22.828+0000 7f4047236640 1 -- 172.21.15.6:0/1065058367 wait complete. 2024-09-12T03:23:23.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:23 smithi006 ceph-mon[30991]: pgmap v403: 97 pgs: 97 active+clean; 13 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 747 KiB/s rd, 331 MiB/s wr, 25.62k op/s 2024-09-12T03:23:23.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:23 smithi161 ceph-mon[35485]: pgmap v403: 97 pgs: 97 active+clean; 13 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 747 KiB/s rd, 331 MiB/s wr, 25.62k op/s 2024-09-12T03:23:23.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:23 smithi118 ceph-mon[35048]: pgmap v403: 97 pgs: 97 active+clean; 13 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 747 KiB/s rd, 331 MiB/s wr, 25.62k op/s 2024-09-12T03:23:24.180 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:23:24.744 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:25.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:25 smithi006 ceph-mon[30991]: pgmap v404: 97 pgs: 97 active+clean; 13 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 362 MiB/s wr, 27.55k op/s 2024-09-12T03:23:25.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:25 smithi161 ceph-mon[35485]: pgmap v404: 97 pgs: 97 active+clean; 13 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 362 MiB/s wr, 27.55k op/s 2024-09-12T03:23:25.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:25 smithi118 ceph-mon[35048]: pgmap v404: 97 pgs: 97 active+clean; 13 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 362 MiB/s wr, 27.55k op/s 2024-09-12T03:23:25.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.793+0000 7fe21c4e5640 1 -- 172.21.15.6:0/2272301985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2141083f0 msgr2=0x7fe2141087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.793+0000 7fe21c4e5640 1 --2- 172.21.15.6:0/2272301985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2141083f0 0x7fe2141087d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe210009860 tx=0x7fe21002a380 comp rx=0 tx=0).stop 2024-09-12T03:23:25.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.794+0000 7fe21c4e5640 1 -- 172.21.15.6:0/2272301985 shutdown_connections 2024-09-12T03:23:25.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.794+0000 7fe21c4e5640 1 --2- 172.21.15.6:0/2272301985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe21410d520 0x7fe2141150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.794+0000 7fe21c4e5640 1 --2- 172.21.15.6:0/2272301985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe214108da0 0x7fe21410cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.794+0000 7fe21c4e5640 1 --2- 172.21.15.6:0/2272301985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2141083f0 0x7fe2141087d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.794+0000 7fe21c4e5640 1 -- 172.21.15.6:0/2272301985 >> 172.21.15.6:0/2272301985 conn(0x7fe214077af0 msgr2=0x7fe214079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:25.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.794+0000 7fe21c4e5640 1 -- 172.21.15.6:0/2272301985 shutdown_connections 2024-09-12T03:23:25.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.795+0000 7fe21c4e5640 1 -- 172.21.15.6:0/2272301985 wait complete. 2024-09-12T03:23:25.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.796+0000 7fe21c4e5640 1 Processor -- start 2024-09-12T03:23:25.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.796+0000 7fe21c4e5640 1 -- start start 2024-09-12T03:23:25.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.796+0000 7fe21c4e5640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2141083f0 0x7fe214082900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.796+0000 7fe21c4e5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe214108da0 0x7fe21407b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.796+0000 7fe21c4e5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe21410d520 0x7fe21407bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.796+0000 7fe21c4e5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe214118610 con 0x7fe21410d520 2024-09-12T03:23:25.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.796+0000 7fe21c4e5640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe214118490 con 0x7fe2141083f0 2024-09-12T03:23:25.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.796+0000 7fe21c4e5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe214118790 con 0x7fe214108da0 2024-09-12T03:23:25.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.797+0000 7fe21a281640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2141083f0 0x7fe214082900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.797+0000 7fe21a281640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2141083f0 0x7fe214082900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:43472/0 (socket says 172.21.15.6:43472) 2024-09-12T03:23:25.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.797+0000 7fe21a281640 1 -- 172.21.15.6:0/2027837331 learned_addr learned my addr 172.21.15.6:0/2027837331 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:25.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.797+0000 7fe219a80640 1 --2- 172.21.15.6:0/2027837331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe214108da0 0x7fe21407b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.797+0000 7fe21a281640 1 -- 172.21.15.6:0/2027837331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe214108da0 msgr2=0x7fe21407b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.797+0000 7fe21a281640 1 --2- 172.21.15.6:0/2027837331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe214108da0 0x7fe21407b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.797+0000 7fe21a281640 1 -- 172.21.15.6:0/2027837331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe21410d520 msgr2=0x7fe21407bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.797+0000 7fe21a281640 1 --2- 172.21.15.6:0/2027837331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe21410d520 0x7fe21407bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.797+0000 7fe21a281640 1 -- 172.21.15.6:0/2027837331 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe21407c570 con 0x7fe2141083f0 2024-09-12T03:23:25.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.798+0000 7fe21a281640 1 --2- 172.21.15.6:0/2027837331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2141083f0 0x7fe214082900 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe21002a8f0 tx=0x7fe210033d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:25.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.798+0000 7fe20b7fe640 1 -- 172.21.15.6:0/2027837331 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210037040 con 0x7fe2141083f0 2024-09-12T03:23:25.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.798+0000 7fe21c4e5640 1 -- 172.21.15.6:0/2027837331 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe214080430 con 0x7fe2141083f0 2024-09-12T03:23:25.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.798+0000 7fe21c4e5640 1 -- 172.21.15.6:0/2027837331 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe214080960 con 0x7fe2141083f0 2024-09-12T03:23:25.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.799+0000 7fe20b7fe640 1 -- 172.21.15.6:0/2027837331 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe210040070 con 0x7fe2141083f0 2024-09-12T03:23:25.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.799+0000 7fe20b7fe640 1 -- 172.21.15.6:0/2027837331 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2100052b0 con 0x7fe2141083f0 2024-09-12T03:23:25.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.802+0000 7fe21e8be740 1 Processor -- start 2024-09-12T03:23:25.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.803+0000 7fe20b7fe640 1 -- 172.21.15.6:0/2027837331 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe2100305b0 con 0x7fe2141083f0 2024-09-12T03:23:25.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.803+0000 7fe21e8be740 1 -- start start 2024-09-12T03:23:25.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.804+0000 7fe21e8be740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5591cb112b40 0x5591cb111090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.804+0000 7fe21e8be740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 0x5591cb111a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.804+0000 7fe21e8be740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb11c370 0x5591cb11e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.804+0000 7fe21e8be740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5591cb0772f0 con 0x5591cb112b40 2024-09-12T03:23:25.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.804+0000 7fe21e8be740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5591cb01ecf0 con 0x5591cb11c370 2024-09-12T03:23:25.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.804+0000 7fe21e8be740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5591cb0732b0 con 0x5591cb1115d0 2024-09-12T03:23:25.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.804+0000 7fe219a80640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb11c370 0x5591cb11e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.804+0000 7fe219a80640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb11c370 0x5591cb11e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:43482/0 (socket says 172.21.15.6:43482) 2024-09-12T03:23:25.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.804+0000 7fe219a80640 1 -- 172.21.15.6:0/185219569 learned_addr learned my addr 172.21.15.6:0/185219569 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:25.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.804+0000 7fe21aa82640 1 --2- 172.21.15.6:0/185219569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5591cb112b40 0x5591cb111090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.804+0000 7fe21a281640 1 --2- 172.21.15.6:0/185219569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 0x5591cb111a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.804+0000 7fe219a80640 1 -- 172.21.15.6:0/185219569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 msgr2=0x5591cb111a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:25.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.804+0000 7fe219a80640 1 --2- 172.21.15.6:0/185219569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 0x5591cb111a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.805+0000 7fe219a80640 1 -- 172.21.15.6:0/185219569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5591cb112b40 msgr2=0x5591cb111090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:25.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.805+0000 7fe219a80640 1 --2- 172.21.15.6:0/185219569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5591cb112b40 0x5591cb111090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.805+0000 7fe219a80640 1 -- 172.21.15.6:0/185219569 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591cb0033a0 con 0x5591cb11c370 2024-09-12T03:23:25.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.805+0000 7fe21a281640 1 --2- 172.21.15.6:0/185219569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 0x5591cb111a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:25.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.805+0000 7fe219a80640 1 --2- 172.21.15.6:0/185219569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb11c370 0x5591cb11e760 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009ef0 tx=0x7fe20c0107e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d144bb64a9dac54 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:25.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.805+0000 7fe208ff9640 1 -- 172.21.15.6:0/185219569 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c01d070 con 0x5591cb11c370 2024-09-12T03:23:25.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.807+0000 7fe20b7fe640 1 --2- 172.21.15.6:0/2027837331 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe2000a2c40 0x7fe2000a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.808+0000 7fe20b7fe640 1 -- 172.21.15.6:0/2027837331 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe2100eeef0 con 0x7fe2141083f0 2024-09-12T03:23:25.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.808+0000 7fe208ff9640 1 -- 172.21.15.6:0/185219569 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe20c00ed40 con 0x5591cb11c370 2024-09-12T03:23:25.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.808+0000 7fe208ff9640 1 -- 172.21.15.6:0/185219569 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c0183f0 con 0x5591cb11c370 2024-09-12T03:23:25.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.811+0000 7fe21e8be740 1 -- 172.21.15.6:0/185219569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb11c370 msgr2=0x5591cb11e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.811+0000 7fe21e8be740 1 --2- 172.21.15.6:0/185219569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb11c370 0x5591cb11e760 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009ef0 tx=0x7fe20c0107e0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.811+0000 7fe21aa82640 1 --2- 172.21.15.6:0/2027837331 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe2000a2c40 0x7fe2000a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.812+0000 7fe21aa82640 1 --2- 172.21.15.6:0/2027837331 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe2000a2c40 0x7fe2000a5100 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe204009910 tx=0x7fe204006f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:25.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.813+0000 7fe21e8be740 1 -- 172.21.15.6:0/185219569 shutdown_connections 2024-09-12T03:23:25.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.813+0000 7fe21e8be740 1 --2- 172.21.15.6:0/185219569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb11c370 0x5591cb11e760 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.813+0000 7fe21e8be740 1 --2- 172.21.15.6:0/185219569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 0x5591cb111a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.813+0000 7fe21e8be740 1 --2- 172.21.15.6:0/185219569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5591cb112b40 0x5591cb111090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.813+0000 7fe21e8be740 1 -- 172.21.15.6:0/185219569 >> 172.21.15.6:0/185219569 conn(0x5591cb0da310 msgr2=0x5591cb10dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:25.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.813+0000 7fe21e8be740 1 -- 172.21.15.6:0/185219569 shutdown_connections 2024-09-12T03:23:25.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.813+0000 7fe21e8be740 1 -- 172.21.15.6:0/185219569 wait complete. 2024-09-12T03:23:25.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.814+0000 7fe21e8be740 1 -- start start 2024-09-12T03:23:25.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.814+0000 7fe21e8be740 1 Processor -- start 2024-09-12T03:23:25.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.814+0000 7fe21e8be740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 0x5591cb11bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.814+0000 7fe21e8be740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5591cb112b40 0x5591cb118f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.814+0000 7fe21e8be740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb11c370 0x5591cb1194d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.814+0000 7fe21e8be740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5591cb11f070 con 0x5591cb112b40 2024-09-12T03:23:25.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.815+0000 7fe21e8be740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5591cb032060 con 0x5591cb11c370 2024-09-12T03:23:25.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.815+0000 7fe21e8be740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5591cb11f1f0 con 0x5591cb1115d0 2024-09-12T03:23:25.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.815+0000 7fe21e8be740 10 client.0 _ll_register_callbacks cb 0x5591cad858c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:25.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.815+0000 7fe21a281640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 0x5591cb11bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.815+0000 7fe21a281640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 0x5591cb11bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:59036/0 (socket says 172.21.15.6:59036) 2024-09-12T03:23:25.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.815+0000 7fe21a281640 1 -- 172.21.15.6:0/3339373521 learned_addr learned my addr 172.21.15.6:0/3339373521 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:25.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.816+0000 7fe21a281640 1 -- 172.21.15.6:0/3339373521 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb11c370 msgr2=0x5591cb1194d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:23:25.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.816+0000 7fe21a281640 1 --2- 172.21.15.6:0/3339373521 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb11c370 0x5591cb1194d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.816+0000 7fe21a281640 1 -- 172.21.15.6:0/3339373521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5591cb112b40 msgr2=0x5591cb118f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:25.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.816+0000 7fe21a281640 1 --2- 172.21.15.6:0/3339373521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5591cb112b40 0x5591cb118f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.816+0000 7fe21a281640 1 -- 172.21.15.6:0/3339373521 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591cade83c0 con 0x5591cb1115d0 2024-09-12T03:23:25.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.817+0000 7fe21a281640 1 --2- 172.21.15.6:0/3339373521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 0x5591cb11bcc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe2100bb6b0 tx=0x7fe2100bb5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da1719dadcebfa80 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:25.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.817+0000 7fe1fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:23:25.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.817+0000 7fe1fdffb640 1 -- 172.21.15.6:0/3339373521 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2100b9860 con 0x5591cb1115d0 2024-09-12T03:23:25.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.817+0000 7fe1fdffb640 1 -- 172.21.15.6:0/3339373521 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe2100c3f00 con 0x5591cb1115d0 2024-09-12T03:23:25.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.817+0000 7fe21e8be740 1 -- 172.21.15.6:0/3339373521 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe20c00ed40 con 0x5591cb1115d0 2024-09-12T03:23:25.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.818+0000 7fe1fdffb640 1 -- 172.21.15.6:0/3339373521 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe2100a60b0 con 0x5591cb1115d0 2024-09-12T03:23:25.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.819+0000 7fe21e8be740 10 client.25264 fetch_fsmap learned FSMap version 26 2024-09-12T03:23:25.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.819+0000 7fe21e8be740 1 -- 172.21.15.6:0/3339373521 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5591cb11f070 con 0x5591cb1115d0 2024-09-12T03:23:25.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.820+0000 7fe1fdffb640 1 -- 172.21.15.6:0/3339373521 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe20c00ed40 con 0x5591cb1115d0 2024-09-12T03:23:25.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.821+0000 7fe1fdffb640 1 -- 172.21.15.6:0/3339373521 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe21003a070 con 0x5591cb1115d0 2024-09-12T03:23:25.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.822+0000 7fe21e8be740 10 client.25264 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:23:25.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.822+0000 7fe21e8be740 10 client.25264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:23:25.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.822+0000 7fe21e8be740 20 client.25264 populate_metadata read hostname 'smithi006' 2024-09-12T03:23:25.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.822+0000 7fe21e8be740 1 --2- 172.21.15.6:0/3339373521 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5591cb12c0b0 0x5591cb12e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.822+0000 7fe21e8be740 4 client.25264 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:23:25.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.822+0000 7fe21e8be740 1 -- 172.21.15.6:0/3339373521 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591cb084860 con 0x5591cb12c0b0 2024-09-12T03:23:25.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.823+0000 7fe219a80640 1 --2- 172.21.15.6:0/3339373521 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5591cb12c0b0 0x5591cb12e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.824+0000 7fe219a80640 1 --2- 172.21.15.6:0/3339373521 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5591cb12c0b0 0x5591cb12e570 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a04c6edff2d71cb8 server_cookie=2255c7d0520bd8a4 in_seq=0 out_seq=0 2024-09-12T03:23:25.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.825+0000 7fe1fdffb640 10 client.25264 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:25.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.829+0000 7fe1fdffb640 1 -- 172.21.15.6:0/3339373521 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5591cb084860 con 0x5591cb12c0b0 2024-09-12T03:23:25.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.829+0000 7fe1fdffb640 10 client.25264 handle_command_reply: tid=0 2024-09-12T03:23:25.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.830+0000 7fe21e8be740 1 client.25264 shutdown 2024-09-12T03:23:25.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.830+0000 7fe21e8be740 1 -- 172.21.15.6:0/3339373521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 msgr2=0x5591cb11bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.830+0000 7fe21e8be740 1 --2- 172.21.15.6:0/3339373521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 0x5591cb11bcc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe2100bb6b0 tx=0x7fe2100bb5b0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.830+0000 7fe21e8be740 1 -- 172.21.15.6:0/3339373521 shutdown_connections 2024-09-12T03:23:25.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.830+0000 7fe21e8be740 1 --2- 172.21.15.6:0/3339373521 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5591cb12c0b0 0x5591cb12e570 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.830+0000 7fe21e8be740 1 --2- 172.21.15.6:0/3339373521 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb11c370 0x5591cb1194d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.830+0000 7fe21e8be740 1 --2- 172.21.15.6:0/3339373521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5591cb112b40 0x5591cb118f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.830+0000 7fe21e8be740 1 --2- 172.21.15.6:0/3339373521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 0x5591cb11bcc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.830+0000 7fe21e8be740 1 -- 172.21.15.6:0/3339373521 >> 172.21.15.6:0/3339373521 conn(0x5591cb0da310 msgr2=0x5591cb113930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:25.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.830+0000 7fe1fdffb640 0 client.25264 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:25.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.831+0000 7fe21e8be740 1 -- 172.21.15.6:0/3339373521 shutdown_connections 2024-09-12T03:23:25.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.832+0000 7fe21e8be740 1 -- 172.21.15.6:0/3339373521 wait complete. 2024-09-12T03:23:25.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.832+0000 7fe21e8be740 20 client.25264 trim_cache size 0 max 16384 2024-09-12T03:23:25.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.860+0000 7fe21e8be740 1 Processor -- start 2024-09-12T03:23:25.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.860+0000 7fe21e8be740 1 -- start start 2024-09-12T03:23:25.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.861+0000 7fe21e8be740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 0x5591cb1283a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.861+0000 7fe21e8be740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb112b40 0x5591cb117840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.862+0000 7fe21e8be740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5591cb11c370 0x5591cb117d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.862+0000 7fe21a281640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 0x5591cb1283a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.862+0000 7fe21a281640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 0x5591cb1283a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:59048/0 (socket says 172.21.15.6:59048) 2024-09-12T03:23:25.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.865+0000 7fe219a80640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb112b40 0x5591cb117840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.865+0000 7fe219a80640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb112b40 0x5591cb117840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:43492/0 (socket says 172.21.15.6:43492) 2024-09-12T03:23:25.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.865+0000 7fe21a281640 1 -- 172.21.15.6:0/4146120318 learned_addr learned my addr 172.21.15.6:0/4146120318 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:25.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.866+0000 7fe21aa82640 1 --2- 172.21.15.6:0/4146120318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5591cb11c370 0x5591cb117d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.866+0000 7fe21e8be740 1 -- 172.21.15.6:0/4146120318 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f8010b10 con 0x5591cb11c370 2024-09-12T03:23:25.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.866+0000 7fe21e8be740 1 -- 172.21.15.6:0/4146120318 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f800e3e0 con 0x5591cb112b40 2024-09-12T03:23:25.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.866+0000 7fe21e8be740 1 -- 172.21.15.6:0/4146120318 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f800c2e0 con 0x5591cb1115d0 2024-09-12T03:23:25.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.867+0000 7fe21aa82640 1 -- 172.21.15.6:0/4146120318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 msgr2=0x5591cb1283a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:25.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.867+0000 7fe21aa82640 1 --2- 172.21.15.6:0/4146120318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 0x5591cb1283a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.867+0000 7fe21aa82640 1 -- 172.21.15.6:0/4146120318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb112b40 msgr2=0x5591cb117840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:25.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.867+0000 7fe21aa82640 1 --2- 172.21.15.6:0/4146120318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb112b40 0x5591cb117840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.867+0000 7fe21aa82640 1 -- 172.21.15.6:0/4146120318 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591cb10df20 con 0x5591cb11c370 2024-09-12T03:23:25.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.869+0000 7fe21aa82640 1 --2- 172.21.15.6:0/4146120318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5591cb11c370 0x5591cb117d80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x5591cb10ef00 tx=0x7fe204011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2df1011d9c2ef0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:25.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.869+0000 7fe1fffff640 1 -- 172.21.15.6:0/4146120318 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20401b040 con 0x5591cb11c370 2024-09-12T03:23:25.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.870+0000 7fe1fffff640 1 -- 172.21.15.6:0/4146120318 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe204024070 con 0x5591cb11c370 2024-09-12T03:23:25.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.871+0000 7fe1fffff640 1 -- 172.21.15.6:0/4146120318 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20401f3f0 con 0x5591cb11c370 2024-09-12T03:23:25.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.872+0000 7fe21e8be740 1 -- 172.21.15.6:0/4146120318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5591cb11c370 msgr2=0x5591cb117d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.872+0000 7fe21e8be740 1 --2- 172.21.15.6:0/4146120318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5591cb11c370 0x5591cb117d80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x5591cb10ef00 tx=0x7fe204011000 comp rx=0 tx=0).stop 2024-09-12T03:23:25.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.872+0000 7fe21e8be740 1 -- 172.21.15.6:0/4146120318 shutdown_connections 2024-09-12T03:23:25.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.873+0000 7fe21e8be740 1 --2- 172.21.15.6:0/4146120318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5591cb11c370 0x5591cb117d80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.873+0000 7fe21e8be740 1 --2- 172.21.15.6:0/4146120318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb112b40 0x5591cb117840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.873+0000 7fe21e8be740 1 --2- 172.21.15.6:0/4146120318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 0x5591cb1283a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.873+0000 7fe21e8be740 1 -- 172.21.15.6:0/4146120318 >> 172.21.15.6:0/4146120318 conn(0x5591cb0da310 msgr2=0x5591cb114260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:25.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.875+0000 7fe21e8be740 1 -- 172.21.15.6:0/4146120318 shutdown_connections 2024-09-12T03:23:25.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.876+0000 7fe21e8be740 1 -- 172.21.15.6:0/4146120318 wait complete. 2024-09-12T03:23:25.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.876+0000 7fe21e8be740 1 -- start start 2024-09-12T03:23:25.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.877+0000 7fe21e8be740 1 Processor -- start 2024-09-12T03:23:25.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.877+0000 7fe21e8be740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 0x5591cb124a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.878+0000 7fe21e8be740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb112b40 0x5591cb11fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.878+0000 7fe21e8be740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5591cb11c370 0x5591cb1202b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.878+0000 7fe21e8be740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f800c8a0 con 0x5591cb11c370 2024-09-12T03:23:25.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.878+0000 7fe21e8be740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f800dd10 con 0x5591cb112b40 2024-09-12T03:23:25.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.878+0000 7fe21e8be740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f800cf70 con 0x5591cb1115d0 2024-09-12T03:23:25.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.879+0000 7fe21a281640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 0x5591cb124a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.879+0000 7fe21a281640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 0x5591cb124a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:59058/0 (socket says 172.21.15.6:59058) 2024-09-12T03:23:25.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.879+0000 7fe21a281640 1 -- 172.21.15.6:0/1544775907 learned_addr learned my addr 172.21.15.6:0/1544775907 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:25.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.881+0000 7fe21aa82640 1 --2- 172.21.15.6:0/1544775907 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5591cb11c370 0x5591cb1202b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.881+0000 7fe219a80640 1 --2- 172.21.15.6:0/1544775907 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb112b40 0x5591cb11fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.881+0000 7fe21e8be740 10 client.0 _ll_register_callbacks cb 0x7fe2100097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:25.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.882+0000 7fe219a80640 1 -- 172.21.15.6:0/1544775907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 msgr2=0x5591cb124a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:25.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.882+0000 7fe219a80640 1 --2- 172.21.15.6:0/1544775907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 0x5591cb124a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.882+0000 7fe219a80640 1 -- 172.21.15.6:0/1544775907 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5591cb11c370 msgr2=0x5591cb1202b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:25.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.882+0000 7fe219a80640 1 --2- 172.21.15.6:0/1544775907 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5591cb11c370 0x5591cb1202b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.882+0000 7fe219a80640 1 -- 172.21.15.6:0/1544775907 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591cb1208a0 con 0x5591cb112b40 2024-09-12T03:23:25.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.883+0000 7fe219a80640 1 --2- 172.21.15.6:0/1544775907 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb112b40 0x5591cb11fd70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x5591cb118800 tx=0x7fe20c009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fbf79c2a8a85d96 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:25.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.883+0000 7fe1fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:23:25.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.883+0000 7fe1fdffb640 1 -- 172.21.15.6:0/1544775907 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c01d040 con 0x5591cb112b40 2024-09-12T03:23:25.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.884+0000 7fe1fdffb640 1 -- 172.21.15.6:0/1544775907 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe20c00cad0 con 0x5591cb112b40 2024-09-12T03:23:25.889 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.888+0000 7fe21e8be740 1 -- 172.21.15.6:0/1544775907 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe204024070 con 0x5591cb112b40 2024-09-12T03:23:25.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.890+0000 7fe1fdffb640 1 -- 172.21.15.6:0/1544775907 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe20c00c680 con 0x5591cb112b40 2024-09-12T03:23:25.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.895+0000 7fe21e8be740 10 client.25229 fetch_fsmap learned FSMap version 26 2024-09-12T03:23:25.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.895+0000 7fe21e8be740 1 -- 172.21.15.6:0/1544775907 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5591cb196c30 con 0x5591cb112b40 2024-09-12T03:23:25.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.897+0000 7fe1fdffb640 1 -- 172.21.15.6:0/1544775907 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe204024070 con 0x5591cb112b40 2024-09-12T03:23:25.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.897+0000 7fe1fdffb640 1 -- 172.21.15.6:0/1544775907 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe20c017030 con 0x5591cb112b40 2024-09-12T03:23:25.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.897+0000 7fe21e8be740 10 client.25229 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:23:25.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.897+0000 7fe21e8be740 10 client.25229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:23:25.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.897+0000 7fe21e8be740 20 client.25229 populate_metadata read hostname 'smithi006' 2024-09-12T03:23:25.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.897+0000 7fe21e8be740 1 --2- 172.21.15.6:0/1544775907 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5591cb197110 0x5591cb199530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.897+0000 7fe21e8be740 4 client.25229 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:23:25.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.898+0000 7fe21e8be740 1 -- 172.21.15.6:0/1544775907 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591cb07ed70 con 0x5591cb197110 2024-09-12T03:23:25.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.900+0000 7fe21a281640 1 --2- 172.21.15.6:0/1544775907 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5591cb197110 0x5591cb199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.903+0000 7fe21a281640 1 --2- 172.21.15.6:0/1544775907 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5591cb197110 0x5591cb199530 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d915b27bda9e7b02 server_cookie=8d4f67027aac6ab7 in_seq=0 out_seq=0 2024-09-12T03:23:25.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.903+0000 7fe1fdffb640 10 client.25229 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:25.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.912+0000 7fe1fdffb640 1 -- 172.21.15.6:0/1544775907 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591cb07ed70 con 0x5591cb197110 2024-09-12T03:23:25.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.912+0000 7fe1fdffb640 10 client.25229 handle_command_reply: tid=0 2024-09-12T03:23:25.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.912+0000 7fe21e8be740 1 client.25229 shutdown 2024-09-12T03:23:25.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.912+0000 7fe21e8be740 1 -- 172.21.15.6:0/1544775907 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb112b40 msgr2=0x5591cb11fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.912+0000 7fe21e8be740 1 --2- 172.21.15.6:0/1544775907 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb112b40 0x5591cb11fd70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x5591cb118800 tx=0x7fe20c009560 comp rx=0 tx=0).stop 2024-09-12T03:23:25.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.912+0000 7fe21e8be740 1 -- 172.21.15.6:0/1544775907 shutdown_connections 2024-09-12T03:23:25.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.912+0000 7fe21e8be740 1 --2- 172.21.15.6:0/1544775907 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5591cb197110 0x5591cb199530 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.913+0000 7fe1fdffb640 0 client.25229 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:25.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.913+0000 7fe21e8be740 1 --2- 172.21.15.6:0/1544775907 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5591cb11c370 0x5591cb1202b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.913+0000 7fe21e8be740 1 --2- 172.21.15.6:0/1544775907 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5591cb112b40 0x5591cb11fd70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.921 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-09-12T03:23:25.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.913+0000 7fe21e8be740 1 --2- 172.21.15.6:0/1544775907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591cb1115d0 0x5591cb124a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.913+0000 7fe21e8be740 1 -- 172.21.15.6:0/1544775907 >> 172.21.15.6:0/1544775907 conn(0x5591cb0da310 msgr2=0x5591cb117800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:25.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.913+0000 7fe21e8be740 1 -- 172.21.15.6:0/1544775907 shutdown_connections 2024-09-12T03:23:25.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.913+0000 7fe21e8be740 1 -- 172.21.15.6:0/1544775907 wait complete. 2024-09-12T03:23:25.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.913+0000 7fe21e8be740 20 client.25229 trim_cache size 0 max 16384 2024-09-12T03:23:25.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.920+0000 7fe21c4e5640 1 -- 172.21.15.6:0/2027837331 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe2000a2c40 msgr2=0x7fe2000a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.920+0000 7fe21c4e5640 1 --2- 172.21.15.6:0/2027837331 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe2000a2c40 0x7fe2000a5100 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe204009910 tx=0x7fe204006f90 comp rx=0 tx=0).stop 2024-09-12T03:23:25.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.920+0000 7fe21c4e5640 1 -- 172.21.15.6:0/2027837331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2141083f0 msgr2=0x7fe214082900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.920+0000 7fe21c4e5640 1 --2- 172.21.15.6:0/2027837331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2141083f0 0x7fe214082900 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe21002a8f0 tx=0x7fe210033d10 comp rx=0 tx=0).stop 2024-09-12T03:23:25.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.920+0000 7fe21c4e5640 1 -- 172.21.15.6:0/2027837331 shutdown_connections 2024-09-12T03:23:25.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.920+0000 7fe21c4e5640 1 --2- 172.21.15.6:0/2027837331 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe2000a2c40 0x7fe2000a5100 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.920+0000 7fe21c4e5640 1 --2- 172.21.15.6:0/2027837331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe21410d520 0x7fe21407bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.920+0000 7fe21c4e5640 1 --2- 172.21.15.6:0/2027837331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe214108da0 0x7fe21407b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.920+0000 7fe21c4e5640 1 --2- 172.21.15.6:0/2027837331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2141083f0 0x7fe214082900 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.920+0000 7fe21c4e5640 1 -- 172.21.15.6:0/2027837331 >> 172.21.15.6:0/2027837331 conn(0x7fe214077af0 msgr2=0x7fe214078f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:25.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.921+0000 7fe21c4e5640 1 -- 172.21.15.6:0/2027837331 shutdown_connections 2024-09-12T03:23:25.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:25.921+0000 7fe21c4e5640 1 -- 172.21.15.6:0/2027837331 wait complete. 2024-09-12T03:23:27.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:27 smithi006 ceph-mon[30991]: pgmap v405: 97 pgs: 97 active+clean; 13 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 579 KiB/s rd, 319 MiB/s wr, 23.80k op/s 2024-09-12T03:23:27.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:27 smithi161 ceph-mon[35485]: pgmap v405: 97 pgs: 97 active+clean; 13 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 579 KiB/s rd, 319 MiB/s wr, 23.80k op/s 2024-09-12T03:23:27.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:27 smithi118 ceph-mon[35048]: pgmap v405: 97 pgs: 97 active+clean; 13 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 579 KiB/s rd, 319 MiB/s wr, 23.80k op/s 2024-09-12T03:23:28.184 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:23:28.184 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:23:28.790 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:29.237 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:29 smithi006 ceph-mon[30991]: pgmap v406: 97 pgs: 97 active+clean; 13 GiB data, 54 GiB used, 1018 GiB / 1.0 TiB avail; 578 KiB/s rd, 343 MiB/s wr, 26.41k op/s 2024-09-12T03:23:29.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:29 smithi161 ceph-mon[35485]: pgmap v406: 97 pgs: 97 active+clean; 13 GiB data, 54 GiB used, 1018 GiB / 1.0 TiB avail; 578 KiB/s rd, 343 MiB/s wr, 26.41k op/s 2024-09-12T03:23:29.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:29 smithi118 ceph-mon[35048]: pgmap v406: 97 pgs: 97 active+clean; 13 GiB data, 54 GiB used, 1018 GiB / 1.0 TiB avail; 578 KiB/s rd, 343 MiB/s wr, 26.41k op/s 2024-09-12T03:23:30.008 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:23:30.008 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2024-09-12T03:23:30.008 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "2b96baee-7790-41fc-bcdb-7a0b692990e8", 2024-09-12T03:23:30.008 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2024-09-12T03:23:30.008 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:23:30.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.804+0000 7fc2864b2640 1 -- 172.21.15.6:0/2028763216 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc280102d30 msgr2=0x7fc280103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.804+0000 7fc2864b2640 1 --2- 172.21.15.6:0/2028763216 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc280102d30 0x7fc280103110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc278009be0 tx=0x7fc27802e910 comp rx=0 tx=0).stop 2024-09-12T03:23:30.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.811+0000 7fc2864b2640 1 -- 172.21.15.6:0/2028763216 shutdown_connections 2024-09-12T03:23:30.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.811+0000 7fc2864b2640 1 --2- 172.21.15.6:0/2028763216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc28010c160 0x7fc28010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.811+0000 7fc2864b2640 1 --2- 172.21.15.6:0/2028763216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc280103650 0x7fc28010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.811+0000 7fc2864b2640 1 --2- 172.21.15.6:0/2028763216 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc280102d30 0x7fc280103110 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.811+0000 7fc2864b2640 1 -- 172.21.15.6:0/2028763216 >> 172.21.15.6:0/2028763216 conn(0x7fc2800fc7d0 msgr2=0x7fc2800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:30.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.814+0000 7fc2864b2640 1 -- 172.21.15.6:0/2028763216 shutdown_connections 2024-09-12T03:23:30.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.814+0000 7fc2864b2640 1 -- 172.21.15.6:0/2028763216 wait complete. 2024-09-12T03:23:30.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.815+0000 7fc2864b2640 1 Processor -- start 2024-09-12T03:23:30.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.816+0000 7fc2864b2640 1 -- start start 2024-09-12T03:23:30.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.817+0000 7fc2864b2640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc280102d30 0x7fc2800739d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.817+0000 7fc2864b2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc280103650 0x7fc28006ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.817+0000 7fc2864b2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc28010c160 0x7fc28006cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.817+0000 7fc2864b2640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc28010fdc0 con 0x7fc280103650 2024-09-12T03:23:30.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.817+0000 7fc2864b2640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fc28010fc40 con 0x7fc280102d30 2024-09-12T03:23:30.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.817+0000 7fc2864b2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc28010ff40 con 0x7fc28010c160 2024-09-12T03:23:30.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.818+0000 7fc27f7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc280103650 0x7fc28006ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.818+0000 7fc27f7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc280103650 0x7fc28006ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54486/0 (socket says 172.21.15.6:54486) 2024-09-12T03:23:30.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.818+0000 7fc27f7fe640 1 -- 172.21.15.6:0/4090177420 learned_addr learned my addr 172.21.15.6:0/4090177420 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:30.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.818+0000 7fc284a4f640 1 --2- 172.21.15.6:0/4090177420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc28010c160 0x7fc28006cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.819+0000 7fc284a4f640 1 -- 172.21.15.6:0/4090177420 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc280102d30 msgr2=0x7fc2800739d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:23:30.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.819+0000 7fc284a4f640 1 --2- 172.21.15.6:0/4090177420 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc280102d30 0x7fc2800739d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.819+0000 7fc284a4f640 1 -- 172.21.15.6:0/4090177420 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc280103650 msgr2=0x7fc28006ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.819+0000 7fc284a4f640 1 --2- 172.21.15.6:0/4090177420 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc280103650 0x7fc28006ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.819+0000 7fc284a4f640 1 -- 172.21.15.6:0/4090177420 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28006d530 con 0x7fc28010c160 2024-09-12T03:23:30.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.820+0000 7fc284a4f640 1 --2- 172.21.15.6:0/4090177420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc28010c160 0x7fc28006cfa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc274009e70 tx=0x7fc27400ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:30.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.822+0000 7fc27d7fa640 1 -- 172.21.15.6:0/4090177420 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274004020 con 0x7fc28010c160 2024-09-12T03:23:30.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.822+0000 7fc2864b2640 1 -- 172.21.15.6:0/4090177420 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc28006d820 con 0x7fc28010c160 2024-09-12T03:23:30.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.822+0000 7fc2864b2640 1 -- 172.21.15.6:0/4090177420 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc280071810 con 0x7fc28010c160 2024-09-12T03:23:30.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.822+0000 7fc27d7fa640 1 -- 172.21.15.6:0/4090177420 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc27400cd90 con 0x7fc28010c160 2024-09-12T03:23:30.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.823+0000 7fc27d7fa640 1 -- 172.21.15.6:0/4090177420 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274023020 con 0x7fc28010c160 2024-09-12T03:23:30.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.826+0000 7fc27d7fa640 1 -- 172.21.15.6:0/4090177420 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc274026020 con 0x7fc28010c160 2024-09-12T03:23:30.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.837+0000 7fc27d7fa640 1 --2- 172.21.15.6:0/4090177420 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc2680a2c40 0x7fc2680a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.837+0000 7fc27d7fa640 1 -- 172.21.15.6:0/4090177420 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fc2740d85f0 con 0x7fc28010c160 2024-09-12T03:23:30.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.839+0000 7fc27ffff640 1 --2- 172.21.15.6:0/4090177420 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc2680a2c40 0x7fc2680a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.839+0000 7fc28888b740 1 Processor -- start 2024-09-12T03:23:30.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.840+0000 7fc27ffff640 1 --2- 172.21.15.6:0/4090177420 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc2680a2c40 0x7fc2680a5100 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc2780099a0 tx=0x7fc278005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:30.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.844+0000 7fc28888b740 1 -- start start 2024-09-12T03:23:30.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.844+0000 7fc28888b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddf4b60 0x564cdddf4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.844+0000 7fc28888b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564cdddf5480 0x564cdddff800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.844+0000 7fc28888b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddffd40 0x564cdde02130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.844+0000 7fc28888b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x564cddd5a280 con 0x564cdddf5480 2024-09-12T03:23:30.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.844+0000 7fc28888b740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x564cddd01b10 con 0x564cdddf4b60 2024-09-12T03:23:30.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.844+0000 7fc28888b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564cddd56240 con 0x564cdddffd40 2024-09-12T03:23:30.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.848+0000 7fc284a4f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddffd40 0x564cdde02130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.848+0000 7fc284a4f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddffd40 0x564cdde02130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:52030/0 (socket says 172.21.15.6:52030) 2024-09-12T03:23:30.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.848+0000 7fc284a4f640 1 -- 172.21.15.6:0/1506044996 learned_addr learned my addr 172.21.15.6:0/1506044996 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:30.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.849+0000 7fc284a4f640 1 -- 172.21.15.6:0/1506044996 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddf4b60 msgr2=0x564cdddf4f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:23:30.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.849+0000 7fc284a4f640 1 --2- 172.21.15.6:0/1506044996 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddf4b60 0x564cdddf4f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.849+0000 7fc284a4f640 1 -- 172.21.15.6:0/1506044996 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564cdddf5480 msgr2=0x564cdddff800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:30.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.849+0000 7fc284a4f640 1 --2- 172.21.15.6:0/1506044996 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564cdddf5480 0x564cdddff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.849+0000 7fc284a4f640 1 -- 172.21.15.6:0/1506044996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cddd0e1f0 con 0x564cdddffd40 2024-09-12T03:23:30.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.849+0000 7fc284a4f640 1 --2- 172.21.15.6:0/1506044996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddffd40 0x564cdde02130 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc27400c6a0 tx=0x7fc2740a4780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9700165cb17a7451 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:30.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.852+0000 7fc2667fc640 1 -- 172.21.15.6:0/1506044996 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2740a91d0 con 0x564cdddffd40 2024-09-12T03:23:30.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.852+0000 7fc2667fc640 1 -- 172.21.15.6:0/1506044996 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2740ada90 con 0x564cdddffd40 2024-09-12T03:23:30.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.853+0000 7fc28888b740 1 -- 172.21.15.6:0/1506044996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddffd40 msgr2=0x564cdde02130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.853+0000 7fc28888b740 1 --2- 172.21.15.6:0/1506044996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddffd40 0x564cdde02130 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc27400c6a0 tx=0x7fc2740a4780 comp rx=0 tx=0).stop 2024-09-12T03:23:30.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.852+0000 7fc2667fc640 1 -- 172.21.15.6:0/1506044996 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2740a52e0 con 0x564cdddffd40 2024-09-12T03:23:30.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.853+0000 7fc28888b740 1 -- 172.21.15.6:0/1506044996 shutdown_connections 2024-09-12T03:23:30.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.853+0000 7fc28888b740 1 --2- 172.21.15.6:0/1506044996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddffd40 0x564cdde02130 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.853+0000 7fc28888b740 1 --2- 172.21.15.6:0/1506044996 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564cdddf5480 0x564cdddff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.853+0000 7fc28888b740 1 --2- 172.21.15.6:0/1506044996 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddf4b60 0x564cdddf4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.853+0000 7fc28888b740 1 -- 172.21.15.6:0/1506044996 >> 172.21.15.6:0/1506044996 conn(0x564cdddf10f0 msgr2=0x564cdddf14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:30.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.860+0000 7fc28888b740 1 -- 172.21.15.6:0/1506044996 shutdown_connections 2024-09-12T03:23:30.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.860+0000 7fc28888b740 1 -- 172.21.15.6:0/1506044996 wait complete. 2024-09-12T03:23:30.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.860+0000 7fc28888b740 1 -- start start 2024-09-12T03:23:30.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.861+0000 7fc28888b740 1 Processor -- start 2024-09-12T03:23:30.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.861+0000 7fc28888b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddf4b60 0x564cdddfcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.861+0000 7fc28888b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564cdddf5480 0x564cdddfd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.861+0000 7fc28888b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddffd40 0x564cdddfd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.861+0000 7fc28888b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x564cdde02960 con 0x564cdddf5480 2024-09-12T03:23:30.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.861+0000 7fc28888b740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x564cddd14f10 con 0x564cdddffd40 2024-09-12T03:23:30.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.861+0000 7fc28888b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564cdde02ae0 con 0x564cdddf4b60 2024-09-12T03:23:30.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.861+0000 7fc28888b740 10 client.0 _ll_register_callbacks cb 0x564cddcdda40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:30.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.862+0000 7fc284a4f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddffd40 0x564cdddfd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.862+0000 7fc284a4f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddffd40 0x564cdddfd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:55850/0 (socket says 172.21.15.6:55850) 2024-09-12T03:23:30.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.862+0000 7fc284a4f640 1 -- 172.21.15.6:0/728082285 learned_addr learned my addr 172.21.15.6:0/728082285 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:30.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.863+0000 7fc284a4f640 1 -- 172.21.15.6:0/728082285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddf4b60 msgr2=0x564cdddfcbc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:23:30.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.863+0000 7fc284a4f640 1 --2- 172.21.15.6:0/728082285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddf4b60 0x564cdddfcbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.863+0000 7fc284a4f640 1 -- 172.21.15.6:0/728082285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564cdddf5480 msgr2=0x564cdddfd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:30.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.863+0000 7fc284a4f640 1 --2- 172.21.15.6:0/728082285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564cdddf5480 0x564cdddfd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.863+0000 7fc284a4f640 1 -- 172.21.15.6:0/728082285 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cddacb6a0 con 0x564cdddffd40 2024-09-12T03:23:30.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.863+0000 7fc284a4f640 1 --2- 172.21.15.6:0/728082285 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddffd40 0x564cdddfd640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc274005240 tx=0x7fc2740a4f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69d5cf0180bd4fbd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:30.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.865+0000 7fc25f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:23:30.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.865+0000 7fc25f7fe640 1 -- 172.21.15.6:0/728082285 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2740c2030 con 0x564cdddffd40 2024-09-12T03:23:30.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.865+0000 7fc28888b740 1 -- 172.21.15.6:0/728082285 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2740ada90 con 0x564cdddffd40 2024-09-12T03:23:30.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.865+0000 7fc25f7fe640 1 -- 172.21.15.6:0/728082285 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fc2740d61d0 con 0x564cdddffd40 2024-09-12T03:23:30.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.869+0000 7fc25f7fe640 1 -- 172.21.15.6:0/728082285 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc2740ad600 con 0x564cdddffd40 2024-09-12T03:23:30.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.869+0000 7fc28888b740 10 client.25235 fetch_fsmap learned FSMap version 26 2024-09-12T03:23:30.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.869+0000 7fc28888b740 1 -- 172.21.15.6:0/728082285 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564cdde02960 con 0x564cdddffd40 2024-09-12T03:23:30.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.871+0000 7fc25f7fe640 1 -- 172.21.15.6:0/728082285 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2740ada90 con 0x564cdddffd40 2024-09-12T03:23:30.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.871+0000 7fc25f7fe640 1 -- 172.21.15.6:0/728082285 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc2740041c0 con 0x564cdddffd40 2024-09-12T03:23:30.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.872+0000 7fc28888b740 10 client.25235 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:23:30.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.872+0000 7fc28888b740 10 client.25235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:23:30.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.872+0000 7fc28888b740 20 client.25235 populate_metadata read hostname 'smithi006' 2024-09-12T03:23:30.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.872+0000 7fc28888b740 1 --2- 172.21.15.6:0/728082285 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x564cdde0f740 0x564cdde11b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.872+0000 7fc28888b740 4 client.25235 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:23:30.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.872+0000 7fc28888b740 1 -- 172.21.15.6:0/728082285 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cddd677f0 con 0x564cdde0f740 2024-09-12T03:23:30.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.872+0000 7fc27ffff640 1 --2- 172.21.15.6:0/728082285 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x564cdde0f740 0x564cdde11b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.884+0000 7fc25f7fe640 10 client.25235 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:30.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.884+0000 7fc27ffff640 1 --2- 172.21.15.6:0/728082285 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x564cdde0f740 0x564cdde11b30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19599e38afdf4322 server_cookie=7196afaf717753fd in_seq=0 out_seq=0 2024-09-12T03:23:30.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.888+0000 7fc25f7fe640 1 -- 172.21.15.6:0/728082285 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564cddd677f0 con 0x564cdde0f740 2024-09-12T03:23:30.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.888+0000 7fc25f7fe640 10 client.25235 handle_command_reply: tid=0 2024-09-12T03:23:30.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.888+0000 7fc28888b740 1 client.25235 shutdown 2024-09-12T03:23:30.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.888+0000 7fc28888b740 1 -- 172.21.15.6:0/728082285 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddffd40 msgr2=0x564cdddfd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.888+0000 7fc28888b740 1 --2- 172.21.15.6:0/728082285 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddffd40 0x564cdddfd640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc274005240 tx=0x7fc2740a4f90 comp rx=0 tx=0).stop 2024-09-12T03:23:30.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.888+0000 7fc28888b740 1 -- 172.21.15.6:0/728082285 shutdown_connections 2024-09-12T03:23:30.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.888+0000 7fc28888b740 1 --2- 172.21.15.6:0/728082285 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x564cdde0f740 0x564cdde11b30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.888+0000 7fc28888b740 1 --2- 172.21.15.6:0/728082285 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddffd40 0x564cdddfd640 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.888+0000 7fc28888b740 1 --2- 172.21.15.6:0/728082285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564cdddf5480 0x564cdddfd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.888+0000 7fc25f7fe640 0 client.25235 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:30.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.888+0000 7fc28888b740 1 --2- 172.21.15.6:0/728082285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddf4b60 0x564cdddfcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.888+0000 7fc28888b740 1 -- 172.21.15.6:0/728082285 >> 172.21.15.6:0/728082285 conn(0x564cdddf10f0 msgr2=0x564cdddf6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:30.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.888+0000 7fc28888b740 1 -- 172.21.15.6:0/728082285 shutdown_connections 2024-09-12T03:23:30.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.888+0000 7fc28888b740 1 -- 172.21.15.6:0/728082285 wait complete. 2024-09-12T03:23:30.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.889+0000 7fc28888b740 20 client.25235 trim_cache size 0 max 16384 2024-09-12T03:23:30.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.934+0000 7fc28888b740 1 Processor -- start 2024-09-12T03:23:30.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.934+0000 7fc28888b740 1 -- start start 2024-09-12T03:23:30.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.935+0000 7fc28888b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddf4b60 0x564cdddf6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.935+0000 7fc28888b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddf5480 0x564cdde172a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.935+0000 7fc28888b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564cdddffd40 0x564cdde177e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.936+0000 7fc27f7fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddf5480 0x564cdde172a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.936+0000 7fc27f7fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddf5480 0x564cdde172a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:55858/0 (socket says 172.21.15.6:55858) 2024-09-12T03:23:30.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.936+0000 7fc27f7fe640 1 -- 172.21.15.6:0/4203325369 learned_addr learned my addr 172.21.15.6:0/4203325369 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:30.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.936+0000 7fc28888b740 1 -- 172.21.15.6:0/4203325369 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c010d60 con 0x564cdddffd40 2024-09-12T03:23:30.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.936+0000 7fc28888b740 1 -- 172.21.15.6:0/4203325369 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c00e630 con 0x564cdddf5480 2024-09-12T03:23:30.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.936+0000 7fc28888b740 1 -- 172.21.15.6:0/4203325369 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c00c480 con 0x564cdddf4b60 2024-09-12T03:23:30.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.937+0000 7fc27ffff640 1 --2- 172.21.15.6:0/4203325369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddf4b60 0x564cdddf6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.937+0000 7fc27ffff640 1 -- 172.21.15.6:0/4203325369 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddf5480 msgr2=0x564cdde172a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:30.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.937+0000 7fc27ffff640 1 --2- 172.21.15.6:0/4203325369 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddf5480 0x564cdde172a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.937+0000 7fc27ffff640 1 -- 172.21.15.6:0/4203325369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564cdddffd40 msgr2=0x564cdde177e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:30.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.937+0000 7fc27ffff640 1 --2- 172.21.15.6:0/4203325369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564cdddffd40 0x564cdde177e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.937+0000 7fc27ffff640 1 -- 172.21.15.6:0/4203325369 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cdddf0f20 con 0x564cdddf4b60 2024-09-12T03:23:30.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.937+0000 7fc27ffff640 1 --2- 172.21.15.6:0/4203325369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddf4b60 0x564cdddf6d10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x564cdde12210 tx=0x7fc27803c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23a68a0f06ca3d34 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:30.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.938+0000 7fc265ffb640 1 -- 172.21.15.6:0/4203325369 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278033d40 con 0x564cdddf4b60 2024-09-12T03:23:30.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.938+0000 7fc265ffb640 1 -- 172.21.15.6:0/4203325369 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc278042070 con 0x564cdddf4b60 2024-09-12T03:23:30.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.939+0000 7fc28888b740 1 -- 172.21.15.6:0/4203325369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddf4b60 msgr2=0x564cdddf6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.939+0000 7fc28888b740 1 --2- 172.21.15.6:0/4203325369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddf4b60 0x564cdddf6d10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x564cdde12210 tx=0x7fc27803c000 comp rx=0 tx=0).stop 2024-09-12T03:23:30.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.940+0000 7fc28888b740 1 -- 172.21.15.6:0/4203325369 shutdown_connections 2024-09-12T03:23:30.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.940+0000 7fc28888b740 1 --2- 172.21.15.6:0/4203325369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564cdddffd40 0x564cdde177e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.940+0000 7fc28888b740 1 --2- 172.21.15.6:0/4203325369 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddf5480 0x564cdde172a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.940+0000 7fc28888b740 1 --2- 172.21.15.6:0/4203325369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddf4b60 0x564cdddf6d10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.940+0000 7fc28888b740 1 -- 172.21.15.6:0/4203325369 >> 172.21.15.6:0/4203325369 conn(0x564cdddf10f0 msgr2=0x564cdddf6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:30.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.940+0000 7fc28888b740 1 -- 172.21.15.6:0/4203325369 shutdown_connections 2024-09-12T03:23:30.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.940+0000 7fc28888b740 1 -- 172.21.15.6:0/4203325369 wait complete. 2024-09-12T03:23:30.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.941+0000 7fc28888b740 1 -- start start 2024-09-12T03:23:30.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.941+0000 7fc28888b740 1 Processor -- start 2024-09-12T03:23:30.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.941+0000 7fc28888b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddf4b60 0x564cdde07fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.941+0000 7fc28888b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddf5480 0x564cdde08510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.941+0000 7fc28888b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564cdddffd40 0x564cdde02da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.941+0000 7fc28888b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c00caf0 con 0x564cdddffd40 2024-09-12T03:23:30.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.941+0000 7fc28888b740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c00df60 con 0x564cdddf5480 2024-09-12T03:23:30.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.941+0000 7fc28888b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c00d1c0 con 0x564cdddf4b60 2024-09-12T03:23:30.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.942+0000 7fc28888b740 10 client.0 _ll_register_callbacks cb 0x7fc2740a4ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:30.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.943+0000 7fc284a4f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564cdddffd40 0x564cdde02da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.943+0000 7fc284a4f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564cdddffd40 0x564cdde02da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54536/0 (socket says 172.21.15.6:54536) 2024-09-12T03:23:30.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.943+0000 7fc284a4f640 1 -- 172.21.15.6:0/3102179018 learned_addr learned my addr 172.21.15.6:0/3102179018 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:30.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.944+0000 7fc27f7fe640 1 --2- 172.21.15.6:0/3102179018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddf5480 0x564cdde08510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.944+0000 7fc27ffff640 1 --2- 172.21.15.6:0/3102179018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddf4b60 0x564cdde07fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.944+0000 7fc27f7fe640 1 -- 172.21.15.6:0/3102179018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddf4b60 msgr2=0x564cdde07fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:30.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.944+0000 7fc27f7fe640 1 --2- 172.21.15.6:0/3102179018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddf4b60 0x564cdde07fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.945+0000 7fc27f7fe640 1 -- 172.21.15.6:0/3102179018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564cdddffd40 msgr2=0x564cdde02da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:30.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.945+0000 7fc27f7fe640 1 --2- 172.21.15.6:0/3102179018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564cdddffd40 0x564cdde02da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.945+0000 7fc27f7fe640 1 -- 172.21.15.6:0/3102179018 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cdde035b0 con 0x564cdddf5480 2024-09-12T03:23:30.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.946+0000 7fc27f7fe640 1 --2- 172.21.15.6:0/3102179018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddf5480 0x564cdde08510 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x564cddde31d0 tx=0x7fc270012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48cccaacd6ea2ca2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:30.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.946+0000 7fc25f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:23:30.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.946+0000 7fc25f7fe640 1 -- 172.21.15.6:0/3102179018 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc270011b00 con 0x564cdddf5480 2024-09-12T03:23:30.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.946+0000 7fc28888b740 1 -- 172.21.15.6:0/3102179018 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc278042070 con 0x564cdddf5480 2024-09-12T03:23:30.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.946+0000 7fc25f7fe640 1 -- 172.21.15.6:0/3102179018 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fc270011ca0 con 0x564cdddf5480 2024-09-12T03:23:30.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.947+0000 7fc25f7fe640 1 -- 172.21.15.6:0/3102179018 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc270011ea0 con 0x564cdddf5480 2024-09-12T03:23:30.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.947+0000 7fc28888b740 10 client.25241 fetch_fsmap learned FSMap version 26 2024-09-12T03:23:30.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.947+0000 7fc28888b740 1 -- 172.21.15.6:0/3102179018 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564cdde8be70 con 0x564cdddf5480 2024-09-12T03:23:30.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.948+0000 7fc25f7fe640 1 -- 172.21.15.6:0/3102179018 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc278042070 con 0x564cdddf5480 2024-09-12T03:23:30.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.949+0000 7fc25f7fe640 1 -- 172.21.15.6:0/3102179018 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc270005030 con 0x564cdddf5480 2024-09-12T03:23:30.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.949+0000 7fc28888b740 10 client.25241 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:23:30.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.949+0000 7fc28888b740 10 client.25241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:23:30.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.949+0000 7fc28888b740 20 client.25241 populate_metadata read hostname 'smithi006' 2024-09-12T03:23:30.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.949+0000 7fc28888b740 1 --2- 172.21.15.6:0/3102179018 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x564cdde8c600 0x564cdde8ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.949+0000 7fc28888b740 4 client.25241 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:23:30.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.949+0000 7fc28888b740 1 -- 172.21.15.6:0/3102179018 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564cddd61d00 con 0x564cdde8c600 2024-09-12T03:23:30.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.951+0000 7fc27ffff640 1 --2- 172.21.15.6:0/3102179018 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x564cdde8c600 0x564cdde8ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.952+0000 7fc27ffff640 1 --2- 172.21.15.6:0/3102179018 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x564cdde8c600 0x564cdde8ea20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c916b83cc0db95be server_cookie=b4d88875146e1783 in_seq=0 out_seq=0 2024-09-12T03:23:30.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.952+0000 7fc25f7fe640 10 client.25241 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:30.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.956+0000 7fc25f7fe640 1 -- 172.21.15.6:0/3102179018 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564cddd61d00 con 0x564cdde8c600 2024-09-12T03:23:30.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.956+0000 7fc25f7fe640 10 client.25241 handle_command_reply: tid=0 2024-09-12T03:23:30.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.956+0000 7fc28888b740 1 client.25241 shutdown 2024-09-12T03:23:30.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.956+0000 7fc28888b740 1 -- 172.21.15.6:0/3102179018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddf5480 msgr2=0x564cdde08510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.956+0000 7fc28888b740 1 --2- 172.21.15.6:0/3102179018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddf5480 0x564cdde08510 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x564cddde31d0 tx=0x7fc270012b60 comp rx=0 tx=0).stop 2024-09-12T03:23:30.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.956+0000 7fc28888b740 1 -- 172.21.15.6:0/3102179018 shutdown_connections 2024-09-12T03:23:30.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.956+0000 7fc28888b740 1 --2- 172.21.15.6:0/3102179018 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x564cdde8c600 0x564cdde8ea20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.956+0000 7fc28888b740 1 --2- 172.21.15.6:0/3102179018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564cdddffd40 0x564cdde02da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.956+0000 7fc28888b740 1 --2- 172.21.15.6:0/3102179018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564cdddf5480 0x564cdde08510 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.956+0000 7fc28888b740 1 --2- 172.21.15.6:0/3102179018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564cdddf4b60 0x564cdde07fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.956+0000 7fc28888b740 1 -- 172.21.15.6:0/3102179018 >> 172.21.15.6:0/3102179018 conn(0x564cdddf10f0 msgr2=0x564cdddf0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:30.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.957+0000 7fc25f7fe640 0 client.25241 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:30.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.957+0000 7fc28888b740 1 -- 172.21.15.6:0/3102179018 shutdown_connections 2024-09-12T03:23:30.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.957+0000 7fc28888b740 1 -- 172.21.15.6:0/3102179018 wait complete. 2024-09-12T03:23:30.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.957+0000 7fc28888b740 20 client.25241 trim_cache size 0 max 16384 2024-09-12T03:23:30.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.962+0000 7fc2864b2640 1 -- 172.21.15.6:0/4090177420 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc2680a2c40 msgr2=0x7fc2680a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.962+0000 7fc2864b2640 1 --2- 172.21.15.6:0/4090177420 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc2680a2c40 0x7fc2680a5100 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc2780099a0 tx=0x7fc278005fd0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.962+0000 7fc2864b2640 1 -- 172.21.15.6:0/4090177420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc28010c160 msgr2=0x7fc28006cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.962+0000 7fc2864b2640 1 --2- 172.21.15.6:0/4090177420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc28010c160 0x7fc28006cfa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc274009e70 tx=0x7fc27400ed10 comp rx=0 tx=0).stop 2024-09-12T03:23:30.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.965+0000 7fc2864b2640 1 -- 172.21.15.6:0/4090177420 shutdown_connections 2024-09-12T03:23:30.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.965+0000 7fc2864b2640 1 --2- 172.21.15.6:0/4090177420 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc2680a2c40 0x7fc2680a5100 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.965+0000 7fc2864b2640 1 --2- 172.21.15.6:0/4090177420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc28010c160 0x7fc28006cfa0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.965+0000 7fc2864b2640 1 --2- 172.21.15.6:0/4090177420 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc280103650 0x7fc28006ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.965+0000 7fc2864b2640 1 --2- 172.21.15.6:0/4090177420 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc280102d30 0x7fc2800739d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.965+0000 7fc2864b2640 1 -- 172.21.15.6:0/4090177420 >> 172.21.15.6:0/4090177420 conn(0x7fc2800fc7d0 msgr2=0x7fc2800fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:30.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.965+0000 7fc2864b2640 1 -- 172.21.15.6:0/4090177420 shutdown_connections 2024-09-12T03:23:30.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:29.965+0000 7fc2864b2640 1 -- 172.21.15.6:0/4090177420 wait complete. 2024-09-12T03:23:30.526 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:30 smithi006 ceph-mon[30991]: scrub queued for path: / 2024-09-12T03:23:30.526 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:30 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:30.526 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:30 smithi006 ceph-mon[30991]: scrub summary: active paths [/] 2024-09-12T03:23:30.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:30 smithi161 ceph-mon[35485]: scrub queued for path: / 2024-09-12T03:23:30.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:30 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:30.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:30 smithi161 ceph-mon[35485]: scrub summary: active paths [/] 2024-09-12T03:23:30.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:30 smithi118 ceph-mon[35048]: scrub queued for path: / 2024-09-12T03:23:30.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:30 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:30.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:30 smithi118 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-12T03:23:31.510 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:31 smithi006 ceph-mon[30991]: pgmap v407: 97 pgs: 97 active+clean; 13 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 897 KiB/s rd, 330 MiB/s wr, 26.21k op/s 2024-09-12T03:23:31.513 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:23:31.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:31 smithi161 ceph-mon[35485]: pgmap v407: 97 pgs: 97 active+clean; 13 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 897 KiB/s rd, 330 MiB/s wr, 26.21k op/s 2024-09-12T03:23:31.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:31 smithi118 ceph-mon[35048]: pgmap v407: 97 pgs: 97 active+clean; 13 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 897 KiB/s rd, 330 MiB/s wr, 26.21k op/s 2024-09-12T03:23:32.095 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:33.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.948+0000 7fa4a1198640 1 -- 172.21.15.6:0/1230889485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa49c11b140 msgr2=0x7fa49c11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:33.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.948+0000 7fa4a1198640 1 --2- 172.21.15.6:0/1230889485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa49c11b140 0x7fa49c11d530 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa494009ca0 tx=0x7fa49402a8f0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.948+0000 7fa4a1198640 1 -- 172.21.15.6:0/1230889485 shutdown_connections 2024-09-12T03:23:33.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.948+0000 7fa4a1198640 1 --2- 172.21.15.6:0/1230889485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa49c11b140 0x7fa49c11d530 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.948+0000 7fa4a1198640 1 --2- 172.21.15.6:0/1230889485 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa49c109830 0x7fa49c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.948+0000 7fa4a1198640 1 --2- 172.21.15.6:0/1230889485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c109450 0x7fa49c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.948+0000 7fa4a1198640 1 -- 172.21.15.6:0/1230889485 >> 172.21.15.6:0/1230889485 conn(0x7fa49c06e140 msgr2=0x7fa49c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:33.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.948+0000 7fa4a1198640 1 -- 172.21.15.6:0/1230889485 shutdown_connections 2024-09-12T03:23:33.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.948+0000 7fa4a1198640 1 -- 172.21.15.6:0/1230889485 wait complete. 2024-09-12T03:23:33.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.949+0000 7fa4a1198640 1 Processor -- start 2024-09-12T03:23:33.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.950+0000 7fa4a1198640 1 -- start start 2024-09-12T03:23:33.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.950+0000 7fa4a1198640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa49c109450 0x7fa49c1a71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:33.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.950+0000 7fa4a1198640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c109830 0x7fa49c1a7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:33.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.950+0000 7fa4a1198640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa49c11b140 0x7fa49c1abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:33.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.950+0000 7fa4a1198640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c120e10 con 0x7fa49c109450 2024-09-12T03:23:33.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.950+0000 7fa4a1198640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c120c90 con 0x7fa49c11b140 2024-09-12T03:23:33.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.950+0000 7fa4a1198640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c120f90 con 0x7fa49c109830 2024-09-12T03:23:33.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.951+0000 7fa49ad9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa49c109450 0x7fa49c1a71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:33.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.951+0000 7fa49b59e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa49c11b140 0x7fa49c1abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:33.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.951+0000 7fa49b59e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa49c11b140 0x7fa49c1abaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:55894/0 (socket says 172.21.15.6:55894) 2024-09-12T03:23:33.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.951+0000 7fa49b59e640 1 -- 172.21.15.6:0/2457361679 learned_addr learned my addr 172.21.15.6:0/2457361679 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:33.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.951+0000 7fa49a59c640 1 --2- 172.21.15.6:0/2457361679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c109830 0x7fa49c1a7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:33.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.951+0000 7fa49b59e640 1 -- 172.21.15.6:0/2457361679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c109830 msgr2=0x7fa49c1a7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:33.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.951+0000 7fa49b59e640 1 --2- 172.21.15.6:0/2457361679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c109830 0x7fa49c1a7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.951+0000 7fa49b59e640 1 -- 172.21.15.6:0/2457361679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa49c109450 msgr2=0x7fa49c1a71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:33.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.951+0000 7fa49b59e640 1 --2- 172.21.15.6:0/2457361679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa49c109450 0x7fa49c1a71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.951+0000 7fa49b59e640 1 -- 172.21.15.6:0/2457361679 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49c1ac180 con 0x7fa49c11b140 2024-09-12T03:23:33.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.952+0000 7fa49b59e640 1 --2- 172.21.15.6:0/2457361679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa49c11b140 0x7fa49c1abaa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa49402fc00 tx=0x7fa49402fc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:33.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.952+0000 7fa483fff640 1 -- 172.21.15.6:0/2457361679 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4940307f0 con 0x7fa49c11b140 2024-09-12T03:23:33.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.953+0000 7fa483fff640 1 -- 172.21.15.6:0/2457361679 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa49402d070 con 0x7fa49c11b140 2024-09-12T03:23:33.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.953+0000 7fa4a1198640 1 -- 172.21.15.6:0/2457361679 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa49c1ac410 con 0x7fa49c11b140 2024-09-12T03:23:33.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.953+0000 7fa4a1198640 1 -- 172.21.15.6:0/2457361679 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa49c1b3d50 con 0x7fa49c11b140 2024-09-12T03:23:33.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.954+0000 7fa483fff640 1 -- 172.21.15.6:0/2457361679 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494045bc0 con 0x7fa49c11b140 2024-09-12T03:23:33.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.961+0000 7fa483fff640 1 -- 172.21.15.6:0/2457361679 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa49403b400 con 0x7fa49c11b140 2024-09-12T03:23:33.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.961+0000 7fa4a3571740 1 Processor -- start 2024-09-12T03:23:33.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.961+0000 7fa4a3571740 1 -- start start 2024-09-12T03:23:33.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.962+0000 7fa483fff640 1 --2- 172.21.15.6:0/2457361679 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa4840a2d90 0x7fa4840a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:33.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.963+0000 7fa483fff640 1 -- 172.21.15.6:0/2457361679 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa4940f16c0 con 0x7fa49c11b140 2024-09-12T03:23:33.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.965+0000 7fa49a59c640 1 --2- 172.21.15.6:0/2457361679 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa4840a2d90 0x7fa4840a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:33.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.966+0000 7fa4a3571740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e94a0 0x55b8720e9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:33.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.966+0000 7fa4a3571740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8720e9dc0 0x55b8720f4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:33.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.966+0000 7fa4a3571740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720f4680 0x55b8720f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:33.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.966+0000 7fa4a3571740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b87204f2e0 con 0x55b8720e9dc0 2024-09-12T03:23:33.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.966+0000 7fa4a3571740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55b871ff6ce0 con 0x55b8720f4680 2024-09-12T03:23:33.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.966+0000 7fa4a3571740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b87204b2a0 con 0x55b8720e94a0 2024-09-12T03:23:33.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.966+0000 7fa49ad9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e94a0 0x55b8720e9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:33.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.967+0000 7fa49ad9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e94a0 0x55b8720e9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:52090/0 (socket says 172.21.15.6:52090) 2024-09-12T03:23:33.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.967+0000 7fa49ad9d640 1 -- 172.21.15.6:0/3267748204 learned_addr learned my addr 172.21.15.6:0/3267748204 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:33.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.967+0000 7fa49b59e640 1 --2- 172.21.15.6:0/3267748204 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720f4680 0x55b8720f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:33.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.967+0000 7fa49a59c640 1 --2- 172.21.15.6:0/3267748204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8720e9dc0 0x55b8720f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:33.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.967+0000 7fa49ad9d640 1 -- 172.21.15.6:0/3267748204 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720f4680 msgr2=0x55b8720f6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:33.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.967+0000 7fa49ad9d640 1 --2- 172.21.15.6:0/3267748204 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720f4680 0x55b8720f6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.967+0000 7fa49ad9d640 1 -- 172.21.15.6:0/3267748204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8720e9dc0 msgr2=0x55b8720f4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:33.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.967+0000 7fa49ad9d640 1 --2- 172.21.15.6:0/3267748204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8720e9dc0 0x55b8720f4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.967+0000 7fa49ad9d640 1 -- 172.21.15.6:0/3267748204 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b871fdb440 con 0x55b8720e94a0 2024-09-12T03:23:33.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.968+0000 7fa49a59c640 1 --2- 172.21.15.6:0/2457361679 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa4840a2d90 0x7fa4840a5250 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa49c11ae30 tx=0x7fa49000a750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:33.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.968+0000 7fa49ad9d640 1 --2- 172.21.15.6:0/3267748204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e94a0 0x55b8720e9880 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009a70 tx=0x7fa48c009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77d87593e8be7bf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:33.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.968+0000 7fa4817fa640 1 -- 172.21.15.6:0/3267748204 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c017070 con 0x55b8720e94a0 2024-09-12T03:23:33.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.968+0000 7fa4817fa640 1 -- 172.21.15.6:0/3267748204 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa48c013470 con 0x55b8720e94a0 2024-09-12T03:23:33.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.970+0000 7fa4a3571740 1 -- 172.21.15.6:0/3267748204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e94a0 msgr2=0x55b8720e9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:33.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.970+0000 7fa4a3571740 1 --2- 172.21.15.6:0/3267748204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e94a0 0x55b8720e9880 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009a70 tx=0x7fa48c009f30 comp rx=0 tx=0).stop 2024-09-12T03:23:33.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.970+0000 7fa4a3571740 1 -- 172.21.15.6:0/3267748204 shutdown_connections 2024-09-12T03:23:33.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.970+0000 7fa4a3571740 1 --2- 172.21.15.6:0/3267748204 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720f4680 0x55b8720f6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.970+0000 7fa4a3571740 1 --2- 172.21.15.6:0/3267748204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8720e9dc0 0x55b8720f4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.970+0000 7fa4a3571740 1 --2- 172.21.15.6:0/3267748204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e94a0 0x55b8720e9880 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.970+0000 7fa4a3571740 1 -- 172.21.15.6:0/3267748204 >> 172.21.15.6:0/3267748204 conn(0x55b8720e5a30 msgr2=0x55b8720e5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:33.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.971+0000 7fa4a3571740 1 -- 172.21.15.6:0/3267748204 shutdown_connections 2024-09-12T03:23:33.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.972+0000 7fa4a3571740 1 -- 172.21.15.6:0/3267748204 wait complete. 2024-09-12T03:23:33.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.973+0000 7fa4a3571740 1 -- start start 2024-09-12T03:23:33.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.974+0000 7fa4a3571740 1 Processor -- start 2024-09-12T03:23:33.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.974+0000 7fa4a3571740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e94a0 0x55b8720f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:33.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.975+0000 7fa4a3571740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720e9dc0 0x55b8720f10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:33.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.975+0000 7fa4a3571740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8720f4680 0x55b8720f1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:33.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.975+0000 7fa4a3571740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b8720f72a0 con 0x55b8720f4680 2024-09-12T03:23:33.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.975+0000 7fa4a3571740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55b87200a050 con 0x55b8720e9dc0 2024-09-12T03:23:33.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.975+0000 7fa4a3571740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b8720f7420 con 0x55b8720e94a0 2024-09-12T03:23:33.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.975+0000 7fa4a3571740 10 client.0 _ll_register_callbacks cb 0x55b87204a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.976+0000 7fa49a59c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720e9dc0 0x55b8720f10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.976+0000 7fa49a59c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720e9dc0 0x55b8720f10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:55916/0 (socket says 172.21.15.6:55916) 2024-09-12T03:23:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.976+0000 7fa49a59c640 1 -- 172.21.15.6:0/2628525625 learned_addr learned my addr 172.21.15.6:0/2628525625 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.976+0000 7fa49ad9d640 1 --2- 172.21.15.6:0/2628525625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e94a0 0x55b8720f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.977+0000 7fa49b59e640 1 --2- 172.21.15.6:0/2628525625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8720f4680 0x55b8720f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.977+0000 7fa49b59e640 1 -- 172.21.15.6:0/2628525625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e94a0 msgr2=0x55b8720f3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.977+0000 7fa49b59e640 1 --2- 172.21.15.6:0/2628525625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e94a0 0x55b8720f3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.977+0000 7fa49b59e640 1 -- 172.21.15.6:0/2628525625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720e9dc0 msgr2=0x55b8720f10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.977+0000 7fa49b59e640 1 --2- 172.21.15.6:0/2628525625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720e9dc0 0x55b8720f10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.978+0000 7fa49b59e640 1 -- 172.21.15.6:0/2628525625 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b871dc03c0 con 0x55b8720f4680 2024-09-12T03:23:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.978+0000 7fa49b59e640 1 --2- 172.21.15.6:0/2628525625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8720f4680 0x55b8720f1600 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa4940ba350 tx=0x7fa49402f340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f9b326b4de2c4d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.978+0000 7fa47eff5640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:23:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.979+0000 7fa47eff5640 1 -- 172.21.15.6:0/2628525625 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4940d6020 con 0x55b8720f4680 2024-09-12T03:23:33.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.979+0000 7fa4a3571740 1 -- 172.21.15.6:0/2628525625 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa48c013470 con 0x55b8720f4680 2024-09-12T03:23:33.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.979+0000 7fa47eff5640 1 -- 172.21.15.6:0/2628525625 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa4940ba560 con 0x55b8720f4680 2024-09-12T03:23:33.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.981+0000 7fa47eff5640 1 -- 172.21.15.6:0/2628525625 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa4940bdef0 con 0x55b8720f4680 2024-09-12T03:23:33.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.981+0000 7fa4a3571740 10 client.15693 fetch_fsmap learned FSMap version 26 2024-09-12T03:23:33.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.981+0000 7fa4a3571740 1 -- 172.21.15.6:0/2628525625 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b872102b40 con 0x55b8720f4680 2024-09-12T03:23:33.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.981+0000 7fa47eff5640 1 -- 172.21.15.6:0/2628525625 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa48c013470 con 0x55b8720f4680 2024-09-12T03:23:33.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.982+0000 7fa47eff5640 1 -- 172.21.15.6:0/2628525625 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa49404d070 con 0x55b8720f4680 2024-09-12T03:23:33.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.983+0000 7fa4a3571740 10 client.15693 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:23:33.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.983+0000 7fa4a3571740 10 client.15693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:23:33.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.983+0000 7fa4a3571740 20 client.15693 populate_metadata read hostname 'smithi006' 2024-09-12T03:23:33.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.983+0000 7fa4a3571740 1 --2- 172.21.15.6:0/2628525625 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b872104210 0x55b872106690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:33.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.983+0000 7fa4a3571740 4 client.15693 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:23:33.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.983+0000 7fa4a3571740 1 -- 172.21.15.6:0/2628525625 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b87205c850 con 0x55b872104210 2024-09-12T03:23:33.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:32.986+0000 7fa49ad9d640 1 --2- 172.21.15.6:0/2628525625 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b872104210 0x55b872106690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:33.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.002+0000 7fa47eff5640 10 client.15693 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:33.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.002+0000 7fa49ad9d640 1 --2- 172.21.15.6:0/2628525625 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b872104210 0x55b872106690 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ea09c0d0a4a72a9 server_cookie=3e61e2b39f4e6752 in_seq=0 out_seq=0 2024-09-12T03:23:33.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.007+0000 7fa47eff5640 1 -- 172.21.15.6:0/2628525625 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b87205c850 con 0x55b872104210 2024-09-12T03:23:33.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.007+0000 7fa47eff5640 10 client.15693 handle_command_reply: tid=0 2024-09-12T03:23:33.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.007+0000 7fa4a3571740 1 client.15693 shutdown 2024-09-12T03:23:33.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.008+0000 7fa4a3571740 1 -- 172.21.15.6:0/2628525625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8720f4680 msgr2=0x55b8720f1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:33.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.008+0000 7fa4a3571740 1 --2- 172.21.15.6:0/2628525625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8720f4680 0x55b8720f1600 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa4940ba350 tx=0x7fa49402f340 comp rx=0 tx=0).stop 2024-09-12T03:23:33.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.008+0000 7fa4a3571740 1 -- 172.21.15.6:0/2628525625 shutdown_connections 2024-09-12T03:23:33.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.008+0000 7fa4a3571740 1 --2- 172.21.15.6:0/2628525625 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b872104210 0x55b872106690 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.008+0000 7fa4a3571740 1 --2- 172.21.15.6:0/2628525625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8720f4680 0x55b8720f1600 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.008+0000 7fa4a3571740 1 --2- 172.21.15.6:0/2628525625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720e9dc0 0x55b8720f10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.008+0000 7fa4a3571740 1 --2- 172.21.15.6:0/2628525625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e94a0 0x55b8720f3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.008+0000 7fa4a3571740 1 -- 172.21.15.6:0/2628525625 >> 172.21.15.6:0/2628525625 conn(0x55b8720e5a30 msgr2=0x55b8720eb760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:33.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.008+0000 7fa47eff5640 0 client.15693 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:33.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.008+0000 7fa4a3571740 1 -- 172.21.15.6:0/2628525625 shutdown_connections 2024-09-12T03:23:33.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.009+0000 7fa4a3571740 1 -- 172.21.15.6:0/2628525625 wait complete. 2024-09-12T03:23:33.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.009+0000 7fa4a3571740 20 client.15693 trim_cache size 0 max 16384 2024-09-12T03:23:33.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.048+0000 7fa4a3571740 1 Processor -- start 2024-09-12T03:23:33.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.048+0000 7fa4a3571740 1 -- start start 2024-09-12T03:23:33.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.049+0000 7fa4a3571740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e94a0 0x55b8720ef990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:33.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.049+0000 7fa4a3571740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8720e9dc0 0x55b8720f3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:33.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.049+0000 7fa4a3571740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720f4680 0x55b87210bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:33.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.049+0000 7fa4a3571740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa488010db0 con 0x55b8720e9dc0 2024-09-12T03:23:33.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.049+0000 7fa4a3571740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa48800e680 con 0x55b8720f4680 2024-09-12T03:23:33.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.049+0000 7fa4a3571740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa48800c4d0 con 0x55b8720e94a0 2024-09-12T03:23:33.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.049+0000 7fa49a59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8720e9dc0 0x55b8720f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:33.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.050+0000 7fa49a59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8720e9dc0 0x55b8720f3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54562/0 (socket says 172.21.15.6:54562) 2024-09-12T03:23:33.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.050+0000 7fa49a59c640 1 -- 172.21.15.6:0/3946872820 learned_addr learned my addr 172.21.15.6:0/3946872820 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:33.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.050+0000 7fa49b59e640 1 --2- 172.21.15.6:0/3946872820 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720f4680 0x55b87210bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:33.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.051+0000 7fa49b59e640 1 -- 172.21.15.6:0/3946872820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e94a0 msgr2=0x55b8720ef990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:33.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.051+0000 7fa49b59e640 1 --2- 172.21.15.6:0/3946872820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e94a0 0x55b8720ef990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.051+0000 7fa49b59e640 1 -- 172.21.15.6:0/3946872820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8720e9dc0 msgr2=0x55b8720f3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:33.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.051+0000 7fa49b59e640 1 --2- 172.21.15.6:0/3946872820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8720e9dc0 0x55b8720f3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.051+0000 7fa49b59e640 1 -- 172.21.15.6:0/3946872820 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8720f0070 con 0x55b8720f4680 2024-09-12T03:23:33.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.051+0000 7fa49b59e640 1 --2- 172.21.15.6:0/3946872820 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720f4680 0x55b87210bf60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa4940c6630 tx=0x7fa49402f470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ff9a2685c46caef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:33.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.053+0000 7fa480ff9640 1 -- 172.21.15.6:0/3946872820 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4940d6030 con 0x55b8720f4680 2024-09-12T03:23:33.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.053+0000 7fa480ff9640 1 -- 172.21.15.6:0/3946872820 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa494036030 con 0x55b8720f4680 2024-09-12T03:23:33.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.053+0000 7fa480ff9640 1 -- 172.21.15.6:0/3946872820 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4940c22a0 con 0x55b8720f4680 2024-09-12T03:23:33.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.055+0000 7fa4a3571740 1 -- 172.21.15.6:0/3946872820 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720f4680 msgr2=0x55b87210bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:33.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.055+0000 7fa4a3571740 1 --2- 172.21.15.6:0/3946872820 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720f4680 0x55b87210bf60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa4940c6630 tx=0x7fa49402f470 comp rx=0 tx=0).stop 2024-09-12T03:23:33.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.055+0000 7fa4a3571740 1 -- 172.21.15.6:0/3946872820 shutdown_connections 2024-09-12T03:23:33.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.055+0000 7fa4a3571740 1 --2- 172.21.15.6:0/3946872820 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720f4680 0x55b87210bf60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.055+0000 7fa4a3571740 1 --2- 172.21.15.6:0/3946872820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8720e9dc0 0x55b8720f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.055+0000 7fa4a3571740 1 --2- 172.21.15.6:0/3946872820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e94a0 0x55b8720ef990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.055+0000 7fa4a3571740 1 -- 172.21.15.6:0/3946872820 >> 172.21.15.6:0/3946872820 conn(0x55b8720e5a30 msgr2=0x55b8720f2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:33.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.058+0000 7fa4a3571740 1 -- 172.21.15.6:0/3946872820 shutdown_connections 2024-09-12T03:23:33.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.058+0000 7fa4a3571740 1 -- 172.21.15.6:0/3946872820 wait complete. 2024-09-12T03:23:33.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.058+0000 7fa4a3571740 1 -- start start 2024-09-12T03:23:33.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.059+0000 7fa4a3571740 1 Processor -- start 2024-09-12T03:23:33.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.059+0000 7fa4a3571740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8720e94a0 0x55b8720eebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:33.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.059+0000 7fa4a3571740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e9dc0 0x55b8720fc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:33.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.059+0000 7fa4a3571740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720f4680 0x55b8720fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:33.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.059+0000 7fa4a3571740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa48800cb40 con 0x55b8720e94a0 2024-09-12T03:23:33.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.059+0000 7fa4a3571740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa48800dfb0 con 0x55b8720f4680 2024-09-12T03:23:33.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.059+0000 7fa4a3571740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa48800d210 con 0x55b8720e9dc0 2024-09-12T03:23:33.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.059+0000 7fa4a3571740 10 client.0 _ll_register_callbacks cb 0x7fa494005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:33.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.061+0000 7fa49a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e9dc0 0x55b8720fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:33.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.062+0000 7fa49a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e9dc0 0x55b8720fc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:52116/0 (socket says 172.21.15.6:52116) 2024-09-12T03:23:33.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.062+0000 7fa49a59c640 1 -- 172.21.15.6:0/3713502231 learned_addr learned my addr 172.21.15.6:0/3713502231 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:33.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.063+0000 7fa49b59e640 1 --2- 172.21.15.6:0/3713502231 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720f4680 0x55b8720fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:33.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.063+0000 7fa49a59c640 1 -- 172.21.15.6:0/3713502231 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720f4680 msgr2=0x55b8720fcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:33.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.063+0000 7fa49a59c640 1 --2- 172.21.15.6:0/3713502231 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720f4680 0x55b8720fcc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.063+0000 7fa49a59c640 1 -- 172.21.15.6:0/3713502231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8720e94a0 msgr2=0x55b8720eebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:33.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.063+0000 7fa49a59c640 1 --2- 172.21.15.6:0/3713502231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8720e94a0 0x55b8720eebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.063+0000 7fa49a59c640 1 -- 172.21.15.6:0/3713502231 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8720f55c0 con 0x55b8720e9dc0 2024-09-12T03:23:33.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.066+0000 7fa49a59c640 1 --2- 172.21.15.6:0/3713502231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e9dc0 0x55b8720fc720 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa490011890 tx=0x7fa490019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63288956680062d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:33.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.071+0000 7fa47eff5640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:23:33.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.071+0000 7fa47eff5640 1 -- 172.21.15.6:0/3713502231 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49000fbb0 con 0x55b8720e9dc0 2024-09-12T03:23:33.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.071+0000 7fa4a3571740 1 -- 172.21.15.6:0/3713502231 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa494036030 con 0x55b8720e9dc0 2024-09-12T03:23:33.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.072+0000 7fa47eff5640 1 -- 172.21.15.6:0/3713502231 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa49000fd50 con 0x55b8720e9dc0 2024-09-12T03:23:33.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.072+0000 7fa47eff5640 1 -- 172.21.15.6:0/3713502231 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa490005020 con 0x55b8720e9dc0 2024-09-12T03:23:33.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.072+0000 7fa4a3571740 10 client.25294 fetch_fsmap learned FSMap version 26 2024-09-12T03:23:33.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.072+0000 7fa4a3571740 1 -- 172.21.15.6:0/3713502231 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b87216f080 con 0x55b8720e9dc0 2024-09-12T03:23:33.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.074+0000 7fa47eff5640 1 -- 172.21.15.6:0/3713502231 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa494036030 con 0x55b8720e9dc0 2024-09-12T03:23:33.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.074+0000 7fa47eff5640 1 -- 172.21.15.6:0/3713502231 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa490004c60 con 0x55b8720e9dc0 2024-09-12T03:23:33.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.074+0000 7fa4a3571740 10 client.25294 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:23:33.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.074+0000 7fa4a3571740 10 client.25294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:23:33.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.074+0000 7fa4a3571740 20 client.25294 populate_metadata read hostname 'smithi006' 2024-09-12T03:23:33.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.074+0000 7fa4a3571740 1 --2- 172.21.15.6:0/3713502231 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b87216f890 0x55b872171cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:33.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.074+0000 7fa4a3571740 4 client.25294 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:23:33.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.074+0000 7fa4a3571740 1 -- 172.21.15.6:0/3713502231 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b872056d60 con 0x55b87216f890 2024-09-12T03:23:33.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.081+0000 7fa49ad9d640 1 --2- 172.21.15.6:0/3713502231 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b87216f890 0x55b872171cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:33.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.084+0000 7fa49ad9d640 1 --2- 172.21.15.6:0/3713502231 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b87216f890 0x55b872171cb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4702cb47f4d5ea40 server_cookie=3d0b5a46747b2f8 in_seq=0 out_seq=0 2024-09-12T03:23:33.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.084+0000 7fa47eff5640 10 client.25294 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:33.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.085+0000 7fa47eff5640 1 -- 172.21.15.6:0/3713502231 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55b872056d60 con 0x55b87216f890 2024-09-12T03:23:33.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.085+0000 7fa47eff5640 10 client.25294 handle_command_reply: tid=0 2024-09-12T03:23:33.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.085+0000 7fa4a3571740 1 client.25294 shutdown 2024-09-12T03:23:33.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.087+0000 7fa4a3571740 1 -- 172.21.15.6:0/3713502231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e9dc0 msgr2=0x55b8720fc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:33.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.087+0000 7fa4a3571740 1 --2- 172.21.15.6:0/3713502231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e9dc0 0x55b8720fc720 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa490011890 tx=0x7fa490019000 comp rx=0 tx=0).stop 2024-09-12T03:23:33.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.087+0000 7fa4a3571740 1 -- 172.21.15.6:0/3713502231 shutdown_connections 2024-09-12T03:23:33.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.087+0000 7fa4a3571740 1 --2- 172.21.15.6:0/3713502231 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b87216f890 0x55b872171cb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.087+0000 7fa4a3571740 1 --2- 172.21.15.6:0/3713502231 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8720f4680 0x55b8720fcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.087+0000 7fa4a3571740 1 --2- 172.21.15.6:0/3713502231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8720e9dc0 0x55b8720fc720 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.087+0000 7fa4a3571740 1 --2- 172.21.15.6:0/3713502231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8720e94a0 0x55b8720eebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.087+0000 7fa4a3571740 1 -- 172.21.15.6:0/3713502231 >> 172.21.15.6:0/3713502231 conn(0x55b8720e5a30 msgr2=0x55b8720f3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:33.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.087+0000 7fa47eff5640 0 client.25294 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:33.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.088+0000 7fa4a3571740 1 -- 172.21.15.6:0/3713502231 shutdown_connections 2024-09-12T03:23:33.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.088+0000 7fa4a3571740 1 -- 172.21.15.6:0/3713502231 wait complete. 2024-09-12T03:23:33.089 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:23:33.089 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "scrub active (196 inodes in the stack)", 2024-09-12T03:23:33.090 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2024-09-12T03:23:33.090 INFO:teuthology.orchestra.run.smithi006.stdout: "2b96baee-7790-41fc-bcdb-7a0b692990e8": { 2024-09-12T03:23:33.090 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-09-12T03:23:33.090 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "2b96baee-7790-41fc-bcdb-7a0b692990e8", 2024-09-12T03:23:33.090 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2024-09-12T03:23:33.090 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:23:33.090 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:23:33.090 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:23:33.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.089+0000 7fa4a3571740 20 client.25294 trim_cache size 0 max 16384 2024-09-12T03:23:33.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.091+0000 7fa4a1198640 1 -- 172.21.15.6:0/2457361679 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa4840a2d90 msgr2=0x7fa4840a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:33.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.091+0000 7fa4a1198640 1 --2- 172.21.15.6:0/2457361679 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa4840a2d90 0x7fa4840a5250 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa49c11ae30 tx=0x7fa49000a750 comp rx=0 tx=0).stop 2024-09-12T03:23:33.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.091+0000 7fa4a1198640 1 -- 172.21.15.6:0/2457361679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa49c11b140 msgr2=0x7fa49c1abaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:33.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.091+0000 7fa4a1198640 1 --2- 172.21.15.6:0/2457361679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa49c11b140 0x7fa49c1abaa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa49402fc00 tx=0x7fa49402fc30 comp rx=0 tx=0).stop 2024-09-12T03:23:33.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.092+0000 7fa4a1198640 1 -- 172.21.15.6:0/2457361679 shutdown_connections 2024-09-12T03:23:33.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.092+0000 7fa4a1198640 1 --2- 172.21.15.6:0/2457361679 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa4840a2d90 0x7fa4840a5250 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.092+0000 7fa4a1198640 1 --2- 172.21.15.6:0/2457361679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa49c11b140 0x7fa49c1abaa0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.092+0000 7fa4a1198640 1 --2- 172.21.15.6:0/2457361679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c109830 0x7fa49c1a7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.092+0000 7fa4a1198640 1 --2- 172.21.15.6:0/2457361679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa49c109450 0x7fa49c1a71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.092+0000 7fa4a1198640 1 -- 172.21.15.6:0/2457361679 >> 172.21.15.6:0/2457361679 conn(0x7fa49c06e140 msgr2=0x7fa49c11bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:33.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.092+0000 7fa4a1198640 1 -- 172.21.15.6:0/2457361679 shutdown_connections 2024-09-12T03:23:33.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:33.092+0000 7fa4a1198640 1 -- 172.21.15.6:0/2457361679 wait complete. 2024-09-12T03:23:33.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:33 smithi006 ceph-mon[30991]: pgmap v408: 97 pgs: 97 active+clean; 13 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 614 KiB/s rd, 279 MiB/s wr, 23.01k op/s 2024-09-12T03:23:33.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:33 smithi161 ceph-mon[35485]: pgmap v408: 97 pgs: 97 active+clean; 13 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 614 KiB/s rd, 279 MiB/s wr, 23.01k op/s 2024-09-12T03:23:33.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:33 smithi118 ceph-mon[35048]: pgmap v408: 97 pgs: 97 active+clean; 13 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 614 KiB/s rd, 279 MiB/s wr, 23.01k op/s 2024-09-12T03:23:33.770 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.7 2024-09-12T03:23:33.770 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:23:34.756 INFO:tasks.cephfs.filesystem:scrub status for tag:2b96baee-7790-41fc-bcdb-7a0b692990e8 - {'path': '/', 'tag': '2b96baee-7790-41fc-bcdb-7a0b692990e8', 'options': 'recursive,force'} 2024-09-12T03:23:34.910 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:35.347 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:35 smithi006 ceph-mon[30991]: pgmap v409: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 745 KiB/s rd, 291 MiB/s wr, 24.85k op/s 2024-09-12T03:23:35.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:35 smithi161 ceph-mon[35485]: pgmap v409: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 745 KiB/s rd, 291 MiB/s wr, 24.85k op/s 2024-09-12T03:23:35.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:35 smithi118 ceph-mon[35048]: pgmap v409: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 745 KiB/s rd, 291 MiB/s wr, 24.85k op/s 2024-09-12T03:23:35.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.858+0000 7fa93587e640 1 -- 172.21.15.6:0/575878514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa930109450 msgr2=0x7fa930110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:35.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.858+0000 7fa93587e640 1 --2- 172.21.15.6:0/575878514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa930109450 0x7fa930110250 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa928009ca0 tx=0x7fa92802a8f0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.858+0000 7fa93587e640 1 -- 172.21.15.6:0/575878514 shutdown_connections 2024-09-12T03:23:35.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.858+0000 7fa93587e640 1 --2- 172.21.15.6:0/575878514 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa93011b140 0x7fa93011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.858+0000 7fa93587e640 1 --2- 172.21.15.6:0/575878514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa930109830 0x7fa930109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.858+0000 7fa93587e640 1 --2- 172.21.15.6:0/575878514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa930109450 0x7fa930110250 secure :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa928009ca0 tx=0x7fa92802a8f0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.858+0000 7fa93587e640 1 -- 172.21.15.6:0/575878514 >> 172.21.15.6:0/575878514 conn(0x7fa93006e140 msgr2=0x7fa93006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:35.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.863+0000 7fa93587e640 1 -- 172.21.15.6:0/575878514 shutdown_connections 2024-09-12T03:23:35.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.864+0000 7fa93587e640 1 -- 172.21.15.6:0/575878514 wait complete. 2024-09-12T03:23:35.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.865+0000 7fa93587e640 1 Processor -- start 2024-09-12T03:23:35.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.865+0000 7fa93587e640 1 -- start start 2024-09-12T03:23:35.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.866+0000 7fa93587e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa930109830 0x7fa930118870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:35.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.866+0000 7fa93587e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa93011b140 0x7fa930111900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:35.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.866+0000 7fa93587e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa930111e40 0x7fa9301122f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:35.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.866+0000 7fa93587e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa930120e90 con 0x7fa930109830 2024-09-12T03:23:35.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.866+0000 7fa93587e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa930120d10 con 0x7fa930111e40 2024-09-12T03:23:35.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.866+0000 7fa93587e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa930121010 con 0x7fa93011b140 2024-09-12T03:23:35.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.866+0000 7fa92effd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa930109830 0x7fa930118870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:35.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.866+0000 7fa92effd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa930109830 0x7fa930118870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54580/0 (socket says 172.21.15.6:54580) 2024-09-12T03:23:35.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.866+0000 7fa92effd640 1 -- 172.21.15.6:0/3350318732 learned_addr learned my addr 172.21.15.6:0/3350318732 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:35.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.867+0000 7fa92effd640 1 -- 172.21.15.6:0/3350318732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa93011b140 msgr2=0x7fa930111900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:35.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.867+0000 7fa92effd640 1 --2- 172.21.15.6:0/3350318732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa93011b140 0x7fa930111900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.867+0000 7fa92effd640 1 -- 172.21.15.6:0/3350318732 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa930111e40 msgr2=0x7fa9301122f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:23:35.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.867+0000 7fa92effd640 1 --2- 172.21.15.6:0/3350318732 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa930111e40 0x7fa9301122f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.867+0000 7fa92effd640 1 -- 172.21.15.6:0/3350318732 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9301164a0 con 0x7fa930109830 2024-09-12T03:23:35.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.867+0000 7fa92effd640 1 --2- 172.21.15.6:0/3350318732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa930109830 0x7fa930118870 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa92802ae60 tx=0x7fa928031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:35.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.868+0000 7fa93487c640 1 -- 172.21.15.6:0/3350318732 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928004240 con 0x7fa930109830 2024-09-12T03:23:35.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.868+0000 7fa93587e640 1 -- 172.21.15.6:0/3350318732 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa930116730 con 0x7fa930109830 2024-09-12T03:23:35.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.868+0000 7fa93587e640 1 -- 172.21.15.6:0/3350318732 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa930116c70 con 0x7fa930109830 2024-09-12T03:23:35.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.868+0000 7fa93487c640 1 -- 172.21.15.6:0/3350318732 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa928036070 con 0x7fa930109830 2024-09-12T03:23:35.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.868+0000 7fa93487c640 1 -- 172.21.15.6:0/3350318732 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92802f5a0 con 0x7fa930109830 2024-09-12T03:23:35.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.871+0000 7fa93587e640 1 -- 172.21.15.6:0/3350318732 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa91c005290 con 0x7fa930109830 2024-09-12T03:23:35.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.872+0000 7fa93487c640 1 -- 172.21.15.6:0/3350318732 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa9280305f0 con 0x7fa930109830 2024-09-12T03:23:35.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.874+0000 7fa93487c640 1 --2- 172.21.15.6:0/3350318732 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa9180a2d90 0x7fa9180a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:35.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.874+0000 7fa92e7fc640 1 --2- 172.21.15.6:0/3350318732 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa9180a2d90 0x7fa9180a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:35.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.874+0000 7fa93487c640 1 -- 172.21.15.6:0/3350318732 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa9280f02a0 con 0x7fa930109830 2024-09-12T03:23:35.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.875+0000 7fa92e7fc640 1 --2- 172.21.15.6:0/3350318732 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa9180a2d90 0x7fa9180a5250 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa9301130c0 tx=0x7fa924006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:35.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:35.882+0000 7fa93487c640 1 -- 172.21.15.6:0/3350318732 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa9280bd1d0 con 0x7fa930109830 2024-09-12T03:23:36.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:36.336+0000 7fa93587e640 1 -- 172.21.15.6:0/3350318732 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa91c005580 con 0x7fa930109830 2024-09-12T03:23:36.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:36.337+0000 7fa93487c640 1 -- 172.21.15.6:0/3350318732 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11911 (secure 0 0 0) 0x7fa9280bcad0 con 0x7fa930109830 2024-09-12T03:23:36.338 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:23:36.338 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":26,"btime":"2024-09-12T03:22:16:132848+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:22:16.064679+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24463,24464,24494]},"id":1}]} 2024-09-12T03:23:36.338 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 26 2024-09-12T03:23:36.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:36.343+0000 7fa93587e640 1 -- 172.21.15.6:0/3350318732 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa9180a2d90 msgr2=0x7fa9180a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:36.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:36.343+0000 7fa93587e640 1 --2- 172.21.15.6:0/3350318732 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa9180a2d90 0x7fa9180a5250 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa9301130c0 tx=0x7fa924006c90 comp rx=0 tx=0).stop 2024-09-12T03:23:36.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:36.343+0000 7fa93587e640 1 -- 172.21.15.6:0/3350318732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa930109830 msgr2=0x7fa930118870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:36.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:36.343+0000 7fa93587e640 1 --2- 172.21.15.6:0/3350318732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa930109830 0x7fa930118870 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa92802ae60 tx=0x7fa928031b60 comp rx=0 tx=0).stop 2024-09-12T03:23:36.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:36.344+0000 7fa93587e640 1 -- 172.21.15.6:0/3350318732 shutdown_connections 2024-09-12T03:23:36.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:36.344+0000 7fa93587e640 1 --2- 172.21.15.6:0/3350318732 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa9180a2d90 0x7fa9180a5250 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:36.344+0000 7fa93587e640 1 --2- 172.21.15.6:0/3350318732 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa930111e40 0x7fa9301122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:36.344+0000 7fa93587e640 1 --2- 172.21.15.6:0/3350318732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa93011b140 0x7fa930111900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:36.344+0000 7fa93587e640 1 --2- 172.21.15.6:0/3350318732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa930109830 0x7fa930118870 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:36.344+0000 7fa93587e640 1 -- 172.21.15.6:0/3350318732 >> 172.21.15.6:0/3350318732 conn(0x7fa93006e140 msgr2=0x7fa93011b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:36.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:36.344+0000 7fa93587e640 1 -- 172.21.15.6:0/3350318732 shutdown_connections 2024-09-12T03:23:36.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:36.344+0000 7fa93587e640 1 -- 172.21.15.6:0/3350318732 wait complete. 2024-09-12T03:23:36.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:36 smithi006 ceph-mon[30991]: pgmap v410: 97 pgs: 97 active+clean; 13 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 663 KiB/s rd, 265 MiB/s wr, 23.00k op/s 2024-09-12T03:23:36.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:36 smithi161 ceph-mon[35485]: pgmap v410: 97 pgs: 97 active+clean; 13 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 663 KiB/s rd, 265 MiB/s wr, 23.00k op/s 2024-09-12T03:23:36.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:36 smithi118 ceph-mon[35048]: pgmap v410: 97 pgs: 97 active+clean; 13 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 663 KiB/s rd, 265 MiB/s wr, 23.00k op/s 2024-09-12T03:23:37.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:37 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3350318732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:37.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:37 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3350318732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:37.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:37 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3350318732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:37.905 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0' 2024-09-12T03:23:37.905 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0 2024-09-12T03:23:38.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:38 smithi006 ceph-mon[30991]: pgmap v411: 97 pgs: 97 active+clean; 13 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 662 KiB/s rd, 283 MiB/s wr, 24.41k op/s 2024-09-12T03:23:38.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:38 smithi161 ceph-mon[35485]: pgmap v411: 97 pgs: 97 active+clean; 13 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 662 KiB/s rd, 283 MiB/s wr, 24.41k op/s 2024-09-12T03:23:38.649 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:38.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:38 smithi118 ceph-mon[35048]: pgmap v411: 97 pgs: 97 active+clean; 13 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 662 KiB/s rd, 283 MiB/s wr, 24.41k op/s 2024-09-12T03:23:39.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.683+0000 7fd793024640 1 -- 172.21.15.6:0/3549916440 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd78c102790 msgr2=0x7fd78c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:39.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.683+0000 7fd793024640 1 --2- 172.21.15.6:0/3549916440 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd78c102790 0x7fd78c1067e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd780009900 tx=0x7fd78002a400 comp rx=0 tx=0).stop 2024-09-12T03:23:39.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.685+0000 7fd793024640 1 -- 172.21.15.6:0/3549916440 shutdown_connections 2024-09-12T03:23:39.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.685+0000 7fd793024640 1 --2- 172.21.15.6:0/3549916440 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd78c106f10 0x7fd78c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.685+0000 7fd793024640 1 --2- 172.21.15.6:0/3549916440 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd78c102790 0x7fd78c1067e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.685+0000 7fd793024640 1 --2- 172.21.15.6:0/3549916440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd78c101de0 0x7fd78c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.685+0000 7fd793024640 1 -- 172.21.15.6:0/3549916440 >> 172.21.15.6:0/3549916440 conn(0x7fd78c0fd880 msgr2=0x7fd78c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:39.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.685+0000 7fd793024640 1 -- 172.21.15.6:0/3549916440 shutdown_connections 2024-09-12T03:23:39.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.686+0000 7fd793024640 1 -- 172.21.15.6:0/3549916440 wait complete. 2024-09-12T03:23:39.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.686+0000 7fd793024640 1 Processor -- start 2024-09-12T03:23:39.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.689+0000 7fd793024640 1 -- start start 2024-09-12T03:23:39.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.689+0000 7fd793024640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd78c101de0 0x7fd78c10ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.689+0000 7fd793024640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd78c102790 0x7fd78c10bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.689+0000 7fd793024640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd78c106f10 0x7fd78c10c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.689+0000 7fd793024640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c0760e0 con 0x7fd78c102790 2024-09-12T03:23:39.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.689+0000 7fd793024640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c075f60 con 0x7fd78c101de0 2024-09-12T03:23:39.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.689+0000 7fd793024640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c076260 con 0x7fd78c106f10 2024-09-12T03:23:39.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.691+0000 7fd78bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd78c102790 0x7fd78c10bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:39.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.691+0000 7fd78bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd78c102790 0x7fd78c10bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46436/0 (socket says 172.21.15.6:46436) 2024-09-12T03:23:39.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.691+0000 7fd78bfff640 1 -- 172.21.15.6:0/1837370088 learned_addr learned my addr 172.21.15.6:0/1837370088 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:39.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.692+0000 7fd790dc0640 1 --2- 172.21.15.6:0/1837370088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd78c101de0 0x7fd78c10ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:39.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.692+0000 7fd7915c1640 1 --2- 172.21.15.6:0/1837370088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd78c106f10 0x7fd78c10c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:39.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.692+0000 7fd78bfff640 1 -- 172.21.15.6:0/1837370088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd78c106f10 msgr2=0x7fd78c10c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:39.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.692+0000 7fd78bfff640 1 --2- 172.21.15.6:0/1837370088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd78c106f10 0x7fd78c10c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.692+0000 7fd78bfff640 1 -- 172.21.15.6:0/1837370088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd78c101de0 msgr2=0x7fd78c10ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:39.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.692+0000 7fd78bfff640 1 --2- 172.21.15.6:0/1837370088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd78c101de0 0x7fd78c10ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.692+0000 7fd78bfff640 1 -- 172.21.15.6:0/1837370088 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78c072c60 con 0x7fd78c102790 2024-09-12T03:23:39.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.693+0000 7fd78bfff640 1 --2- 172.21.15.6:0/1837370088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd78c102790 0x7fd78c10bad0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd780006010 tx=0x7fd780004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:39.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.694+0000 7fd789ffb640 1 -- 172.21.15.6:0/1837370088 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd780040070 con 0x7fd78c102790 2024-09-12T03:23:39.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.694+0000 7fd789ffb640 1 -- 172.21.15.6:0/1837370088 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7800314d0 con 0x7fd78c102790 2024-09-12T03:23:39.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.695+0000 7fd793024640 1 -- 172.21.15.6:0/1837370088 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd78c072ef0 con 0x7fd78c102790 2024-09-12T03:23:39.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.695+0000 7fd793024640 1 -- 172.21.15.6:0/1837370088 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd78c073370 con 0x7fd78c102790 2024-09-12T03:23:39.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.698+0000 7fd789ffb640 1 -- 172.21.15.6:0/1837370088 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78004a060 con 0x7fd78c102790 2024-09-12T03:23:39.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.702+0000 7fd7953fd740 1 Processor -- start 2024-09-12T03:23:39.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.704+0000 7fd7953fd740 1 -- start start 2024-09-12T03:23:39.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.704+0000 7fd7953fd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d09760 0x55b977d09b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.705+0000 7fd7953fd740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b977d0a080 0x55b977d14400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.705+0000 7fd7953fd740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d14940 0x55b977d16d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.705+0000 7fd7953fd740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b977c6f2a0 con 0x55b977d0a080 2024-09-12T03:23:39.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.705+0000 7fd7953fd740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55b977c16b10 con 0x55b977d14940 2024-09-12T03:23:39.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.705+0000 7fd7953fd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b977c6b260 con 0x55b977d09760 2024-09-12T03:23:39.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.705+0000 7fd7915c1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d09760 0x55b977d09b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:39.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.705+0000 7fd7915c1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d09760 0x55b977d09b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:43768/0 (socket says 172.21.15.6:43768) 2024-09-12T03:23:39.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.705+0000 7fd7915c1640 1 -- 172.21.15.6:0/2280028701 learned_addr learned my addr 172.21.15.6:0/2280028701 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:39.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.705+0000 7fd7915c1640 1 -- 172.21.15.6:0/2280028701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d14940 msgr2=0x55b977d16d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:23:39.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.705+0000 7fd7915c1640 1 --2- 172.21.15.6:0/2280028701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d14940 0x55b977d16d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.705+0000 7fd7915c1640 1 -- 172.21.15.6:0/2280028701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b977d0a080 msgr2=0x55b977d14400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:39.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.705+0000 7fd7915c1640 1 --2- 172.21.15.6:0/2280028701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b977d0a080 0x55b977d14400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.705+0000 7fd7915c1640 1 -- 172.21.15.6:0/2280028701 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9779e0b90 con 0x55b977d09760 2024-09-12T03:23:39.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.706+0000 7fd7915c1640 1 --2- 172.21.15.6:0/2280028701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d09760 0x55b977d09b40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd784007ec0 tx=0x7fd78400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97f9311bf22bed86 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:39.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.706+0000 7fd789ffb640 1 -- 172.21.15.6:0/1837370088 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd780043070 con 0x7fd78c102790 2024-09-12T03:23:39.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.706+0000 7fd772ffd640 1 -- 172.21.15.6:0/2280028701 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd784010840 con 0x55b977d09760 2024-09-12T03:23:39.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.706+0000 7fd772ffd640 1 -- 172.21.15.6:0/2280028701 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7840183f0 con 0x55b977d09760 2024-09-12T03:23:39.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.708+0000 7fd789ffb640 1 --2- 172.21.15.6:0/1837370088 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd7740a2c00 0x7fd7740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.708+0000 7fd7953fd740 1 -- 172.21.15.6:0/2280028701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d09760 msgr2=0x55b977d09b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:39.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.708+0000 7fd7953fd740 1 --2- 172.21.15.6:0/2280028701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d09760 0x55b977d09b40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd784007ec0 tx=0x7fd78400f790 comp rx=0 tx=0).stop 2024-09-12T03:23:39.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.708+0000 7fd7953fd740 1 -- 172.21.15.6:0/2280028701 shutdown_connections 2024-09-12T03:23:39.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.708+0000 7fd7953fd740 1 --2- 172.21.15.6:0/2280028701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d14940 0x55b977d16d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.708+0000 7fd7953fd740 1 --2- 172.21.15.6:0/2280028701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b977d0a080 0x55b977d14400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.708+0000 7fd7953fd740 1 --2- 172.21.15.6:0/2280028701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d09760 0x55b977d09b40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.708+0000 7fd7953fd740 1 -- 172.21.15.6:0/2280028701 >> 172.21.15.6:0/2280028701 conn(0x55b977d05cf0 msgr2=0x55b977d060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:39.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.709+0000 7fd789ffb640 1 -- 172.21.15.6:0/1837370088 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd7800efd00 con 0x7fd78c102790 2024-09-12T03:23:39.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.710+0000 7fd790dc0640 1 --2- 172.21.15.6:0/1837370088 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd7740a2c00 0x7fd7740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:39.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.711+0000 7fd7953fd740 1 -- 172.21.15.6:0/2280028701 shutdown_connections 2024-09-12T03:23:39.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.713+0000 7fd7953fd740 1 -- 172.21.15.6:0/2280028701 wait complete. 2024-09-12T03:23:39.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.713+0000 7fd7953fd740 1 -- start start 2024-09-12T03:23:39.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.713+0000 7fd7953fd740 1 Processor -- start 2024-09-12T03:23:39.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.714+0000 7fd7953fd740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b977d0a080 0x55b977d143a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.714+0000 7fd7953fd740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d14940 0x55b977d11650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.714+0000 7fd7953fd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d11d40 0x55b977d20a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.714+0000 7fd7953fd740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b977d17560 con 0x55b977d0a080 2024-09-12T03:23:39.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.714+0000 7fd7953fd740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55b977bf76e0 con 0x55b977d14940 2024-09-12T03:23:39.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.714+0000 7fd7953fd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b977d176e0 con 0x55b977d11d40 2024-09-12T03:23:39.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.714+0000 7fd7953fd740 10 client.0 _ll_register_callbacks cb 0x55b977c1ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:39.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.715+0000 7fd78bfff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d14940 0x55b977d11650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:39.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.715+0000 7fd78bfff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d14940 0x55b977d11650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:43296/0 (socket says 172.21.15.6:43296) 2024-09-12T03:23:39.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.715+0000 7fd78bfff640 1 -- 172.21.15.6:0/4222408370 learned_addr learned my addr 172.21.15.6:0/4222408370 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:39.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.715+0000 7fd78bfff640 1 -- 172.21.15.6:0/4222408370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d11d40 msgr2=0x55b977d20a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:39.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.715+0000 7fd78bfff640 1 --2- 172.21.15.6:0/4222408370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d11d40 0x55b977d20a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.715+0000 7fd78bfff640 1 -- 172.21.15.6:0/4222408370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b977d0a080 msgr2=0x55b977d143a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:39.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.715+0000 7fd78bfff640 1 --2- 172.21.15.6:0/4222408370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b977d0a080 0x55b977d143a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.715+0000 7fd78bfff640 1 -- 172.21.15.6:0/4222408370 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b977d20f90 con 0x55b977d14940 2024-09-12T03:23:39.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.716+0000 7fd78bfff640 1 --2- 172.21.15.6:0/4222408370 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d14940 0x55b977d11650 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd7800c4c70 tx=0x7fd7800b8890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9dd833574325ee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:39.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.716+0000 7fd76bfff640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:23:39.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.717+0000 7fd76bfff640 1 -- 172.21.15.6:0/4222408370 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7800dc050 con 0x55b977d14940 2024-09-12T03:23:39.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.717+0000 7fd7953fd740 1 -- 172.21.15.6:0/4222408370 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b977c7c9d0 con 0x55b977d14940 2024-09-12T03:23:39.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.717+0000 7fd76bfff640 1 -- 172.21.15.6:0/4222408370 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd7800b8ba0 con 0x55b977d14940 2024-09-12T03:23:39.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.718+0000 7fd76bfff640 1 -- 172.21.15.6:0/4222408370 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd780035040 con 0x55b977d14940 2024-09-12T03:23:39.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.719+0000 7fd790dc0640 1 --2- 172.21.15.6:0/1837370088 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd7740a2c00 0x7fd7740a50c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd77c0058c0 tx=0x7fd77c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:39.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.725+0000 7fd7953fd740 10 client.25259 fetch_fsmap learned FSMap version 26 2024-09-12T03:23:39.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.725+0000 7fd7953fd740 1 -- 172.21.15.6:0/4222408370 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b977d23700 con 0x55b977d14940 2024-09-12T03:23:39.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.728+0000 7fd76bfff640 1 -- 172.21.15.6:0/4222408370 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b977c7c9d0 con 0x55b977d14940 2024-09-12T03:23:39.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.729+0000 7fd76bfff640 1 -- 172.21.15.6:0/4222408370 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd78003a040 con 0x55b977d14940 2024-09-12T03:23:39.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.729+0000 7fd7953fd740 10 client.25259 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:23:39.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.729+0000 7fd7953fd740 10 client.25259 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:23:39.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.729+0000 7fd7953fd740 20 client.25259 populate_metadata read hostname 'smithi006' 2024-09-12T03:23:39.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.729+0000 7fd7953fd740 1 --2- 172.21.15.6:0/4222408370 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b977d24e10 0x55b977d27270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.729+0000 7fd7953fd740 4 client.25259 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:23:39.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.729+0000 7fd7953fd740 1 -- 172.21.15.6:0/4222408370 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b977c76d20 con 0x55b977d24e10 2024-09-12T03:23:39.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.730+0000 7fd790dc0640 1 --2- 172.21.15.6:0/4222408370 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b977d24e10 0x55b977d27270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:39.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.732+0000 7fd790dc0640 1 --2- 172.21.15.6:0/4222408370 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b977d24e10 0x55b977d27270 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce0a5fff5e22e231 server_cookie=16e003633e7f4ba in_seq=0 out_seq=0 2024-09-12T03:23:39.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.732+0000 7fd76bfff640 10 client.25259 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:39.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.733+0000 7fd76bfff640 1 -- 172.21.15.6:0/4222408370 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b977c76d20 con 0x55b977d24e10 2024-09-12T03:23:39.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.733+0000 7fd76bfff640 10 client.25259 handle_command_reply: tid=0 2024-09-12T03:23:39.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.733+0000 7fd7953fd740 1 client.25259 shutdown 2024-09-12T03:23:39.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.734+0000 7fd7953fd740 1 -- 172.21.15.6:0/4222408370 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d14940 msgr2=0x55b977d11650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:39.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.734+0000 7fd7953fd740 1 --2- 172.21.15.6:0/4222408370 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d14940 0x55b977d11650 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd7800c4c70 tx=0x7fd7800b8890 comp rx=0 tx=0).stop 2024-09-12T03:23:39.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.736+0000 7fd7953fd740 1 -- 172.21.15.6:0/4222408370 shutdown_connections 2024-09-12T03:23:39.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.736+0000 7fd7953fd740 1 --2- 172.21.15.6:0/4222408370 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b977d24e10 0x55b977d27270 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.736+0000 7fd7953fd740 1 --2- 172.21.15.6:0/4222408370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d11d40 0x55b977d20a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.736+0000 7fd7953fd740 1 --2- 172.21.15.6:0/4222408370 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d14940 0x55b977d11650 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.736+0000 7fd7953fd740 1 --2- 172.21.15.6:0/4222408370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b977d0a080 0x55b977d143a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.736+0000 7fd7953fd740 1 -- 172.21.15.6:0/4222408370 >> 172.21.15.6:0/4222408370 conn(0x55b977d05cf0 msgr2=0x55b977d04650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:39.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.736+0000 7fd76bfff640 0 client.25259 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:39.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.736+0000 7fd7953fd740 1 -- 172.21.15.6:0/4222408370 shutdown_connections 2024-09-12T03:23:39.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.737+0000 7fd7953fd740 1 -- 172.21.15.6:0/4222408370 wait complete. 2024-09-12T03:23:39.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.737+0000 7fd7953fd740 20 client.25259 trim_cache size 0 max 16384 2024-09-12T03:23:39.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.775+0000 7fd7953fd740 1 Processor -- start 2024-09-12T03:23:39.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.775+0000 7fd7953fd740 1 -- start start 2024-09-12T03:23:39.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.775+0000 7fd7953fd740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d0a080 0x55b977d04d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.776+0000 7fd7953fd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d11d40 0x55b977d20840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.776+0000 7fd790dc0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d0a080 0x55b977d04d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:39.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.776+0000 7fd790dc0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d0a080 0x55b977d04d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:43312/0 (socket says 172.21.15.6:43312) 2024-09-12T03:23:39.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.776+0000 7fd78bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d11d40 0x55b977d20840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:39.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.777+0000 7fd78bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d11d40 0x55b977d20840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:43798/0 (socket says 172.21.15.6:43798) 2024-09-12T03:23:39.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.777+0000 7fd790dc0640 1 -- 172.21.15.6:0/3261545001 learned_addr learned my addr 172.21.15.6:0/3261545001 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:39.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.777+0000 7fd7953fd740 1 --2- 172.21.15.6:0/3261545001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b977d14940 0x55b977d15d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.777+0000 7fd7953fd740 1 -- 172.21.15.6:0/3261545001 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c1be8d0 con 0x55b977d14940 2024-09-12T03:23:39.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.777+0000 7fd7915c1640 1 --2- 172.21.15.6:0/3261545001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b977d14940 0x55b977d15d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:39.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.777+0000 7fd7953fd740 1 -- 172.21.15.6:0/3261545001 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c1bc1a0 con 0x55b977d0a080 2024-09-12T03:23:39.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.778+0000 7fd7953fd740 1 -- 172.21.15.6:0/3261545001 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c1b9ff0 con 0x55b977d11d40 2024-09-12T03:23:39.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.778+0000 7fd790dc0640 1 -- 172.21.15.6:0/3261545001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d11d40 msgr2=0x55b977d20840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:39.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.778+0000 7fd790dc0640 1 --2- 172.21.15.6:0/3261545001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d11d40 0x55b977d20840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.778+0000 7fd790dc0640 1 -- 172.21.15.6:0/3261545001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b977d14940 msgr2=0x55b977d15d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:39.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.778+0000 7fd790dc0640 1 --2- 172.21.15.6:0/3261545001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b977d14940 0x55b977d15d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.778+0000 7fd790dc0640 1 -- 172.21.15.6:0/3261545001 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b977d0c330 con 0x55b977d0a080 2024-09-12T03:23:39.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.779+0000 7fd790dc0640 1 --2- 172.21.15.6:0/3261545001 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d0a080 0x55b977d04d60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00fe10 tx=0x7fd77c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be91941a94e17179 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:39.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.780+0000 7fd7727fc640 1 -- 172.21.15.6:0/3261545001 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c016070 con 0x55b977d0a080 2024-09-12T03:23:39.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.780+0000 7fd7727fc640 1 -- 172.21.15.6:0/3261545001 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd77c029070 con 0x55b977d0a080 2024-09-12T03:23:39.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.781+0000 7fd7953fd740 1 -- 172.21.15.6:0/3261545001 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d0a080 msgr2=0x55b977d04d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:39.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.781+0000 7fd7953fd740 1 --2- 172.21.15.6:0/3261545001 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d0a080 0x55b977d04d60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00fe10 tx=0x7fd77c014000 comp rx=0 tx=0).stop 2024-09-12T03:23:39.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.781+0000 7fd7953fd740 1 -- 172.21.15.6:0/3261545001 shutdown_connections 2024-09-12T03:23:39.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.781+0000 7fd7953fd740 1 --2- 172.21.15.6:0/3261545001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b977d14940 0x55b977d15d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.781+0000 7fd7953fd740 1 --2- 172.21.15.6:0/3261545001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d11d40 0x55b977d20840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.781+0000 7fd7953fd740 1 --2- 172.21.15.6:0/3261545001 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d0a080 0x55b977d04d60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.781+0000 7fd7953fd740 1 -- 172.21.15.6:0/3261545001 >> 172.21.15.6:0/3261545001 conn(0x55b977d05cf0 msgr2=0x55b977d04650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:39.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.781+0000 7fd7953fd740 1 -- 172.21.15.6:0/3261545001 shutdown_connections 2024-09-12T03:23:39.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.782+0000 7fd7953fd740 1 -- 172.21.15.6:0/3261545001 wait complete. 2024-09-12T03:23:39.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.782+0000 7fd7953fd740 1 -- start start 2024-09-12T03:23:39.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.782+0000 7fd7953fd740 1 Processor -- start 2024-09-12T03:23:39.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.783+0000 7fd7953fd740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b977d11d40 0x55b977d1c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.783+0000 7fd7953fd740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d14940 0x55b977d1cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.783+0000 7fd7953fd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d1d440 0x55b977d179b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.783+0000 7fd7953fd740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c1ba660 con 0x55b977d11d40 2024-09-12T03:23:39.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.783+0000 7fd7953fd740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c1bbad0 con 0x55b977d14940 2024-09-12T03:23:39.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.783+0000 7fd7953fd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c1bad30 con 0x55b977d1d440 2024-09-12T03:23:39.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.783+0000 7fd7953fd740 10 client.0 _ll_register_callbacks cb 0x7fd7800095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:39.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.784+0000 7fd7915c1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d1d440 0x55b977d179b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:39.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.784+0000 7fd7915c1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d1d440 0x55b977d179b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:43800/0 (socket says 172.21.15.6:43800) 2024-09-12T03:23:39.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.784+0000 7fd7915c1640 1 -- 172.21.15.6:0/3048351704 learned_addr learned my addr 172.21.15.6:0/3048351704 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:39.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.784+0000 7fd78bfff640 1 --2- 172.21.15.6:0/3048351704 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d14940 0x55b977d1cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:39.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.784+0000 7fd790dc0640 1 --2- 172.21.15.6:0/3048351704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b977d11d40 0x55b977d1c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:39.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.785+0000 7fd78bfff640 1 -- 172.21.15.6:0/3048351704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d1d440 msgr2=0x55b977d179b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:39.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.785+0000 7fd78bfff640 1 --2- 172.21.15.6:0/3048351704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d1d440 0x55b977d179b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.785+0000 7fd78bfff640 1 -- 172.21.15.6:0/3048351704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b977d11d40 msgr2=0x55b977d1c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:39.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.785+0000 7fd78bfff640 1 --2- 172.21.15.6:0/3048351704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b977d11d40 0x55b977d1c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.785+0000 7fd78bfff640 1 -- 172.21.15.6:0/3048351704 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b977d181e0 con 0x55b977d14940 2024-09-12T03:23:39.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.786+0000 7fd78bfff640 1 --2- 172.21.15.6:0/3048351704 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d14940 0x55b977d1cf00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd7800bc4c0 tx=0x7fd7800036f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efa6893c7e6168f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:39.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.786+0000 7fd76bfff640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:23:39.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.786+0000 7fd76bfff640 1 -- 172.21.15.6:0/3048351704 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7800dc040 con 0x55b977d14940 2024-09-12T03:23:39.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.786+0000 7fd7953fd740 1 -- 172.21.15.6:0/3048351704 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b977bf2ec0 con 0x55b977d14940 2024-09-12T03:23:39.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.787+0000 7fd76bfff640 1 -- 172.21.15.6:0/3048351704 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd7800d5250 con 0x55b977d14940 2024-09-12T03:23:39.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.787+0000 7fd76bfff640 1 -- 172.21.15.6:0/3048351704 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd7800c08e0 con 0x55b977d14940 2024-09-12T03:23:39.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.787+0000 7fd7953fd740 10 client.25265 fetch_fsmap learned FSMap version 26 2024-09-12T03:23:39.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.787+0000 7fd7953fd740 1 -- 172.21.15.6:0/3048351704 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b977d90720 con 0x55b977d14940 2024-09-12T03:23:39.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.789+0000 7fd76bfff640 1 -- 172.21.15.6:0/3048351704 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b977bf2ec0 con 0x55b977d14940 2024-09-12T03:23:39.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.789+0000 7fd76bfff640 1 -- 172.21.15.6:0/3048351704 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd7800a5ac0 con 0x55b977d14940 2024-09-12T03:23:39.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.789+0000 7fd7953fd740 10 client.25265 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:23:39.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.789+0000 7fd7953fd740 10 client.25265 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:23:39.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.789+0000 7fd7953fd740 20 client.25265 populate_metadata read hostname 'smithi006' 2024-09-12T03:23:39.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.789+0000 7fd7953fd740 1 --2- 172.21.15.6:0/3048351704 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b977d90e20 0x55b977d93240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.789+0000 7fd7953fd740 4 client.25265 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}] 2024-09-12T03:23:39.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.789+0000 7fd7953fd740 1 -- 172.21.15.6:0/3048351704 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}) -- 0x55b977c405e0 con 0x55b977d90e20 2024-09-12T03:23:39.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.790+0000 7fd790dc0640 1 --2- 172.21.15.6:0/3048351704 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b977d90e20 0x55b977d93240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:39.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.790+0000 7fd76bfff640 10 client.25265 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:39.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:39.791+0000 7fd790dc0640 1 --2- 172.21.15.6:0/3048351704 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b977d90e20 0x55b977d93240 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d245bc58f78d64e7 server_cookie=b66cd9886eca8345 in_seq=0 out_seq=0 2024-09-12T03:23:40.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.663+0000 7fd76bfff640 1 -- 172.21.15.6:0/3048351704 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b977c405e0 con 0x55b977d90e20 2024-09-12T03:23:40.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.663+0000 7fd76bfff640 10 client.25265 handle_command_reply: tid=0 2024-09-12T03:23:40.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.663+0000 7fd7953fd740 1 client.25265 shutdown 2024-09-12T03:23:40.663 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:40 smithi006 ceph-mon[30991]: pgmap v412: 97 pgs: 97 active+clean; 13 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 302 MiB/s wr, 25.66k op/s 2024-09-12T03:23:40.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.663+0000 7fd7953fd740 1 -- 172.21.15.6:0/3048351704 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d14940 msgr2=0x55b977d1cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:40.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.663+0000 7fd7953fd740 1 --2- 172.21.15.6:0/3048351704 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d14940 0x55b977d1cf00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd7800bc4c0 tx=0x7fd7800036f0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.663+0000 7fd7953fd740 1 -- 172.21.15.6:0/3048351704 shutdown_connections 2024-09-12T03:23:40.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.663+0000 7fd7953fd740 1 --2- 172.21.15.6:0/3048351704 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b977d90e20 0x55b977d93240 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.663+0000 7fd7953fd740 1 --2- 172.21.15.6:0/3048351704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b977d1d440 0x55b977d179b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.663+0000 7fd7953fd740 1 --2- 172.21.15.6:0/3048351704 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b977d14940 0x55b977d1cf00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.663+0000 7fd7953fd740 1 --2- 172.21.15.6:0/3048351704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b977d11d40 0x55b977d1c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.663+0000 7fd7953fd740 1 -- 172.21.15.6:0/3048351704 >> 172.21.15.6:0/3048351704 conn(0x55b977d05cf0 msgr2=0x55b977d0b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:40.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.663+0000 7fd76bfff640 0 client.25265 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:23:40.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.663+0000 7fd7953fd740 1 -- 172.21.15.6:0/3048351704 shutdown_connections 2024-09-12T03:23:40.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.663+0000 7fd7953fd740 1 -- 172.21.15.6:0/3048351704 wait complete. 2024-09-12T03:23:40.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.663+0000 7fd7953fd740 20 client.25265 trim_cache size 0 max 16384 2024-09-12T03:23:40.665 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:23:40.665 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 26, 2024-09-12T03:23:40.665 INFO:teuthology.orchestra.run.smithi006.stdout: "leader": 24494, 2024-09-12T03:23:40.666 INFO:teuthology.orchestra.run.smithi006.stdout: "set_version": 10, 2024-09-12T03:23:40.666 INFO:teuthology.orchestra.run.smithi006.stdout: "sets": { 2024-09-12T03:23:40.666 INFO:teuthology.orchestra.run.smithi006.stdout: "f395d0fc": { 2024-09-12T03:23:40.666 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 10, 2024-09-12T03:23:40.666 INFO:teuthology.orchestra.run.smithi006.stdout: "age_ref": 0.0, 2024-09-12T03:23:40.666 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:23:40.666 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "QUIESCED", 2024-09-12T03:23:40.666 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:23:40.666 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:23:40.666 INFO:teuthology.orchestra.run.smithi006.stdout: "timeout": 90.0, 2024-09-12T03:23:40.666 INFO:teuthology.orchestra.run.smithi006.stdout: "expiration": 144.7, 2024-09-12T03:23:40.666 INFO:teuthology.orchestra.run.smithi006.stdout: "members": { 2024-09-12T03:23:40.666 INFO:teuthology.orchestra.run.smithi006.stdout: "file:/": { 2024-09-12T03:23:40.666 INFO:teuthology.orchestra.run.smithi006.stdout: "excluded": false, 2024-09-12T03:23:40.667 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:23:40.667 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "QUIESCED", 2024-09-12T03:23:40.667 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:23:40.667 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:23:40.667 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:23:40.667 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:23:40.667 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:23:40.667 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:23:40.667 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:23:40.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.664+0000 7fd793024640 1 -- 172.21.15.6:0/1837370088 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd7740a2c00 msgr2=0x7fd7740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:40.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.664+0000 7fd793024640 1 --2- 172.21.15.6:0/1837370088 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd7740a2c00 0x7fd7740a50c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd77c0058c0 tx=0x7fd77c005850 comp rx=0 tx=0).stop 2024-09-12T03:23:40.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.664+0000 7fd793024640 1 -- 172.21.15.6:0/1837370088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd78c102790 msgr2=0x7fd78c10bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:40.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.664+0000 7fd793024640 1 --2- 172.21.15.6:0/1837370088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd78c102790 0x7fd78c10bad0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd780006010 tx=0x7fd780004b80 comp rx=0 tx=0).stop 2024-09-12T03:23:40.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.664+0000 7fd793024640 1 -- 172.21.15.6:0/1837370088 shutdown_connections 2024-09-12T03:23:40.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.664+0000 7fd793024640 1 --2- 172.21.15.6:0/1837370088 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd7740a2c00 0x7fd7740a50c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.664+0000 7fd793024640 1 --2- 172.21.15.6:0/1837370088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd78c106f10 0x7fd78c10c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.664+0000 7fd793024640 1 --2- 172.21.15.6:0/1837370088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd78c102790 0x7fd78c10bad0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.664+0000 7fd793024640 1 --2- 172.21.15.6:0/1837370088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd78c101de0 0x7fd78c10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.664+0000 7fd793024640 1 -- 172.21.15.6:0/1837370088 >> 172.21.15.6:0/1837370088 conn(0x7fd78c0fd880 msgr2=0x7fd78c0fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:40.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.664+0000 7fd793024640 1 -- 172.21.15.6:0/1837370088 shutdown_connections 2024-09-12T03:23:40.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:40.664+0000 7fd793024640 1 -- 172.21.15.6:0/1837370088 wait complete. 2024-09-12T03:23:40.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:40 smithi118 ceph-mon[35048]: pgmap v412: 97 pgs: 97 active+clean; 13 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 302 MiB/s wr, 25.66k op/s 2024-09-12T03:23:40.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:40 smithi161 ceph-mon[35485]: pgmap v412: 97 pgs: 97 active+clean; 13 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 302 MiB/s wr, 25.66k op/s 2024-09-12T03:23:41.444 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f395d0fc', quiesce took 7.7 seconds. Will release after: 17.0 2024-09-12T03:23:41.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:41 smithi118 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-12T03:23:41.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:41 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:41.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:41 smithi118 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-12T03:23:41.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:41 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:41.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:41 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:41.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:41 smithi006 ceph-mon[30991]: scrub summary: active paths [/] 2024-09-12T03:23:41.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:41 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:41.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:41 smithi006 ceph-mon[30991]: scrub summary: active paths [/] 2024-09-12T03:23:41.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:41 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:41.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:41 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:41.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:41 smithi161 ceph-mon[35485]: scrub summary: active paths [/] 2024-09-12T03:23:41.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:41 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:41.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:41 smithi161 ceph-mon[35485]: scrub summary: active paths [/] 2024-09-12T03:23:41.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:41 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:41.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:41 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:42.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:42 smithi118 ceph-mon[35048]: pgmap v413: 97 pgs: 97 active+clean; 13 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 816 KiB/s rd, 255 MiB/s wr, 21.67k op/s 2024-09-12T03:23:42.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:42 smithi006 ceph-mon[30991]: pgmap v413: 97 pgs: 97 active+clean; 13 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 816 KiB/s rd, 255 MiB/s wr, 21.67k op/s 2024-09-12T03:23:42.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:42 smithi161 ceph-mon[35485]: pgmap v413: 97 pgs: 97 active+clean; 13 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 816 KiB/s rd, 255 MiB/s wr, 21.67k op/s 2024-09-12T03:23:44.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:44 smithi118 ceph-mon[35048]: pgmap v414: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1010 GiB / 1.0 TiB avail; 1009 KiB/s rd, 267 MiB/s wr, 23.25k op/s 2024-09-12T03:23:44.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:44 smithi006 ceph-mon[30991]: pgmap v414: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1010 GiB / 1.0 TiB avail; 1009 KiB/s rd, 267 MiB/s wr, 23.25k op/s 2024-09-12T03:23:44.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:44 smithi161 ceph-mon[35485]: pgmap v414: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1010 GiB / 1.0 TiB avail; 1009 KiB/s rd, 267 MiB/s wr, 23.25k op/s 2024-09-12T03:23:46.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:46 smithi118 ceph-mon[35048]: pgmap v415: 97 pgs: 97 active+clean; 13 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 880 KiB/s rd, 222 MiB/s wr, 19.55k op/s 2024-09-12T03:23:46.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:46 smithi118 ceph-mon[35048]: scrub summary: idle 2024-09-12T03:23:46.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:46 smithi118 ceph-mon[35048]: scrub summary: idle 2024-09-12T03:23:46.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:46 smithi006 ceph-mon[30991]: pgmap v415: 97 pgs: 97 active+clean; 13 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 880 KiB/s rd, 222 MiB/s wr, 19.55k op/s 2024-09-12T03:23:46.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:46 smithi006 ceph-mon[30991]: scrub summary: idle 2024-09-12T03:23:46.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:46 smithi006 ceph-mon[30991]: scrub summary: idle 2024-09-12T03:23:46.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:46 smithi161 ceph-mon[35485]: pgmap v415: 97 pgs: 97 active+clean; 13 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 880 KiB/s rd, 222 MiB/s wr, 19.55k op/s 2024-09-12T03:23:46.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:46 smithi161 ceph-mon[35485]: scrub summary: idle 2024-09-12T03:23:46.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:46 smithi161 ceph-mon[35485]: scrub summary: idle 2024-09-12T03:23:47.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:47 smithi118 ceph-mon[35048]: scrub summary: idle 2024-09-12T03:23:47.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:47 smithi006 ceph-mon[30991]: scrub summary: idle 2024-09-12T03:23:47.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:47 smithi161 ceph-mon[35485]: scrub summary: idle 2024-09-12T03:23:48.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:48 smithi118 ceph-mon[35048]: pgmap v416: 97 pgs: 97 active+clean; 13 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 667 KiB/s rd, 188 MiB/s wr, 16.57k op/s 2024-09-12T03:23:48.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:48 smithi006 ceph-mon[30991]: pgmap v416: 97 pgs: 97 active+clean; 13 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 667 KiB/s rd, 188 MiB/s wr, 16.57k op/s 2024-09-12T03:23:48.862 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:48 smithi161 ceph-mon[35485]: pgmap v416: 97 pgs: 97 active+clean; 13 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 667 KiB/s rd, 188 MiB/s wr, 16.57k op/s 2024-09-12T03:23:50.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:50 smithi118 ceph-mon[35048]: pgmap v417: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 670 KiB/s rd, 159 MiB/s wr, 14.02k op/s 2024-09-12T03:23:50.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:50 smithi006 ceph-mon[30991]: pgmap v417: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 670 KiB/s rd, 159 MiB/s wr, 14.02k op/s 2024-09-12T03:23:50.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:50 smithi161 ceph-mon[35485]: pgmap v417: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 670 KiB/s rd, 159 MiB/s wr, 14.02k op/s 2024-09-12T03:23:52.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:52 smithi118 ceph-mon[35048]: pgmap v418: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 197 KiB/s rd, 83 MiB/s wr, 7.64k op/s 2024-09-12T03:23:52.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:52 smithi006 ceph-mon[30991]: pgmap v418: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 197 KiB/s rd, 83 MiB/s wr, 7.64k op/s 2024-09-12T03:23:52.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:52 smithi161 ceph-mon[35485]: pgmap v418: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 197 KiB/s rd, 83 MiB/s wr, 7.64k op/s 2024-09-12T03:23:54.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:54 smithi118 ceph-mon[35048]: pgmap v419: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 198 KiB/s rd, 81 MiB/s wr, 7.31k op/s 2024-09-12T03:23:54.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:54 smithi006 ceph-mon[30991]: pgmap v419: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 198 KiB/s rd, 81 MiB/s wr, 7.31k op/s 2024-09-12T03:23:54.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:54 smithi161 ceph-mon[35485]: pgmap v419: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 198 KiB/s rd, 81 MiB/s wr, 7.31k op/s 2024-09-12T03:23:56.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:56 smithi118 ceph-mon[35048]: pgmap v420: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 46 MiB/s wr, 3.84k op/s 2024-09-12T03:23:56.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:56 smithi006 ceph-mon[30991]: pgmap v420: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 46 MiB/s wr, 3.84k op/s 2024-09-12T03:23:56.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:56 smithi161 ceph-mon[35485]: pgmap v420: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 46 MiB/s wr, 3.84k op/s 2024-09-12T03:23:58.445 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:23:58.701 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:58.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:23:58 smithi118 ceph-mon[35048]: pgmap v421: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 40 MiB/s wr, 3.10k op/s 2024-09-12T03:23:58.823 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:23:58 smithi006 ceph-mon[30991]: pgmap v421: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 40 MiB/s wr, 3.10k op/s 2024-09-12T03:23:58.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:23:58 smithi161 ceph-mon[35485]: pgmap v421: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 40 MiB/s wr, 3.10k op/s 2024-09-12T03:23:59.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.705+0000 7fbc32314640 1 -- 172.21.15.6:0/3445206533 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc2c102790 msgr2=0x7fbc2c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.705+0000 7fbc32314640 1 --2- 172.21.15.6:0/3445206533 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc2c102790 0x7fbc2c1067e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fbc20009900 tx=0x7fbc2002a410 comp rx=0 tx=0).stop 2024-09-12T03:23:59.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.706+0000 7fbc32314640 1 -- 172.21.15.6:0/3445206533 shutdown_connections 2024-09-12T03:23:59.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.706+0000 7fbc32314640 1 --2- 172.21.15.6:0/3445206533 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc2c106f10 0x7fbc2c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.706+0000 7fbc32314640 1 --2- 172.21.15.6:0/3445206533 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc2c102790 0x7fbc2c1067e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.706+0000 7fbc32314640 1 --2- 172.21.15.6:0/3445206533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc2c101de0 0x7fbc2c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.706+0000 7fbc32314640 1 -- 172.21.15.6:0/3445206533 >> 172.21.15.6:0/3445206533 conn(0x7fbc2c0fd8c0 msgr2=0x7fbc2c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:59.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.706+0000 7fbc32314640 1 -- 172.21.15.6:0/3445206533 shutdown_connections 2024-09-12T03:23:59.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.706+0000 7fbc32314640 1 -- 172.21.15.6:0/3445206533 wait complete. 2024-09-12T03:23:59.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.706+0000 7fbc32314640 1 Processor -- start 2024-09-12T03:23:59.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.706+0000 7fbc32314640 1 -- start start 2024-09-12T03:23:59.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.707+0000 7fbc32314640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc2c101de0 0x7fbc2c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.707+0000 7fbc32314640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc2c102790 0x7fbc2c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.707+0000 7fbc32314640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc2c106f10 0x7fbc2c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.707+0000 7fbc32314640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c111f70 con 0x7fbc2c106f10 2024-09-12T03:23:59.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.707+0000 7fbc32314640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c111df0 con 0x7fbc2c101de0 2024-09-12T03:23:59.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.707+0000 7fbc32314640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c1120f0 con 0x7fbc2c102790 2024-09-12T03:23:59.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.707+0000 7fbc308b1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc2c106f10 0x7fbc2c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.707+0000 7fbc308b1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc2c106f10 0x7fbc2c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54408/0 (socket says 172.21.15.6:54408) 2024-09-12T03:23:59.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.707+0000 7fbc308b1640 1 -- 172.21.15.6:0/1023578239 learned_addr learned my addr 172.21.15.6:0/1023578239 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:23:59.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.707+0000 7fbc2b7fe640 1 --2- 172.21.15.6:0/1023578239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc2c102790 0x7fbc2c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.707+0000 7fbc2bfff640 1 --2- 172.21.15.6:0/1023578239 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc2c101de0 0x7fbc2c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.707+0000 7fbc308b1640 1 -- 172.21.15.6:0/1023578239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc2c102790 msgr2=0x7fbc2c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.707+0000 7fbc308b1640 1 --2- 172.21.15.6:0/1023578239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc2c102790 0x7fbc2c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.707+0000 7fbc308b1640 1 -- 172.21.15.6:0/1023578239 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc2c101de0 msgr2=0x7fbc2c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.707+0000 7fbc308b1640 1 --2- 172.21.15.6:0/1023578239 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc2c101de0 0x7fbc2c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.707+0000 7fbc308b1640 1 -- 172.21.15.6:0/1023578239 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2c1a6910 con 0x7fbc2c106f10 2024-09-12T03:23:59.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.707+0000 7fbc2bfff640 1 --2- 172.21.15.6:0/1023578239 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc2c101de0 0x7fbc2c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:59.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.707+0000 7fbc308b1640 1 --2- 172.21.15.6:0/1023578239 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc2c106f10 0x7fbc2c074f70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400eb10 tx=0x7fbc2400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:59.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.708+0000 7fbc297fa640 1 -- 172.21.15.6:0/1023578239 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24004b20 con 0x7fbc2c106f10 2024-09-12T03:23:59.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.708+0000 7fbc32314640 1 -- 172.21.15.6:0/1023578239 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2c1a6c00 con 0x7fbc2c106f10 2024-09-12T03:23:59.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.708+0000 7fbc297fa640 1 -- 172.21.15.6:0/1023578239 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc24004cc0 con 0x7fbc2c106f10 2024-09-12T03:23:59.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.708+0000 7fbc297fa640 1 -- 172.21.15.6:0/1023578239 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24010760 con 0x7fbc2c106f10 2024-09-12T03:23:59.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.708+0000 7fbc32314640 1 -- 172.21.15.6:0/1023578239 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc2c1a70c0 con 0x7fbc2c106f10 2024-09-12T03:23:59.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.709+0000 7fbc32314640 1 -- 172.21.15.6:0/1023578239 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc2c1021c0 con 0x7fbc2c106f10 2024-09-12T03:23:59.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.710+0000 7fbc297fa640 1 -- 172.21.15.6:0/1023578239 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fbc24005020 con 0x7fbc2c106f10 2024-09-12T03:23:59.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.711+0000 7fbc297fa640 1 --2- 172.21.15.6:0/1023578239 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fbc140a2b80 0x7fbc140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.711+0000 7fbc297fa640 1 -- 172.21.15.6:0/1023578239 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fbc240cf5b0 con 0x7fbc2c106f10 2024-09-12T03:23:59.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.711+0000 7fbc2bfff640 1 --2- 172.21.15.6:0/1023578239 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fbc140a2b80 0x7fbc140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.711+0000 7fbc2bfff640 1 --2- 172.21.15.6:0/1023578239 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fbc140a2b80 0x7fbc140a5040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009c20 tx=0x7fbc1c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:59.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.716+0000 7fbc297fa640 1 -- 172.21.15.6:0/1023578239 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc2409c540 con 0x7fbc2c106f10 2024-09-12T03:23:59.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.933+0000 7fbc32314640 1 -- 172.21.15.6:0/1023578239 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc2c075a20 con 0x7fbc2c106f10 2024-09-12T03:23:59.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.934+0000 7fbc297fa640 1 -- 172.21.15.6:0/1023578239 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11911 (secure 0 0 0) 0x7fbc2409be40 con 0x7fbc2c106f10 2024-09-12T03:23:59.934 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:23:59.935 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":26,"btime":"2024-09-12T03:22:16:132848+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:22:16.064679+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24463,24464,24494]},"id":1}]} 2024-09-12T03:23:59.935 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 26 2024-09-12T03:23:59.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.936+0000 7fbc32314640 1 -- 172.21.15.6:0/1023578239 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fbc140a2b80 msgr2=0x7fbc140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.936+0000 7fbc32314640 1 --2- 172.21.15.6:0/1023578239 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fbc140a2b80 0x7fbc140a5040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009c20 tx=0x7fbc1c005f90 comp rx=0 tx=0).stop 2024-09-12T03:23:59.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.936+0000 7fbc32314640 1 -- 172.21.15.6:0/1023578239 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc2c106f10 msgr2=0x7fbc2c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.936+0000 7fbc32314640 1 --2- 172.21.15.6:0/1023578239 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc2c106f10 0x7fbc2c074f70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400eb10 tx=0x7fbc2400c2d0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.937+0000 7fbc32314640 1 -- 172.21.15.6:0/1023578239 shutdown_connections 2024-09-12T03:23:59.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.937+0000 7fbc32314640 1 --2- 172.21.15.6:0/1023578239 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fbc140a2b80 0x7fbc140a5040 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.937+0000 7fbc32314640 1 --2- 172.21.15.6:0/1023578239 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc2c106f10 0x7fbc2c074f70 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.937+0000 7fbc32314640 1 --2- 172.21.15.6:0/1023578239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc2c102790 0x7fbc2c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.937+0000 7fbc32314640 1 --2- 172.21.15.6:0/1023578239 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc2c101de0 0x7fbc2c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.937+0000 7fbc32314640 1 -- 172.21.15.6:0/1023578239 >> 172.21.15.6:0/1023578239 conn(0x7fbc2c0fd8c0 msgr2=0x7fbc2c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:59.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.937+0000 7fbc32314640 1 -- 172.21.15.6:0/1023578239 shutdown_connections 2024-09-12T03:23:59.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:23:59.937+0000 7fbc32314640 1 -- 172.21.15.6:0/1023578239 wait complete. 2024-09-12T03:24:00.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:00 smithi006 ceph-mon[30991]: pgmap v422: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 44 MiB/s wr, 3.54k op/s 2024-09-12T03:24:00.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:00 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1023578239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:00.703 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id f395d0fc --release --await' 2024-09-12T03:24:00.703 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id f395d0fc --release --await 2024-09-12T03:24:00.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:00 smithi118 ceph-mon[35048]: pgmap v422: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 44 MiB/s wr, 3.54k op/s 2024-09-12T03:24:00.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:00 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1023578239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:00.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:00 smithi161 ceph-mon[35485]: pgmap v422: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 44 MiB/s wr, 3.54k op/s 2024-09-12T03:24:00.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:00 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1023578239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:00.962 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:01.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.477+0000 7f3025a99640 1 -- 172.21.15.6:0/3367851800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3020102d30 msgr2=0x7f3020103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:01.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.477+0000 7f3025a99640 1 --2- 172.21.15.6:0/3367851800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3020102d30 0x7f3020103110 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3010009930 tx=0x7f301002a420 comp rx=0 tx=0).stop 2024-09-12T03:24:01.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.477+0000 7f3025a99640 1 -- 172.21.15.6:0/3367851800 shutdown_connections 2024-09-12T03:24:01.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.477+0000 7f3025a99640 1 --2- 172.21.15.6:0/3367851800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30201082a0 0x7f302010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.477+0000 7f3025a99640 1 --2- 172.21.15.6:0/3367851800 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f30201036e0 0x7f3020107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.477+0000 7f3025a99640 1 --2- 172.21.15.6:0/3367851800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3020102d30 0x7f3020103110 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.477+0000 7f3025a99640 1 -- 172.21.15.6:0/3367851800 >> 172.21.15.6:0/3367851800 conn(0x7f30200fc7d0 msgr2=0x7f30200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:01.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.477+0000 7f3025a99640 1 -- 172.21.15.6:0/3367851800 shutdown_connections 2024-09-12T03:24:01.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.477+0000 7f3025a99640 1 -- 172.21.15.6:0/3367851800 wait complete. 2024-09-12T03:24:01.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.477+0000 7f3025a99640 1 Processor -- start 2024-09-12T03:24:01.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.478+0000 7f3025a99640 1 -- start start 2024-09-12T03:24:01.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.478+0000 7f3025a99640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3020102d30 0x7f30201a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.478+0000 7f3025a99640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30201036e0 0x7f30201a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.478+0000 7f3025a99640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f30201082a0 0x7f302019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.478+0000 7f3025a99640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f30201120a0 con 0x7f3020102d30 2024-09-12T03:24:01.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.478+0000 7f3025a99640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f3020111f20 con 0x7f30201082a0 2024-09-12T03:24:01.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.478+0000 7f3025a99640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3020112220 con 0x7f30201036e0 2024-09-12T03:24:01.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.478+0000 7f301f7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3020102d30 0x7f30201a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.478+0000 7f301f7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3020102d30 0x7f30201a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54424/0 (socket says 172.21.15.6:54424) 2024-09-12T03:24:01.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.478+0000 7f301f7fe640 1 -- 172.21.15.6:0/1935526458 learned_addr learned my addr 172.21.15.6:0/1935526458 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:01.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.478+0000 7f301effd640 1 --2- 172.21.15.6:0/1935526458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30201036e0 0x7f30201a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.478+0000 7f301f7fe640 1 -- 172.21.15.6:0/1935526458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30201036e0 msgr2=0x7f30201a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:01.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.478+0000 7f301f7fe640 1 --2- 172.21.15.6:0/1935526458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30201036e0 0x7f30201a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.478+0000 7f301ffff640 1 --2- 172.21.15.6:0/1935526458 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f30201082a0 0x7f302019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.478+0000 7f301f7fe640 1 -- 172.21.15.6:0/1935526458 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f30201082a0 msgr2=0x7f302019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:01.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.478+0000 7f301f7fe640 1 --2- 172.21.15.6:0/1935526458 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f30201082a0 0x7f302019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.478+0000 7f301f7fe640 1 -- 172.21.15.6:0/1935526458 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302019ad00 con 0x7f3020102d30 2024-09-12T03:24:01.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.478+0000 7f301f7fe640 1 --2- 172.21.15.6:0/1935526458 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3020102d30 0x7f30201a0410 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3010009230 tx=0x7f3010004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:01.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.479+0000 7f301cff9640 1 -- 172.21.15.6:0/1935526458 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301003f030 con 0x7f3020102d30 2024-09-12T03:24:01.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.479+0000 7f301cff9640 1 -- 172.21.15.6:0/1935526458 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3010003ad0 con 0x7f3020102d30 2024-09-12T03:24:01.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.479+0000 7f3025a99640 1 -- 172.21.15.6:0/1935526458 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302019af90 con 0x7f3020102d30 2024-09-12T03:24:01.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.479+0000 7f301cff9640 1 -- 172.21.15.6:0/1935526458 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010035070 con 0x7f3020102d30 2024-09-12T03:24:01.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.479+0000 7f3025a99640 1 -- 172.21.15.6:0/1935526458 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30201a72e0 con 0x7f3020102d30 2024-09-12T03:24:01.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.482+0000 7f301cff9640 1 -- 172.21.15.6:0/1935526458 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3010042030 con 0x7f3020102d30 2024-09-12T03:24:01.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.482+0000 7f3027e72740 1 Processor -- start 2024-09-12T03:24:01.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.482+0000 7f3027e72740 1 -- start start 2024-09-12T03:24:01.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.482+0000 7f3027e72740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f060 0x558144c2f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.482+0000 7f3027e72740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c2f980 0x558144c39d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.482+0000 7f3027e72740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c3a240 0x558144c3c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.482+0000 7f3027e72740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x558144b94620 con 0x558144c3a240 2024-09-12T03:24:01.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.482+0000 7f3027e72740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x558144b3be50 con 0x558144c2f060 2024-09-12T03:24:01.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.482+0000 7f3027e72740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558144b905e0 con 0x558144c2f980 2024-09-12T03:24:01.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.482+0000 7f301effd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c3a240 0x558144c3c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.482+0000 7f301f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c2f980 0x558144c39d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.482+0000 7f301effd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c3a240 0x558144c3c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54438/0 (socket says 172.21.15.6:54438) 2024-09-12T03:24:01.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.482+0000 7f301effd640 1 -- 172.21.15.6:0/2034578419 learned_addr learned my addr 172.21.15.6:0/2034578419 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:01.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.482+0000 7f301ffff640 1 --2- 172.21.15.6:0/2034578419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f060 0x558144c2f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.483+0000 7f301effd640 1 -- 172.21.15.6:0/2034578419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c2f980 msgr2=0x558144c39d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:01.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.483+0000 7f301effd640 1 --2- 172.21.15.6:0/2034578419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c2f980 0x558144c39d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.483+0000 7f301effd640 1 -- 172.21.15.6:0/2034578419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f060 msgr2=0x558144c2f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:01.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.483+0000 7f301effd640 1 --2- 172.21.15.6:0/2034578419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f060 0x558144c2f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.483+0000 7f301effd640 1 -- 172.21.15.6:0/2034578419 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558144b2cd40 con 0x558144c3a240 2024-09-12T03:24:01.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.483+0000 7f301ffff640 1 --2- 172.21.15.6:0/2034578419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f060 0x558144c2f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:01.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.483+0000 7f301cff9640 1 --2- 172.21.15.6:0/1935526458 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f30080a2b80 0x7f30080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.483+0000 7f301effd640 1 --2- 172.21.15.6:0/2034578419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c3a240 0x558144c3c630 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f30200697d0 tx=0x7f30140107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6cab523f182df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:01.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.483+0000 7f301ffff640 1 --2- 172.21.15.6:0/1935526458 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f30080a2b80 0x7f30080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.483+0000 7f301cff9640 1 -- 172.21.15.6:0/1935526458 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f30100ef690 con 0x7f3020102d30 2024-09-12T03:24:01.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.483+0000 7f3005ffb640 1 -- 172.21.15.6:0/2034578419 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301401c480 con 0x558144c3a240 2024-09-12T03:24:01.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.483+0000 7f3005ffb640 1 -- 172.21.15.6:0/2034578419 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f301401cb00 con 0x558144c3a240 2024-09-12T03:24:01.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.483+0000 7f3005ffb640 1 -- 172.21.15.6:0/2034578419 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3014011c50 con 0x558144c3a240 2024-09-12T03:24:01.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.484+0000 7f3027e72740 1 -- 172.21.15.6:0/2034578419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c3a240 msgr2=0x558144c3c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:01.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.484+0000 7f301ffff640 1 --2- 172.21.15.6:0/1935526458 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f30080a2b80 0x7f30080a5040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3018005fd0 tx=0x7f3018009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:01.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.484+0000 7f3027e72740 1 --2- 172.21.15.6:0/2034578419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c3a240 0x558144c3c630 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f30200697d0 tx=0x7f30140107a0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.484+0000 7f3027e72740 1 -- 172.21.15.6:0/2034578419 shutdown_connections 2024-09-12T03:24:01.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.484+0000 7f3027e72740 1 --2- 172.21.15.6:0/2034578419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c3a240 0x558144c3c630 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.484+0000 7f3027e72740 1 --2- 172.21.15.6:0/2034578419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c2f980 0x558144c39d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.484+0000 7f3027e72740 1 --2- 172.21.15.6:0/2034578419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f060 0x558144c2f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.484+0000 7f3027e72740 1 -- 172.21.15.6:0/2034578419 >> 172.21.15.6:0/2034578419 conn(0x558144c2b5f0 msgr2=0x558144c2b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:01.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.484+0000 7f3027e72740 1 -- 172.21.15.6:0/2034578419 shutdown_connections 2024-09-12T03:24:01.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.484+0000 7f3027e72740 1 -- 172.21.15.6:0/2034578419 wait complete. 2024-09-12T03:24:01.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.484+0000 7f3027e72740 1 -- start start 2024-09-12T03:24:01.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.485+0000 7f3027e72740 1 Processor -- start 2024-09-12T03:24:01.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.485+0000 7f3027e72740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f060 0x558144c370b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.485+0000 7f3027e72740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c2f980 0x558144c375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.485+0000 7f3027e72740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c3a240 0x558144c37b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.485+0000 7f3027e72740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x558144c3ce10 con 0x558144c3a240 2024-09-12T03:24:01.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.485+0000 7f3027e72740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x558144b1cf10 con 0x558144c2f060 2024-09-12T03:24:01.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.485+0000 7f3027e72740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558144c3cf90 con 0x558144c2f980 2024-09-12T03:24:01.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.485+0000 7f3027e72740 10 client.0 _ll_register_callbacks cb 0x558144b40db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:01.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.485+0000 7f301ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c3a240 0x558144c37b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.485+0000 7f301ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c3a240 0x558144c37b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54444/0 (socket says 172.21.15.6:54444) 2024-09-12T03:24:01.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.485+0000 7f301ffff640 1 -- 172.21.15.6:0/2713352230 learned_addr learned my addr 172.21.15.6:0/2713352230 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:01.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.485+0000 7f301effd640 1 --2- 172.21.15.6:0/2713352230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c2f980 0x558144c375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.485+0000 7f301f7fe640 1 --2- 172.21.15.6:0/2713352230 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f060 0x558144c370b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.485+0000 7f301ffff640 1 -- 172.21.15.6:0/2713352230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c2f980 msgr2=0x558144c375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:01.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.485+0000 7f301ffff640 1 --2- 172.21.15.6:0/2713352230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c2f980 0x558144c375f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.485+0000 7f301ffff640 1 -- 172.21.15.6:0/2713352230 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f060 msgr2=0x558144c370b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:01.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.485+0000 7f301ffff640 1 --2- 172.21.15.6:0/2713352230 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f060 0x558144c370b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.485+0000 7f301ffff640 1 -- 172.21.15.6:0/2713352230 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581448a3580 con 0x558144c3a240 2024-09-12T03:24:01.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.485+0000 7f301effd640 1 --2- 172.21.15.6:0/2713352230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c2f980 0x558144c375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:01.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.485+0000 7f301ffff640 1 --2- 172.21.15.6:0/2713352230 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c3a240 0x558144c37b30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3018012890 tx=0x7f3018004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43aa064b906108a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:01.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.485+0000 7f2ffeffd640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:24:01.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.486+0000 7f2ffeffd640 1 -- 172.21.15.6:0/2713352230 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301800fce0 con 0x558144c3a240 2024-09-12T03:24:01.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.486+0000 7f3027e72740 1 -- 172.21.15.6:0/2713352230 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f301401cb00 con 0x558144c3a240 2024-09-12T03:24:01.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.486+0000 7f2ffeffd640 1 -- 172.21.15.6:0/2713352230 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f301801f420 con 0x558144c3a240 2024-09-12T03:24:01.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.486+0000 7f2ffeffd640 1 -- 172.21.15.6:0/2713352230 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f301800fe80 con 0x558144c3a240 2024-09-12T03:24:01.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.486+0000 7f3027e72740 10 client.15732 fetch_fsmap learned FSMap version 26 2024-09-12T03:24:01.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.486+0000 7f3027e72740 1 -- 172.21.15.6:0/2713352230 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55814496d960 con 0x558144c3a240 2024-09-12T03:24:01.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.486+0000 7f2ffeffd640 1 -- 172.21.15.6:0/2713352230 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f301401cb00 con 0x558144c3a240 2024-09-12T03:24:01.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.486+0000 7f2ffeffd640 1 -- 172.21.15.6:0/2713352230 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3018027760 con 0x558144c3a240 2024-09-12T03:24:01.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.486+0000 7f3027e72740 10 client.15732 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:24:01.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.486+0000 7f3027e72740 10 client.15732 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:24:01.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.486+0000 7f3027e72740 20 client.15732 populate_metadata read hostname 'smithi006' 2024-09-12T03:24:01.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.486+0000 7f3027e72740 1 --2- 172.21.15.6:0/2713352230 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x558144c49e60 0x558144c4c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.486+0000 7f3027e72740 4 client.15732 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:24:01.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.486+0000 7f3027e72740 1 -- 172.21.15.6:0/2713352230 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558144ba1b90 con 0x558144c49e60 2024-09-12T03:24:01.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.487+0000 7f301f7fe640 1 --2- 172.21.15.6:0/2713352230 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x558144c49e60 0x558144c4c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.487+0000 7f301f7fe640 1 --2- 172.21.15.6:0/2713352230 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x558144c49e60 0x558144c4c250 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0f431255bbfcc2a server_cookie=81b5b734fb4964ea in_seq=0 out_seq=0 2024-09-12T03:24:01.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.487+0000 7f2ffeffd640 10 client.15732 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:01.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.488+0000 7f2ffeffd640 1 -- 172.21.15.6:0/2713352230 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558144ba1b90 con 0x558144c49e60 2024-09-12T03:24:01.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.488+0000 7f2ffeffd640 10 client.15732 handle_command_reply: tid=0 2024-09-12T03:24:01.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.488+0000 7f3027e72740 1 client.15732 shutdown 2024-09-12T03:24:01.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.488+0000 7f3027e72740 1 -- 172.21.15.6:0/2713352230 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c3a240 msgr2=0x558144c37b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:01.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.488+0000 7f3027e72740 1 --2- 172.21.15.6:0/2713352230 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c3a240 0x558144c37b30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3018012890 tx=0x7f3018004a40 comp rx=0 tx=0).stop 2024-09-12T03:24:01.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.488+0000 7f3027e72740 1 -- 172.21.15.6:0/2713352230 shutdown_connections 2024-09-12T03:24:01.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.488+0000 7f3027e72740 1 --2- 172.21.15.6:0/2713352230 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x558144c49e60 0x558144c4c250 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.489+0000 7f2ffeffd640 0 client.15732 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:01.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.489+0000 7f3027e72740 1 --2- 172.21.15.6:0/2713352230 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c3a240 0x558144c37b30 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.489+0000 7f3027e72740 1 --2- 172.21.15.6:0/2713352230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c2f980 0x558144c375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.489+0000 7f3027e72740 1 --2- 172.21.15.6:0/2713352230 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f060 0x558144c370b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.489+0000 7f3027e72740 1 -- 172.21.15.6:0/2713352230 >> 172.21.15.6:0/2713352230 conn(0x558144c2b5f0 msgr2=0x558144c312d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:01.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.489+0000 7f3027e72740 1 -- 172.21.15.6:0/2713352230 shutdown_connections 2024-09-12T03:24:01.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.489+0000 7f3027e72740 1 -- 172.21.15.6:0/2713352230 wait complete. 2024-09-12T03:24:01.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.489+0000 7f3027e72740 20 client.15732 trim_cache size 0 max 16384 2024-09-12T03:24:01.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.507+0000 7f3027e72740 1 Processor -- start 2024-09-12T03:24:01.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.507+0000 7f3027e72740 1 -- start start 2024-09-12T03:24:01.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.507+0000 7f3027e72740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c2f060 0x558144c39020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.507+0000 7f3027e72740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f980 0x558144c51b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.507+0000 7f3027e72740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c3a240 0x558144c520a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.507+0000 7f3027e72740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f30201b1fb0 con 0x558144c2f060 2024-09-12T03:24:01.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.507+0000 7f3027e72740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f30201af880 con 0x558144c2f980 2024-09-12T03:24:01.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.507+0000 7f3027e72740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f30201ad6d0 con 0x558144c3a240 2024-09-12T03:24:01.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.507+0000 7f301f7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c2f060 0x558144c39020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.507+0000 7f301f7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c2f060 0x558144c39020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54452/0 (socket says 172.21.15.6:54452) 2024-09-12T03:24:01.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.507+0000 7f301f7fe640 1 -- 172.21.15.6:0/2051578359 learned_addr learned my addr 172.21.15.6:0/2051578359 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:01.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.507+0000 7f301ffff640 1 --2- 172.21.15.6:0/2051578359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c3a240 0x558144c520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.508+0000 7f301effd640 1 --2- 172.21.15.6:0/2051578359 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f980 0x558144c51b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.508+0000 7f301f7fe640 1 -- 172.21.15.6:0/2051578359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c3a240 msgr2=0x558144c520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:01.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.508+0000 7f301f7fe640 1 --2- 172.21.15.6:0/2051578359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c3a240 0x558144c520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.508+0000 7f301f7fe640 1 -- 172.21.15.6:0/2051578359 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f980 msgr2=0x558144c51b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:01.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.508+0000 7f301f7fe640 1 --2- 172.21.15.6:0/2051578359 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f980 0x558144c51b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.508+0000 7f301f7fe640 1 -- 172.21.15.6:0/2051578359 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558144c4c5c0 con 0x558144c2f060 2024-09-12T03:24:01.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.508+0000 7f301ffff640 1 --2- 172.21.15.6:0/2051578359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c3a240 0x558144c520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:01.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.508+0000 7f301effd640 1 --2- 172.21.15.6:0/2051578359 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f980 0x558144c51b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:01.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.508+0000 7f301f7fe640 1 --2- 172.21.15.6:0/2051578359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c2f060 0x558144c39020 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f30100bbeb0 tx=0x7f30100f5550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b64358b525f5194 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:01.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.508+0000 7f30057fa640 1 -- 172.21.15.6:0/2051578359 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010049040 con 0x558144c2f060 2024-09-12T03:24:01.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.508+0000 7f30057fa640 1 -- 172.21.15.6:0/2051578359 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30100bc620 con 0x558144c2f060 2024-09-12T03:24:01.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.508+0000 7f30057fa640 1 -- 172.21.15.6:0/2051578359 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30100ba220 con 0x558144c2f060 2024-09-12T03:24:01.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.509+0000 7f3027e72740 1 -- 172.21.15.6:0/2051578359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c2f060 msgr2=0x558144c39020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:01.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.509+0000 7f3027e72740 1 --2- 172.21.15.6:0/2051578359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c2f060 0x558144c39020 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f30100bbeb0 tx=0x7f30100f5550 comp rx=0 tx=0).stop 2024-09-12T03:24:01.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.509+0000 7f3027e72740 1 -- 172.21.15.6:0/2051578359 shutdown_connections 2024-09-12T03:24:01.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.509+0000 7f3027e72740 1 --2- 172.21.15.6:0/2051578359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c3a240 0x558144c520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.509+0000 7f3027e72740 1 --2- 172.21.15.6:0/2051578359 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f980 0x558144c51b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.509+0000 7f3027e72740 1 --2- 172.21.15.6:0/2051578359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c2f060 0x558144c39020 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.509+0000 7f3027e72740 1 -- 172.21.15.6:0/2051578359 >> 172.21.15.6:0/2051578359 conn(0x558144c2b5f0 msgr2=0x558144c38910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:01.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.509+0000 7f3027e72740 1 -- 172.21.15.6:0/2051578359 shutdown_connections 2024-09-12T03:24:01.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.509+0000 7f3027e72740 1 -- 172.21.15.6:0/2051578359 wait complete. 2024-09-12T03:24:01.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.509+0000 7f3027e72740 1 -- start start 2024-09-12T03:24:01.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.510+0000 7f3027e72740 1 Processor -- start 2024-09-12T03:24:01.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.510+0000 7f3027e72740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c2f060 0x558144c42210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.510+0000 7f3027e72740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f980 0x558144c42750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.510+0000 7f3027e72740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c3a240 0x558144c3f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.510+0000 7f3027e72740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f30201add40 con 0x558144c3a240 2024-09-12T03:24:01.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.510+0000 7f3027e72740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f30201af1b0 con 0x558144c2f980 2024-09-12T03:24:01.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.510+0000 7f3027e72740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f30201ae410 con 0x558144c2f060 2024-09-12T03:24:01.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.510+0000 7f3027e72740 10 client.0 _ll_register_callbacks cb 0x7f3018012540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:01.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.510+0000 7f301ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c3a240 0x558144c3f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.510+0000 7f301ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c3a240 0x558144c3f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54458/0 (socket says 172.21.15.6:54458) 2024-09-12T03:24:01.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.510+0000 7f301ffff640 1 -- 172.21.15.6:0/1345339914 learned_addr learned my addr 172.21.15.6:0/1345339914 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:01.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.510+0000 7f301f7fe640 1 --2- 172.21.15.6:0/1345339914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c2f060 0x558144c42210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.510+0000 7f301effd640 1 --2- 172.21.15.6:0/1345339914 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f980 0x558144c42750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.510+0000 7f301ffff640 1 -- 172.21.15.6:0/1345339914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c2f060 msgr2=0x558144c42210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:01.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.510+0000 7f301ffff640 1 --2- 172.21.15.6:0/1345339914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c2f060 0x558144c42210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.510+0000 7f301ffff640 1 -- 172.21.15.6:0/1345339914 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f980 msgr2=0x558144c42750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:01.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.510+0000 7f301ffff640 1 --2- 172.21.15.6:0/1345339914 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f980 0x558144c42750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.510+0000 7f301ffff640 1 -- 172.21.15.6:0/1345339914 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558144c3f750 con 0x558144c3a240 2024-09-12T03:24:01.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.510+0000 7f301f7fe640 1 --2- 172.21.15.6:0/1345339914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c2f060 0x558144c42210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:01.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.511+0000 7f301effd640 1 --2- 172.21.15.6:0/1345339914 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f980 0x558144c42750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:01.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.511+0000 7f301ffff640 1 --2- 172.21.15.6:0/1345339914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c3a240 0x558144c3f1e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3018027b00 tx=0x7f3018027630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf88f2089173b553 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:01.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.511+0000 7f2ffeffd640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:24:01.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.511+0000 7f2ffeffd640 1 -- 172.21.15.6:0/1345339914 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3018009590 con 0x558144c3a240 2024-09-12T03:24:01.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.511+0000 7f2ffeffd640 1 -- 172.21.15.6:0/1345339914 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f3018003970 con 0x558144c3a240 2024-09-12T03:24:01.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.511+0000 7f3027e72740 1 -- 172.21.15.6:0/1345339914 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30100bc620 con 0x558144c3a240 2024-09-12T03:24:01.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.511+0000 7f2ffeffd640 1 -- 172.21.15.6:0/1345339914 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3018046050 con 0x558144c3a240 2024-09-12T03:24:01.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.511+0000 7f3027e72740 10 client.15738 fetch_fsmap learned FSMap version 26 2024-09-12T03:24:01.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.511+0000 7f3027e72740 1 -- 172.21.15.6:0/1345339914 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558144cb53c0 con 0x558144c3a240 2024-09-12T03:24:01.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.511+0000 7f2ffeffd640 1 -- 172.21.15.6:0/1345339914 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30100bc620 con 0x558144c3a240 2024-09-12T03:24:01.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.511+0000 7f2ffeffd640 1 -- 172.21.15.6:0/1345339914 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3018004580 con 0x558144c3a240 2024-09-12T03:24:01.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.512+0000 7f3027e72740 10 client.15738 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:24:01.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.512+0000 7f3027e72740 10 client.15738 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:24:01.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.512+0000 7f3027e72740 20 client.15738 populate_metadata read hostname 'smithi006' 2024-09-12T03:24:01.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.512+0000 7f3027e72740 1 --2- 172.21.15.6:0/1345339914 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x558144c49e60 0x558144cb7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.512+0000 7f3027e72740 4 client.15738 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f395d0fc", "release": true, "await": true}] 2024-09-12T03:24:01.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.512+0000 7f3027e72740 1 -- 172.21.15.6:0/1345339914 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f395d0fc", "release": true, "await": true}) -- 0x558144b9c0a0 con 0x558144c49e60 2024-09-12T03:24:01.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.512+0000 7f301f7fe640 1 --2- 172.21.15.6:0/1345339914 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x558144c49e60 0x558144cb7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.512+0000 7f301f7fe640 1 --2- 172.21.15.6:0/1345339914 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x558144c49e60 0x558144cb7be0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa77cb19b392d692 server_cookie=15882f42e98d4a1b in_seq=0 out_seq=0 2024-09-12T03:24:01.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.512+0000 7f2ffeffd640 10 client.15738 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:01.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.562+0000 7f2ffeffd640 1 -- 172.21.15.6:0/1345339914 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558144b9c0a0 con 0x558144c49e60 2024-09-12T03:24:01.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.562+0000 7f2ffeffd640 10 client.15738 handle_command_reply: tid=0 2024-09-12T03:24:01.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.563+0000 7f3027e72740 1 client.15738 shutdown 2024-09-12T03:24:01.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.568+0000 7f3027e72740 1 -- 172.21.15.6:0/1345339914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c3a240 msgr2=0x558144c3f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:01.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.568+0000 7f3027e72740 1 --2- 172.21.15.6:0/1345339914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c3a240 0x558144c3f1e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3018027b00 tx=0x7f3018027630 comp rx=0 tx=0).stop 2024-09-12T03:24:01.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.568+0000 7f3027e72740 1 -- 172.21.15.6:0/1345339914 shutdown_connections 2024-09-12T03:24:01.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.568+0000 7f3027e72740 1 --2- 172.21.15.6:0/1345339914 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x558144c49e60 0x558144cb7be0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.571 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:24:01.571 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 26, 2024-09-12T03:24:01.571 INFO:teuthology.orchestra.run.smithi006.stdout: "leader": 24494, 2024-09-12T03:24:01.571 INFO:teuthology.orchestra.run.smithi006.stdout: "set_version": 12, 2024-09-12T03:24:01.571 INFO:teuthology.orchestra.run.smithi006.stdout: "sets": { 2024-09-12T03:24:01.571 INFO:teuthology.orchestra.run.smithi006.stdout: "f395d0fc": { 2024-09-12T03:24:01.571 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 12, 2024-09-12T03:24:01.572 INFO:teuthology.orchestra.run.smithi006.stdout: "age_ref": 0.0, 2024-09-12T03:24:01.572 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:24:01.572 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RELEASED", 2024-09-12T03:24:01.572 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:24:01.572 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:24:01.572 INFO:teuthology.orchestra.run.smithi006.stdout: "timeout": 90.0, 2024-09-12T03:24:01.572 INFO:teuthology.orchestra.run.smithi006.stdout: "expiration": 144.7, 2024-09-12T03:24:01.572 INFO:teuthology.orchestra.run.smithi006.stdout: "members": { 2024-09-12T03:24:01.572 INFO:teuthology.orchestra.run.smithi006.stdout: "file:/": { 2024-09-12T03:24:01.572 INFO:teuthology.orchestra.run.smithi006.stdout: "excluded": false, 2024-09-12T03:24:01.572 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:24:01.572 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RELEASED", 2024-09-12T03:24:01.572 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:24:01.573 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:24:01.573 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:24:01.573 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:24:01.573 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:24:01.573 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:24:01.573 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:24:01.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.568+0000 7f3027e72740 1 --2- 172.21.15.6:0/1345339914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558144c3a240 0x558144c3f1e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.568+0000 7f2ffeffd640 0 client.15738 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:01.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.568+0000 7f3027e72740 1 --2- 172.21.15.6:0/1345339914 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558144c2f980 0x558144c42750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.568+0000 7f3027e72740 1 --2- 172.21.15.6:0/1345339914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558144c2f060 0x558144c42210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.568+0000 7f3027e72740 1 -- 172.21.15.6:0/1345339914 >> 172.21.15.6:0/1345339914 conn(0x558144c2b5f0 msgr2=0x558144c39130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:01.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.568+0000 7f3027e72740 1 -- 172.21.15.6:0/1345339914 shutdown_connections 2024-09-12T03:24:01.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.568+0000 7f3027e72740 1 -- 172.21.15.6:0/1345339914 wait complete. 2024-09-12T03:24:01.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.568+0000 7f3027e72740 20 client.15738 trim_cache size 0 max 16384 2024-09-12T03:24:01.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.571+0000 7f3025a99640 1 -- 172.21.15.6:0/1935526458 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f30080a2b80 msgr2=0x7f30080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:01.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.571+0000 7f3025a99640 1 --2- 172.21.15.6:0/1935526458 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f30080a2b80 0x7f30080a5040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3018005fd0 tx=0x7f3018009500 comp rx=0 tx=0).stop 2024-09-12T03:24:01.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.571+0000 7f3025a99640 1 -- 172.21.15.6:0/1935526458 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3020102d30 msgr2=0x7f30201a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:01.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.571+0000 7f3025a99640 1 --2- 172.21.15.6:0/1935526458 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3020102d30 0x7f30201a0410 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3010009230 tx=0x7f3010004270 comp rx=0 tx=0).stop 2024-09-12T03:24:01.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.571+0000 7f3025a99640 1 -- 172.21.15.6:0/1935526458 shutdown_connections 2024-09-12T03:24:01.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.571+0000 7f3025a99640 1 --2- 172.21.15.6:0/1935526458 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f30080a2b80 0x7f30080a5040 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.571+0000 7f3025a99640 1 --2- 172.21.15.6:0/1935526458 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f30201082a0 0x7f302019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.571+0000 7f3025a99640 1 --2- 172.21.15.6:0/1935526458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30201036e0 0x7f30201a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.571+0000 7f3025a99640 1 --2- 172.21.15.6:0/1935526458 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3020102d30 0x7f30201a0410 secure :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3010009230 tx=0x7f3010004270 comp rx=0 tx=0).stop 2024-09-12T03:24:01.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.571+0000 7f3025a99640 1 -- 172.21.15.6:0/1935526458 >> 172.21.15.6:0/1935526458 conn(0x7f30200fc7d0 msgr2=0x7f302010d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:01.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.572+0000 7f3025a99640 1 -- 172.21.15.6:0/1935526458 shutdown_connections 2024-09-12T03:24:01.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:01.572+0000 7f3025a99640 1 -- 172.21.15.6:0/1935526458 wait complete. 2024-09-12T03:24:02.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:02 smithi006 ceph-mon[30991]: pgmap v423: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 35 MiB/s wr, 3.04k op/s 2024-09-12T03:24:02.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:02 smithi161 ceph-mon[35485]: pgmap v423: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 35 MiB/s wr, 3.04k op/s 2024-09-12T03:24:02.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:02 smithi118 ceph-mon[35048]: pgmap v423: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 35 MiB/s wr, 3.04k op/s 2024-09-12T03:24:03.205 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f395d0fc', total seconds elapsed: 29.4 2024-09-12T03:24:04.757 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:24:04.810 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:04 smithi006 ceph-mon[30991]: pgmap v424: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1010 GiB / 1.0 TiB avail; 6.4 KiB/s rd, 59 MiB/s wr, 5.49k op/s 2024-09-12T03:24:04.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:04 smithi161 ceph-mon[35485]: pgmap v424: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1010 GiB / 1.0 TiB avail; 6.4 KiB/s rd, 59 MiB/s wr, 5.49k op/s 2024-09-12T03:24:04.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:04 smithi118 ceph-mon[35048]: pgmap v424: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1010 GiB / 1.0 TiB avail; 6.4 KiB/s rd, 59 MiB/s wr, 5.49k op/s 2024-09-12T03:24:05.321 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:06.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.168+0000 7fc9a159a640 1 -- 172.21.15.6:0/837518150 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9940319b0 con 0x7fc9a4100e20 2024-09-12T03:24:06.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.168+0000 7fc9a8893640 1 -- 172.21.15.6:0/837518150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9a4100e20 msgr2=0x7fc9a4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.168+0000 7fc9a8893640 1 --2- 172.21.15.6:0/837518150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9a4100e20 0x7fc9a4101200 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc994009930 tx=0x7fc99402a4d0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.171+0000 7fc9a8893640 1 -- 172.21.15.6:0/837518150 shutdown_connections 2024-09-12T03:24:06.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.171+0000 7fc9a8893640 1 --2- 172.21.15.6:0/837518150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9a4102ef0 0x7fc9a410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.171+0000 7fc9a8893640 1 --2- 172.21.15.6:0/837518150 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc9a4102570 0x7fc9a41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.171+0000 7fc9a8893640 1 --2- 172.21.15.6:0/837518150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9a4100e20 0x7fc9a4101200 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.171+0000 7fc9a8893640 1 -- 172.21.15.6:0/837518150 >> 172.21.15.6:0/837518150 conn(0x7fc9a40fc7d0 msgr2=0x7fc9a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:06.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.173+0000 7fc9a8893640 1 -- 172.21.15.6:0/837518150 shutdown_connections 2024-09-12T03:24:06.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.173+0000 7fc9a8893640 1 -- 172.21.15.6:0/837518150 wait complete. 2024-09-12T03:24:06.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.174+0000 7fc9a8893640 1 Processor -- start 2024-09-12T03:24:06.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.175+0000 7fc9a8893640 1 -- start start 2024-09-12T03:24:06.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.175+0000 7fc9a8893640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc9a4100e20 0x7fc9a4109c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.175+0000 7fc9a8893640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9a4102570 0x7fc9a410a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.175+0000 7fc9a8893640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9a4102ef0 0x7fc9a410a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.175+0000 7fc9a8893640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a410fe80 con 0x7fc9a4100e20 2024-09-12T03:24:06.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.175+0000 7fc9a8893640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a410fd00 con 0x7fc9a4102570 2024-09-12T03:24:06.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.175+0000 7fc9a8893640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a4110000 con 0x7fc9a4102ef0 2024-09-12T03:24:06.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.175+0000 7fc9a2d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9a4102ef0 0x7fc9a410a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.175+0000 7fc9a2d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9a4102ef0 0x7fc9a410a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:39198/0 (socket says 172.21.15.6:39198) 2024-09-12T03:24:06.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.175+0000 7fc9a2d9d640 1 -- 172.21.15.6:0/4287373058 learned_addr learned my addr 172.21.15.6:0/4287373058 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:06.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.176+0000 7fc9a1d9b640 1 --2- 172.21.15.6:0/4287373058 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9a4102570 0x7fc9a410a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.176+0000 7fc9a2d9d640 1 -- 172.21.15.6:0/4287373058 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9a4102570 msgr2=0x7fc9a410a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.176+0000 7fc9a2d9d640 1 --2- 172.21.15.6:0/4287373058 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9a4102570 0x7fc9a410a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.176+0000 7fc9a2d9d640 1 -- 172.21.15.6:0/4287373058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc9a4100e20 msgr2=0x7fc9a4109c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.176+0000 7fc9a2d9d640 1 --2- 172.21.15.6:0/4287373058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc9a4100e20 0x7fc9a4109c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.176+0000 7fc9a2d9d640 1 -- 172.21.15.6:0/4287373058 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a41aad90 con 0x7fc9a4102ef0 2024-09-12T03:24:06.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.176+0000 7fc9a2d9d640 1 --2- 172.21.15.6:0/4287373058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9a4102ef0 0x7fc9a410a6c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00ead0 tx=0x7fc99c00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:06.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.177+0000 7fc9937fe640 1 -- 172.21.15.6:0/4287373058 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c00cde0 con 0x7fc9a4102ef0 2024-09-12T03:24:06.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.177+0000 7fc9a8893640 1 -- 172.21.15.6:0/4287373058 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a41ab080 con 0x7fc9a4102ef0 2024-09-12T03:24:06.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.177+0000 7fc9a8893640 1 -- 172.21.15.6:0/4287373058 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a41ab5c0 con 0x7fc9a4102ef0 2024-09-12T03:24:06.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.182+0000 7fc9aac6c740 1 Processor -- start 2024-09-12T03:24:06.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.182+0000 7fc9aac6c740 1 -- start start 2024-09-12T03:24:06.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.183+0000 7fc9aac6c740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5561fe21c790 0x5561fe21cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.183+0000 7fc9aac6c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561fe21d0b0 0x5561fe227430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.183+0000 7fc9aac6c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe227970 0x5561fe229d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.183+0000 7fc9aac6c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5561fe1822e0 con 0x5561fe227970 2024-09-12T03:24:06.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.183+0000 7fc9aac6c740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5561fe129ce0 con 0x5561fe21c790 2024-09-12T03:24:06.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.183+0000 7fc9aac6c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5561fe17e2a0 con 0x5561fe21d0b0 2024-09-12T03:24:06.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.185+0000 7fc9937fe640 1 -- 172.21.15.6:0/4287373058 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc99c005020 con 0x7fc9a4102ef0 2024-09-12T03:24:06.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.186+0000 7fc9a1d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe227970 0x5561fe229d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.186+0000 7fc9a1d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe227970 0x5561fe229d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54498/0 (socket says 172.21.15.6:54498) 2024-09-12T03:24:06.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.186+0000 7fc9a1d9b640 1 -- 172.21.15.6:0/2583359240 learned_addr learned my addr 172.21.15.6:0/2583359240 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:06.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.186+0000 7fc9a1d9b640 1 --2- 172.21.15.6:0/2583359240 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5561fe21c790 0x5561fe21cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.186+0000 7fc9a1d9b640 1 -- 172.21.15.6:0/2583359240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561fe21d0b0 msgr2=0x5561fe227430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:06.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.186+0000 7fc9a1d9b640 1 --2- 172.21.15.6:0/2583359240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561fe21d0b0 0x5561fe227430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.186+0000 7fc9a1d9b640 1 -- 172.21.15.6:0/2583359240 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5561fe21c790 msgr2=0x5561fe21cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:06.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.186+0000 7fc9a1d9b640 1 --2- 172.21.15.6:0/2583359240 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5561fe21c790 0x5561fe21cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.186+0000 7fc9937fe640 1 -- 172.21.15.6:0/4287373058 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c0116d0 con 0x7fc9a4102ef0 2024-09-12T03:24:06.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.186+0000 7fc9937fe640 1 -- 172.21.15.6:0/4287373058 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc99c0118f0 con 0x7fc9a4102ef0 2024-09-12T03:24:06.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.187+0000 7fc9a1d9b640 1 -- 172.21.15.6:0/2583359240 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561fe10e440 con 0x5561fe227970 2024-09-12T03:24:06.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.188+0000 7fc9937fe640 1 --2- 172.21.15.6:0/4287373058 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc98c0b3dd0 0x7fc98c0b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.188+0000 7fc9937fe640 1 -- 172.21.15.6:0/4287373058 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fc99c0d0280 con 0x7fc9a4102ef0 2024-09-12T03:24:06.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.189+0000 7fc9a259c640 1 --2- 172.21.15.6:0/4287373058 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc98c0b3dd0 0x7fc98c0b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.190+0000 7fc9a259c640 1 --2- 172.21.15.6:0/4287373058 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc98c0b3dd0 0x7fc98c0b6290 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc994009230 tx=0x7fc994005760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:06.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.193+0000 7fc9a1d9b640 1 --2- 172.21.15.6:0/2583359240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe227970 0x5561fe229d60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc99800cfc0 tx=0x7fc99800c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12da5b492fd51317 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:06.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.196+0000 7fc98bfff640 1 -- 172.21.15.6:0/2583359240 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc998018030 con 0x5561fe227970 2024-09-12T03:24:06.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.196+0000 7fc98bfff640 1 -- 172.21.15.6:0/2583359240 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc998005020 con 0x5561fe227970 2024-09-12T03:24:06.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.197+0000 7fc9aac6c740 1 -- 172.21.15.6:0/2583359240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe227970 msgr2=0x5561fe229d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.197+0000 7fc9aac6c740 1 --2- 172.21.15.6:0/2583359240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe227970 0x5561fe229d60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc99800cfc0 tx=0x7fc99800c5a0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.198+0000 7fc9aac6c740 1 -- 172.21.15.6:0/2583359240 shutdown_connections 2024-09-12T03:24:06.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.198+0000 7fc9aac6c740 1 --2- 172.21.15.6:0/2583359240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe227970 0x5561fe229d60 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.198+0000 7fc9aac6c740 1 --2- 172.21.15.6:0/2583359240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561fe21d0b0 0x5561fe227430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.198+0000 7fc9aac6c740 1 --2- 172.21.15.6:0/2583359240 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5561fe21c790 0x5561fe21cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.198+0000 7fc9aac6c740 1 -- 172.21.15.6:0/2583359240 >> 172.21.15.6:0/2583359240 conn(0x5561fe218d20 msgr2=0x5561fe219100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:06.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.200+0000 7fc9aac6c740 1 -- 172.21.15.6:0/2583359240 shutdown_connections 2024-09-12T03:24:06.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.201+0000 7fc9aac6c740 1 -- 172.21.15.6:0/2583359240 wait complete. 2024-09-12T03:24:06.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.202+0000 7fc9aac6c740 1 -- start start 2024-09-12T03:24:06.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.202+0000 7fc9aac6c740 1 Processor -- start 2024-09-12T03:24:06.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.203+0000 7fc9aac6c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe21c790 0x5561fe227030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.203+0000 7fc9aac6c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561fe21d0b0 0x5561fe2242e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.203+0000 7fc9aac6c740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5561fe227970 0x5561fe224820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.203+0000 7fc9aac6c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5561fe23b4c0 con 0x5561fe21c790 2024-09-12T03:24:06.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.203+0000 7fc9aac6c740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5561fe13d050 con 0x5561fe227970 2024-09-12T03:24:06.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.203+0000 7fc9aac6c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5561fe23b640 con 0x5561fe21d0b0 2024-09-12T03:24:06.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.203+0000 7fc9aac6c740 10 client.0 _ll_register_callbacks cb 0x5561fe17d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:06.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.206+0000 7fc9a259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe21c790 0x5561fe227030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.206+0000 7fc9a259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe21c790 0x5561fe227030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54510/0 (socket says 172.21.15.6:54510) 2024-09-12T03:24:06.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.206+0000 7fc9a259c640 1 -- 172.21.15.6:0/1115618613 learned_addr learned my addr 172.21.15.6:0/1115618613 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:06.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.208+0000 7fc9a259c640 1 -- 172.21.15.6:0/1115618613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561fe21d0b0 msgr2=0x5561fe2242e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:24:06.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.208+0000 7fc9a259c640 1 --2- 172.21.15.6:0/1115618613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561fe21d0b0 0x5561fe2242e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.208+0000 7fc9a259c640 1 -- 172.21.15.6:0/1115618613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5561fe227970 msgr2=0x5561fe224820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:24:06.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.208+0000 7fc9a259c640 1 --2- 172.21.15.6:0/1115618613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5561fe227970 0x5561fe224820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.208+0000 7fc9a259c640 1 -- 172.21.15.6:0/1115618613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561fdef33c0 con 0x5561fe21c790 2024-09-12T03:24:06.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.208+0000 7fc9a259c640 1 --2- 172.21.15.6:0/1115618613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe21c790 0x5561fe227030 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc9940044b0 tx=0x7fc994005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19904f0c3c5d617 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:06.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.210+0000 7fc989ffb640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:24:06.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.210+0000 7fc989ffb640 1 -- 172.21.15.6:0/1115618613 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99404b020 con 0x5561fe21c790 2024-09-12T03:24:06.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.210+0000 7fc989ffb640 1 -- 172.21.15.6:0/1115618613 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fc994003db0 con 0x5561fe21c790 2024-09-12T03:24:06.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.211+0000 7fc9aac6c740 1 -- 172.21.15.6:0/1115618613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc998005020 con 0x5561fe21c790 2024-09-12T03:24:06.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.211+0000 7fc989ffb640 1 -- 172.21.15.6:0/1115618613 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc994035040 con 0x5561fe21c790 2024-09-12T03:24:06.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.211+0000 7fc9aac6c740 10 client.15744 fetch_fsmap learned FSMap version 26 2024-09-12T03:24:06.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.212+0000 7fc9aac6c740 1 -- 172.21.15.6:0/1115618613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5561fe246ce0 con 0x5561fe21c790 2024-09-12T03:24:06.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.212+0000 7fc989ffb640 1 -- 172.21.15.6:0/1115618613 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc998005020 con 0x5561fe21c790 2024-09-12T03:24:06.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.214+0000 7fc989ffb640 1 -- 172.21.15.6:0/1115618613 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc994047750 con 0x5561fe21c790 2024-09-12T03:24:06.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.214+0000 7fc9aac6c740 10 client.15744 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:24:06.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.214+0000 7fc9aac6c740 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:24:06.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.214+0000 7fc9aac6c740 20 client.15744 populate_metadata read hostname 'smithi006' 2024-09-12T03:24:06.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.214+0000 7fc9aac6c740 1 --2- 172.21.15.6:0/1115618613 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5561fe248680 0x5561fe24ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.214+0000 7fc9aac6c740 4 client.15744 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:24:06.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.214+0000 7fc9aac6c740 1 -- 172.21.15.6:0/1115618613 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561fe18f850 con 0x5561fe248680 2024-09-12T03:24:06.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.215+0000 7fc9a1d9b640 1 --2- 172.21.15.6:0/1115618613 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5561fe248680 0x5561fe24ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.219+0000 7fc989ffb640 10 client.15744 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:06.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.219+0000 7fc9a1d9b640 1 --2- 172.21.15.6:0/1115618613 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5561fe248680 0x5561fe24ab40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90361563dc96ba60 server_cookie=a9b6f36c2d755c7e in_seq=0 out_seq=0 2024-09-12T03:24:06.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.230+0000 7fc989ffb640 1 -- 172.21.15.6:0/1115618613 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5561fe18f850 con 0x5561fe248680 2024-09-12T03:24:06.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.230+0000 7fc989ffb640 10 client.15744 handle_command_reply: tid=0 2024-09-12T03:24:06.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.230+0000 7fc9aac6c740 1 client.15744 shutdown 2024-09-12T03:24:06.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.233+0000 7fc9aac6c740 1 -- 172.21.15.6:0/1115618613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe21c790 msgr2=0x5561fe227030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.233+0000 7fc9aac6c740 1 --2- 172.21.15.6:0/1115618613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe21c790 0x5561fe227030 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc9940044b0 tx=0x7fc994005fd0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.233+0000 7fc9aac6c740 1 -- 172.21.15.6:0/1115618613 shutdown_connections 2024-09-12T03:24:06.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.233+0000 7fc9aac6c740 1 --2- 172.21.15.6:0/1115618613 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5561fe248680 0x5561fe24ab40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.233+0000 7fc9aac6c740 1 --2- 172.21.15.6:0/1115618613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5561fe227970 0x5561fe224820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.233+0000 7fc9aac6c740 1 --2- 172.21.15.6:0/1115618613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561fe21d0b0 0x5561fe2242e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.233+0000 7fc9aac6c740 1 --2- 172.21.15.6:0/1115618613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe21c790 0x5561fe227030 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.233+0000 7fc9aac6c740 1 -- 172.21.15.6:0/1115618613 >> 172.21.15.6:0/1115618613 conn(0x5561fe218d20 msgr2=0x5561fe217ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:06.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.233+0000 7fc989ffb640 0 client.15744 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:06.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.238+0000 7fc9aac6c740 1 -- 172.21.15.6:0/1115618613 shutdown_connections 2024-09-12T03:24:06.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.238+0000 7fc9aac6c740 1 -- 172.21.15.6:0/1115618613 wait complete. 2024-09-12T03:24:06.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.239+0000 7fc9aac6c740 20 client.15744 trim_cache size 0 max 16384 2024-09-12T03:24:06.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.286+0000 7fc9aac6c740 1 Processor -- start 2024-09-12T03:24:06.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.286+0000 7fc9aac6c740 1 -- start start 2024-09-12T03:24:06.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.287+0000 7fc9aac6c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe21c790 0x5561fe21fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.287+0000 7fc9aac6c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561fe21d0b0 0x5561fe21e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.287+0000 7fc9a259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe21c790 0x5561fe21fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.287+0000 7fc9aac6c740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5561fe227970 0x5561fe250450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.287+0000 7fc9aac6c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc984010d60 con 0x5561fe21c790 2024-09-12T03:24:06.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.288+0000 7fc9a259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe21c790 0x5561fe21fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54514/0 (socket says 172.21.15.6:54514) 2024-09-12T03:24:06.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.288+0000 7fc9a259c640 1 -- 172.21.15.6:0/4175065926 learned_addr learned my addr 172.21.15.6:0/4175065926 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:06.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.288+0000 7fc9a1d9b640 1 --2- 172.21.15.6:0/4175065926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561fe21d0b0 0x5561fe21e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.288+0000 7fc9aac6c740 1 -- 172.21.15.6:0/4175065926 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fc98400e630 con 0x5561fe227970 2024-09-12T03:24:06.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.288+0000 7fc9aac6c740 1 -- 172.21.15.6:0/4175065926 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc98400c480 con 0x5561fe21d0b0 2024-09-12T03:24:06.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.288+0000 7fc9a259c640 1 -- 172.21.15.6:0/4175065926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561fe21d0b0 msgr2=0x5561fe21e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:06.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.288+0000 7fc9a259c640 1 --2- 172.21.15.6:0/4175065926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561fe21d0b0 0x5561fe21e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.288+0000 7fc9a259c640 1 -- 172.21.15.6:0/4175065926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5561fe227970 msgr2=0x5561fe250450 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:24:06.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.288+0000 7fc9a259c640 1 --2- 172.21.15.6:0/4175065926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5561fe227970 0x5561fe250450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.288+0000 7fc9a259c640 1 -- 172.21.15.6:0/4175065926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561fe224c40 con 0x5561fe21c790 2024-09-12T03:24:06.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.288+0000 7fc9a1d9b640 1 --2- 172.21.15.6:0/4175065926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561fe21d0b0 0x5561fe21e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:06.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.292+0000 7fc9a259c640 1 --2- 172.21.15.6:0/4175065926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe21c790 0x5561fe21fbf0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc994003f20 tx=0x7fc994005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe9c5f3f9dfc8fa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:06.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.298+0000 7fc98b7fe640 1 -- 172.21.15.6:0/4175065926 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99405d070 con 0x5561fe21c790 2024-09-12T03:24:06.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.299+0000 7fc98b7fe640 1 -- 172.21.15.6:0/4175065926 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc994040030 con 0x5561fe21c790 2024-09-12T03:24:06.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.299+0000 7fc98b7fe640 1 -- 172.21.15.6:0/4175065926 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99404b040 con 0x5561fe21c790 2024-09-12T03:24:06.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.300+0000 7fc9aac6c740 1 -- 172.21.15.6:0/4175065926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe21c790 msgr2=0x5561fe21fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.300+0000 7fc9aac6c740 1 --2- 172.21.15.6:0/4175065926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe21c790 0x5561fe21fbf0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc994003f20 tx=0x7fc994005e00 comp rx=0 tx=0).stop 2024-09-12T03:24:06.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.301+0000 7fc9aac6c740 1 -- 172.21.15.6:0/4175065926 shutdown_connections 2024-09-12T03:24:06.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.301+0000 7fc9aac6c740 1 --2- 172.21.15.6:0/4175065926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5561fe227970 0x5561fe250450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.301+0000 7fc9aac6c740 1 --2- 172.21.15.6:0/4175065926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561fe21d0b0 0x5561fe21e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.301+0000 7fc9aac6c740 1 --2- 172.21.15.6:0/4175065926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe21c790 0x5561fe21fbf0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.301+0000 7fc9aac6c740 1 -- 172.21.15.6:0/4175065926 >> 172.21.15.6:0/4175065926 conn(0x5561fe218d20 msgr2=0x5561fe21e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:06.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.301+0000 7fc9aac6c740 1 -- 172.21.15.6:0/4175065926 shutdown_connections 2024-09-12T03:24:06.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.301+0000 7fc9aac6c740 1 -- 172.21.15.6:0/4175065926 wait complete. 2024-09-12T03:24:06.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.302+0000 7fc9aac6c740 1 -- start start 2024-09-12T03:24:06.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.302+0000 7fc9aac6c740 1 Processor -- start 2024-09-12T03:24:06.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.303+0000 7fc9aac6c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe21c790 0x5561fe2214f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.303+0000 7fc9aac6c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561fe21d0b0 0x5561fe221a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.303+0000 7fc9aac6c740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5561fe227970 0x5561fe221f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.303+0000 7fc9aac6c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc98400caf0 con 0x5561fe21c790 2024-09-12T03:24:06.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.304+0000 7fc9aac6c740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fc98400df60 con 0x5561fe227970 2024-09-12T03:24:06.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.304+0000 7fc9a1d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561fe21d0b0 0x5561fe221a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.304+0000 7fc9a1d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561fe21d0b0 0x5561fe221a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:39220/0 (socket says 172.21.15.6:39220) 2024-09-12T03:24:06.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.304+0000 7fc9a1d9b640 1 -- 172.21.15.6:0/1480741758 learned_addr learned my addr 172.21.15.6:0/1480741758 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:06.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.304+0000 7fc9aac6c740 1 -- 172.21.15.6:0/1480741758 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc98400d1c0 con 0x5561fe21d0b0 2024-09-12T03:24:06.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.304+0000 7fc9aac6c740 10 client.0 _ll_register_callbacks cb 0x7fc994009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:06.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.307+0000 7fc9a2d9d640 1 --2- 172.21.15.6:0/1480741758 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5561fe227970 0x5561fe221f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.307+0000 7fc9a259c640 1 --2- 172.21.15.6:0/1480741758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe21c790 0x5561fe2214f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.307+0000 7fc9a1d9b640 1 -- 172.21.15.6:0/1480741758 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5561fe227970 msgr2=0x5561fe221f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:06.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.307+0000 7fc9a1d9b640 1 --2- 172.21.15.6:0/1480741758 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5561fe227970 0x5561fe221f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.307+0000 7fc9a1d9b640 1 -- 172.21.15.6:0/1480741758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe21c790 msgr2=0x5561fe2214f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:06.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.307+0000 7fc9a1d9b640 1 --2- 172.21.15.6:0/1480741758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe21c790 0x5561fe2214f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.307+0000 7fc9a1d9b640 1 -- 172.21.15.6:0/1480741758 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561fe1eade0 con 0x5561fe21d0b0 2024-09-12T03:24:06.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.308+0000 7fc9a1d9b640 1 --2- 172.21.15.6:0/1480741758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561fe21d0b0 0x5561fe221a30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc998012fd0 tx=0x7fc99800cad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e5c29724fdcf6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:06.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.308+0000 7fc9897fa640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:24:06.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.308+0000 7fc9897fa640 1 -- 172.21.15.6:0/1480741758 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc998018040 con 0x5561fe21d0b0 2024-09-12T03:24:06.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.308+0000 7fc9897fa640 1 -- 172.21.15.6:0/1480741758 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fc9980253a0 con 0x5561fe21d0b0 2024-09-12T03:24:06.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.308+0000 7fc9aac6c740 1 -- 172.21.15.6:0/1480741758 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc994040030 con 0x5561fe21d0b0 2024-09-12T03:24:06.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.308+0000 7fc9897fa640 1 -- 172.21.15.6:0/1480741758 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc9980255a0 con 0x5561fe21d0b0 2024-09-12T03:24:06.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.310+0000 7fc9aac6c740 10 client.25339 fetch_fsmap learned FSMap version 26 2024-09-12T03:24:06.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.310+0000 7fc9aac6c740 1 -- 172.21.15.6:0/1480741758 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5561fe2b3690 con 0x5561fe21d0b0 2024-09-12T03:24:06.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.314+0000 7fc9897fa640 1 -- 172.21.15.6:0/1480741758 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc99802b070 con 0x5561fe21d0b0 2024-09-12T03:24:06.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.315+0000 7fc9897fa640 1 -- 172.21.15.6:0/1480741758 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc998031460 con 0x5561fe21d0b0 2024-09-12T03:24:06.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.315+0000 7fc9aac6c740 10 client.25339 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:24:06.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.315+0000 7fc9aac6c740 10 client.25339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:24:06.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.315+0000 7fc9aac6c740 20 client.25339 populate_metadata read hostname 'smithi006' 2024-09-12T03:24:06.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.315+0000 7fc9aac6c740 1 --2- 172.21.15.6:0/1480741758 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5561fe2b3b60 0x5561fe2b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.315+0000 7fc9aac6c740 4 client.25339 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:24:06.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.316+0000 7fc9aac6c740 1 -- 172.21.15.6:0/1480741758 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fc994040030 con 0x5561fe2b3b60 2024-09-12T03:24:06.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.319+0000 7fc9a259c640 1 --2- 172.21.15.6:0/1480741758 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5561fe2b3b60 0x5561fe2b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.322+0000 7fc9897fa640 10 client.25339 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:06.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.322+0000 7fc9a259c640 1 --2- 172.21.15.6:0/1480741758 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5561fe2b3b60 0x5561fe2b5f80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4807e0beda759cf server_cookie=4cf059403b26e204 in_seq=0 out_seq=0 2024-09-12T03:24:06.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.326+0000 7fc9897fa640 1 -- 172.21.15.6:0/1480741758 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fc994040030 con 0x5561fe2b3b60 2024-09-12T03:24:06.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.326+0000 7fc9897fa640 10 client.25339 handle_command_reply: tid=0 2024-09-12T03:24:06.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.326+0000 7fc9aac6c740 1 client.25339 shutdown 2024-09-12T03:24:06.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.328+0000 7fc9aac6c740 1 -- 172.21.15.6:0/1480741758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561fe21d0b0 msgr2=0x5561fe221a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.328+0000 7fc9aac6c740 1 --2- 172.21.15.6:0/1480741758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561fe21d0b0 0x5561fe221a30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc998012fd0 tx=0x7fc99800cad0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.330+0000 7fc9aac6c740 1 -- 172.21.15.6:0/1480741758 shutdown_connections 2024-09-12T03:24:06.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.330+0000 7fc9aac6c740 1 --2- 172.21.15.6:0/1480741758 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5561fe2b3b60 0x5561fe2b5f80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.331+0000 7fc9aac6c740 1 --2- 172.21.15.6:0/1480741758 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5561fe227970 0x5561fe221f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.331+0000 7fc9aac6c740 1 --2- 172.21.15.6:0/1480741758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561fe21d0b0 0x5561fe221a30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.436 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:24:06.436 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2024-09-12T03:24:06.436 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2024-09-12T03:24:06.436 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:24:06.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.331+0000 7fc9aac6c740 1 --2- 172.21.15.6:0/1480741758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5561fe21c790 0x5561fe2214f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.331+0000 7fc9897fa640 0 client.25339 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:06.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.331+0000 7fc9aac6c740 1 -- 172.21.15.6:0/1480741758 >> 172.21.15.6:0/1480741758 conn(0x5561fe218d20 msgr2=0x5561fe21fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:06.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.331+0000 7fc9aac6c740 1 -- 172.21.15.6:0/1480741758 shutdown_connections 2024-09-12T03:24:06.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.331+0000 7fc9aac6c740 1 -- 172.21.15.6:0/1480741758 wait complete. 2024-09-12T03:24:06.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.331+0000 7fc9aac6c740 20 client.25339 trim_cache size 0 max 16384 2024-09-12T03:24:06.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.332+0000 7fc9917fa640 1 -- 172.21.15.6:0/4287373058 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc98c0b3dd0 msgr2=0x7fc98c0b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.332+0000 7fc9917fa640 1 --2- 172.21.15.6:0/4287373058 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc98c0b3dd0 0x7fc98c0b6290 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc994009230 tx=0x7fc994005760 comp rx=0 tx=0).stop 2024-09-12T03:24:06.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.332+0000 7fc9917fa640 1 -- 172.21.15.6:0/4287373058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9a4102ef0 msgr2=0x7fc9a410a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.332+0000 7fc9917fa640 1 --2- 172.21.15.6:0/4287373058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9a4102ef0 0x7fc9a410a6c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00ead0 tx=0x7fc99c00ef90 comp rx=0 tx=0).stop 2024-09-12T03:24:06.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.332+0000 7fc9917fa640 1 -- 172.21.15.6:0/4287373058 shutdown_connections 2024-09-12T03:24:06.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.332+0000 7fc9917fa640 1 --2- 172.21.15.6:0/4287373058 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc98c0b3dd0 0x7fc98c0b6290 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.332+0000 7fc9917fa640 1 --2- 172.21.15.6:0/4287373058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9a4102ef0 0x7fc9a410a6c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.332+0000 7fc9917fa640 1 --2- 172.21.15.6:0/4287373058 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9a4102570 0x7fc9a410a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.332+0000 7fc9917fa640 1 --2- 172.21.15.6:0/4287373058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc9a4100e20 0x7fc9a4109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.332+0000 7fc9917fa640 1 -- 172.21.15.6:0/4287373058 >> 172.21.15.6:0/4287373058 conn(0x7fc9a40fc7d0 msgr2=0x7fc9a4104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:06.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.332+0000 7fc9917fa640 1 -- 172.21.15.6:0/4287373058 shutdown_connections 2024-09-12T03:24:06.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:06.332+0000 7fc9917fa640 1 -- 172.21.15.6:0/4287373058 wait complete. 2024-09-12T03:24:06.764 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:06 smithi118 ceph-mon[35048]: pgmap v425: 97 pgs: 97 active+clean; 13 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 6.9 KiB/s rd, 68 MiB/s wr, 6.56k op/s 2024-09-12T03:24:06.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:06 smithi006 ceph-mon[30991]: pgmap v425: 97 pgs: 97 active+clean; 13 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 6.9 KiB/s rd, 68 MiB/s wr, 6.56k op/s 2024-09-12T03:24:06.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:06 smithi161 ceph-mon[35485]: pgmap v425: 97 pgs: 97 active+clean; 13 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 6.9 KiB/s rd, 68 MiB/s wr, 6.56k op/s 2024-09-12T03:24:08.144 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:24:08.144 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:24:08.700 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:08.844 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:08 smithi006 ceph-mon[30991]: pgmap v426: 97 pgs: 97 active+clean; 13 GiB data, 63 GiB used, 1009 GiB / 1.0 TiB avail; 48 KiB/s rd, 92 MiB/s wr, 8.60k op/s 2024-09-12T03:24:08.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:08 smithi161 ceph-mon[35485]: pgmap v426: 97 pgs: 97 active+clean; 13 GiB data, 63 GiB used, 1009 GiB / 1.0 TiB avail; 48 KiB/s rd, 92 MiB/s wr, 8.60k op/s 2024-09-12T03:24:08.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:08 smithi118 ceph-mon[35048]: pgmap v426: 97 pgs: 97 active+clean; 13 GiB data, 63 GiB used, 1009 GiB / 1.0 TiB avail; 48 KiB/s rd, 92 MiB/s wr, 8.60k op/s 2024-09-12T03:24:09.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.721+0000 7fe525ac3640 1 -- 172.21.15.6:0/230303589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe52011b140 msgr2=0x7fe52011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:09.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.721+0000 7fe525ac3640 1 --2- 172.21.15.6:0/230303589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe52011b140 0x7fe52011d530 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe51801acb0 tx=0x7fe51803b8e0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.722+0000 7fe525ac3640 1 -- 172.21.15.6:0/230303589 shutdown_connections 2024-09-12T03:24:09.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.722+0000 7fe525ac3640 1 --2- 172.21.15.6:0/230303589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe52011b140 0x7fe52011d530 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.722+0000 7fe525ac3640 1 --2- 172.21.15.6:0/230303589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52006ea30 0x7fe52006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.722+0000 7fe525ac3640 1 --2- 172.21.15.6:0/230303589 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe520110190 0x7fe52006e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.722+0000 7fe525ac3640 1 -- 172.21.15.6:0/230303589 >> 172.21.15.6:0/230303589 conn(0x7fe520073180 msgr2=0x7fe520073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:09.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.723+0000 7fe525ac3640 1 -- 172.21.15.6:0/230303589 shutdown_connections 2024-09-12T03:24:09.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.723+0000 7fe525ac3640 1 -- 172.21.15.6:0/230303589 wait complete. 2024-09-12T03:24:09.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.724+0000 7fe525ac3640 1 Processor -- start 2024-09-12T03:24:09.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.724+0000 7fe525ac3640 1 -- start start 2024-09-12T03:24:09.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.724+0000 7fe525ac3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52006ea30 0x7fe52011aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:09.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.724+0000 7fe525ac3640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe520110190 0x7fe520116ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:09.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.724+0000 7fe525ac3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe52011b140 0x7fe520116ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:09.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.725+0000 7fe525ac3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe520113370 con 0x7fe52011b140 2024-09-12T03:24:09.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.725+0000 7fe525ac3640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe5201131f0 con 0x7fe520110190 2024-09-12T03:24:09.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.725+0000 7fe525ac3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe5201134f0 con 0x7fe52006ea30 2024-09-12T03:24:09.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.726+0000 7fe5252c2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe52011b140 0x7fe520116ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:09.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.726+0000 7fe5252c2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe52011b140 0x7fe520116ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59008/0 (socket says 172.21.15.6:59008) 2024-09-12T03:24:09.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.726+0000 7fe5252c2640 1 -- 172.21.15.6:0/75989949 learned_addr learned my addr 172.21.15.6:0/75989949 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:09.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.726+0000 7fe51ffff640 1 --2- 172.21.15.6:0/75989949 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe520110190 0x7fe520116ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:09.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.726+0000 7fe524ac1640 1 --2- 172.21.15.6:0/75989949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52006ea30 0x7fe52011aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:09.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.728+0000 7fe524ac1640 1 -- 172.21.15.6:0/75989949 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe520110190 msgr2=0x7fe520116ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:09.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.728+0000 7fe524ac1640 1 --2- 172.21.15.6:0/75989949 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe520110190 0x7fe520116ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.728+0000 7fe524ac1640 1 -- 172.21.15.6:0/75989949 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe52011b140 msgr2=0x7fe520116ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:09.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.728+0000 7fe524ac1640 1 --2- 172.21.15.6:0/75989949 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe52011b140 0x7fe520116ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.728+0000 7fe524ac1640 1 -- 172.21.15.6:0/75989949 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe520117880 con 0x7fe52006ea30 2024-09-12T03:24:09.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.729+0000 7fe524ac1640 1 --2- 172.21.15.6:0/75989949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52006ea30 0x7fe52011aa10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe51000b3f0 tx=0x7fe51000b7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:09.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.729+0000 7fe51dffb640 1 -- 172.21.15.6:0/75989949 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510015020 con 0x7fe52006ea30 2024-09-12T03:24:09.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.729+0000 7fe525ac3640 1 -- 172.21.15.6:0/75989949 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52007a4b0 con 0x7fe52006ea30 2024-09-12T03:24:09.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.730+0000 7fe525ac3640 1 -- 172.21.15.6:0/75989949 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52007a9f0 con 0x7fe52006ea30 2024-09-12T03:24:09.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.731+0000 7fe51dffb640 1 -- 172.21.15.6:0/75989949 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe510011d10 con 0x7fe52006ea30 2024-09-12T03:24:09.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.731+0000 7fe51dffb640 1 -- 172.21.15.6:0/75989949 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510010e60 con 0x7fe52006ea30 2024-09-12T03:24:09.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.732+0000 7fe525ac3640 1 -- 172.21.15.6:0/75989949 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe50c005290 con 0x7fe52006ea30 2024-09-12T03:24:09.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.739+0000 7fe51dffb640 1 -- 172.21.15.6:0/75989949 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe510018020 con 0x7fe52006ea30 2024-09-12T03:24:09.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.740+0000 7fe51dffb640 1 --2- 172.21.15.6:0/75989949 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5080a2d50 0x7fe5080a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:09.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.741+0000 7fe51dffb640 1 -- 172.21.15.6:0/75989949 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe5100cea10 con 0x7fe52006ea30 2024-09-12T03:24:09.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.742+0000 7fe51ffff640 1 --2- 172.21.15.6:0/75989949 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5080a2d50 0x7fe5080a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:09.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.742+0000 7fe51dffb640 1 -- 172.21.15.6:0/75989949 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe51001f020 con 0x7fe52006ea30 2024-09-12T03:24:09.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:09.755+0000 7fe51ffff640 1 --2- 172.21.15.6:0/75989949 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5080a2d50 0x7fe5080a5210 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe520118270 tx=0x7fe514009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:10.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:10.264+0000 7fe525ac3640 1 -- 172.21.15.6:0/75989949 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe50c005580 con 0x7fe52006ea30 2024-09-12T03:24:10.269 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:24:10.269 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":26,"btime":"2024-09-12T03:22:16:132848+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:22:16.064679+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24463,24464,24494]},"id":1}]} 2024-09-12T03:24:10.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:10.268+0000 7fe51dffb640 1 -- 172.21.15.6:0/75989949 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11911 (secure 0 0 0) 0x7fe51009b9c0 con 0x7fe52006ea30 2024-09-12T03:24:10.269 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 26 2024-09-12T03:24:10.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:10.274+0000 7fe525ac3640 1 -- 172.21.15.6:0/75989949 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5080a2d50 msgr2=0x7fe5080a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:10.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:10.274+0000 7fe525ac3640 1 --2- 172.21.15.6:0/75989949 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5080a2d50 0x7fe5080a5210 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe520118270 tx=0x7fe514009110 comp rx=0 tx=0).stop 2024-09-12T03:24:10.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:10.274+0000 7fe525ac3640 1 -- 172.21.15.6:0/75989949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52006ea30 msgr2=0x7fe52011aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:10.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:10.274+0000 7fe525ac3640 1 --2- 172.21.15.6:0/75989949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52006ea30 0x7fe52011aa10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe51000b3f0 tx=0x7fe51000b7f0 comp rx=0 tx=0).stop 2024-09-12T03:24:10.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:10.274+0000 7fe525ac3640 1 -- 172.21.15.6:0/75989949 shutdown_connections 2024-09-12T03:24:10.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:10.275+0000 7fe525ac3640 1 --2- 172.21.15.6:0/75989949 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5080a2d50 0x7fe5080a5210 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:10.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:10.275+0000 7fe525ac3640 1 --2- 172.21.15.6:0/75989949 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe52011b140 0x7fe520116ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:10.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:10.275+0000 7fe525ac3640 1 --2- 172.21.15.6:0/75989949 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe520110190 0x7fe520116ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:10.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:10.275+0000 7fe525ac3640 1 --2- 172.21.15.6:0/75989949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52006ea30 0x7fe52011aa10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:10.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:10.275+0000 7fe525ac3640 1 -- 172.21.15.6:0/75989949 >> 172.21.15.6:0/75989949 conn(0x7fe520073180 msgr2=0x7fe52011b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:10.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:10.275+0000 7fe525ac3640 1 -- 172.21.15.6:0/75989949 shutdown_connections 2024-09-12T03:24:10.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:10.275+0000 7fe525ac3640 1 -- 172.21.15.6:0/75989949 wait complete. 2024-09-12T03:24:10.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:10 smithi006 ceph-mon[30991]: pgmap v427: 97 pgs: 97 active+clean; 13 GiB data, 65 GiB used, 1007 GiB / 1.0 TiB avail; 193 KiB/s rd, 165 MiB/s wr, 15.22k op/s 2024-09-12T03:24:10.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:10 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/75989949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:10.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:10 smithi161 ceph-mon[35485]: pgmap v427: 97 pgs: 97 active+clean; 13 GiB data, 65 GiB used, 1007 GiB / 1.0 TiB avail; 193 KiB/s rd, 165 MiB/s wr, 15.22k op/s 2024-09-12T03:24:10.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:10 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/75989949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:10.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:10 smithi118 ceph-mon[35048]: pgmap v427: 97 pgs: 97 active+clean; 13 GiB data, 65 GiB used, 1007 GiB / 1.0 TiB avail; 193 KiB/s rd, 165 MiB/s wr, 15.22k op/s 2024-09-12T03:24:10.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:10 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/75989949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:11.743 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:24:12.293 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:12.812 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:12 smithi006 ceph-mon[30991]: pgmap v428: 97 pgs: 97 active+clean; 13 GiB data, 66 GiB used, 1007 GiB / 1.0 TiB avail; 490 KiB/s rd, 172 MiB/s wr, 15.72k op/s 2024-09-12T03:24:12.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:12 smithi161 ceph-mon[35485]: pgmap v428: 97 pgs: 97 active+clean; 13 GiB data, 66 GiB used, 1007 GiB / 1.0 TiB avail; 490 KiB/s rd, 172 MiB/s wr, 15.72k op/s 2024-09-12T03:24:12.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:12 smithi118 ceph-mon[35048]: pgmap v428: 97 pgs: 97 active+clean; 13 GiB data, 66 GiB used, 1007 GiB / 1.0 TiB avail; 490 KiB/s rd, 172 MiB/s wr, 15.72k op/s 2024-09-12T03:24:13.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.293+0000 7f614417e640 1 -- 172.21.15.6:0/2636007628 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f613c109830 msgr2=0x7f613c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:13.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.293+0000 7f614417e640 1 --2- 172.21.15.6:0/2636007628 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f613c109830 0x7f613c109c90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6134009ca0 tx=0x7f613402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.295+0000 7f614417e640 1 -- 172.21.15.6:0/2636007628 shutdown_connections 2024-09-12T03:24:13.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.295+0000 7f614417e640 1 --2- 172.21.15.6:0/2636007628 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f613c11b140 0x7f613c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.295+0000 7f614417e640 1 --2- 172.21.15.6:0/2636007628 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f613c109830 0x7f613c109c90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.296+0000 7f614417e640 1 --2- 172.21.15.6:0/2636007628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f613c109450 0x7f613c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.296+0000 7f614417e640 1 -- 172.21.15.6:0/2636007628 >> 172.21.15.6:0/2636007628 conn(0x7f613c06e140 msgr2=0x7f613c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:13.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.296+0000 7f614417e640 1 -- 172.21.15.6:0/2636007628 shutdown_connections 2024-09-12T03:24:13.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.297+0000 7f614417e640 1 -- 172.21.15.6:0/2636007628 wait complete. 2024-09-12T03:24:13.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.301+0000 7f614417e640 1 Processor -- start 2024-09-12T03:24:13.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.301+0000 7f614417e640 1 -- start start 2024-09-12T03:24:13.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.302+0000 7f614417e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f613c109450 0x7f613c1a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:13.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.302+0000 7f614417e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f613c109830 0x7f613c1a7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:13.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.302+0000 7f6141f1a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f613c109450 0x7f613c1a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:13.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.302+0000 7f6141f1a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f613c109450 0x7f613c1a72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:43140/0 (socket says 172.21.15.6:43140) 2024-09-12T03:24:13.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.302+0000 7f6141f1a640 1 -- 172.21.15.6:0/661905415 learned_addr learned my addr 172.21.15.6:0/661905415 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:13.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.302+0000 7f614417e640 1 --2- 172.21.15.6:0/661905415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f613c11b140 0x7f613c1abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:13.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.302+0000 7f614417e640 1 -- 172.21.15.6:0/661905415 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f613c120e30 con 0x7f613c11b140 2024-09-12T03:24:13.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.303+0000 7f614417e640 1 -- 172.21.15.6:0/661905415 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f613c120cb0 con 0x7f613c109830 2024-09-12T03:24:13.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.303+0000 7f614417e640 1 -- 172.21.15.6:0/661905415 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f613c120fb0 con 0x7f613c109450 2024-09-12T03:24:13.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.304+0000 7f614271b640 1 --2- 172.21.15.6:0/661905415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f613c11b140 0x7f613c1abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:13.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.304+0000 7f6141f1a640 1 -- 172.21.15.6:0/661905415 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f613c109830 msgr2=0x7f613c1a7830 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:24:13.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.304+0000 7f6141f1a640 1 --2- 172.21.15.6:0/661905415 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f613c109830 0x7f613c1a7830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.304+0000 7f6141f1a640 1 -- 172.21.15.6:0/661905415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f613c11b140 msgr2=0x7f613c1abbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:13.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.304+0000 7f6141f1a640 1 --2- 172.21.15.6:0/661905415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f613c11b140 0x7f613c1abbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.304+0000 7f6141f1a640 1 -- 172.21.15.6:0/661905415 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613c1ac2a0 con 0x7f613c109450 2024-09-12T03:24:13.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.305+0000 7f6141f1a640 1 --2- 172.21.15.6:0/661905415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f613c109450 0x7f613c1a72f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f612c00b450 tx=0x7f612c00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:13.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.305+0000 7f6132ffd640 1 -- 172.21.15.6:0/661905415 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612c015020 con 0x7f613c109450 2024-09-12T03:24:13.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.306+0000 7f6132ffd640 1 -- 172.21.15.6:0/661905415 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f612c011d70 con 0x7f613c109450 2024-09-12T03:24:13.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.306+0000 7f614417e640 1 -- 172.21.15.6:0/661905415 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f613c1ac590 con 0x7f613c109450 2024-09-12T03:24:13.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.306+0000 7f614417e640 1 -- 172.21.15.6:0/661905415 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f613c1b3e70 con 0x7f613c109450 2024-09-12T03:24:13.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.306+0000 7f6132ffd640 1 -- 172.21.15.6:0/661905415 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612c015020 con 0x7f613c109450 2024-09-12T03:24:13.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.310+0000 7f6146557740 1 Processor -- start 2024-09-12T03:24:13.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.311+0000 7f6146557740 1 -- start start 2024-09-12T03:24:13.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.311+0000 7f6146557740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededb8b40 0x55dededb7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:13.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.311+0000 7f6146557740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dededb75d0 0x55dededb7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:13.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.311+0000 7f6146557740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededc2370 0x55dededc4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:13.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.311+0000 7f6146557740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55deded1d2f0 con 0x55dededb75d0 2024-09-12T03:24:13.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.311+0000 7f6146557740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55dedecc4cf0 con 0x55dededc2370 2024-09-12T03:24:13.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.311+0000 7f6146557740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55deded192b0 con 0x55dededb8b40 2024-09-12T03:24:13.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.313+0000 7f6132ffd640 1 -- 172.21.15.6:0/661905415 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f612c018020 con 0x7f613c109450 2024-09-12T03:24:13.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.313+0000 7f6141719640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededc2370 0x55dededc4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:13.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.313+0000 7f6141719640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededc2370 0x55dededc4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:43866/0 (socket says 172.21.15.6:43866) 2024-09-12T03:24:13.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.313+0000 7f6141719640 1 -- 172.21.15.6:0/1678002199 learned_addr learned my addr 172.21.15.6:0/1678002199 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:13.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.313+0000 7f614271b640 1 --2- 172.21.15.6:0/1678002199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededb8b40 0x55dededb7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:13.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.314+0000 7f6141719640 1 -- 172.21.15.6:0/1678002199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededb8b40 msgr2=0x55dededb7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:13.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.314+0000 7f6141719640 1 --2- 172.21.15.6:0/1678002199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededb8b40 0x55dededb7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.314+0000 7f6141719640 1 -- 172.21.15.6:0/1678002199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dededb75d0 msgr2=0x55dededb7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:13.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.316+0000 7f6132ffd640 1 --2- 172.21.15.6:0/661905415 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f61280a2f90 0x7f61280a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:13.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.317+0000 7f6132ffd640 1 -- 172.21.15.6:0/661905415 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f612c011690 con 0x7f613c109450 2024-09-12T03:24:13.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.317+0000 7f614271b640 1 --2- 172.21.15.6:0/661905415 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f61280a2f90 0x7f61280a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:13.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.317+0000 7f6141f1a640 1 --2- 172.21.15.6:0/1678002199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dededb75d0 0x55dededb7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:13.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.317+0000 7f6141719640 1 --2- 172.21.15.6:0/1678002199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dededb75d0 0x55dededb7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.318+0000 7f6141719640 1 -- 172.21.15.6:0/1678002199 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dedeca93a0 con 0x55dededc2370 2024-09-12T03:24:13.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.318+0000 7f614271b640 1 --2- 172.21.15.6:0/661905415 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f61280a2f90 0x7f61280a5450 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6138009ce0 tx=0x7f6138005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:13.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.320+0000 7f6141719640 1 --2- 172.21.15.6:0/1678002199 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededc2370 0x55dededc4760 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6134005e30 tx=0x7f6134005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dd6d6fc501f23d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:13.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.321+0000 7f6127fff640 1 -- 172.21.15.6:0/1678002199 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134031770 con 0x55dededc2370 2024-09-12T03:24:13.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.321+0000 7f6127fff640 1 -- 172.21.15.6:0/1678002199 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6134036040 con 0x55dededc2370 2024-09-12T03:24:13.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.322+0000 7f6146557740 1 -- 172.21.15.6:0/1678002199 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededc2370 msgr2=0x55dededc4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:13.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.322+0000 7f6146557740 1 --2- 172.21.15.6:0/1678002199 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededc2370 0x55dededc4760 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6134005e30 tx=0x7f6134005fd0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.323+0000 7f6146557740 1 -- 172.21.15.6:0/1678002199 shutdown_connections 2024-09-12T03:24:13.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.323+0000 7f6146557740 1 --2- 172.21.15.6:0/1678002199 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededc2370 0x55dededc4760 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.323+0000 7f6146557740 1 --2- 172.21.15.6:0/1678002199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dededb75d0 0x55dededb7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.323+0000 7f6146557740 1 --2- 172.21.15.6:0/1678002199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededb8b40 0x55dededb7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.323+0000 7f6146557740 1 -- 172.21.15.6:0/1678002199 >> 172.21.15.6:0/1678002199 conn(0x55deded80310 msgr2=0x55dededb3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:13.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.323+0000 7f6146557740 1 -- 172.21.15.6:0/1678002199 shutdown_connections 2024-09-12T03:24:13.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.324+0000 7f6146557740 1 -- 172.21.15.6:0/1678002199 wait complete. 2024-09-12T03:24:13.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.325+0000 7f6146557740 1 -- start start 2024-09-12T03:24:13.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.325+0000 7f6146557740 1 Processor -- start 2024-09-12T03:24:13.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.326+0000 7f6146557740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dededb75d0 0x55dededc4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:13.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.326+0000 7f6146557740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededb8b40 0x55dededc16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:13.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.326+0000 7f6146557740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededc1c10 0x55dededc2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:13.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.326+0000 7f6146557740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55dededc50b0 con 0x55dededb75d0 2024-09-12T03:24:13.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.326+0000 7f6146557740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55dedecd8060 con 0x55dededb8b40 2024-09-12T03:24:13.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.326+0000 7f6146557740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55dededc5230 con 0x55dededc1c10 2024-09-12T03:24:13.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.326+0000 7f6146557740 10 client.0 _ll_register_callbacks cb 0x55dedea2b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:13.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.327+0000 7f6141f1a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dededb75d0 0x55dededc4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:13.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.327+0000 7f614271b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededc1c10 0x55dededc2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:13.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.327+0000 7f614271b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededc1c10 0x55dededc2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:43158/0 (socket says 172.21.15.6:43158) 2024-09-12T03:24:13.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.327+0000 7f614271b640 1 -- 172.21.15.6:0/4193076288 learned_addr learned my addr 172.21.15.6:0/4193076288 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:13.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.327+0000 7f6141719640 1 --2- 172.21.15.6:0/4193076288 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededb8b40 0x55dededc16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:13.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.327+0000 7f6141f1a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dededb75d0 0x55dededc4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59050/0 (socket says 172.21.15.6:59050) 2024-09-12T03:24:13.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.327+0000 7f614271b640 1 -- 172.21.15.6:0/4193076288 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededb8b40 msgr2=0x55dededc16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:13.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.327+0000 7f614271b640 1 --2- 172.21.15.6:0/4193076288 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededb8b40 0x55dededc16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.327+0000 7f614271b640 1 -- 172.21.15.6:0/4193076288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dededb75d0 msgr2=0x55dededc4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:13.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.327+0000 7f614271b640 1 --2- 172.21.15.6:0/4193076288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dededb75d0 0x55dededc4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.327+0000 7f614271b640 1 -- 172.21.15.6:0/4193076288 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dedea8e3c0 con 0x55dededc1c10 2024-09-12T03:24:13.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.328+0000 7f614271b640 1 --2- 172.21.15.6:0/4193076288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededc1c10 0x55dededc2090 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f61380209c0 tx=0x7f6138004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3db6f6ca77676b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:13.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.328+0000 7f61257fa640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:24:13.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.328+0000 7f61257fa640 1 -- 172.21.15.6:0/4193076288 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613802d060 con 0x55dededc1c10 2024-09-12T03:24:13.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.328+0000 7f6146557740 1 -- 172.21.15.6:0/4193076288 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55deded2a860 con 0x55dededc1c10 2024-09-12T03:24:13.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.328+0000 7f61257fa640 1 -- 172.21.15.6:0/4193076288 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f6138018030 con 0x55dededc1c10 2024-09-12T03:24:13.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.328+0000 7f61257fa640 1 -- 172.21.15.6:0/4193076288 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f61380119c0 con 0x55dededc1c10 2024-09-12T03:24:13.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.328+0000 7f6146557740 10 client.25351 fetch_fsmap learned FSMap version 26 2024-09-12T03:24:13.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.328+0000 7f6146557740 1 -- 172.21.15.6:0/4193076288 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dedede1e50 con 0x55dededc1c10 2024-09-12T03:24:13.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.329+0000 7f61257fa640 1 -- 172.21.15.6:0/4193076288 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55deded2a860 con 0x55dededc1c10 2024-09-12T03:24:13.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.330+0000 7f61257fa640 1 -- 172.21.15.6:0/4193076288 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6138028c70 con 0x55dededc1c10 2024-09-12T03:24:13.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.330+0000 7f6146557740 10 client.25351 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:24:13.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.330+0000 7f6146557740 10 client.25351 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-12T03:24:13.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.330+0000 7f6146557740 20 client.25351 populate_metadata read hostname 'smithi006' 2024-09-12T03:24:13.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.330+0000 7f6146557740 1 --2- 172.21.15.6:0/4193076288 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55dedede36a0 0x55dedede5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:13.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.330+0000 7f6146557740 4 client.25351 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:24:13.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.330+0000 7f6146557740 1 -- 172.21.15.6:0/4193076288 --> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55deded24d70 con 0x55dedede36a0 2024-09-12T03:24:13.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.331+0000 7f6141f1a640 1 --2- 172.21.15.6:0/4193076288 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55dedede36a0 0x55dedede5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:13.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.338+0000 7f6141f1a640 1 --2- 172.21.15.6:0/4193076288 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55dedede36a0 0x55dedede5b20 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4a00bd2445016ae9 server_cookie=ded91eb9057f4333 in_seq=0 out_seq=0 2024-09-12T03:24:13.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.338+0000 7f61257fa640 10 client.25351 ms_handle_connect on v2:172.21.15.161:6838/3645555254 2024-09-12T03:24:13.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.340+0000 7f61257fa640 1 -- 172.21.15.6:0/4193076288 <== mds.2 v2:172.21.15.161:6838/3645555254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55deded24d70 con 0x55dedede36a0 2024-09-12T03:24:13.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.340+0000 7f61257fa640 10 client.25351 handle_command_reply: tid=0 2024-09-12T03:24:13.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.340+0000 7f6146557740 1 client.25351 shutdown 2024-09-12T03:24:13.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.341+0000 7f6146557740 1 -- 172.21.15.6:0/4193076288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededc1c10 msgr2=0x55dededc2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:13.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.341+0000 7f6146557740 1 --2- 172.21.15.6:0/4193076288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededc1c10 0x55dededc2090 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f61380209c0 tx=0x7f6138004730 comp rx=0 tx=0).stop 2024-09-12T03:24:13.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.341+0000 7f6146557740 1 -- 172.21.15.6:0/4193076288 shutdown_connections 2024-09-12T03:24:13.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.341+0000 7f6146557740 1 --2- 172.21.15.6:0/4193076288 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55dedede36a0 0x55dedede5b20 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.341+0000 7f6146557740 1 --2- 172.21.15.6:0/4193076288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededc1c10 0x55dededc2090 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.341+0000 7f6146557740 1 --2- 172.21.15.6:0/4193076288 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededb8b40 0x55dededc16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.341+0000 7f6146557740 1 --2- 172.21.15.6:0/4193076288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dededb75d0 0x55dededc4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.341+0000 7f6146557740 1 -- 172.21.15.6:0/4193076288 >> 172.21.15.6:0/4193076288 conn(0x55deded80310 msgr2=0x55dededb9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:13.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.341+0000 7f61257fa640 0 client.25351 ms_handle_reset on v2:172.21.15.161:6838/3645555254 2024-09-12T03:24:13.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.341+0000 7f6146557740 1 -- 172.21.15.6:0/4193076288 shutdown_connections 2024-09-12T03:24:13.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.341+0000 7f6146557740 1 -- 172.21.15.6:0/4193076288 wait complete. 2024-09-12T03:24:13.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.342+0000 7f6146557740 20 client.25351 trim_cache size 0 max 16384 2024-09-12T03:24:13.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.375+0000 7f6146557740 1 Processor -- start 2024-09-12T03:24:13.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.375+0000 7f6146557740 1 -- start start 2024-09-12T03:24:13.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.376+0000 7f6146557740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededb75d0 0x55dededb8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:13.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.376+0000 7f6146557740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededb8b40 0x55dededb3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:13.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.376+0000 7f6141f1a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededb75d0 0x55dededb8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:13.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.376+0000 7f6141f1a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededb75d0 0x55dededb8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:43162/0 (socket says 172.21.15.6:43162) 2024-09-12T03:24:13.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.377+0000 7f6141f1a640 1 -- 172.21.15.6:0/3633450956 learned_addr learned my addr 172.21.15.6:0/3633450956 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:13.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.377+0000 7f6146557740 1 --2- 172.21.15.6:0/3633450956 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dededc1c10 0x55dededc3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:13.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.377+0000 7f6146557740 1 -- 172.21.15.6:0/3633450956 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1b6430 con 0x55dededc1c10 2024-09-12T03:24:13.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.377+0000 7f6146557740 1 -- 172.21.15.6:0/3633450956 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f613c118d90 con 0x55dededb8b40 2024-09-12T03:24:13.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.378+0000 7f614271b640 1 --2- 172.21.15.6:0/3633450956 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dededc1c10 0x55dededc3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:13.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.378+0000 7f6146557740 1 -- 172.21.15.6:0/3633450956 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f613c116c90 con 0x55dededb75d0 2024-09-12T03:24:13.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.378+0000 7f6141f1a640 1 -- 172.21.15.6:0/3633450956 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededb8b40 msgr2=0x55dededb3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:13.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.378+0000 7f6141f1a640 1 --2- 172.21.15.6:0/3633450956 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededb8b40 0x55dededb3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.378+0000 7f6141f1a640 1 -- 172.21.15.6:0/3633450956 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dededc1c10 msgr2=0x55dededc3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:13.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.378+0000 7f6141f1a640 1 --2- 172.21.15.6:0/3633450956 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dededc1c10 0x55dededc3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.379+0000 7f6141f1a640 1 -- 172.21.15.6:0/3633450956 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dededdf990 con 0x55dededb75d0 2024-09-12T03:24:13.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.380+0000 7f6141f1a640 1 --2- 172.21.15.6:0/3633450956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededb75d0 0x55dededb8410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f612c0977f0 tx=0x7f612c010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58be4193828ed0c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:13.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.380+0000 7f61277fe640 1 -- 172.21.15.6:0/3633450956 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612c015040 con 0x55dededb75d0 2024-09-12T03:24:13.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.380+0000 7f61277fe640 1 -- 172.21.15.6:0/3633450956 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f612c09f1d0 con 0x55dededb75d0 2024-09-12T03:24:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.381+0000 7f6146557740 1 -- 172.21.15.6:0/3633450956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededb75d0 msgr2=0x55dededb8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.381+0000 7f6146557740 1 --2- 172.21.15.6:0/3633450956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededb75d0 0x55dededb8410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f612c0977f0 tx=0x7f612c010830 comp rx=0 tx=0).stop 2024-09-12T03:24:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.382+0000 7f6146557740 1 -- 172.21.15.6:0/3633450956 shutdown_connections 2024-09-12T03:24:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.382+0000 7f6146557740 1 --2- 172.21.15.6:0/3633450956 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dededc1c10 0x55dededc3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.382+0000 7f6146557740 1 --2- 172.21.15.6:0/3633450956 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededb8b40 0x55dededb3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.382+0000 7f6146557740 1 --2- 172.21.15.6:0/3633450956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededb75d0 0x55dededb8410 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.382+0000 7f6146557740 1 -- 172.21.15.6:0/3633450956 >> 172.21.15.6:0/3633450956 conn(0x55deded80310 msgr2=0x55dededc2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:13.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.385+0000 7f6146557740 1 -- 172.21.15.6:0/3633450956 shutdown_connections 2024-09-12T03:24:13.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.385+0000 7f6146557740 1 -- 172.21.15.6:0/3633450956 wait complete. 2024-09-12T03:24:13.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.385+0000 7f6146557740 1 -- start start 2024-09-12T03:24:13.386 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.385+0000 7f6146557740 1 Processor -- start 2024-09-12T03:24:13.386 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.386+0000 7f6146557740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dededb75d0 0x55dededca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:13.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.386+0000 7f6146557740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededb8b40 0x55dededc7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:13.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.386+0000 7f6146557740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededc1c10 0x55dededc8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:13.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.386+0000 7f6146557740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f613c117250 con 0x55dededb75d0 2024-09-12T03:24:13.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.386+0000 7f6146557740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1186c0 con 0x55dededb8b40 2024-09-12T03:24:13.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.386+0000 7f6146557740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f613c117920 con 0x55dededc1c10 2024-09-12T03:24:13.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.386+0000 7f6146557740 10 client.0 _ll_register_callbacks cb 0x7f61380095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:13.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.386+0000 7f614271b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededc1c10 0x55dededc8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:13.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.386+0000 7f614271b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededc1c10 0x55dededc8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:43176/0 (socket says 172.21.15.6:43176) 2024-09-12T03:24:13.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.386+0000 7f614271b640 1 -- 172.21.15.6:0/426375030 learned_addr learned my addr 172.21.15.6:0/426375030 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:13.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.387+0000 7f6141719640 1 --2- 172.21.15.6:0/426375030 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededb8b40 0x55dededc7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:13.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.387+0000 7f6141719640 1 -- 172.21.15.6:0/426375030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededc1c10 msgr2=0x55dededc8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:13.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.387+0000 7f6141719640 1 --2- 172.21.15.6:0/426375030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededc1c10 0x55dededc8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.387+0000 7f6141719640 1 -- 172.21.15.6:0/426375030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dededb75d0 msgr2=0x55dededca9f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:24:13.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.387+0000 7f6141719640 1 --2- 172.21.15.6:0/426375030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dededb75d0 0x55dededca9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.387+0000 7f6141719640 1 -- 172.21.15.6:0/426375030 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dededc8840 con 0x55dededb8b40 2024-09-12T03:24:13.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.388+0000 7f6141719640 1 --2- 172.21.15.6:0/426375030 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededb8b40 0x55dededc7d40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f613402aed0 tx=0x7f6134004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c06384a109a60f5f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:13.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.388+0000 7f61257fa640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:24:13.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.388+0000 7f61257fa640 1 -- 172.21.15.6:0/426375030 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134031e10 con 0x55dededb8b40 2024-09-12T03:24:13.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.388+0000 7f6146557740 1 -- 172.21.15.6:0/426375030 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f612c09f1d0 con 0x55dededb8b40 2024-09-12T03:24:13.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.389+0000 7f61257fa640 1 -- 172.21.15.6:0/426375030 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f6134003970 con 0x55dededb8b40 2024-09-12T03:24:13.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.390+0000 7f61257fa640 1 -- 172.21.15.6:0/426375030 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6134003b70 con 0x55dededb8b40 2024-09-12T03:24:13.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.390+0000 7f6146557740 10 client.25310 fetch_fsmap learned FSMap version 26 2024-09-12T03:24:13.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.390+0000 7f6146557740 1 -- 172.21.15.6:0/426375030 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dedee4e1a0 con 0x55dededb8b40 2024-09-12T03:24:13.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.391+0000 7f61257fa640 1 -- 172.21.15.6:0/426375030 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f612c09f1d0 con 0x55dededb8b40 2024-09-12T03:24:13.392 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.392+0000 7f61257fa640 1 -- 172.21.15.6:0/426375030 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6134052450 con 0x55dededb8b40 2024-09-12T03:24:13.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.392+0000 7f6146557740 10 client.25310 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:24:13.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.392+0000 7f6146557740 10 client.25310 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-12T03:24:13.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.392+0000 7f6146557740 20 client.25310 populate_metadata read hostname 'smithi006' 2024-09-12T03:24:13.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.392+0000 7f6146557740 1 --2- 172.21.15.6:0/426375030 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55dedee4e920 0x55dedee50d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:13.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.392+0000 7f6146557740 4 client.25310 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:24:13.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.392+0000 7f6146557740 1 -- 172.21.15.6:0/426375030 --> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dedeca1170 con 0x55dedee4e920 2024-09-12T03:24:13.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.392+0000 7f6141f1a640 1 --2- 172.21.15.6:0/426375030 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55dedee4e920 0x55dedee50d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:13.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.393+0000 7f6141f1a640 1 --2- 172.21.15.6:0/426375030 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55dedee4e920 0x55dedee50d40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=824292bdb857b46f server_cookie=126c312d3070928e in_seq=0 out_seq=0 2024-09-12T03:24:13.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.393+0000 7f61257fa640 10 client.25310 ms_handle_connect on v2:172.21.15.161:6838/3645555254 2024-09-12T03:24:13.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.393+0000 7f61257fa640 1 -- 172.21.15.6:0/426375030 <== mds.2 v2:172.21.15.161:6838/3645555254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dedeca1170 con 0x55dedee4e920 2024-09-12T03:24:13.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.393+0000 7f61257fa640 10 client.25310 handle_command_reply: tid=0 2024-09-12T03:24:13.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.393+0000 7f6146557740 1 client.25310 shutdown 2024-09-12T03:24:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.394+0000 7f6146557740 1 -- 172.21.15.6:0/426375030 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededb8b40 msgr2=0x55dededc7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.394+0000 7f6146557740 1 --2- 172.21.15.6:0/426375030 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededb8b40 0x55dededc7d40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f613402aed0 tx=0x7f6134004750 comp rx=0 tx=0).stop 2024-09-12T03:24:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.394+0000 7f6146557740 1 -- 172.21.15.6:0/426375030 shutdown_connections 2024-09-12T03:24:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.394+0000 7f6146557740 1 --2- 172.21.15.6:0/426375030 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55dedee4e920 0x55dedee50d40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.394+0000 7f6146557740 1 --2- 172.21.15.6:0/426375030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dededc1c10 0x55dededc8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.394+0000 7f6146557740 1 --2- 172.21.15.6:0/426375030 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55dededb8b40 0x55dededc7d40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.394+0000 7f61257fa640 0 client.25310 ms_handle_reset on v2:172.21.15.161:6838/3645555254 2024-09-12T03:24:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.394+0000 7f6146557740 1 --2- 172.21.15.6:0/426375030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dededb75d0 0x55dededca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.394+0000 7f6146557740 1 -- 172.21.15.6:0/426375030 >> 172.21.15.6:0/426375030 conn(0x55deded80310 msgr2=0x55dededc3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.394+0000 7f6146557740 1 -- 172.21.15.6:0/426375030 shutdown_connections 2024-09-12T03:24:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.395+0000 7f6146557740 1 -- 172.21.15.6:0/426375030 wait complete. 2024-09-12T03:24:13.397 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-09-12T03:24:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.396+0000 7f6146557740 20 client.25310 trim_cache size 0 max 16384 2024-09-12T03:24:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.397+0000 7f614417e640 1 -- 172.21.15.6:0/661905415 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f61280a2f90 msgr2=0x7f61280a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.397+0000 7f614417e640 1 --2- 172.21.15.6:0/661905415 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f61280a2f90 0x7f61280a5450 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6138009ce0 tx=0x7f6138005f90 comp rx=0 tx=0).stop 2024-09-12T03:24:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.397+0000 7f614417e640 1 -- 172.21.15.6:0/661905415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f613c109450 msgr2=0x7f613c1a72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.397+0000 7f614417e640 1 --2- 172.21.15.6:0/661905415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f613c109450 0x7f613c1a72f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f612c00b450 tx=0x7f612c00b910 comp rx=0 tx=0).stop 2024-09-12T03:24:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.397+0000 7f614417e640 1 -- 172.21.15.6:0/661905415 shutdown_connections 2024-09-12T03:24:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.398+0000 7f614417e640 1 --2- 172.21.15.6:0/661905415 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f61280a2f90 0x7f61280a5450 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.398+0000 7f614417e640 1 --2- 172.21.15.6:0/661905415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f613c11b140 0x7f613c1abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.398+0000 7f614417e640 1 --2- 172.21.15.6:0/661905415 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f613c109830 0x7f613c1a7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.398+0000 7f614417e640 1 --2- 172.21.15.6:0/661905415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f613c109450 0x7f613c1a72f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.398+0000 7f614417e640 1 -- 172.21.15.6:0/661905415 >> 172.21.15.6:0/661905415 conn(0x7f613c06e140 msgr2=0x7f613c11c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.398+0000 7f614417e640 1 -- 172.21.15.6:0/661905415 shutdown_connections 2024-09-12T03:24:13.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:13.398+0000 7f614417e640 1 -- 172.21.15.6:0/661905415 wait complete. 2024-09-12T03:24:14.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:14 smithi006 ceph-mon[30991]: pgmap v429: 97 pgs: 97 active+clean; 13 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 831 KiB/s rd, 219 MiB/s wr, 19.65k op/s 2024-09-12T03:24:14.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:14 smithi161 ceph-mon[35485]: pgmap v429: 97 pgs: 97 active+clean; 13 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 831 KiB/s rd, 219 MiB/s wr, 19.65k op/s 2024-09-12T03:24:14.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:14 smithi118 ceph-mon[35048]: pgmap v429: 97 pgs: 97 active+clean; 13 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 831 KiB/s rd, 219 MiB/s wr, 19.65k op/s 2024-09-12T03:24:15.001 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:24:15.568 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:16.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.455+0000 7f1bdd06c640 1 -- 172.21.15.6:0/3490300707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bd8109e00 msgr2=0x7f1bd810e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.455+0000 7f1bdd06c640 1 --2- 172.21.15.6:0/3490300707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bd8109e00 0x7f1bd810e350 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000dd10 tx=0x7f1bc0032980 comp rx=0 tx=0).stop 2024-09-12T03:24:16.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.455+0000 7f1bdd06c640 1 -- 172.21.15.6:0/3490300707 shutdown_connections 2024-09-12T03:24:16.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.455+0000 7f1bdd06c640 1 --2- 172.21.15.6:0/3490300707 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1bd810ea80 0x7f1bd81154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.455+0000 7f1bdd06c640 1 --2- 172.21.15.6:0/3490300707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bd8109e00 0x7f1bd810e350 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.455+0000 7f1bdd06c640 1 --2- 172.21.15.6:0/3490300707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bd8109450 0x7f1bd8109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.455+0000 7f1bdd06c640 1 -- 172.21.15.6:0/3490300707 >> 172.21.15.6:0/3490300707 conn(0x7f1bd806e2d0 msgr2=0x7f1bd806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:16.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.456+0000 7f1bdd06c640 1 -- 172.21.15.6:0/3490300707 shutdown_connections 2024-09-12T03:24:16.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.456+0000 7f1bdd06c640 1 -- 172.21.15.6:0/3490300707 wait complete. 2024-09-12T03:24:16.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.457+0000 7f1bdd06c640 1 Processor -- start 2024-09-12T03:24:16.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.457+0000 7f1bdd06c640 1 -- start start 2024-09-12T03:24:16.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.458+0000 7f1bdd06c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bd8109450 0x7f1bd81b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.458+0000 7f1bdd06c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1bd8109e00 0x7f1bd81b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.458+0000 7f1bdd06c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bd810ea80 0x7f1bd81af670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.458+0000 7f1bdd06c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd8118570 con 0x7f1bd8109450 2024-09-12T03:24:16.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.458+0000 7f1bdd06c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81183f0 con 0x7f1bd8109e00 2024-09-12T03:24:16.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.458+0000 7f1bdd06c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81186f0 con 0x7f1bd810ea80 2024-09-12T03:24:16.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.459+0000 7f1bd759e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bd810ea80 0x7f1bd81af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.459+0000 7f1bd759e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bd810ea80 0x7f1bd81af670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:43190/0 (socket says 172.21.15.6:43190) 2024-09-12T03:24:16.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.459+0000 7f1bd759e640 1 -- 172.21.15.6:0/4206123160 learned_addr learned my addr 172.21.15.6:0/4206123160 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:16.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.459+0000 7f1bce59c640 1 --2- 172.21.15.6:0/4206123160 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1bd8109e00 0x7f1bd81b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.460+0000 7f1bd6d9d640 1 --2- 172.21.15.6:0/4206123160 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bd8109450 0x7f1bd81b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.460+0000 7f1bce59c640 1 -- 172.21.15.6:0/4206123160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bd810ea80 msgr2=0x7f1bd81af670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.460+0000 7f1bce59c640 1 --2- 172.21.15.6:0/4206123160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bd810ea80 0x7f1bd81af670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.460+0000 7f1bce59c640 1 -- 172.21.15.6:0/4206123160 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bd8109450 msgr2=0x7f1bd81b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.460+0000 7f1bce59c640 1 --2- 172.21.15.6:0/4206123160 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bd8109450 0x7f1bd81b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.460+0000 7f1bce59c640 1 -- 172.21.15.6:0/4206123160 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd81afee0 con 0x7f1bd8109e00 2024-09-12T03:24:16.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.460+0000 7f1bd6d9d640 1 --2- 172.21.15.6:0/4206123160 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bd8109450 0x7f1bd81b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:16.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.460+0000 7f1bce59c640 1 --2- 172.21.15.6:0/4206123160 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1bd8109e00 0x7f1bd81b5ae0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0032f70 tx=0x7f1bc0005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:16.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.460+0000 7f1bd4d99640 1 -- 172.21.15.6:0/4206123160 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc000ceb0 con 0x7f1bd8109e00 2024-09-12T03:24:16.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.460+0000 7f1bdd06c640 1 -- 172.21.15.6:0/4206123160 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd81b0170 con 0x7f1bd8109e00 2024-09-12T03:24:16.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.460+0000 7f1bd4d99640 1 -- 172.21.15.6:0/4206123160 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bc00046c0 con 0x7f1bd8109e00 2024-09-12T03:24:16.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.461+0000 7f1bd4d99640 1 -- 172.21.15.6:0/4206123160 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc00171a0 con 0x7f1bd8109e00 2024-09-12T03:24:16.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.461+0000 7f1bdd06c640 1 -- 172.21.15.6:0/4206123160 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd81bc520 con 0x7f1bd8109e00 2024-09-12T03:24:16.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.465+0000 7f1bd4d99640 1 -- 172.21.15.6:0/4206123160 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1bc003f030 con 0x7f1bd8109e00 2024-09-12T03:24:16.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.465+0000 7f1bdf445740 1 Processor -- start 2024-09-12T03:24:16.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.465+0000 7f1bdf445740 1 -- start start 2024-09-12T03:24:16.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.466+0000 7f1bdf445740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d27edb40 0x5640d27ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.466+0000 7f1bdf445740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27ec5d0 0x5640d27eca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.466+0000 7f1bdf445740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5640d2819380 0x5640d281b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.466+0000 7f1bdf445740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5640d27522f0 con 0x5640d2819380 2024-09-12T03:24:16.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.466+0000 7f1bdf445740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5640d26f9cf0 con 0x5640d27ec5d0 2024-09-12T03:24:16.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.466+0000 7f1bdf445740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5640d274e2b0 con 0x5640d27edb40 2024-09-12T03:24:16.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.466+0000 7f1bd759e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d27edb40 0x5640d27ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.467+0000 7f1bd759e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d27edb40 0x5640d27ec090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:43202/0 (socket says 172.21.15.6:43202) 2024-09-12T03:24:16.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.467+0000 7f1bd759e640 1 -- 172.21.15.6:0/4064719069 learned_addr learned my addr 172.21.15.6:0/4064719069 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:16.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.467+0000 7f1bd6d9d640 1 --2- 172.21.15.6:0/4064719069 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27ec5d0 0x5640d27eca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.467+0000 7f1bce59c640 1 --2- 172.21.15.6:0/4064719069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5640d2819380 0x5640d281b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.468+0000 7f1bd4d99640 1 --2- 172.21.15.6:0/4206123160 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1bc40a2cc0 0x7f1bc40a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.468+0000 7f1bd759e640 1 -- 172.21.15.6:0/4064719069 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27ec5d0 msgr2=0x5640d27eca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:16.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.468+0000 7f1bd759e640 1 --2- 172.21.15.6:0/4064719069 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27ec5d0 0x5640d27eca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.468+0000 7f1bd759e640 1 -- 172.21.15.6:0/4064719069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5640d2819380 msgr2=0x5640d281b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:16.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.468+0000 7f1bd759e640 1 --2- 172.21.15.6:0/4064719069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5640d2819380 0x5640d281b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.468+0000 7f1bd759e640 1 -- 172.21.15.6:0/4064719069 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640d26de3a0 con 0x5640d27edb40 2024-09-12T03:24:16.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.468+0000 7f1bd759e640 1 --2- 172.21.15.6:0/4206123160 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1bc40a2cc0 0x7f1bc40a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.468+0000 7f1bd4d99640 1 -- 172.21.15.6:0/4206123160 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f1bc00eed40 con 0x7f1bd8109e00 2024-09-12T03:24:16.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.469+0000 7f1bd759e640 1 --2- 172.21.15.6:0/4064719069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d27edb40 0x5640d27ec090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1bd000bcc0 tx=0x7f1bd00107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae4d4578ece5bff5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:16.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.469+0000 7f1bd759e640 1 --2- 172.21.15.6:0/4206123160 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1bc40a2cc0 0x7f1bc40a5180 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0021000 tx=0x7f1bd0022000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:16.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.469+0000 7f1bcd59a640 1 -- 172.21.15.6:0/4064719069 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0011b90 con 0x5640d27edb40 2024-09-12T03:24:16.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.470+0000 7f1bcd59a640 1 -- 172.21.15.6:0/4064719069 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bd0011d30 con 0x5640d27edb40 2024-09-12T03:24:16.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.470+0000 7f1bcd59a640 1 -- 172.21.15.6:0/4064719069 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd001d660 con 0x5640d27edb40 2024-09-12T03:24:16.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.471+0000 7f1bdf445740 1 -- 172.21.15.6:0/4064719069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d27edb40 msgr2=0x5640d27ec090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.471+0000 7f1bdf445740 1 --2- 172.21.15.6:0/4064719069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d27edb40 0x5640d27ec090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1bd000bcc0 tx=0x7f1bd00107a0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.472+0000 7f1bdf445740 1 -- 172.21.15.6:0/4064719069 shutdown_connections 2024-09-12T03:24:16.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.472+0000 7f1bdf445740 1 --2- 172.21.15.6:0/4064719069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5640d2819380 0x5640d281b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.472+0000 7f1bdf445740 1 --2- 172.21.15.6:0/4064719069 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27ec5d0 0x5640d27eca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.472+0000 7f1bdf445740 1 --2- 172.21.15.6:0/4064719069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d27edb40 0x5640d27ec090 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.472+0000 7f1bdf445740 1 -- 172.21.15.6:0/4064719069 >> 172.21.15.6:0/4064719069 conn(0x5640d27b5310 msgr2=0x5640d27e8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:16.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.472+0000 7f1bdf445740 1 -- 172.21.15.6:0/4064719069 shutdown_connections 2024-09-12T03:24:16.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.473+0000 7f1bdf445740 1 -- 172.21.15.6:0/4064719069 wait complete. 2024-09-12T03:24:16.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.473+0000 7f1bdf445740 1 -- start start 2024-09-12T03:24:16.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.473+0000 7f1bdf445740 1 Processor -- start 2024-09-12T03:24:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.474+0000 7f1bdf445740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27ec5d0 0x5640d27f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.474+0000 7f1bdf445740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5640d27edb40 0x5640d27f4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.474+0000 7f1bdf445740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d2819380 0x5640d27f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.474+0000 7f1bdf445740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5640d281c080 con 0x5640d27edb40 2024-09-12T03:24:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.474+0000 7f1bdf445740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5640d270d060 con 0x5640d27ec5d0 2024-09-12T03:24:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.474+0000 7f1bdf445740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5640d281c200 con 0x5640d2819380 2024-09-12T03:24:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.474+0000 7f1bdf445740 10 client.0 _ll_register_callbacks cb 0x5640d24608c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.474+0000 7f1bd6d9d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27ec5d0 0x5640d27f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.474+0000 7f1bd6d9d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27ec5d0 0x5640d27f6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:43932/0 (socket says 172.21.15.6:43932) 2024-09-12T03:24:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.474+0000 7f1bd6d9d640 1 -- 172.21.15.6:0/1394560273 learned_addr learned my addr 172.21.15.6:0/1394560273 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.474+0000 7f1bd759e640 1 --2- 172.21.15.6:0/1394560273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d2819380 0x5640d27f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.475+0000 7f1bd6d9d640 1 -- 172.21.15.6:0/1394560273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d2819380 msgr2=0x5640d27f45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:16.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.475+0000 7f1bd6d9d640 1 --2- 172.21.15.6:0/1394560273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d2819380 0x5640d27f45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.475+0000 7f1bd6d9d640 1 -- 172.21.15.6:0/1394560273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5640d27edb40 msgr2=0x5640d27f4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:16.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.475+0000 7f1bd6d9d640 1 --2- 172.21.15.6:0/1394560273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5640d27edb40 0x5640d27f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.475+0000 7f1bd6d9d640 1 -- 172.21.15.6:0/1394560273 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640d24c33c0 con 0x5640d27ec5d0 2024-09-12T03:24:16.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.476+0000 7f1bd6d9d640 1 --2- 172.21.15.6:0/1394560273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27ec5d0 0x5640d27f6d50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8012720 tx=0x7f1bc8012be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=373f244a47d35901 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:16.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.476+0000 7f1bbe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:24:16.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.476+0000 7f1bbe7fc640 1 -- 172.21.15.6:0/1394560273 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8010dc0 con 0x5640d27ec5d0 2024-09-12T03:24:16.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.476+0000 7f1bdf445740 1 -- 172.21.15.6:0/1394560273 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bd0011d30 con 0x5640d27ec5d0 2024-09-12T03:24:16.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.476+0000 7f1bbe7fc640 1 -- 172.21.15.6:0/1394560273 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f1bc801a3e0 con 0x5640d27ec5d0 2024-09-12T03:24:16.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.476+0000 7f1bbe7fc640 1 -- 172.21.15.6:0/1394560273 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1bc801a620 con 0x5640d27ec5d0 2024-09-12T03:24:16.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.476+0000 7f1bdf445740 10 client.25322 fetch_fsmap learned FSMap version 26 2024-09-12T03:24:16.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.476+0000 7f1bdf445740 1 -- 172.21.15.6:0/1394560273 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5640d2827c00 con 0x5640d27ec5d0 2024-09-12T03:24:16.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.477+0000 7f1bbe7fc640 1 -- 172.21.15.6:0/1394560273 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bd0011d30 con 0x5640d27ec5d0 2024-09-12T03:24:16.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.477+0000 7f1bbe7fc640 1 -- 172.21.15.6:0/1394560273 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1bc8019070 con 0x5640d27ec5d0 2024-09-12T03:24:16.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.477+0000 7f1bdf445740 10 client.25322 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:24:16.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.477+0000 7f1bdf445740 10 client.25322 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-12T03:24:16.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.477+0000 7f1bdf445740 20 client.25322 populate_metadata read hostname 'smithi006' 2024-09-12T03:24:16.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.477+0000 7f1bdf445740 1 --2- 172.21.15.6:0/1394560273 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x5640d2829010 0x5640d282b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.477+0000 7f1bdf445740 4 client.25322 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:24:16.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.477+0000 7f1bdf445740 1 -- 172.21.15.6:0/1394560273 --> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640d275f860 con 0x5640d2829010 2024-09-12T03:24:16.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.478+0000 7f1bce59c640 1 --2- 172.21.15.6:0/1394560273 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x5640d2829010 0x5640d282b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.479+0000 7f1bce59c640 1 --2- 172.21.15.6:0/1394560273 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x5640d2829010 0x5640d282b4b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1440be2c7c0814ca server_cookie=d0e542fc338fb79b in_seq=0 out_seq=0 2024-09-12T03:24:16.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.479+0000 7f1bbe7fc640 10 client.25322 ms_handle_connect on v2:172.21.15.118:6836/3879191727 2024-09-12T03:24:16.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.481+0000 7f1bbe7fc640 1 -- 172.21.15.6:0/1394560273 <== mds.1 v2:172.21.15.118:6836/3879191727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5640d275f860 con 0x5640d2829010 2024-09-12T03:24:16.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.481+0000 7f1bbe7fc640 10 client.25322 handle_command_reply: tid=0 2024-09-12T03:24:16.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.481+0000 7f1bdf445740 1 client.25322 shutdown 2024-09-12T03:24:16.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.482+0000 7f1bdf445740 1 -- 172.21.15.6:0/1394560273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27ec5d0 msgr2=0x5640d27f6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.482+0000 7f1bdf445740 1 --2- 172.21.15.6:0/1394560273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27ec5d0 0x5640d27f6d50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8012720 tx=0x7f1bc8012be0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.482+0000 7f1bdf445740 1 -- 172.21.15.6:0/1394560273 shutdown_connections 2024-09-12T03:24:16.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.482+0000 7f1bdf445740 1 --2- 172.21.15.6:0/1394560273 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x5640d2829010 0x5640d282b4b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.482+0000 7f1bdf445740 1 --2- 172.21.15.6:0/1394560273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d2819380 0x5640d27f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.482+0000 7f1bdf445740 1 --2- 172.21.15.6:0/1394560273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5640d27edb40 0x5640d27f4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.482+0000 7f1bdf445740 1 --2- 172.21.15.6:0/1394560273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27ec5d0 0x5640d27f6d50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.482+0000 7f1bdf445740 1 -- 172.21.15.6:0/1394560273 >> 172.21.15.6:0/1394560273 conn(0x5640d27b5310 msgr2=0x5640d27ee960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:16.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.482+0000 7f1bbe7fc640 0 client.25322 ms_handle_reset on v2:172.21.15.118:6836/3879191727 2024-09-12T03:24:16.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.482+0000 7f1bdf445740 1 -- 172.21.15.6:0/1394560273 shutdown_connections 2024-09-12T03:24:16.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.482+0000 7f1bdf445740 1 -- 172.21.15.6:0/1394560273 wait complete. 2024-09-12T03:24:16.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.482+0000 7f1bdf445740 20 client.25322 trim_cache size 0 max 16384 2024-09-12T03:24:16.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.511+0000 7f1bdf445740 1 Processor -- start 2024-09-12T03:24:16.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.511+0000 7f1bdf445740 1 -- start start 2024-09-12T03:24:16.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.512+0000 7f1bdf445740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5640d27ec5d0 0x5640d2825370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.512+0000 7f1bdf445740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27edb40 0x5640d27f28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.513+0000 7f1bd6d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5640d27ec5d0 0x5640d2825370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.513+0000 7f1bd6d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5640d27ec5d0 0x5640d2825370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59122/0 (socket says 172.21.15.6:59122) 2024-09-12T03:24:16.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.513+0000 7f1bd6d9d640 1 -- 172.21.15.6:0/4057343627 learned_addr learned my addr 172.21.15.6:0/4057343627 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:16.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.513+0000 7f1bce59c640 1 --2- 172.21.15.6:0/4057343627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27edb40 0x5640d27f28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.513+0000 7f1bdf445740 1 --2- 172.21.15.6:0/4057343627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d2819380 0x5640d27f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.513+0000 7f1bdf445740 1 -- 172.21.15.6:0/4057343627 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81a84d0 con 0x5640d27ec5d0 2024-09-12T03:24:16.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.514+0000 7f1bdf445740 1 -- 172.21.15.6:0/4057343627 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81a5de0 con 0x5640d27edb40 2024-09-12T03:24:16.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.514+0000 7f1bdf445740 1 -- 172.21.15.6:0/4057343627 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd806c9b0 con 0x5640d2819380 2024-09-12T03:24:16.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.514+0000 7f1bd759e640 1 --2- 172.21.15.6:0/4057343627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d2819380 0x5640d27f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.515+0000 7f1bce59c640 1 -- 172.21.15.6:0/4057343627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d2819380 msgr2=0x5640d27f2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:16.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.515+0000 7f1bce59c640 1 --2- 172.21.15.6:0/4057343627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d2819380 0x5640d27f2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.515+0000 7f1bce59c640 1 -- 172.21.15.6:0/4057343627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5640d27ec5d0 msgr2=0x5640d2825370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:16.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.515+0000 7f1bce59c640 1 --2- 172.21.15.6:0/4057343627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5640d27ec5d0 0x5640d2825370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.515+0000 7f1bce59c640 1 -- 172.21.15.6:0/4057343627 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640d27f4940 con 0x5640d27edb40 2024-09-12T03:24:16.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.516+0000 7f1bce59c640 1 --2- 172.21.15.6:0/4057343627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27edb40 0x5640d27f28d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1bc00b79d0 tx=0x7f1bc00a6040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28ed02d41cf5fa8f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:16.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.518+0000 7f1bccd99640 1 -- 172.21.15.6:0/4057343627 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0046040 con 0x5640d27edb40 2024-09-12T03:24:16.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.519+0000 7f1bccd99640 1 -- 172.21.15.6:0/4057343627 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bc00f2020 con 0x5640d27edb40 2024-09-12T03:24:16.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.519+0000 7f1bccd99640 1 -- 172.21.15.6:0/4057343627 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc00b9870 con 0x5640d27edb40 2024-09-12T03:24:16.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.520+0000 7f1bdf445740 1 -- 172.21.15.6:0/4057343627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27edb40 msgr2=0x5640d27f28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.520+0000 7f1bdf445740 1 --2- 172.21.15.6:0/4057343627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27edb40 0x5640d27f28d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1bc00b79d0 tx=0x7f1bc00a6040 comp rx=0 tx=0).stop 2024-09-12T03:24:16.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.521+0000 7f1bdf445740 1 -- 172.21.15.6:0/4057343627 shutdown_connections 2024-09-12T03:24:16.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.521+0000 7f1bdf445740 1 --2- 172.21.15.6:0/4057343627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d2819380 0x5640d27f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.521+0000 7f1bdf445740 1 --2- 172.21.15.6:0/4057343627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27edb40 0x5640d27f28d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.521+0000 7f1bdf445740 1 --2- 172.21.15.6:0/4057343627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5640d27ec5d0 0x5640d2825370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.521+0000 7f1bdf445740 1 -- 172.21.15.6:0/4057343627 >> 172.21.15.6:0/4057343627 conn(0x5640d27b5310 msgr2=0x5640d27ef2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:16.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.521+0000 7f1bdf445740 1 -- 172.21.15.6:0/4057343627 shutdown_connections 2024-09-12T03:24:16.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.521+0000 7f1bdf445740 1 -- 172.21.15.6:0/4057343627 wait complete. 2024-09-12T03:24:16.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.522+0000 7f1bdf445740 1 -- start start 2024-09-12T03:24:16.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.522+0000 7f1bdf445740 1 Processor -- start 2024-09-12T03:24:16.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.523+0000 7f1bdf445740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27ec5d0 0x5640d2821910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.523+0000 7f1bdf445740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d27edb40 0x5640d281cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.523+0000 7f1bd6d9d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27ec5d0 0x5640d2821910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.524+0000 7f1bce59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d27edb40 0x5640d281cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.524+0000 7f1bce59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d27edb40 0x5640d281cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:43228/0 (socket says 172.21.15.6:43228) 2024-09-12T03:24:16.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.524+0000 7f1bce59c640 1 -- 172.21.15.6:0/2627749415 learned_addr learned my addr 172.21.15.6:0/2627749415 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:16.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.524+0000 7f1bdf445740 1 --2- 172.21.15.6:0/2627749415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5640d2819380 0x5640d281d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.524+0000 7f1bdf445740 1 -- 172.21.15.6:0/2627749415 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81a42a0 con 0x5640d2819380 2024-09-12T03:24:16.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.524+0000 7f1bdf445740 1 -- 172.21.15.6:0/2627749415 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81a5710 con 0x5640d27ec5d0 2024-09-12T03:24:16.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.524+0000 7f1bdf445740 1 -- 172.21.15.6:0/2627749415 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81a4970 con 0x5640d27edb40 2024-09-12T03:24:16.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.525+0000 7f1bd759e640 1 --2- 172.21.15.6:0/2627749415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5640d2819380 0x5640d281d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.525+0000 7f1bd6d9d640 1 -- 172.21.15.6:0/2627749415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d27edb40 msgr2=0x5640d281cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:16.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.525+0000 7f1bd6d9d640 1 --2- 172.21.15.6:0/2627749415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d27edb40 0x5640d281cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.525+0000 7f1bd6d9d640 1 -- 172.21.15.6:0/2627749415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5640d2819380 msgr2=0x5640d281d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:16.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.525+0000 7f1bd6d9d640 1 --2- 172.21.15.6:0/2627749415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5640d2819380 0x5640d281d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.525+0000 7f1bd6d9d640 1 -- 172.21.15.6:0/2627749415 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640d281d6c0 con 0x5640d27ec5d0 2024-09-12T03:24:16.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.526+0000 7f1bdf445740 10 client.0 _ll_register_callbacks cb 0x7f1bc8009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:16.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.526+0000 7f1bd6d9d640 1 --2- 172.21.15.6:0/2627749415 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27ec5d0 0x5640d2821910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8012490 tx=0x7f1bc8009c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3623eea409832c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:16.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.526+0000 7f1bbe7fc640 10 client.25328 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:24:16.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.527+0000 7f1bdf445740 1 -- 172.21.15.6:0/2627749415 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bc00f2020 con 0x5640d27ec5d0 2024-09-12T03:24:16.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.535+0000 7f1bbe7fc640 1 -- 172.21.15.6:0/2627749415 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc802a070 con 0x5640d27ec5d0 2024-09-12T03:24:16.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.535+0000 7f1bbe7fc640 1 -- 172.21.15.6:0/2627749415 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f1bc8012220 con 0x5640d27ec5d0 2024-09-12T03:24:16.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.535+0000 7f1bbe7fc640 1 -- 172.21.15.6:0/2627749415 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1bc8024420 con 0x5640d27ec5d0 2024-09-12T03:24:16.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.536+0000 7f1bdf445740 10 client.25328 fetch_fsmap learned FSMap version 26 2024-09-12T03:24:16.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.536+0000 7f1bdf445740 1 -- 172.21.15.6:0/2627749415 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5640d2893900 con 0x5640d27ec5d0 2024-09-12T03:24:16.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.537+0000 7f1bbe7fc640 1 -- 172.21.15.6:0/2627749415 <== mon.2 v2:172.21.15.118:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1bc8019070 con 0x5640d27ec5d0 2024-09-12T03:24:16.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.537+0000 7f1bdf445740 10 client.25328 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:24:16.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.537+0000 7f1bdf445740 10 client.25328 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-12T03:24:16.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.537+0000 7f1bdf445740 20 client.25328 populate_metadata read hostname 'smithi006' 2024-09-12T03:24:16.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.537+0000 7f1bdf445740 1 --2- 172.21.15.6:0/2627749415 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x5640d2893f40 0x5640d28963a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.537+0000 7f1bdf445740 4 client.25328 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:24:16.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.537+0000 7f1bdf445740 1 -- 172.21.15.6:0/2627749415 --> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f1bc00f2020 con 0x5640d2893f40 2024-09-12T03:24:16.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.537+0000 7f1bce59c640 1 --2- 172.21.15.6:0/2627749415 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x5640d2893f40 0x5640d28963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.538+0000 7f1bce59c640 1 --2- 172.21.15.6:0/2627749415 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x5640d2893f40 0x5640d28963a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f387a20ecd265146 server_cookie=3ee92bf580e6194b in_seq=0 out_seq=0 2024-09-12T03:24:16.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.538+0000 7f1bbe7fc640 10 client.25328 ms_handle_connect on v2:172.21.15.118:6836/3879191727 2024-09-12T03:24:16.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.538+0000 7f1bbe7fc640 1 -- 172.21.15.6:0/2627749415 <== mds.1 v2:172.21.15.118:6836/3879191727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f1bc00f2020 con 0x5640d2893f40 2024-09-12T03:24:16.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.538+0000 7f1bbe7fc640 10 client.25328 handle_command_reply: tid=0 2024-09-12T03:24:16.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.538+0000 7f1bdf445740 1 client.25328 shutdown 2024-09-12T03:24:16.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.539+0000 7f1bdf445740 1 -- 172.21.15.6:0/2627749415 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27ec5d0 msgr2=0x5640d2821910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.539+0000 7f1bdf445740 1 --2- 172.21.15.6:0/2627749415 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27ec5d0 0x5640d2821910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8012490 tx=0x7f1bc8009c70 comp rx=0 tx=0).stop 2024-09-12T03:24:16.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.539+0000 7f1bdf445740 1 -- 172.21.15.6:0/2627749415 shutdown_connections 2024-09-12T03:24:16.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.539+0000 7f1bdf445740 1 --2- 172.21.15.6:0/2627749415 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x5640d2893f40 0x5640d28963a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.539+0000 7f1bdf445740 1 --2- 172.21.15.6:0/2627749415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5640d2819380 0x5640d281d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.539+0000 7f1bdf445740 1 --2- 172.21.15.6:0/2627749415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5640d27edb40 0x5640d281cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.539+0000 7f1bbe7fc640 0 client.25328 ms_handle_reset on v2:172.21.15.118:6836/3879191727 2024-09-12T03:24:16.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.540+0000 7f1bdf445740 1 --2- 172.21.15.6:0/2627749415 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5640d27ec5d0 0x5640d2821910 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.540+0000 7f1bdf445740 1 -- 172.21.15.6:0/2627749415 >> 172.21.15.6:0/2627749415 conn(0x5640d27b5310 msgr2=0x5640d27f2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:16.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.540+0000 7f1bdf445740 1 -- 172.21.15.6:0/2627749415 shutdown_connections 2024-09-12T03:24:16.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.540+0000 7f1bdf445740 1 -- 172.21.15.6:0/2627749415 wait complete. 2024-09-12T03:24:16.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.540+0000 7f1bdf445740 20 client.25328 trim_cache size 0 max 16384 2024-09-12T03:24:16.541 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-09-12T03:24:16.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.542+0000 7f1bdd06c640 1 -- 172.21.15.6:0/4206123160 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1bc40a2cc0 msgr2=0x7f1bc40a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.542+0000 7f1bdd06c640 1 --2- 172.21.15.6:0/4206123160 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1bc40a2cc0 0x7f1bc40a5180 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0021000 tx=0x7f1bd0022000 comp rx=0 tx=0).stop 2024-09-12T03:24:16.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.542+0000 7f1bdd06c640 1 -- 172.21.15.6:0/4206123160 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1bd8109e00 msgr2=0x7f1bd81b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.542+0000 7f1bdd06c640 1 --2- 172.21.15.6:0/4206123160 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1bd8109e00 0x7f1bd81b5ae0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0032f70 tx=0x7f1bc0005040 comp rx=0 tx=0).stop 2024-09-12T03:24:16.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.542+0000 7f1bdd06c640 1 -- 172.21.15.6:0/4206123160 shutdown_connections 2024-09-12T03:24:16.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.542+0000 7f1bdd06c640 1 --2- 172.21.15.6:0/4206123160 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f1bc40a2cc0 0x7f1bc40a5180 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.542+0000 7f1bdd06c640 1 --2- 172.21.15.6:0/4206123160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bd810ea80 0x7f1bd81af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.542+0000 7f1bdd06c640 1 --2- 172.21.15.6:0/4206123160 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1bd8109e00 0x7f1bd81b5ae0 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0032f70 tx=0x7f1bc0005040 comp rx=0 tx=0).stop 2024-09-12T03:24:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.543+0000 7f1bdd06c640 1 --2- 172.21.15.6:0/4206123160 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bd8109450 0x7f1bd81b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.543+0000 7f1bdd06c640 1 -- 172.21.15.6:0/4206123160 >> 172.21.15.6:0/4206123160 conn(0x7f1bd806e2d0 msgr2=0x7f1bd8072730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.544+0000 7f1bdd06c640 1 -- 172.21.15.6:0/4206123160 shutdown_connections 2024-09-12T03:24:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:16.544+0000 7f1bdd06c640 1 -- 172.21.15.6:0/4206123160 wait complete. 2024-09-12T03:24:16.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:16 smithi006 ceph-mon[30991]: pgmap v430: 97 pgs: 97 active+clean; 13 GiB data, 68 GiB used, 1004 GiB / 1.0 TiB avail; 829 KiB/s rd, 251 MiB/s wr, 21.86k op/s 2024-09-12T03:24:16.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:16 smithi161 ceph-mon[35485]: pgmap v430: 97 pgs: 97 active+clean; 13 GiB data, 68 GiB used, 1004 GiB / 1.0 TiB avail; 829 KiB/s rd, 251 MiB/s wr, 21.86k op/s 2024-09-12T03:24:16.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:16 smithi118 ceph-mon[35048]: pgmap v430: 97 pgs: 97 active+clean; 13 GiB data, 68 GiB used, 1004 GiB / 1.0 TiB avail; 829 KiB/s rd, 251 MiB/s wr, 21.86k op/s 2024-09-12T03:24:18.224 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:24:18.856 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:18.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:18 smithi006 ceph-mon[30991]: pgmap v431: 97 pgs: 97 active+clean; 13 GiB data, 69 GiB used, 1004 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 255 MiB/s wr, 21.97k op/s 2024-09-12T03:24:18.867 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:18 smithi161 ceph-mon[35485]: pgmap v431: 97 pgs: 97 active+clean; 13 GiB data, 69 GiB used, 1004 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 255 MiB/s wr, 21.97k op/s 2024-09-12T03:24:18.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:18 smithi118 ceph-mon[35048]: pgmap v431: 97 pgs: 97 active+clean; 13 GiB data, 69 GiB used, 1004 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 255 MiB/s wr, 21.97k op/s 2024-09-12T03:24:19.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:19 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:19.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.742+0000 7f384e574640 1 -- 172.21.15.6:0/2508281214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f384811b140 msgr2=0x7f384811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.742+0000 7f384e574640 1 --2- 172.21.15.6:0/2508281214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f384811b140 0x7f384811d530 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f384001acb0 tx=0x7f384003b920 comp rx=0 tx=0).stop 2024-09-12T03:24:19.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.743+0000 7f384e574640 1 -- 172.21.15.6:0/2508281214 shutdown_connections 2024-09-12T03:24:19.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.743+0000 7f384e574640 1 --2- 172.21.15.6:0/2508281214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f384811b140 0x7f384811d530 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.743+0000 7f384e574640 1 --2- 172.21.15.6:0/2508281214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f384806ea30 0x7f384806ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.743+0000 7f384e574640 1 --2- 172.21.15.6:0/2508281214 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3848110190 0x7f384806e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.743+0000 7f384e574640 1 -- 172.21.15.6:0/2508281214 >> 172.21.15.6:0/2508281214 conn(0x7f3848073180 msgr2=0x7f3848073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:19.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.750+0000 7f384e574640 1 -- 172.21.15.6:0/2508281214 shutdown_connections 2024-09-12T03:24:19.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.751+0000 7f384e574640 1 -- 172.21.15.6:0/2508281214 wait complete. 2024-09-12T03:24:19.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.753+0000 7f384e574640 1 Processor -- start 2024-09-12T03:24:19.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.759+0000 7f384e574640 1 -- start start 2024-09-12T03:24:19.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.759+0000 7f384e574640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f384806ea30 0x7f384811a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.759+0000 7f384e574640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3848110190 0x7f38481169c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.759+0000 7f384e574640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f384811b140 0x7f3848116f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.759+0000 7f384e574640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3848113310 con 0x7f384806ea30 2024-09-12T03:24:19.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.759+0000 7f384e574640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f3848113190 con 0x7f384811b140 2024-09-12T03:24:19.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.759+0000 7f384e574640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3848113490 con 0x7f3848110190 2024-09-12T03:24:19.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.760+0000 7f384dd73640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f384811b140 0x7f3848116f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.760+0000 7f384dd73640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f384811b140 0x7f3848116f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:39604/0 (socket says 172.21.15.6:39604) 2024-09-12T03:24:19.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.760+0000 7f384dd73640 1 -- 172.21.15.6:0/1129875448 learned_addr learned my addr 172.21.15.6:0/1129875448 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:19.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.760+0000 7f384cd71640 1 --2- 172.21.15.6:0/1129875448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3848110190 0x7f38481169c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.761+0000 7f384d572640 1 --2- 172.21.15.6:0/1129875448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f384806ea30 0x7f384811a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.762+0000 7f384dd73640 1 -- 172.21.15.6:0/1129875448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3848110190 msgr2=0x7f38481169c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.762+0000 7f384dd73640 1 --2- 172.21.15.6:0/1129875448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3848110190 0x7f38481169c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.762+0000 7f384dd73640 1 -- 172.21.15.6:0/1129875448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f384806ea30 msgr2=0x7f384811a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.762+0000 7f384dd73640 1 --2- 172.21.15.6:0/1129875448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f384806ea30 0x7f384811a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.762+0000 7f384dd73640 1 -- 172.21.15.6:0/1129875448 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3848117680 con 0x7f384811b140 2024-09-12T03:24:19.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.762+0000 7f384dd73640 1 --2- 172.21.15.6:0/1129875448 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f384811b140 0x7f3848116f00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f384001ac00 tx=0x7f3840040aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:19.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.763+0000 7f383e7fc640 1 -- 172.21.15.6:0/1129875448 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3840045070 con 0x7f384811b140 2024-09-12T03:24:19.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.763+0000 7f384e574640 1 -- 172.21.15.6:0/1129875448 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38481aaf60 con 0x7f384811b140 2024-09-12T03:24:19.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.763+0000 7f384e574640 1 -- 172.21.15.6:0/1129875448 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38481ab410 con 0x7f384811b140 2024-09-12T03:24:19.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.763+0000 7f383e7fc640 1 -- 172.21.15.6:0/1129875448 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3840041a60 con 0x7f384811b140 2024-09-12T03:24:19.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.764+0000 7f383e7fc640 1 -- 172.21.15.6:0/1129875448 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3840045070 con 0x7f384811b140 2024-09-12T03:24:19.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.767+0000 7f3851baf740 1 Processor -- start 2024-09-12T03:24:19.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.768+0000 7f383e7fc640 1 -- 172.21.15.6:0/1129875448 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3840048020 con 0x7f384811b140 2024-09-12T03:24:19.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.768+0000 7f3851baf740 1 -- start start 2024-09-12T03:24:19.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.769+0000 7f3851baf740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd3a0d20 0x55a5dd39f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.769+0000 7f3851baf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5dd39f7b0 0x55a5dd39fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.769+0000 7f3851baf740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5dd3aa550 0x55a5dd3ac940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.769+0000 7f3851baf740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a5dd3052f0 con 0x55a5dd3aa550 2024-09-12T03:24:19.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.769+0000 7f3851baf740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55a5dd2accf0 con 0x55a5dd3a0d20 2024-09-12T03:24:19.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.769+0000 7f3851baf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a5dd3012b0 con 0x55a5dd39f7b0 2024-09-12T03:24:19.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.769+0000 7f383e7fc640 1 --2- 172.21.15.6:0/1129875448 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38340a2dd0 0x7f38340a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.770+0000 7f383e7fc640 1 -- 172.21.15.6:0/1129875448 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f3840100920 con 0x7f384811b140 2024-09-12T03:24:19.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.770+0000 7f384dd73640 1 --2- 172.21.15.6:0/1129875448 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38340a2dd0 0x7f38340a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.770+0000 7f384cd71640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd3a0d20 0x55a5dd39f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.770+0000 7f384cd71640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd3a0d20 0x55a5dd39f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:39614/0 (socket says 172.21.15.6:39614) 2024-09-12T03:24:19.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.770+0000 7f384cd71640 1 -- 172.21.15.6:0/3412659378 learned_addr learned my addr 172.21.15.6:0/3412659378 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:19.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.770+0000 7f384cd71640 1 --2- 172.21.15.6:0/3412659378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5dd3aa550 0x55a5dd3ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.771+0000 7f384cd71640 1 -- 172.21.15.6:0/3412659378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5dd39f7b0 msgr2=0x55a5dd39fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:19.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.771+0000 7f384cd71640 1 --2- 172.21.15.6:0/3412659378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5dd39f7b0 0x55a5dd39fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.771+0000 7f384cd71640 1 -- 172.21.15.6:0/3412659378 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd3a0d20 msgr2=0x55a5dd39f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:19.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.771+0000 7f384cd71640 1 --2- 172.21.15.6:0/3412659378 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd3a0d20 0x55a5dd39f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.771+0000 7f384cd71640 1 -- 172.21.15.6:0/3412659378 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5dd2913a0 con 0x55a5dd3aa550 2024-09-12T03:24:19.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.771+0000 7f384cd71640 1 --2- 172.21.15.6:0/3412659378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5dd3aa550 0x55a5dd3ac940 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3838017700 tx=0x7f38380090e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d251904e086f08c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:19.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.773+0000 7f38337fe640 1 -- 172.21.15.6:0/3412659378 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838015760 con 0x55a5dd3aa550 2024-09-12T03:24:19.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.773+0000 7f38337fe640 1 -- 172.21.15.6:0/3412659378 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383801d070 con 0x55a5dd3aa550 2024-09-12T03:24:19.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.773+0000 7f38337fe640 1 -- 172.21.15.6:0/3412659378 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838018b90 con 0x55a5dd3aa550 2024-09-12T03:24:19.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.774+0000 7f3851baf740 1 -- 172.21.15.6:0/3412659378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5dd3aa550 msgr2=0x55a5dd3ac940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.774+0000 7f3851baf740 1 --2- 172.21.15.6:0/3412659378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5dd3aa550 0x55a5dd3ac940 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3838017700 tx=0x7f38380090e0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.774+0000 7f384dd73640 1 --2- 172.21.15.6:0/1129875448 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38340a2dd0 0x7f38340a5290 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f38400c9610 tx=0x7f384001a5d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:19.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.776+0000 7f3851baf740 1 -- 172.21.15.6:0/3412659378 shutdown_connections 2024-09-12T03:24:19.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.776+0000 7f3851baf740 1 --2- 172.21.15.6:0/3412659378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5dd3aa550 0x55a5dd3ac940 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.776+0000 7f3851baf740 1 --2- 172.21.15.6:0/3412659378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5dd39f7b0 0x55a5dd39fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.776+0000 7f3851baf740 1 --2- 172.21.15.6:0/3412659378 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd3a0d20 0x55a5dd39f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.776+0000 7f3851baf740 1 -- 172.21.15.6:0/3412659378 >> 172.21.15.6:0/3412659378 conn(0x55a5dd368310 msgr2=0x55a5dd39be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:19.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.776+0000 7f3851baf740 1 -- 172.21.15.6:0/3412659378 shutdown_connections 2024-09-12T03:24:19.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.776+0000 7f3851baf740 1 -- 172.21.15.6:0/3412659378 wait complete. 2024-09-12T03:24:19.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.777+0000 7f3851baf740 1 -- start start 2024-09-12T03:24:19.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.777+0000 7f3851baf740 1 Processor -- start 2024-09-12T03:24:19.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.778+0000 7f3851baf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5dd39f7b0 0x55a5dd3a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.778+0000 7f3851baf740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd3a0d20 0x55a5dd3a7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.778+0000 7f3851baf740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5dd3aa550 0x55a5dd3a79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.778+0000 7f3851baf740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a5dd3ad1f0 con 0x55a5dd3aa550 2024-09-12T03:24:19.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.779+0000 7f3851baf740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55a5dd2c0060 con 0x55a5dd3a0d20 2024-09-12T03:24:19.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.779+0000 7f384d572640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd3a0d20 0x55a5dd3a7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.779+0000 7f384d572640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd3a0d20 0x55a5dd3a7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:39626/0 (socket says 172.21.15.6:39626) 2024-09-12T03:24:19.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.779+0000 7f384d572640 1 -- 172.21.15.6:0/3363720537 learned_addr learned my addr 172.21.15.6:0/3363720537 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:19.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.779+0000 7f3851baf740 1 -- 172.21.15.6:0/3363720537 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a5dd3ad370 con 0x55a5dd39f7b0 2024-09-12T03:24:19.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.779+0000 7f384cd71640 1 --2- 172.21.15.6:0/3363720537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5dd39f7b0 0x55a5dd3a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.779+0000 7f3851baf740 10 client.0 _ll_register_callbacks cb 0x55a5dd0138c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:19.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.779+0000 7f384cd71640 1 --2- 172.21.15.6:0/3363720537 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5dd3aa550 0x55a5dd3a79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.780+0000 7f384cd71640 1 -- 172.21.15.6:0/3363720537 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd3a0d20 msgr2=0x55a5dd3a7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:19.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.780+0000 7f384cd71640 1 --2- 172.21.15.6:0/3363720537 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd3a0d20 0x55a5dd3a7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.780+0000 7f384cd71640 1 -- 172.21.15.6:0/3363720537 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5dd3aa550 msgr2=0x55a5dd3a79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:19.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.780+0000 7f384cd71640 1 --2- 172.21.15.6:0/3363720537 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5dd3aa550 0x55a5dd3a79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.780+0000 7f384cd71640 1 -- 172.21.15.6:0/3363720537 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5dd0763c0 con 0x55a5dd39f7b0 2024-09-12T03:24:19.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.781+0000 7f384cd71640 1 --2- 172.21.15.6:0/3363720537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5dd39f7b0 0x55a5dd3a9f80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3838018770 tx=0x7f3838009a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=160640a990e5c5bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:19.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.781+0000 7f3830ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:24:19.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.781+0000 7f3830ff9640 1 -- 172.21.15.6:0/3363720537 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838017e90 con 0x55a5dd39f7b0 2024-09-12T03:24:19.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.782+0000 7f3851baf740 1 -- 172.21.15.6:0/3363720537 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f383801d070 con 0x55a5dd39f7b0 2024-09-12T03:24:19.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.782+0000 7f3830ff9640 1 -- 172.21.15.6:0/3363720537 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f3838015730 con 0x55a5dd39f7b0 2024-09-12T03:24:19.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.782+0000 7f3830ff9640 1 -- 172.21.15.6:0/3363720537 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3838015930 con 0x55a5dd39f7b0 2024-09-12T03:24:19.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.782+0000 7f3851baf740 10 client.25381 fetch_fsmap learned FSMap version 26 2024-09-12T03:24:19.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.782+0000 7f3851baf740 1 -- 172.21.15.6:0/3363720537 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a5dd3b8d20 con 0x55a5dd39f7b0 2024-09-12T03:24:19.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.783+0000 7f3830ff9640 1 -- 172.21.15.6:0/3363720537 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3838029030 con 0x55a5dd39f7b0 2024-09-12T03:24:19.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.785+0000 7f3830ff9640 1 -- 172.21.15.6:0/3363720537 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f383802ea40 con 0x55a5dd39f7b0 2024-09-12T03:24:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.785+0000 7f3851baf740 10 client.25381 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:24:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.785+0000 7f3851baf740 10 client.25381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:24:19.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.785+0000 7f3851baf740 20 client.25381 populate_metadata read hostname 'smithi006' 2024-09-12T03:24:19.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.786+0000 7f3851baf740 1 --2- 172.21.15.6:0/3363720537 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a5dd3ba4f0 0x55a5dd3bc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.786+0000 7f3851baf740 4 client.25381 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:24:19.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.786+0000 7f3851baf740 1 -- 172.21.15.6:0/3363720537 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f383801d070 con 0x55a5dd3ba4f0 2024-09-12T03:24:19.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.790+0000 7f384d572640 1 --2- 172.21.15.6:0/3363720537 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a5dd3ba4f0 0x55a5dd3bc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.791+0000 7f384d572640 1 --2- 172.21.15.6:0/3363720537 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a5dd3ba4f0 0x55a5dd3bc970 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf6e97fd0dc1cb65 server_cookie=fc04ab613d104354 in_seq=0 out_seq=0 2024-09-12T03:24:19.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.792+0000 7f3830ff9640 10 client.25381 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:19.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.794+0000 7f3830ff9640 1 -- 172.21.15.6:0/3363720537 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f383801d070 con 0x55a5dd3ba4f0 2024-09-12T03:24:19.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.794+0000 7f3830ff9640 10 client.25381 handle_command_reply: tid=0 2024-09-12T03:24:19.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.794+0000 7f3851baf740 1 client.25381 shutdown 2024-09-12T03:24:19.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.796+0000 7f3851baf740 1 -- 172.21.15.6:0/3363720537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5dd39f7b0 msgr2=0x55a5dd3a9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.796+0000 7f3851baf740 1 --2- 172.21.15.6:0/3363720537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5dd39f7b0 0x55a5dd3a9f80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3838018770 tx=0x7f3838009a50 comp rx=0 tx=0).stop 2024-09-12T03:24:19.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.797+0000 7f3851baf740 1 -- 172.21.15.6:0/3363720537 shutdown_connections 2024-09-12T03:24:19.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.797+0000 7f3851baf740 1 --2- 172.21.15.6:0/3363720537 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a5dd3ba4f0 0x55a5dd3bc970 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.797+0000 7f3851baf740 1 --2- 172.21.15.6:0/3363720537 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5dd3aa550 0x55a5dd3a79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.797+0000 7f3851baf740 1 --2- 172.21.15.6:0/3363720537 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd3a0d20 0x55a5dd3a7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.797+0000 7f3851baf740 1 --2- 172.21.15.6:0/3363720537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5dd39f7b0 0x55a5dd3a9f80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.797+0000 7f3851baf740 1 -- 172.21.15.6:0/3363720537 >> 172.21.15.6:0/3363720537 conn(0x55a5dd368310 msgr2=0x55a5dd3a1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:19.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.797+0000 7f3830ff9640 0 client.25381 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:19.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.797+0000 7f3851baf740 1 -- 172.21.15.6:0/3363720537 shutdown_connections 2024-09-12T03:24:19.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.797+0000 7f3851baf740 1 -- 172.21.15.6:0/3363720537 wait complete. 2024-09-12T03:24:19.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.798+0000 7f3851baf740 20 client.25381 trim_cache size 0 max 16384 2024-09-12T03:24:19.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.825+0000 7f3851baf740 1 Processor -- start 2024-09-12T03:24:19.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.825+0000 7f3851baf740 1 -- start start 2024-09-12T03:24:19.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.825+0000 7f3851baf740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd39f7b0 0x55a5dd3a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.826+0000 7f3851baf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5dd3a0d20 0x55a5dd3a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.826+0000 7f384cd71640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd39f7b0 0x55a5dd3a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.826+0000 7f3851baf740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5dd3aa550 0x55a5dd3a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.826+0000 7f3851baf740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f382c010ab0 con 0x55a5dd3aa550 2024-09-12T03:24:19.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.826+0000 7f3851baf740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f382c00e380 con 0x55a5dd39f7b0 2024-09-12T03:24:19.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.826+0000 7f3851baf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f382c00c280 con 0x55a5dd3a0d20 2024-09-12T03:24:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.826+0000 7f384cd71640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd39f7b0 0x55a5dd3a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:39642/0 (socket says 172.21.15.6:39642) 2024-09-12T03:24:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.826+0000 7f384cd71640 1 -- 172.21.15.6:0/2338808569 learned_addr learned my addr 172.21.15.6:0/2338808569 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.827+0000 7f384d572640 1 --2- 172.21.15.6:0/2338808569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5dd3a0d20 0x55a5dd3a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.827+0000 7f384cd71640 1 --2- 172.21.15.6:0/2338808569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5dd3aa550 0x55a5dd3a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.827+0000 7f384cd71640 1 -- 172.21.15.6:0/2338808569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5dd3a0d20 msgr2=0x55a5dd3a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.827+0000 7f384cd71640 1 --2- 172.21.15.6:0/2338808569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5dd3a0d20 0x55a5dd3a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.827+0000 7f384cd71640 1 -- 172.21.15.6:0/2338808569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5dd3aa550 msgr2=0x55a5dd3a27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.827+0000 7f384cd71640 1 --2- 172.21.15.6:0/2338808569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5dd3aa550 0x55a5dd3a27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.827+0000 7f384cd71640 1 -- 172.21.15.6:0/2338808569 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5dd39f4b0 con 0x55a5dd39f7b0 2024-09-12T03:24:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.828+0000 7f384cd71640 1 --2- 172.21.15.6:0/2338808569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd39f7b0 0x55a5dd3a7200 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3838017700 tx=0x7f3838009670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b299d04c561605f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.828+0000 7f3832ffd640 1 -- 172.21.15.6:0/2338808569 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838018450 con 0x55a5dd39f7b0 2024-09-12T03:24:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.828+0000 7f3832ffd640 1 -- 172.21.15.6:0/2338808569 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3838029070 con 0x55a5dd39f7b0 2024-09-12T03:24:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.828+0000 7f3832ffd640 1 -- 172.21.15.6:0/2338808569 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38380187a0 con 0x55a5dd39f7b0 2024-09-12T03:24:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.829+0000 7f3851baf740 1 -- 172.21.15.6:0/2338808569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd39f7b0 msgr2=0x55a5dd3a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.829+0000 7f3851baf740 1 --2- 172.21.15.6:0/2338808569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd39f7b0 0x55a5dd3a7200 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3838017700 tx=0x7f3838009670 comp rx=0 tx=0).stop 2024-09-12T03:24:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.829+0000 7f3851baf740 1 -- 172.21.15.6:0/2338808569 shutdown_connections 2024-09-12T03:24:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.829+0000 7f3851baf740 1 --2- 172.21.15.6:0/2338808569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5dd3aa550 0x55a5dd3a27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.829+0000 7f3851baf740 1 --2- 172.21.15.6:0/2338808569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5dd3a0d20 0x55a5dd3a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.829+0000 7f3851baf740 1 --2- 172.21.15.6:0/2338808569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd39f7b0 0x55a5dd3a7200 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.829+0000 7f3851baf740 1 -- 172.21.15.6:0/2338808569 >> 172.21.15.6:0/2338808569 conn(0x55a5dd368310 msgr2=0x55a5dd3a6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.829+0000 7f3851baf740 1 -- 172.21.15.6:0/2338808569 shutdown_connections 2024-09-12T03:24:19.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.829+0000 7f3851baf740 1 -- 172.21.15.6:0/2338808569 wait complete. 2024-09-12T03:24:19.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.830+0000 7f3851baf740 1 -- start start 2024-09-12T03:24:19.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.830+0000 7f3851baf740 1 Processor -- start 2024-09-12T03:24:19.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.831+0000 7f3851baf740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd39f7b0 0x55a5dd3a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.831+0000 7f3851baf740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5dd3a0d20 0x55a5dd3b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.831+0000 7f3851baf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5dd3aa550 0x55a5dd3b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.831+0000 7f3851baf740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f382c00c840 con 0x55a5dd3a0d20 2024-09-12T03:24:19.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.831+0000 7f384cd71640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd39f7b0 0x55a5dd3a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.831+0000 7f384cd71640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd39f7b0 0x55a5dd3a4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:39654/0 (socket says 172.21.15.6:39654) 2024-09-12T03:24:19.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.832+0000 7f384cd71640 1 -- 172.21.15.6:0/2286927568 learned_addr learned my addr 172.21.15.6:0/2286927568 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:19.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.832+0000 7f3851baf740 1 -- 172.21.15.6:0/2286927568 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f382c00dcb0 con 0x55a5dd39f7b0 2024-09-12T03:24:19.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.832+0000 7f3851baf740 1 -- 172.21.15.6:0/2286927568 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f382c00cf10 con 0x55a5dd3aa550 2024-09-12T03:24:19.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.832+0000 7f3851baf740 10 client.0 _ll_register_callbacks cb 0x7f3838009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:19.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.832+0000 7f384cd71640 1 --2- 172.21.15.6:0/2286927568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5dd3aa550 0x55a5dd3b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.832+0000 7f384d572640 1 --2- 172.21.15.6:0/2286927568 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5dd3a0d20 0x55a5dd3b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.832+0000 7f384cd71640 1 -- 172.21.15.6:0/2286927568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5dd3aa550 msgr2=0x55a5dd3b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:19.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.832+0000 7f384cd71640 1 --2- 172.21.15.6:0/2286927568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5dd3aa550 0x55a5dd3b2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.832+0000 7f384cd71640 1 -- 172.21.15.6:0/2286927568 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5dd3a0d20 msgr2=0x55a5dd3b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:19.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.832+0000 7f384cd71640 1 --2- 172.21.15.6:0/2286927568 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5dd3a0d20 0x55a5dd3b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.832+0000 7f384cd71640 1 -- 172.21.15.6:0/2286927568 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5dd3a34c0 con 0x55a5dd39f7b0 2024-09-12T03:24:19.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.833+0000 7f384cd71640 1 --2- 172.21.15.6:0/2286927568 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd39f7b0 0x55a5dd3a4de0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f38380157c0 tx=0x7f3838006a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2770bf4818b5d05 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:19.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.833+0000 7f3830ff9640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:24:19.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.833+0000 7f3830ff9640 1 -- 172.21.15.6:0/2286927568 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383804b060 con 0x55a5dd39f7b0 2024-09-12T03:24:19.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.833+0000 7f3851baf740 1 -- 172.21.15.6:0/2286927568 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3838029070 con 0x55a5dd39f7b0 2024-09-12T03:24:19.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.834+0000 7f3830ff9640 1 -- 172.21.15.6:0/2286927568 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f3838019da0 con 0x55a5dd39f7b0 2024-09-12T03:24:19.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.834+0000 7f3830ff9640 1 -- 172.21.15.6:0/2286927568 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3838018450 con 0x55a5dd39f7b0 2024-09-12T03:24:19.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.834+0000 7f3851baf740 10 client.25346 fetch_fsmap learned FSMap version 26 2024-09-12T03:24:19.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.834+0000 7f3851baf740 1 -- 172.21.15.6:0/2286927568 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a5dd424360 con 0x55a5dd39f7b0 2024-09-12T03:24:19.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.834+0000 7f3830ff9640 1 -- 172.21.15.6:0/2286927568 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3838029070 con 0x55a5dd39f7b0 2024-09-12T03:24:19.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.835+0000 7f3830ff9640 1 -- 172.21.15.6:0/2286927568 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3838042050 con 0x55a5dd39f7b0 2024-09-12T03:24:19.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.835+0000 7f3851baf740 10 client.25346 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:24:19.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.835+0000 7f3851baf740 10 client.25346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:24:19.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.835+0000 7f3851baf740 20 client.25346 populate_metadata read hostname 'smithi006' 2024-09-12T03:24:19.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.835+0000 7f3851baf740 1 --2- 172.21.15.6:0/2286927568 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a5dd4249b0 0x55a5dd426da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.835+0000 7f3851baf740 4 client.25346 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:24:19.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.835+0000 7f3851baf740 1 -- 172.21.15.6:0/2286927568 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a5dd312860 con 0x55a5dd4249b0 2024-09-12T03:24:19.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.837+0000 7f384d572640 1 --2- 172.21.15.6:0/2286927568 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a5dd4249b0 0x55a5dd426da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.860+0000 7f384d572640 1 --2- 172.21.15.6:0/2286927568 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a5dd4249b0 0x55a5dd426da0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5ae5e249e97a1c8 server_cookie=e82ca1d4f3ba86dc in_seq=0 out_seq=0 2024-09-12T03:24:19.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.860+0000 7f3830ff9640 10 client.25346 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:19.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:19 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:19.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.876+0000 7f3830ff9640 1 -- 172.21.15.6:0/2286927568 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a5dd312860 con 0x55a5dd4249b0 2024-09-12T03:24:19.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.876+0000 7f3830ff9640 10 client.25346 handle_command_reply: tid=0 2024-09-12T03:24:19.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.876+0000 7f3851baf740 1 client.25346 shutdown 2024-09-12T03:24:19.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.880+0000 7f3851baf740 1 -- 172.21.15.6:0/2286927568 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd39f7b0 msgr2=0x55a5dd3a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.880+0000 7f3851baf740 1 --2- 172.21.15.6:0/2286927568 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd39f7b0 0x55a5dd3a4de0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f38380157c0 tx=0x7f3838006a80 comp rx=0 tx=0).stop 2024-09-12T03:24:19.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.882+0000 7f3851baf740 1 -- 172.21.15.6:0/2286927568 shutdown_connections 2024-09-12T03:24:19.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.882+0000 7f3851baf740 1 --2- 172.21.15.6:0/2286927568 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a5dd4249b0 0x55a5dd426da0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.882+0000 7f3830ff9640 0 client.25346 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:19.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.882+0000 7f3851baf740 1 --2- 172.21.15.6:0/2286927568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5dd3aa550 0x55a5dd3b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.882+0000 7f3851baf740 1 --2- 172.21.15.6:0/2286927568 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5dd3a0d20 0x55a5dd3b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.882+0000 7f3851baf740 1 --2- 172.21.15.6:0/2286927568 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a5dd39f7b0 0x55a5dd3a4de0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.882+0000 7f3851baf740 1 -- 172.21.15.6:0/2286927568 >> 172.21.15.6:0/2286927568 conn(0x55a5dd368310 msgr2=0x55a5dd39a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:19.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.882+0000 7f3851baf740 1 -- 172.21.15.6:0/2286927568 shutdown_connections 2024-09-12T03:24:19.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.882+0000 7f3851baf740 1 -- 172.21.15.6:0/2286927568 wait complete. 2024-09-12T03:24:19.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.883+0000 7f3851baf740 20 client.25346 trim_cache size 0 max 16384 2024-09-12T03:24:19.884 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-09-12T03:24:19.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.883+0000 7f384e574640 1 -- 172.21.15.6:0/1129875448 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38340a2dd0 msgr2=0x7f38340a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.883+0000 7f384e574640 1 --2- 172.21.15.6:0/1129875448 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38340a2dd0 0x7f38340a5290 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f38400c9610 tx=0x7f384001a5d0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.883+0000 7f384e574640 1 -- 172.21.15.6:0/1129875448 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f384811b140 msgr2=0x7f3848116f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.883+0000 7f384e574640 1 --2- 172.21.15.6:0/1129875448 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f384811b140 0x7f3848116f00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f384001ac00 tx=0x7f3840040aa0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.884+0000 7f384e574640 1 -- 172.21.15.6:0/1129875448 shutdown_connections 2024-09-12T03:24:19.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.884+0000 7f384e574640 1 --2- 172.21.15.6:0/1129875448 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f38340a2dd0 0x7f38340a5290 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.884+0000 7f384e574640 1 --2- 172.21.15.6:0/1129875448 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f384811b140 0x7f3848116f00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.884+0000 7f384e574640 1 --2- 172.21.15.6:0/1129875448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3848110190 0x7f38481169c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.884+0000 7f384e574640 1 --2- 172.21.15.6:0/1129875448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f384806ea30 0x7f384811a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.884+0000 7f384e574640 1 -- 172.21.15.6:0/1129875448 >> 172.21.15.6:0/1129875448 conn(0x7f3848073180 msgr2=0x7f384811b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:19.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.884+0000 7f384e574640 1 -- 172.21.15.6:0/1129875448 shutdown_connections 2024-09-12T03:24:19.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:19.884+0000 7f384e574640 1 -- 172.21.15.6:0/1129875448 wait complete. 2024-09-12T03:24:19.964 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:19 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:20.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:20 smithi006 ceph-mon[30991]: pgmap v432: 97 pgs: 97 active+clean; 13 GiB data, 71 GiB used, 1002 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 302 MiB/s wr, 25.83k op/s 2024-09-12T03:24:20.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:20 smithi161 ceph-mon[35485]: pgmap v432: 97 pgs: 97 active+clean; 13 GiB data, 71 GiB used, 1002 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 302 MiB/s wr, 25.83k op/s 2024-09-12T03:24:20.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:20 smithi118 ceph-mon[35048]: pgmap v432: 97 pgs: 97 active+clean; 13 GiB data, 71 GiB used, 1002 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 302 MiB/s wr, 25.83k op/s 2024-09-12T03:24:21.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:21 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:21.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:21 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:21.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:21 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:24:21.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:21 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:21.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:21 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:21.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:21 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:24:21.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:21 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:21.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:21 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:21.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:21 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:24:22.677 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:24:22.677 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:24:23.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:22 smithi006 ceph-mon[30991]: pgmap v433: 97 pgs: 97 active+clean; 13 GiB data, 71 GiB used, 1001 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 246 MiB/s wr, 20.39k op/s 2024-09-12T03:24:23.185 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:23.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:22 smithi118 ceph-mon[35048]: pgmap v433: 97 pgs: 97 active+clean; 13 GiB data, 71 GiB used, 1001 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 246 MiB/s wr, 20.39k op/s 2024-09-12T03:24:23.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:22 smithi161 ceph-mon[35485]: pgmap v433: 97 pgs: 97 active+clean; 13 GiB data, 71 GiB used, 1001 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 246 MiB/s wr, 20.39k op/s 2024-09-12T03:24:24.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.343+0000 7f299ac2a640 1 -- 172.21.15.6:0/2047673380 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2994074590 msgr2=0x7f2994074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:24.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.343+0000 7f299ac2a640 1 --2- 172.21.15.6:0/2047673380 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2994074590 0x7f2994074970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2984009900 tx=0x7f298402a4d0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.344+0000 7f299ac2a640 1 -- 172.21.15.6:0/2047673380 shutdown_connections 2024-09-12T03:24:24.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.344+0000 7f299ac2a640 1 --2- 172.21.15.6:0/2047673380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2994112b90 0x7f2994114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.344+0000 7f299ac2a640 1 --2- 172.21.15.6:0/2047673380 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2994074eb0 0x7f2994112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.344+0000 7f299ac2a640 1 --2- 172.21.15.6:0/2047673380 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2994074590 0x7f2994074970 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.344+0000 7f299ac2a640 1 -- 172.21.15.6:0/2047673380 >> 172.21.15.6:0/2047673380 conn(0x7f29940fd8e0 msgr2=0x7f29940ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:24.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.344+0000 7f299ac2a640 1 -- 172.21.15.6:0/2047673380 shutdown_connections 2024-09-12T03:24:24.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.345+0000 7f299ac2a640 1 -- 172.21.15.6:0/2047673380 wait complete. 2024-09-12T03:24:24.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.345+0000 7f299ac2a640 1 Processor -- start 2024-09-12T03:24:24.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.346+0000 7f299ac2a640 1 -- start start 2024-09-12T03:24:24.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.346+0000 7f299ac2a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2994074590 0x7f2994071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:24.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.346+0000 7f299ac2a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2994074eb0 0x7f2994071560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:24.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.346+0000 7f299ac2a640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2994112b90 0x7f2994071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:24.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.346+0000 7f299ac2a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f29941185a0 con 0x7f2994074eb0 2024-09-12T03:24:24.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.346+0000 7f299ac2a640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f2994118420 con 0x7f2994112b90 2024-09-12T03:24:24.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.346+0000 7f299ac2a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2994118720 con 0x7f2994074590 2024-09-12T03:24:24.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.346+0000 7f29991c7640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2994112b90 0x7f2994071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:24.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.346+0000 7f29991c7640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2994112b90 0x7f2994071aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:39682/0 (socket says 172.21.15.6:39682) 2024-09-12T03:24:24.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.346+0000 7f29991c7640 1 -- 172.21.15.6:0/1671336577 learned_addr learned my addr 172.21.15.6:0/1671336577 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:24.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.347+0000 7f29989c6640 1 --2- 172.21.15.6:0/1671336577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2994074590 0x7f2994071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:24.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.347+0000 7f29991c7640 1 -- 172.21.15.6:0/1671336577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2994074590 msgr2=0x7f2994071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:24.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.347+0000 7f29991c7640 1 --2- 172.21.15.6:0/1671336577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2994074590 0x7f2994071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.347+0000 7f29991c7640 1 -- 172.21.15.6:0/1671336577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2994074eb0 msgr2=0x7f2994071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:24.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.347+0000 7f29991c7640 1 --2- 172.21.15.6:0/1671336577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2994074eb0 0x7f2994071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.347+0000 7f29991c7640 1 -- 172.21.15.6:0/1671336577 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299410ee00 con 0x7f2994112b90 2024-09-12T03:24:24.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.348+0000 7f29991c7640 1 --2- 172.21.15.6:0/1671336577 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2994112b90 0x7f2994071aa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f298c00eb70 tx=0x7f298c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:24.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.350+0000 7f2991ffb640 1 -- 172.21.15.6:0/1671336577 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c004d20 con 0x7f2994112b90 2024-09-12T03:24:24.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.350+0000 7f2991ffb640 1 -- 172.21.15.6:0/1671336577 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f298c00ce70 con 0x7f2994112b90 2024-09-12T03:24:24.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.350+0000 7f299ac2a640 1 -- 172.21.15.6:0/1671336577 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299410f0f0 con 0x7f2994112b90 2024-09-12T03:24:24.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.350+0000 7f299ac2a640 1 -- 172.21.15.6:0/1671336577 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299410f630 con 0x7f2994112b90 2024-09-12T03:24:24.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.350+0000 7f2991ffb640 1 -- 172.21.15.6:0/1671336577 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c010810 con 0x7f2994112b90 2024-09-12T03:24:24.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.355+0000 7f299d003740 1 Processor -- start 2024-09-12T03:24:24.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.357+0000 7f299d003740 1 -- start start 2024-09-12T03:24:24.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.357+0000 7f299d003740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7c0b60 0x55c9aa7c0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:24.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.357+0000 7f299d003740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aa7c1480 0x55c9aa7cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:24.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.357+0000 7f299d003740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7cbf20 0x55c9aa7ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:24.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.357+0000 7f299d003740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55c9aa726280 con 0x55c9aa7c1480 2024-09-12T03:24:24.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.357+0000 7f299d003740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55c9aa6cdb10 con 0x55c9aa7c0b60 2024-09-12T03:24:24.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.357+0000 7f299d003740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c9aa722240 con 0x55c9aa7cbf20 2024-09-12T03:24:24.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.358+0000 7f2991ffb640 1 -- 172.21.15.6:0/1671336577 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f298c004fc0 con 0x7f2994112b90 2024-09-12T03:24:24.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.358+0000 7f2993fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7c0b60 0x55c9aa7c0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:24.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.358+0000 7f2993fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7c0b60 0x55c9aa7c0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:39696/0 (socket says 172.21.15.6:39696) 2024-09-12T03:24:24.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.358+0000 7f2993fff640 1 -- 172.21.15.6:0/1604022939 learned_addr learned my addr 172.21.15.6:0/1604022939 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:24.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.358+0000 7f2993fff640 1 --2- 172.21.15.6:0/1604022939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7cbf20 0x55c9aa7ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:24.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.358+0000 7f2993fff640 1 -- 172.21.15.6:0/1604022939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7cbf20 msgr2=0x55c9aa7ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:24.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.358+0000 7f2993fff640 1 --2- 172.21.15.6:0/1604022939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7cbf20 0x55c9aa7ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.359+0000 7f2993fff640 1 -- 172.21.15.6:0/1604022939 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aa7c1480 msgr2=0x55c9aa7cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:24.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.359+0000 7f2993fff640 1 --2- 172.21.15.6:0/1604022939 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aa7c1480 0x55c9aa7cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.359+0000 7f2993fff640 1 -- 172.21.15.6:0/1604022939 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9aa6da1f0 con 0x55c9aa7c0b60 2024-09-12T03:24:24.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.359+0000 7f2993fff640 1 --2- 172.21.15.6:0/1604022939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7c0b60 0x55c9aa7c0f40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2988005fd0 tx=0x7f298800d3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=849288b64e52fa23 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:24.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.360+0000 7f2991ffb640 1 --2- 172.21.15.6:0/1671336577 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f297c0a2d90 0x7f297c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:24.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.361+0000 7f29989c6640 1 --2- 172.21.15.6:0/1671336577 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f297c0a2d90 0x7f297c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:24.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.361+0000 7f2991ffb640 1 -- 172.21.15.6:0/1671336577 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f298c0cf750 con 0x7f2994112b90 2024-09-12T03:24:24.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.363+0000 7f299ac2a640 1 -- 172.21.15.6:0/1604022939 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298800a690 con 0x55c9aa7c0b60 2024-09-12T03:24:24.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.363+0000 7f299ac2a640 1 -- 172.21.15.6:0/1604022939 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f298800ad10 con 0x55c9aa7c0b60 2024-09-12T03:24:24.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.364+0000 7f299d003740 1 -- 172.21.15.6:0/1604022939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7c0b60 msgr2=0x55c9aa7c0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:24.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.364+0000 7f299d003740 1 --2- 172.21.15.6:0/1604022939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7c0b60 0x55c9aa7c0f40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2988005fd0 tx=0x7f298800d3c0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.364+0000 7f29989c6640 1 --2- 172.21.15.6:0/1671336577 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f297c0a2d90 0x7f297c0a5250 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f298402aac0 tx=0x7f298402a9d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:24.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.364+0000 7f299d003740 1 -- 172.21.15.6:0/1604022939 shutdown_connections 2024-09-12T03:24:24.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.364+0000 7f299d003740 1 --2- 172.21.15.6:0/1604022939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7cbf20 0x55c9aa7ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.364+0000 7f299d003740 1 --2- 172.21.15.6:0/1604022939 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aa7c1480 0x55c9aa7cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.364+0000 7f299d003740 1 --2- 172.21.15.6:0/1604022939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7c0b60 0x55c9aa7c0f40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.364+0000 7f299d003740 1 -- 172.21.15.6:0/1604022939 >> 172.21.15.6:0/1604022939 conn(0x55c9aa7bd2d0 msgr2=0x55c9aa7bd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:24.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.364+0000 7f299d003740 1 -- 172.21.15.6:0/1604022939 shutdown_connections 2024-09-12T03:24:24.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.364+0000 7f299d003740 1 -- 172.21.15.6:0/1604022939 wait complete. 2024-09-12T03:24:24.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.365+0000 7f299d003740 1 -- start start 2024-09-12T03:24:24.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.365+0000 7f299d003740 1 Processor -- start 2024-09-12T03:24:24.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.366+0000 7f299d003740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7c0b60 0x55c9aa7cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:24.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.366+0000 7f299d003740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aa7c1480 0x55c9aa7c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:24.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.366+0000 7f299d003740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7cbf20 0x55c9aa7c9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:24.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.366+0000 7f299d003740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55c9aa7ce970 con 0x55c9aa7c1480 2024-09-12T03:24:24.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.366+0000 7f299d003740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55c9aa6e0f10 con 0x55c9aa7c0b60 2024-09-12T03:24:24.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.366+0000 7f299d003740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c9aa7ceaf0 con 0x55c9aa7cbf20 2024-09-12T03:24:24.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.366+0000 7f299d003740 10 client.0 _ll_register_callbacks cb 0x55c9aa6a9a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:24.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.366+0000 7f29991c7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7cbf20 0x55c9aa7c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:24.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.366+0000 7f29991c7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7cbf20 0x55c9aa7c9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:49918/0 (socket says 172.21.15.6:49918) 2024-09-12T03:24:24.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.366+0000 7f29991c7640 1 -- 172.21.15.6:0/2785800189 learned_addr learned my addr 172.21.15.6:0/2785800189 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:24.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.367+0000 7f29991c7640 1 -- 172.21.15.6:0/2785800189 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7c0b60 msgr2=0x55c9aa7cb9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:24:24.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.367+0000 7f29991c7640 1 --2- 172.21.15.6:0/2785800189 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7c0b60 0x55c9aa7cb9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.367+0000 7f29991c7640 1 -- 172.21.15.6:0/2785800189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aa7c1480 msgr2=0x55c9aa7c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:24.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.367+0000 7f29991c7640 1 --2- 172.21.15.6:0/2785800189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aa7c1480 0x55c9aa7c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.367+0000 7f29991c7640 1 -- 172.21.15.6:0/2785800189 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9aa4976a0 con 0x55c9aa7cbf20 2024-09-12T03:24:24.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.369+0000 7f29991c7640 1 --2- 172.21.15.6:0/2785800189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7cbf20 0x55c9aa7c9430 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f298c0983e0 tx=0x7f298c0bc110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76db83e32ff34c6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:24.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.369+0000 7f2973fff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:24:24.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.369+0000 7f2973fff640 1 -- 172.21.15.6:0/2785800189 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c0b4070 con 0x55c9aa7cbf20 2024-09-12T03:24:24.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.370+0000 7f299d003740 1 -- 172.21.15.6:0/2785800189 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f298800ad10 con 0x55c9aa7cbf20 2024-09-12T03:24:24.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.370+0000 7f2973fff640 1 -- 172.21.15.6:0/2785800189 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f298c00b2c0 con 0x55c9aa7cbf20 2024-09-12T03:24:24.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.370+0000 7f2973fff640 1 -- 172.21.15.6:0/2785800189 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f298c0ab050 con 0x55c9aa7cbf20 2024-09-12T03:24:24.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.372+0000 7f299d003740 10 client.25399 fetch_fsmap learned FSMap version 26 2024-09-12T03:24:24.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.372+0000 7f299d003740 1 -- 172.21.15.6:0/2785800189 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c9aa7da5d0 con 0x55c9aa7cbf20 2024-09-12T03:24:24.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.372+0000 7f2973fff640 1 -- 172.21.15.6:0/2785800189 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f298800ad10 con 0x55c9aa7cbf20 2024-09-12T03:24:24.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.374+0000 7f2973fff640 1 -- 172.21.15.6:0/2785800189 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f298c0babc0 con 0x55c9aa7cbf20 2024-09-12T03:24:24.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.374+0000 7f299d003740 10 client.25399 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:24:24.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.374+0000 7f299d003740 10 client.25399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:24:24.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.374+0000 7f299d003740 20 client.25399 populate_metadata read hostname 'smithi006' 2024-09-12T03:24:24.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.374+0000 7f299d003740 1 --2- 172.21.15.6:0/2785800189 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55c9aa7dbf20 0x55c9aa7de3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:24.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.374+0000 7f299d003740 4 client.25399 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:24:24.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.374+0000 7f299d003740 1 -- 172.21.15.6:0/2785800189 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9aa7337f0 con 0x55c9aa7dbf20 2024-09-12T03:24:24.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.375+0000 7f29989c6640 1 --2- 172.21.15.6:0/2785800189 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55c9aa7dbf20 0x55c9aa7de3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:24.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.375+0000 7f29989c6640 1 --2- 172.21.15.6:0/2785800189 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55c9aa7dbf20 0x55c9aa7de3e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bac18a8505d6e633 server_cookie=1d3422ef9ef348fc in_seq=0 out_seq=0 2024-09-12T03:24:24.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.375+0000 7f2973fff640 10 client.25399 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:24.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.376+0000 7f2973fff640 1 -- 172.21.15.6:0/2785800189 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9aa7337f0 con 0x55c9aa7dbf20 2024-09-12T03:24:24.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.376+0000 7f2973fff640 10 client.25399 handle_command_reply: tid=0 2024-09-12T03:24:24.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.376+0000 7f299d003740 1 client.25399 shutdown 2024-09-12T03:24:24.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.377+0000 7f299d003740 1 -- 172.21.15.6:0/2785800189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7cbf20 msgr2=0x55c9aa7c9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:24.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.377+0000 7f299d003740 1 --2- 172.21.15.6:0/2785800189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7cbf20 0x55c9aa7c9430 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f298c0983e0 tx=0x7f298c0bc110 comp rx=0 tx=0).stop 2024-09-12T03:24:24.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.377+0000 7f299d003740 1 -- 172.21.15.6:0/2785800189 shutdown_connections 2024-09-12T03:24:24.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.377+0000 7f299d003740 1 --2- 172.21.15.6:0/2785800189 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55c9aa7dbf20 0x55c9aa7de3e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.377+0000 7f299d003740 1 --2- 172.21.15.6:0/2785800189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7cbf20 0x55c9aa7c9430 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.377+0000 7f299d003740 1 --2- 172.21.15.6:0/2785800189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aa7c1480 0x55c9aa7c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.377+0000 7f299d003740 1 --2- 172.21.15.6:0/2785800189 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7c0b60 0x55c9aa7cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.377+0000 7f299d003740 1 -- 172.21.15.6:0/2785800189 >> 172.21.15.6:0/2785800189 conn(0x55c9aa7bd2d0 msgr2=0x55c9aa7c2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:24.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.377+0000 7f2973fff640 0 client.25399 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:24.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.378+0000 7f299d003740 1 -- 172.21.15.6:0/2785800189 shutdown_connections 2024-09-12T03:24:24.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.378+0000 7f299d003740 1 -- 172.21.15.6:0/2785800189 wait complete. 2024-09-12T03:24:24.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.380+0000 7f299d003740 20 client.25399 trim_cache size 0 max 16384 2024-09-12T03:24:24.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.418+0000 7f299d003740 1 Processor -- start 2024-09-12T03:24:24.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.418+0000 7f299d003740 1 -- start start 2024-09-12T03:24:24.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.418+0000 7f299d003740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7bd2d0 0x55c9aa7d7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:24.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.418+0000 7f299d003740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aa7c0b60 0x55c9aa7c2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:24.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.419+0000 7f299d003740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7c1480 0x55c9aa7e3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:24.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.419+0000 7f29989c6640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7bd2d0 0x55c9aa7d7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:24.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.419+0000 7f299d003740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2980010b00 con 0x55c9aa7c0b60 2024-09-12T03:24:24.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.419+0000 7f299d003740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f298000e3d0 con 0x55c9aa7bd2d0 2024-09-12T03:24:24.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.419+0000 7f299d003740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f298000c2d0 con 0x55c9aa7c1480 2024-09-12T03:24:24.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.419+0000 7f29989c6640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7bd2d0 0x55c9aa7d7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:39714/0 (socket says 172.21.15.6:39714) 2024-09-12T03:24:24.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.419+0000 7f29989c6640 1 -- 172.21.15.6:0/2771637539 learned_addr learned my addr 172.21.15.6:0/2771637539 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:24.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.419+0000 7f29991c7640 1 --2- 172.21.15.6:0/2771637539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7c1480 0x55c9aa7e3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:24.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.419+0000 7f29991c7640 1 -- 172.21.15.6:0/2771637539 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7bd2d0 msgr2=0x55c9aa7d7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:24.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.419+0000 7f29991c7640 1 --2- 172.21.15.6:0/2771637539 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7bd2d0 0x55c9aa7d7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.419+0000 7f29991c7640 1 -- 172.21.15.6:0/2771637539 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aa7c0b60 msgr2=0x55c9aa7c2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:24.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.419+0000 7f29991c7640 1 --2- 172.21.15.6:0/2771637539 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aa7c0b60 0x55c9aa7c2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.419+0000 7f29991c7640 1 -- 172.21.15.6:0/2771637539 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9aa7d81d0 con 0x55c9aa7c1480 2024-09-12T03:24:24.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.420+0000 7f29991c7640 1 --2- 172.21.15.6:0/2771637539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7c1480 0x55c9aa7e3cf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f298c00eb40 tx=0x7f298c09be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a33b49f705ff7b06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:24.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.420+0000 7f297a7fc640 1 -- 172.21.15.6:0/2771637539 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c0b4070 con 0x55c9aa7c1480 2024-09-12T03:24:24.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.420+0000 7f297a7fc640 1 -- 172.21.15.6:0/2771637539 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f298c0a6e50 con 0x55c9aa7c1480 2024-09-12T03:24:24.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.421+0000 7f299d003740 1 -- 172.21.15.6:0/2771637539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7c1480 msgr2=0x55c9aa7e3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:24.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.421+0000 7f299d003740 1 --2- 172.21.15.6:0/2771637539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7c1480 0x55c9aa7e3cf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f298c00eb40 tx=0x7f298c09be00 comp rx=0 tx=0).stop 2024-09-12T03:24:24.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.421+0000 7f299d003740 1 -- 172.21.15.6:0/2771637539 shutdown_connections 2024-09-12T03:24:24.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.421+0000 7f299d003740 1 --2- 172.21.15.6:0/2771637539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7c1480 0x55c9aa7e3cf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.421+0000 7f299d003740 1 --2- 172.21.15.6:0/2771637539 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aa7c0b60 0x55c9aa7c2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.421+0000 7f299d003740 1 --2- 172.21.15.6:0/2771637539 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7bd2d0 0x55c9aa7d7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.421+0000 7f299d003740 1 -- 172.21.15.6:0/2771637539 >> 172.21.15.6:0/2771637539 conn(0x55c9aa7c46e0 msgr2=0x55c9aa7c2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:24.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.423+0000 7f299d003740 1 -- 172.21.15.6:0/2771637539 shutdown_connections 2024-09-12T03:24:24.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.423+0000 7f299d003740 1 -- 172.21.15.6:0/2771637539 wait complete. 2024-09-12T03:24:24.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.424+0000 7f299d003740 1 -- start start 2024-09-12T03:24:24.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.424+0000 7f299d003740 1 Processor -- start 2024-09-12T03:24:24.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.425+0000 7f299d003740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7cbf20 0x55c9aa7c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:24.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.425+0000 7f299d003740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aa7d4760 0x55c9aa7c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:24.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.425+0000 7f299d003740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7c6770 0x55c9aa7d2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:24.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.425+0000 7f299d003740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f298000c890 con 0x55c9aa7d4760 2024-09-12T03:24:24.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.425+0000 7f299d003740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f298000dd00 con 0x55c9aa7cbf20 2024-09-12T03:24:24.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.425+0000 7f299d003740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f298000cf60 con 0x55c9aa7c6770 2024-09-12T03:24:24.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.425+0000 7f299d003740 10 client.0 _ll_register_callbacks cb 0x7f298c004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:24.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.426+0000 7f29989c6640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7cbf20 0x55c9aa7c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:24.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.426+0000 7f29989c6640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7cbf20 0x55c9aa7c5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:39724/0 (socket says 172.21.15.6:39724) 2024-09-12T03:24:24.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.426+0000 7f29989c6640 1 -- 172.21.15.6:0/327992332 learned_addr learned my addr 172.21.15.6:0/327992332 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:24.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.426+0000 7f29991c7640 1 --2- 172.21.15.6:0/327992332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7c6770 0x55c9aa7d2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:24.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.427+0000 7f2993fff640 1 --2- 172.21.15.6:0/327992332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aa7d4760 0x55c9aa7c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:24.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.427+0000 7f29991c7640 1 -- 172.21.15.6:0/327992332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7cbf20 msgr2=0x55c9aa7c5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:24.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.427+0000 7f29991c7640 1 --2- 172.21.15.6:0/327992332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7cbf20 0x55c9aa7c5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.427+0000 7f29991c7640 1 -- 172.21.15.6:0/327992332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aa7d4760 msgr2=0x55c9aa7c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:24.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.427+0000 7f29991c7640 1 --2- 172.21.15.6:0/327992332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aa7d4760 0x55c9aa7c60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.427+0000 7f29991c7640 1 -- 172.21.15.6:0/327992332 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9aa7d2a60 con 0x55c9aa7c6770 2024-09-12T03:24:24.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.428+0000 7f29991c7640 1 --2- 172.21.15.6:0/327992332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7c6770 0x55c9aa7d2280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f298c09a230 tx=0x7f298c09b600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=705b74b6c72b1d65 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:24.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.429+0000 7f2973fff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:24:24.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.429+0000 7f2973fff640 1 -- 172.21.15.6:0/327992332 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c0b4070 con 0x55c9aa7c6770 2024-09-12T03:24:24.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.429+0000 7f299d003740 1 -- 172.21.15.6:0/327992332 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c9aa72dd00 con 0x55c9aa7c6770 2024-09-12T03:24:24.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.429+0000 7f2973fff640 1 -- 172.21.15.6:0/327992332 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f298c097cd0 con 0x55c9aa7c6770 2024-09-12T03:24:24.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.429+0000 7f2973fff640 1 -- 172.21.15.6:0/327992332 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f298c098220 con 0x55c9aa7c6770 2024-09-12T03:24:24.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.430+0000 7f299d003740 10 client.25405 fetch_fsmap learned FSMap version 26 2024-09-12T03:24:24.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.430+0000 7f299d003740 1 -- 172.21.15.6:0/327992332 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c9aa847770 con 0x55c9aa7c6770 2024-09-12T03:24:24.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.431+0000 7f2973fff640 1 -- 172.21.15.6:0/327992332 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c9aa72dd00 con 0x55c9aa7c6770 2024-09-12T03:24:24.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.431+0000 7f2973fff640 1 -- 172.21.15.6:0/327992332 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f298c0a1550 con 0x55c9aa7c6770 2024-09-12T03:24:24.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.431+0000 7f299d003740 10 client.25405 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:24:24.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.431+0000 7f299d003740 10 client.25405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:24:24.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.431+0000 7f299d003740 20 client.25405 populate_metadata read hostname 'smithi006' 2024-09-12T03:24:24.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.431+0000 7f299d003740 1 --2- 172.21.15.6:0/327992332 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55c9aa847d70 0x55c9aa84a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:24.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.431+0000 7f299d003740 4 client.25405 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:24:24.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.431+0000 7f299d003740 1 -- 172.21.15.6:0/327992332 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c9aa6aa110 con 0x55c9aa847d70 2024-09-12T03:24:24.520 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:24:24.520 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2024-09-12T03:24:24.520 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "9cc9dee5-d9ac-4c99-b9a3-da95838d23be", 2024-09-12T03:24:24.520 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2024-09-12T03:24:24.521 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:24:24.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.432+0000 7f29989c6640 1 --2- 172.21.15.6:0/327992332 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55c9aa847d70 0x55c9aa84a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:24.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.434+0000 7f29989c6640 1 --2- 172.21.15.6:0/327992332 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55c9aa847d70 0x55c9aa84a190 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7590a84501edb3c8 server_cookie=48a7b60852a2076f in_seq=0 out_seq=0 2024-09-12T03:24:24.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.434+0000 7f2973fff640 10 client.25405 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:24.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.436+0000 7f2973fff640 1 -- 172.21.15.6:0/327992332 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c9aa6aa110 con 0x55c9aa847d70 2024-09-12T03:24:24.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.437+0000 7f2973fff640 10 client.25405 handle_command_reply: tid=0 2024-09-12T03:24:24.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.437+0000 7f299d003740 1 client.25405 shutdown 2024-09-12T03:24:24.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.437+0000 7f299d003740 1 -- 172.21.15.6:0/327992332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7c6770 msgr2=0x55c9aa7d2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:24.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.437+0000 7f299d003740 1 --2- 172.21.15.6:0/327992332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7c6770 0x55c9aa7d2280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f298c09a230 tx=0x7f298c09b600 comp rx=0 tx=0).stop 2024-09-12T03:24:24.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.437+0000 7f299d003740 1 -- 172.21.15.6:0/327992332 shutdown_connections 2024-09-12T03:24:24.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.437+0000 7f299d003740 1 --2- 172.21.15.6:0/327992332 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55c9aa847d70 0x55c9aa84a190 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.437+0000 7f299d003740 1 --2- 172.21.15.6:0/327992332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9aa7c6770 0x55c9aa7d2280 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.437+0000 7f299d003740 1 --2- 172.21.15.6:0/327992332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aa7d4760 0x55c9aa7c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.437+0000 7f299d003740 1 --2- 172.21.15.6:0/327992332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c9aa7cbf20 0x55c9aa7c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.437+0000 7f299d003740 1 -- 172.21.15.6:0/327992332 >> 172.21.15.6:0/327992332 conn(0x55c9aa7c46e0 msgr2=0x55c9aa7c2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:24.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.437+0000 7f2973fff640 0 client.25405 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:24.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.443+0000 7f299d003740 1 -- 172.21.15.6:0/327992332 shutdown_connections 2024-09-12T03:24:24.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.443+0000 7f299d003740 1 -- 172.21.15.6:0/327992332 wait complete. 2024-09-12T03:24:24.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.443+0000 7f299d003740 20 client.25405 trim_cache size 0 max 16384 2024-09-12T03:24:24.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.450+0000 7f297b7fe640 1 -- 172.21.15.6:0/1671336577 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f297c0a2d90 msgr2=0x7f297c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:24.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.450+0000 7f297b7fe640 1 --2- 172.21.15.6:0/1671336577 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f297c0a2d90 0x7f297c0a5250 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f298402aac0 tx=0x7f298402a9d0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.450+0000 7f297b7fe640 1 -- 172.21.15.6:0/1671336577 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2994112b90 msgr2=0x7f2994071aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:24.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.450+0000 7f297b7fe640 1 --2- 172.21.15.6:0/1671336577 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2994112b90 0x7f2994071aa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f298c00eb70 tx=0x7f298c00c6a0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.450+0000 7f297b7fe640 1 -- 172.21.15.6:0/1671336577 shutdown_connections 2024-09-12T03:24:24.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.450+0000 7f297b7fe640 1 --2- 172.21.15.6:0/1671336577 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f297c0a2d90 0x7f297c0a5250 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.450+0000 7f297b7fe640 1 --2- 172.21.15.6:0/1671336577 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2994112b90 0x7f2994071aa0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.450+0000 7f297b7fe640 1 --2- 172.21.15.6:0/1671336577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2994074eb0 0x7f2994071560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.450+0000 7f297b7fe640 1 --2- 172.21.15.6:0/1671336577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2994074590 0x7f2994071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:24.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.450+0000 7f297b7fe640 1 -- 172.21.15.6:0/1671336577 >> 172.21.15.6:0/1671336577 conn(0x7f29940fd8e0 msgr2=0x7f29940ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:24.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.450+0000 7f297b7fe640 1 -- 172.21.15.6:0/1671336577 shutdown_connections 2024-09-12T03:24:24.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:24.450+0000 7f297b7fe640 1 -- 172.21.15.6:0/1671336577 wait complete. 2024-09-12T03:24:25.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:24 smithi118 ceph-mon[35048]: pgmap v434: 97 pgs: 97 active+clean; 13 GiB data, 73 GiB used, 1000 GiB / 1.0 TiB avail; 738 KiB/s rd, 286 MiB/s wr, 23.84k op/s 2024-09-12T03:24:25.320 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:24 smithi006 ceph-mon[30991]: pgmap v434: 97 pgs: 97 active+clean; 13 GiB data, 73 GiB used, 1000 GiB / 1.0 TiB avail; 738 KiB/s rd, 286 MiB/s wr, 23.84k op/s 2024-09-12T03:24:25.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:24 smithi161 ceph-mon[35485]: pgmap v434: 97 pgs: 97 active+clean; 13 GiB data, 73 GiB used, 1000 GiB / 1.0 TiB avail; 738 KiB/s rd, 286 MiB/s wr, 23.84k op/s 2024-09-12T03:24:26.155 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:24:26.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:25 smithi006 ceph-mon[30991]: scrub queued for path: / 2024-09-12T03:24:26.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:25 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:24:26.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:25 smithi006 ceph-mon[30991]: scrub summary: active paths [/] 2024-09-12T03:24:26.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:25 smithi118 ceph-mon[35048]: scrub queued for path: / 2024-09-12T03:24:26.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:25 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:24:26.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:25 smithi118 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-12T03:24:26.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:25 smithi161 ceph-mon[35485]: scrub queued for path: / 2024-09-12T03:24:26.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:25 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:24:26.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:25 smithi161 ceph-mon[35485]: scrub summary: active paths [/] 2024-09-12T03:24:26.712 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:27.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:26 smithi006 ceph-mon[30991]: pgmap v435: 97 pgs: 97 active+clean; 13 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 687 KiB/s rd, 281 MiB/s wr, 23.63k op/s 2024-09-12T03:24:27.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:26 smithi118 ceph-mon[35048]: pgmap v435: 97 pgs: 97 active+clean; 13 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 687 KiB/s rd, 281 MiB/s wr, 23.63k op/s 2024-09-12T03:24:27.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:26 smithi161 ceph-mon[35485]: pgmap v435: 97 pgs: 97 active+clean; 13 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 687 KiB/s rd, 281 MiB/s wr, 23.63k op/s 2024-09-12T03:24:27.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.595+0000 7faefa272640 1 -- 172.21.15.6:0/1490125694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faef41091f0 msgr2=0x7faef41095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.595+0000 7faefa272640 1 --2- 172.21.15.6:0/1490125694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faef41091f0 0x7faef41095d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faee8009900 tx=0x7faee802a490 comp rx=0 tx=0).stop 2024-09-12T03:24:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.595+0000 7faefa272640 1 -- 172.21.15.6:0/1490125694 shutdown_connections 2024-09-12T03:24:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.596+0000 7faefa272640 1 --2- 172.21.15.6:0/1490125694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faef4114ab0 0x7faef4116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.596+0000 7faefa272640 1 --2- 172.21.15.6:0/1490125694 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faef4109b10 0x7faef4114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.596+0000 7faefa272640 1 --2- 172.21.15.6:0/1490125694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faef41091f0 0x7faef41095d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.596+0000 7faefa272640 1 -- 172.21.15.6:0/1490125694 >> 172.21.15.6:0/1490125694 conn(0x7faef406e140 msgr2=0x7faef406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:27.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.599+0000 7faefa272640 1 -- 172.21.15.6:0/1490125694 shutdown_connections 2024-09-12T03:24:27.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.604+0000 7faefa272640 1 -- 172.21.15.6:0/1490125694 wait complete. 2024-09-12T03:24:27.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.606+0000 7faefa272640 1 Processor -- start 2024-09-12T03:24:27.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.606+0000 7faefa272640 1 -- start start 2024-09-12T03:24:27.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.607+0000 7faefa272640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faef41091f0 0x7faef41a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.607+0000 7faefa272640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faef4109b10 0x7faef41a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.607+0000 7faefa272640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faef4114ab0 0x7faef41a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.607+0000 7faefa272640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7faef411a860 con 0x7faef41091f0 2024-09-12T03:24:27.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.607+0000 7faefa272640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7faef411a6e0 con 0x7faef4114ab0 2024-09-12T03:24:27.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.607+0000 7faefa272640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faef411a9e0 con 0x7faef4109b10 2024-09-12T03:24:27.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.607+0000 7faef880f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faef4114ab0 0x7faef41a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:27.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.607+0000 7faef880f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faef4114ab0 0x7faef41a5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:39734/0 (socket says 172.21.15.6:39734) 2024-09-12T03:24:27.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.607+0000 7faef880f640 1 -- 172.21.15.6:0/3371365227 learned_addr learned my addr 172.21.15.6:0/3371365227 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:27.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.608+0000 7faef37fe640 1 --2- 172.21.15.6:0/3371365227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faef4109b10 0x7faef41a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:27.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.608+0000 7faef880f640 1 -- 172.21.15.6:0/3371365227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faef4109b10 msgr2=0x7faef41a12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:27.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.608+0000 7faef880f640 1 --2- 172.21.15.6:0/3371365227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faef4109b10 0x7faef41a12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.608+0000 7faef880f640 1 -- 172.21.15.6:0/3371365227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faef41091f0 msgr2=0x7faef41a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:27.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.608+0000 7faef880f640 1 --2- 172.21.15.6:0/3371365227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faef41091f0 0x7faef41a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.608+0000 7faef880f640 1 -- 172.21.15.6:0/3371365227 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef41a5d50 con 0x7faef4114ab0 2024-09-12T03:24:27.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.609+0000 7faef880f640 1 --2- 172.21.15.6:0/3371365227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faef4114ab0 0x7faef41a5670 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faeec009fa0 tx=0x7faeec00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.609+0000 7faef17fa640 1 -- 172.21.15.6:0/3371365227 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec004b20 con 0x7faef4114ab0 2024-09-12T03:24:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.610+0000 7faef17fa640 1 -- 172.21.15.6:0/3371365227 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faeec004cc0 con 0x7faef4114ab0 2024-09-12T03:24:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.610+0000 7faefa272640 1 -- 172.21.15.6:0/3371365227 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef41a6040 con 0x7faef4114ab0 2024-09-12T03:24:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.610+0000 7faefa272640 1 -- 172.21.15.6:0/3371365227 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef410a300 con 0x7faef4114ab0 2024-09-12T03:24:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.610+0000 7faef17fa640 1 -- 172.21.15.6:0/3371365227 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec005650 con 0x7faef4114ab0 2024-09-12T03:24:27.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.615+0000 7faef17fa640 1 -- 172.21.15.6:0/3371365227 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7faeec0137d0 con 0x7faef4114ab0 2024-09-12T03:24:27.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.617+0000 7faef17fa640 1 --2- 172.21.15.6:0/3371365227 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7faedc0a2c80 0x7faedc0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.618+0000 7faef17fa640 1 -- 172.21.15.6:0/3371365227 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7faeec0d39e0 con 0x7faef4114ab0 2024-09-12T03:24:27.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.619+0000 7faefc64b740 1 Processor -- start 2024-09-12T03:24:27.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.619+0000 7faefc64b740 1 -- start start 2024-09-12T03:24:27.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.620+0000 7faefc64b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55648ab1b4a0 0x55648ab1b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.620+0000 7faefc64b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab1bdc0 0x55648ab26140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.620+0000 7faefc64b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab26680 0x55648ab28a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.620+0000 7faefc64b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55648aa812e0 con 0x55648ab1b4a0 2024-09-12T03:24:27.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.620+0000 7faef37fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab1bdc0 0x55648ab26140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:27.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.620+0000 7faef37fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab1bdc0 0x55648ab26140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:39742/0 (socket says 172.21.15.6:39742) 2024-09-12T03:24:27.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.620+0000 7faef37fe640 1 -- 172.21.15.6:0/4094311182 learned_addr learned my addr 172.21.15.6:0/4094311182 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:27.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.621+0000 7faef880f640 1 --2- 172.21.15.6:0/4094311182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab26680 0x55648ab28a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:27.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.621+0000 7faefc64b740 1 -- 172.21.15.6:0/4094311182 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55648aa28ce0 con 0x55648ab1bdc0 2024-09-12T03:24:27.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.621+0000 7faefc64b740 1 -- 172.21.15.6:0/4094311182 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55648aa7d2a0 con 0x55648ab26680 2024-09-12T03:24:27.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.622+0000 7faef3fff640 1 --2- 172.21.15.6:0/3371365227 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7faedc0a2c80 0x7faedc0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:27.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.622+0000 7faef880f640 1 -- 172.21.15.6:0/4094311182 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab1bdc0 msgr2=0x55648ab26140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:27.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.622+0000 7faef880f640 1 --2- 172.21.15.6:0/4094311182 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab1bdc0 0x55648ab26140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.622+0000 7faef880f640 1 -- 172.21.15.6:0/4094311182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55648ab1b4a0 msgr2=0x55648ab1b880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:24:27.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.623+0000 7faef880f640 1 --2- 172.21.15.6:0/4094311182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55648ab1b4a0 0x55648ab1b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.623+0000 7faef880f640 1 -- 172.21.15.6:0/4094311182 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55648aa0d440 con 0x55648ab26680 2024-09-12T03:24:27.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.623+0000 7faef880f640 1 --2- 172.21.15.6:0/4094311182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab26680 0x55648ab28a70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faeec09c670 tx=0x7faeec09fe10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf5b838f24c94775 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:27.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.623+0000 7faef3fff640 1 --2- 172.21.15.6:0/3371365227 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7faedc0a2c80 0x7faedc0a5140 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7faee80096c0 tx=0x7faee8031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:27.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.623+0000 7faeda7fc640 1 -- 172.21.15.6:0/4094311182 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec0b9070 con 0x55648ab26680 2024-09-12T03:24:27.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.624+0000 7faeda7fc640 1 -- 172.21.15.6:0/4094311182 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faeec0bee50 con 0x55648ab26680 2024-09-12T03:24:27.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.624+0000 7faeda7fc640 1 -- 172.21.15.6:0/4094311182 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec0d1690 con 0x55648ab26680 2024-09-12T03:24:27.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.625+0000 7faefc64b740 1 -- 172.21.15.6:0/4094311182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab26680 msgr2=0x55648ab28a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:27.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.625+0000 7faefc64b740 1 --2- 172.21.15.6:0/4094311182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab26680 0x55648ab28a70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faeec09c670 tx=0x7faeec09fe10 comp rx=0 tx=0).stop 2024-09-12T03:24:27.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.626+0000 7faefc64b740 1 -- 172.21.15.6:0/4094311182 shutdown_connections 2024-09-12T03:24:27.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.626+0000 7faefc64b740 1 --2- 172.21.15.6:0/4094311182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab26680 0x55648ab28a70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.626+0000 7faefc64b740 1 --2- 172.21.15.6:0/4094311182 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab1bdc0 0x55648ab26140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.626+0000 7faefc64b740 1 --2- 172.21.15.6:0/4094311182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55648ab1b4a0 0x55648ab1b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.626+0000 7faefc64b740 1 -- 172.21.15.6:0/4094311182 >> 172.21.15.6:0/4094311182 conn(0x55648ab17a30 msgr2=0x55648ab17e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:27.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.628+0000 7faefc64b740 1 -- 172.21.15.6:0/4094311182 shutdown_connections 2024-09-12T03:24:27.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.628+0000 7faefc64b740 1 -- 172.21.15.6:0/4094311182 wait complete. 2024-09-12T03:24:27.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.628+0000 7faefc64b740 1 -- start start 2024-09-12T03:24:27.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.629+0000 7faefc64b740 1 Processor -- start 2024-09-12T03:24:27.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.629+0000 7faefc64b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab1b4a0 0x55648ab25fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.629+0000 7faefc64b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55648ab1bdc0 0x55648ab23270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.630+0000 7faef3fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab1b4a0 0x55648ab25fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:27.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.630+0000 7faef3fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab1b4a0 0x55648ab25fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:39748/0 (socket says 172.21.15.6:39748) 2024-09-12T03:24:27.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.630+0000 7faef3fff640 1 -- 172.21.15.6:0/3907166200 learned_addr learned my addr 172.21.15.6:0/3907166200 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:27.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.630+0000 7faefc64b740 1 --2- 172.21.15.6:0/3907166200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab26680 0x55648ab237b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.630+0000 7faefc64b740 1 -- 172.21.15.6:0/3907166200 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55648ab29250 con 0x55648ab1bdc0 2024-09-12T03:24:27.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.630+0000 7faef880f640 1 --2- 172.21.15.6:0/3907166200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab26680 0x55648ab237b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:27.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.630+0000 7faefc64b740 1 -- 172.21.15.6:0/3907166200 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55648aa3c050 con 0x55648ab1b4a0 2024-09-12T03:24:27.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.630+0000 7faefc64b740 1 -- 172.21.15.6:0/3907166200 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55648ab293d0 con 0x55648ab26680 2024-09-12T03:24:27.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.631+0000 7faefc64b740 10 client.0 _ll_register_callbacks cb 0x55648aa7c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:27.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.633+0000 7faef37fe640 1 --2- 172.21.15.6:0/3907166200 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55648ab1bdc0 0x55648ab23270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:27.633 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.633+0000 7faef3fff640 1 -- 172.21.15.6:0/3907166200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab26680 msgr2=0x55648ab237b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:27.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.633+0000 7faef3fff640 1 --2- 172.21.15.6:0/3907166200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab26680 0x55648ab237b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.633+0000 7faef3fff640 1 -- 172.21.15.6:0/3907166200 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55648ab1bdc0 msgr2=0x55648ab23270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:27.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.633+0000 7faef3fff640 1 --2- 172.21.15.6:0/3907166200 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55648ab1bdc0 0x55648ab23270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.633+0000 7faef3fff640 1 -- 172.21.15.6:0/3907166200 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55648a7f23c0 con 0x55648ab1b4a0 2024-09-12T03:24:27.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.633+0000 7faef880f640 1 --2- 172.21.15.6:0/3907166200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab26680 0x55648ab237b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:27.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.634+0000 7faef3fff640 1 --2- 172.21.15.6:0/3907166200 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab1b4a0 0x55648ab25fc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faee8033000 tx=0x7faee8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4c31d037842ea4d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:27.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.634+0000 7faed7ff7640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:24:27.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.634+0000 7faed7ff7640 1 -- 172.21.15.6:0/3907166200 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8044050 con 0x55648ab1b4a0 2024-09-12T03:24:27.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.635+0000 7faefc64b740 1 -- 172.21.15.6:0/3907166200 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faeec0bee50 con 0x55648ab1b4a0 2024-09-12T03:24:27.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.635+0000 7faed7ff7640 1 -- 172.21.15.6:0/3907166200 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7faee80319c0 con 0x55648ab1b4a0 2024-09-12T03:24:27.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.635+0000 7faed7ff7640 1 -- 172.21.15.6:0/3907166200 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faee8035040 con 0x55648ab1b4a0 2024-09-12T03:24:27.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.635+0000 7faefc64b740 10 client.25373 fetch_fsmap learned FSMap version 27 2024-09-12T03:24:27.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.635+0000 7faefc64b740 1 -- 172.21.15.6:0/3907166200 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55648ab34ca0 con 0x55648ab1b4a0 2024-09-12T03:24:27.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.636+0000 7faed7ff7640 1 -- 172.21.15.6:0/3907166200 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faee804e070 con 0x55648ab1b4a0 2024-09-12T03:24:27.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.637+0000 7faed7ff7640 1 -- 172.21.15.6:0/3907166200 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7faee803c410 con 0x55648ab1b4a0 2024-09-12T03:24:27.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.637+0000 7faefc64b740 10 client.25373 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:24:27.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.637+0000 7faefc64b740 10 client.25373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:24:27.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.637+0000 7faefc64b740 20 client.25373 populate_metadata read hostname 'smithi006' 2024-09-12T03:24:27.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.637+0000 7faefc64b740 1 --2- 172.21.15.6:0/3907166200 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55648ab36570 0x55648ab389b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.637+0000 7faefc64b740 4 client.25373 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:24:27.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.637+0000 7faefc64b740 1 -- 172.21.15.6:0/3907166200 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7faeec0bee50 con 0x55648ab36570 2024-09-12T03:24:27.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.638+0000 7faef37fe640 1 --2- 172.21.15.6:0/3907166200 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55648ab36570 0x55648ab389b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:27.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.638+0000 7faef37fe640 1 --2- 172.21.15.6:0/3907166200 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55648ab36570 0x55648ab389b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f7f549a631b3ec6 server_cookie=aeeae82144b2fecc in_seq=0 out_seq=0 2024-09-12T03:24:27.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.638+0000 7faed7ff7640 10 client.25373 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:27.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.642+0000 7faed7ff7640 1 -- 172.21.15.6:0/3907166200 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7faeec0bee50 con 0x55648ab36570 2024-09-12T03:24:27.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.642+0000 7faed7ff7640 10 client.25373 handle_command_reply: tid=0 2024-09-12T03:24:27.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.643+0000 7faefc64b740 1 client.25373 shutdown 2024-09-12T03:24:27.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.646+0000 7faefc64b740 1 -- 172.21.15.6:0/3907166200 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab1b4a0 msgr2=0x55648ab25fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:27.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.646+0000 7faefc64b740 1 --2- 172.21.15.6:0/3907166200 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab1b4a0 0x55648ab25fc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faee8033000 tx=0x7faee8009260 comp rx=0 tx=0).stop 2024-09-12T03:24:27.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.646+0000 7faefc64b740 1 -- 172.21.15.6:0/3907166200 shutdown_connections 2024-09-12T03:24:27.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.646+0000 7faefc64b740 1 --2- 172.21.15.6:0/3907166200 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55648ab36570 0x55648ab389b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.646+0000 7faefc64b740 1 --2- 172.21.15.6:0/3907166200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab26680 0x55648ab237b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.646+0000 7faed7ff7640 0 client.25373 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:27.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.646+0000 7faefc64b740 1 --2- 172.21.15.6:0/3907166200 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55648ab1bdc0 0x55648ab23270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.646+0000 7faefc64b740 1 --2- 172.21.15.6:0/3907166200 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab1b4a0 0x55648ab25fc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.647+0000 7faefc64b740 1 -- 172.21.15.6:0/3907166200 >> 172.21.15.6:0/3907166200 conn(0x55648ab17a30 msgr2=0x55648ab1d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:27.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.648+0000 7faefc64b740 1 -- 172.21.15.6:0/3907166200 shutdown_connections 2024-09-12T03:24:27.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.648+0000 7faefc64b740 1 -- 172.21.15.6:0/3907166200 wait complete. 2024-09-12T03:24:27.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.649+0000 7faefc64b740 20 client.25373 trim_cache size 0 max 16384 2024-09-12T03:24:27.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.688+0000 7faefc64b740 1 Processor -- start 2024-09-12T03:24:27.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.688+0000 7faefc64b740 1 -- start start 2024-09-12T03:24:27.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.688+0000 7faefc64b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55648ab1b4a0 0x55648ab32500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.688+0000 7faefc64b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab1bdc0 0x55648ab1d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.688+0000 7faefc64b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab26680 0x55648ab3e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.688+0000 7faefc64b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7faef41b62d0 con 0x55648ab1b4a0 2024-09-12T03:24:27.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.688+0000 7faefc64b740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7faef41b3ba0 con 0x55648ab1bdc0 2024-09-12T03:24:27.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.688+0000 7faefc64b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faef41b1aa0 con 0x55648ab26680 2024-09-12T03:24:27.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.693+0000 7faef37fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab1bdc0 0x55648ab1d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:27.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.693+0000 7faef37fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab1bdc0 0x55648ab1d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:39752/0 (socket says 172.21.15.6:39752) 2024-09-12T03:24:27.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.693+0000 7faef37fe640 1 -- 172.21.15.6:0/2381312598 learned_addr learned my addr 172.21.15.6:0/2381312598 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:27.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.698+0000 7faef880f640 1 --2- 172.21.15.6:0/2381312598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab26680 0x55648ab3e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:27.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.699+0000 7faef3fff640 1 --2- 172.21.15.6:0/2381312598 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55648ab1b4a0 0x55648ab32500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:27.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.699+0000 7faef3fff640 1 -- 172.21.15.6:0/2381312598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab26680 msgr2=0x55648ab3e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:27.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.699+0000 7faef3fff640 1 --2- 172.21.15.6:0/2381312598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab26680 0x55648ab3e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.699+0000 7faef3fff640 1 -- 172.21.15.6:0/2381312598 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab1bdc0 msgr2=0x55648ab1d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:27.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.699+0000 7faef3fff640 1 --2- 172.21.15.6:0/2381312598 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab1bdc0 0x55648ab1d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.699+0000 7faef3fff640 1 -- 172.21.15.6:0/2381312598 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55648ab22220 con 0x55648ab1b4a0 2024-09-12T03:24:27.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.700+0000 7faef3fff640 1 --2- 172.21.15.6:0/2381312598 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55648ab1b4a0 0x55648ab32500 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7faee8009230 tx=0x7faee8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b754785b382059e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:27.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.704+0000 7faed9ffb640 1 -- 172.21.15.6:0/2381312598 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8044040 con 0x55648ab1b4a0 2024-09-12T03:24:27.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.704+0000 7faed9ffb640 1 -- 172.21.15.6:0/2381312598 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faee804e040 con 0x55648ab1b4a0 2024-09-12T03:24:27.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.704+0000 7faed9ffb640 1 -- 172.21.15.6:0/2381312598 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8055460 con 0x55648ab1b4a0 2024-09-12T03:24:27.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.705+0000 7faefc64b740 1 -- 172.21.15.6:0/2381312598 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55648ab1b4a0 msgr2=0x55648ab32500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:27.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.705+0000 7faefc64b740 1 --2- 172.21.15.6:0/2381312598 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55648ab1b4a0 0x55648ab32500 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7faee8009230 tx=0x7faee8005bd0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.707+0000 7faefc64b740 1 -- 172.21.15.6:0/2381312598 shutdown_connections 2024-09-12T03:24:27.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.707+0000 7faefc64b740 1 --2- 172.21.15.6:0/2381312598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab26680 0x55648ab3e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.707+0000 7faefc64b740 1 --2- 172.21.15.6:0/2381312598 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab1bdc0 0x55648ab1d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.707+0000 7faefc64b740 1 --2- 172.21.15.6:0/2381312598 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55648ab1b4a0 0x55648ab32500 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.708+0000 7faefc64b740 1 -- 172.21.15.6:0/2381312598 >> 172.21.15.6:0/2381312598 conn(0x55648ab17a30 msgr2=0x55648ab1cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:27.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.708+0000 7faefc64b740 1 -- 172.21.15.6:0/2381312598 shutdown_connections 2024-09-12T03:24:27.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.708+0000 7faefc64b740 1 -- 172.21.15.6:0/2381312598 wait complete. 2024-09-12T03:24:27.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.709+0000 7faefc64b740 1 -- start start 2024-09-12T03:24:27.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.709+0000 7faefc64b740 1 Processor -- start 2024-09-12T03:24:27.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.710+0000 7faefc64b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab1b4a0 0x55648ab2eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.710+0000 7faefc64b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55648ab1bdc0 0x55648ab29ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.711+0000 7faefc64b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab26680 0x55648ab2a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.711+0000 7faefc64b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7faef41b2060 con 0x55648ab1bdc0 2024-09-12T03:24:27.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.711+0000 7faefc64b740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7faef41b34d0 con 0x55648ab26680 2024-09-12T03:24:27.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.711+0000 7faef880f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab26680 0x55648ab2a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:27.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.711+0000 7faef880f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab26680 0x55648ab2a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:39758/0 (socket says 172.21.15.6:39758) 2024-09-12T03:24:27.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.711+0000 7faef880f640 1 -- 172.21.15.6:0/3418934996 learned_addr learned my addr 172.21.15.6:0/3418934996 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:27.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.711+0000 7faef3fff640 1 --2- 172.21.15.6:0/3418934996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab1b4a0 0x55648ab2eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:27.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.711+0000 7faefc64b740 1 -- 172.21.15.6:0/3418934996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faef41b2730 con 0x55648ab1b4a0 2024-09-12T03:24:27.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.712+0000 7faefc64b740 10 client.0 _ll_register_callbacks cb 0x7faeec0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:27.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.712+0000 7faef880f640 1 -- 172.21.15.6:0/3418934996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab1b4a0 msgr2=0x55648ab2eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:27.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.712+0000 7faef880f640 1 --2- 172.21.15.6:0/3418934996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab1b4a0 0x55648ab2eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.712+0000 7faef880f640 1 -- 172.21.15.6:0/3418934996 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55648ab1bdc0 msgr2=0x55648ab29ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:27.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.712+0000 7faef880f640 1 --2- 172.21.15.6:0/3418934996 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55648ab1bdc0 0x55648ab29ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.712+0000 7faef880f640 1 -- 172.21.15.6:0/3418934996 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55648ab26330 con 0x55648ab26680 2024-09-12T03:24:27.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.727+0000 7faef880f640 1 --2- 172.21.15.6:0/3418934996 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab26680 0x55648ab2a410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faeec09ff10 tx=0x7faeec09fe80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22102caa9f88cf06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:27.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.727+0000 7faed7ff7640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:24:27.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.727+0000 7faed7ff7640 1 -- 172.21.15.6:0/3418934996 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec0b9040 con 0x55648ab26680 2024-09-12T03:24:27.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.727+0000 7faefc64b740 1 -- 172.21.15.6:0/3418934996 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faee804e040 con 0x55648ab26680 2024-09-12T03:24:27.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.728+0000 7faed7ff7640 1 -- 172.21.15.6:0/3418934996 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7faeec0a89f0 con 0x55648ab26680 2024-09-12T03:24:27.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.729+0000 7faed7ff7640 1 -- 172.21.15.6:0/3418934996 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faeec084030 con 0x55648ab26680 2024-09-12T03:24:27.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.729+0000 7faefc64b740 10 client.25379 fetch_fsmap learned FSMap version 27 2024-09-12T03:24:27.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.729+0000 7faefc64b740 1 -- 172.21.15.6:0/3418934996 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55648aba0e40 con 0x55648ab26680 2024-09-12T03:24:27.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.730+0000 7faed7ff7640 1 -- 172.21.15.6:0/3418934996 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faee804e040 con 0x55648ab26680 2024-09-12T03:24:27.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.730+0000 7faed7ff7640 1 -- 172.21.15.6:0/3418934996 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7faeec0893f0 con 0x55648ab26680 2024-09-12T03:24:27.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.730+0000 7faefc64b740 10 client.25379 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:24:27.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.730+0000 7faefc64b740 10 client.25379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:24:27.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.730+0000 7faefc64b740 20 client.25379 populate_metadata read hostname 'smithi006' 2024-09-12T03:24:27.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.730+0000 7faefc64b740 1 --2- 172.21.15.6:0/3418934996 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55648ab36570 0x55648aba3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.730+0000 7faefc64b740 4 client.25379 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:24:27.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.730+0000 7faefc64b740 1 -- 172.21.15.6:0/3418934996 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55648aa8e850 con 0x55648ab36570 2024-09-12T03:24:27.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.731+0000 7faef3fff640 1 --2- 172.21.15.6:0/3418934996 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55648ab36570 0x55648aba3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:27.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.732+0000 7faef3fff640 1 --2- 172.21.15.6:0/3418934996 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55648ab36570 0x55648aba3080 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e255f6b7ea1cf1d7 server_cookie=a011a7edce0bc488 in_seq=0 out_seq=0 2024-09-12T03:24:27.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.732+0000 7faed7ff7640 10 client.25379 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:27.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.738+0000 7faed7ff7640 1 -- 172.21.15.6:0/3418934996 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55648aa8e850 con 0x55648ab36570 2024-09-12T03:24:27.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.738+0000 7faed7ff7640 10 client.25379 handle_command_reply: tid=0 2024-09-12T03:24:27.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.738+0000 7faefc64b740 1 client.25379 shutdown 2024-09-12T03:24:27.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.738+0000 7faefc64b740 1 -- 172.21.15.6:0/3418934996 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab26680 msgr2=0x55648ab2a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:27.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.738+0000 7faefc64b740 1 --2- 172.21.15.6:0/3418934996 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab26680 0x55648ab2a410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faeec09ff10 tx=0x7faeec09fe80 comp rx=0 tx=0).stop 2024-09-12T03:24:27.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.739+0000 7faefc64b740 1 -- 172.21.15.6:0/3418934996 shutdown_connections 2024-09-12T03:24:27.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.739+0000 7faefc64b740 1 --2- 172.21.15.6:0/3418934996 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55648ab36570 0x55648aba3080 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.739+0000 7faefc64b740 1 --2- 172.21.15.6:0/3418934996 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55648ab26680 0x55648ab2a410 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.739+0000 7faefc64b740 1 --2- 172.21.15.6:0/3418934996 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55648ab1bdc0 0x55648ab29ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.739+0000 7faefc64b740 1 --2- 172.21.15.6:0/3418934996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55648ab1b4a0 0x55648ab2eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.739+0000 7faefc64b740 1 -- 172.21.15.6:0/3418934996 >> 172.21.15.6:0/3418934996 conn(0x55648ab17a30 msgr2=0x55648ab16b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:27.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.739+0000 7faed7ff7640 0 client.25379 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:27.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.739+0000 7faefc64b740 1 -- 172.21.15.6:0/3418934996 shutdown_connections 2024-09-12T03:24:27.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.739+0000 7faefc64b740 1 -- 172.21.15.6:0/3418934996 wait complete. 2024-09-12T03:24:27.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.739+0000 7faefc64b740 20 client.25379 trim_cache size 0 max 16384 2024-09-12T03:24:27.741 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:24:27.741 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "scrub active (190 inodes in the stack)", 2024-09-12T03:24:27.741 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2024-09-12T03:24:27.741 INFO:teuthology.orchestra.run.smithi006.stdout: "9cc9dee5-d9ac-4c99-b9a3-da95838d23be": { 2024-09-12T03:24:27.741 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-09-12T03:24:27.741 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "9cc9dee5-d9ac-4c99-b9a3-da95838d23be", 2024-09-12T03:24:27.741 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2024-09-12T03:24:27.741 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:24:27.741 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:24:27.742 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:24:27.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.740+0000 7faefa272640 1 -- 172.21.15.6:0/3371365227 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7faedc0a2c80 msgr2=0x7faedc0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:27.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.740+0000 7faefa272640 1 --2- 172.21.15.6:0/3371365227 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7faedc0a2c80 0x7faedc0a5140 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7faee80096c0 tx=0x7faee8031490 comp rx=0 tx=0).stop 2024-09-12T03:24:27.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.740+0000 7faefa272640 1 -- 172.21.15.6:0/3371365227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faef4114ab0 msgr2=0x7faef41a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:27.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.740+0000 7faefa272640 1 --2- 172.21.15.6:0/3371365227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faef4114ab0 0x7faef41a5670 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faeec009fa0 tx=0x7faeec00c2d0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.740+0000 7faefa272640 1 -- 172.21.15.6:0/3371365227 shutdown_connections 2024-09-12T03:24:27.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.740+0000 7faefa272640 1 --2- 172.21.15.6:0/3371365227 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7faedc0a2c80 0x7faedc0a5140 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.740+0000 7faefa272640 1 --2- 172.21.15.6:0/3371365227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faef4114ab0 0x7faef41a5670 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.740+0000 7faefa272640 1 --2- 172.21.15.6:0/3371365227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faef4109b10 0x7faef41a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.740+0000 7faefa272640 1 --2- 172.21.15.6:0/3371365227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faef41091f0 0x7faef41a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.740+0000 7faefa272640 1 -- 172.21.15.6:0/3371365227 >> 172.21.15.6:0/3371365227 conn(0x7faef406e140 msgr2=0x7faef4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:27.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.740+0000 7faefa272640 1 -- 172.21.15.6:0/3371365227 shutdown_connections 2024-09-12T03:24:27.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:27.741+0000 7faefa272640 1 -- 172.21.15.6:0/3371365227 wait complete. 2024-09-12T03:24:28.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:27 smithi006 ceph-mon[30991]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:24:28.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:27 smithi118 ceph-mon[35048]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:24:28.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:27 smithi161 ceph-mon[35485]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:24:29.012 INFO:tasks.cephfs.filesystem:scrub status for tag:9cc9dee5-d9ac-4c99-b9a3-da95838d23be - {'path': '/', 'tag': '9cc9dee5-d9ac-4c99-b9a3-da95838d23be', 'options': 'recursive,force'} 2024-09-12T03:24:29.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:28 smithi006 ceph-mon[30991]: pgmap v436: 97 pgs: 97 active+clean; 13 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 875 KiB/s rd, 238 MiB/s wr, 20.18k op/s 2024-09-12T03:24:29.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:28 smithi161 ceph-mon[35485]: pgmap v436: 97 pgs: 97 active+clean; 13 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 875 KiB/s rd, 238 MiB/s wr, 20.18k op/s 2024-09-12T03:24:29.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:28 smithi118 ceph-mon[35048]: pgmap v436: 97 pgs: 97 active+clean; 13 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 875 KiB/s rd, 238 MiB/s wr, 20.18k op/s 2024-09-12T03:24:31.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:30 smithi006 ceph-mon[30991]: pgmap v437: 97 pgs: 97 active+clean; 13 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 533 KiB/s rd, 295 MiB/s wr, 25.66k op/s 2024-09-12T03:24:31.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:30 smithi161 ceph-mon[35485]: pgmap v437: 97 pgs: 97 active+clean; 13 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 533 KiB/s rd, 295 MiB/s wr, 25.66k op/s 2024-09-12T03:24:31.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:30 smithi118 ceph-mon[35048]: pgmap v437: 97 pgs: 97 active+clean; 13 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 533 KiB/s rd, 295 MiB/s wr, 25.66k op/s 2024-09-12T03:24:33.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:32 smithi006 ceph-mon[30991]: pgmap v438: 97 pgs: 97 active+clean; 13 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 634 KiB/s rd, 238 MiB/s wr, 21.26k op/s 2024-09-12T03:24:33.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:32 smithi161 ceph-mon[35485]: pgmap v438: 97 pgs: 97 active+clean; 13 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 634 KiB/s rd, 238 MiB/s wr, 21.26k op/s 2024-09-12T03:24:33.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:32 smithi118 ceph-mon[35048]: pgmap v438: 97 pgs: 97 active+clean; 13 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 634 KiB/s rd, 238 MiB/s wr, 21.26k op/s 2024-09-12T03:24:35.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:35 smithi006 ceph-mon[30991]: pgmap v439: 97 pgs: 97 active+clean; 13 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 633 KiB/s rd, 245 MiB/s wr, 22.60k op/s 2024-09-12T03:24:35.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:35 smithi161 ceph-mon[35485]: pgmap v439: 97 pgs: 97 active+clean; 13 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 633 KiB/s rd, 245 MiB/s wr, 22.60k op/s 2024-09-12T03:24:35.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:35 smithi118 ceph-mon[35048]: pgmap v439: 97 pgs: 97 active+clean; 13 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 633 KiB/s rd, 245 MiB/s wr, 22.60k op/s 2024-09-12T03:24:37.263 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:37 smithi118 ceph-mon[35048]: pgmap v440: 97 pgs: 97 active+clean; 13 GiB data, 79 GiB used, 994 GiB / 1.0 TiB avail; 748 KiB/s rd, 248 MiB/s wr, 23.84k op/s 2024-09-12T03:24:37.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:37 smithi006 ceph-mon[30991]: pgmap v440: 97 pgs: 97 active+clean; 13 GiB data, 79 GiB used, 994 GiB / 1.0 TiB avail; 748 KiB/s rd, 248 MiB/s wr, 23.84k op/s 2024-09-12T03:24:37.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:37 smithi161 ceph-mon[35485]: pgmap v440: 97 pgs: 97 active+clean; 13 GiB data, 79 GiB used, 994 GiB / 1.0 TiB avail; 748 KiB/s rd, 248 MiB/s wr, 23.84k op/s 2024-09-12T03:24:39.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:39 smithi006 ceph-mon[30991]: pgmap v441: 97 pgs: 97 active+clean; 13 GiB data, 79 GiB used, 994 GiB / 1.0 TiB avail; 458 KiB/s rd, 214 MiB/s wr, 20.91k op/s 2024-09-12T03:24:39.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:39 smithi161 ceph-mon[35485]: pgmap v441: 97 pgs: 97 active+clean; 13 GiB data, 79 GiB used, 994 GiB / 1.0 TiB avail; 458 KiB/s rd, 214 MiB/s wr, 20.91k op/s 2024-09-12T03:24:39.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:39 smithi118 ceph-mon[35048]: pgmap v441: 97 pgs: 97 active+clean; 13 GiB data, 79 GiB used, 994 GiB / 1.0 TiB avail; 458 KiB/s rd, 214 MiB/s wr, 20.91k op/s 2024-09-12T03:24:41.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:41 smithi006 ceph-mon[30991]: pgmap v442: 97 pgs: 97 active+clean; 13 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 270 KiB/s rd, 243 MiB/s wr, 23.86k op/s 2024-09-12T03:24:41.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:41 smithi161 ceph-mon[35485]: pgmap v442: 97 pgs: 97 active+clean; 13 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 270 KiB/s rd, 243 MiB/s wr, 23.86k op/s 2024-09-12T03:24:41.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:41 smithi118 ceph-mon[35048]: pgmap v442: 97 pgs: 97 active+clean; 13 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 270 KiB/s rd, 243 MiB/s wr, 23.86k op/s 2024-09-12T03:24:43.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:43 smithi006 ceph-mon[30991]: pgmap v443: 97 pgs: 97 active+clean; 13 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 499 KiB/s rd, 203 MiB/s wr, 20.39k op/s 2024-09-12T03:24:43.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:43 smithi161 ceph-mon[35485]: pgmap v443: 97 pgs: 97 active+clean; 13 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 499 KiB/s rd, 203 MiB/s wr, 20.39k op/s 2024-09-12T03:24:43.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:43 smithi118 ceph-mon[35048]: pgmap v443: 97 pgs: 97 active+clean; 13 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 499 KiB/s rd, 203 MiB/s wr, 20.39k op/s 2024-09-12T03:24:45.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:45 smithi006 ceph-mon[30991]: pgmap v444: 97 pgs: 97 active+clean; 13 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 345 KiB/s rd, 218 MiB/s wr, 21.90k op/s 2024-09-12T03:24:45.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:45 smithi161 ceph-mon[35485]: pgmap v444: 97 pgs: 97 active+clean; 13 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 345 KiB/s rd, 218 MiB/s wr, 21.90k op/s 2024-09-12T03:24:45.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:45 smithi118 ceph-mon[35048]: pgmap v444: 97 pgs: 97 active+clean; 13 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 345 KiB/s rd, 218 MiB/s wr, 21.90k op/s 2024-09-12T03:24:47.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:47 smithi006 ceph-mon[30991]: pgmap v445: 97 pgs: 97 active+clean; 13 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 686 KiB/s rd, 246 MiB/s wr, 25.08k op/s 2024-09-12T03:24:47.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:47 smithi161 ceph-mon[35485]: pgmap v445: 97 pgs: 97 active+clean; 13 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 686 KiB/s rd, 246 MiB/s wr, 25.08k op/s 2024-09-12T03:24:47.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:47 smithi118 ceph-mon[35048]: pgmap v445: 97 pgs: 97 active+clean; 13 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 686 KiB/s rd, 246 MiB/s wr, 25.08k op/s 2024-09-12T03:24:49.077 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T03:24:49.077 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-12T03:24:49.078 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:24:49.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:49 smithi006 ceph-mon[30991]: pgmap v446: 97 pgs: 97 active+clean; 13 GiB data, 83 GiB used, 989 GiB / 1.0 TiB avail; 571 KiB/s rd, 202 MiB/s wr, 20.59k op/s 2024-09-12T03:24:49.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:49 smithi161 ceph-mon[35485]: pgmap v446: 97 pgs: 97 active+clean; 13 GiB data, 83 GiB used, 989 GiB / 1.0 TiB avail; 571 KiB/s rd, 202 MiB/s wr, 20.59k op/s 2024-09-12T03:24:49.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:49 smithi118 ceph-mon[35048]: pgmap v446: 97 pgs: 97 active+clean; 13 GiB data, 83 GiB used, 989 GiB / 1.0 TiB avail; 571 KiB/s rd, 202 MiB/s wr, 20.59k op/s 2024-09-12T03:24:49.626 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:50.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.461+0000 7fb7e4ab7640 1 -- 172.21.15.6:0/3476314686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7d8004c80 msgr2=0x7fb7d80050c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:50.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.461+0000 7fb7e4ab7640 1 --2- 172.21.15.6:0/3476314686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7d8004c80 0x7fb7d80050c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009900 tx=0x7fb7cc02a410 comp rx=0 tx=0).stop 2024-09-12T03:24:50.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.462+0000 7fb7e4ab7640 1 -- 172.21.15.6:0/3476314686 shutdown_connections 2024-09-12T03:24:50.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.462+0000 7fb7e4ab7640 1 --2- 172.21.15.6:0/3476314686 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7d8005710 0x7fb7d8097fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:50.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.462+0000 7fb7e4ab7640 1 --2- 172.21.15.6:0/3476314686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7d8004c80 0x7fb7d80050c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:50.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.462+0000 7fb7e4ab7640 1 --2- 172.21.15.6:0/3476314686 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7d809a4c0 0x7fb7d809a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:50.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.462+0000 7fb7e4ab7640 1 -- 172.21.15.6:0/3476314686 >> 172.21.15.6:0/3476314686 conn(0x7fb7d800a1f0 msgr2=0x7fb7d800a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:50.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.463+0000 7fb7e4ab7640 1 -- 172.21.15.6:0/3476314686 shutdown_connections 2024-09-12T03:24:50.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.463+0000 7fb7e4ab7640 1 -- 172.21.15.6:0/3476314686 wait complete. 2024-09-12T03:24:50.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.464+0000 7fb7e4ab7640 1 Processor -- start 2024-09-12T03:24:50.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.465+0000 7fb7e4ab7640 1 -- start start 2024-09-12T03:24:50.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.465+0000 7fb7e4ab7640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7d8004c80 0x7fb7d8132450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:50.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.465+0000 7fb7e4ab7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7d8005710 0x7fb7d8132990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:50.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.465+0000 7fb7e4ab7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7d809a4c0 0x7fb7d8136d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:50.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.465+0000 7fb7df7fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7d8004c80 0x7fb7d8132450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:50.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.465+0000 7fb7df7fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7d8004c80 0x7fb7d8132450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:39570/0 (socket says 172.21.15.6:39570) 2024-09-12T03:24:50.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.466+0000 7fb7df7fe640 1 -- 172.21.15.6:0/4093537279 learned_addr learned my addr 172.21.15.6:0/4093537279 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:50.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.466+0000 7fb7deffd640 1 --2- 172.21.15.6:0/4093537279 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7d8005710 0x7fb7d8132990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:50.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.466+0000 7fb7dffff640 1 --2- 172.21.15.6:0/4093537279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7d809a4c0 0x7fb7d8136d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:50.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.466+0000 7fb7e4ab7640 1 -- 172.21.15.6:0/4093537279 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d80abff0 con 0x7fb7d8005710 2024-09-12T03:24:50.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.466+0000 7fb7e4ab7640 1 -- 172.21.15.6:0/4093537279 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d80abe70 con 0x7fb7d8004c80 2024-09-12T03:24:50.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.466+0000 7fb7e4ab7640 1 -- 172.21.15.6:0/4093537279 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d80ac170 con 0x7fb7d809a4c0 2024-09-12T03:24:50.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.466+0000 7fb7df7fe640 1 -- 172.21.15.6:0/4093537279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7d809a4c0 msgr2=0x7fb7d8136d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:50.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.466+0000 7fb7df7fe640 1 --2- 172.21.15.6:0/4093537279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7d809a4c0 0x7fb7d8136d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:50.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.466+0000 7fb7df7fe640 1 -- 172.21.15.6:0/4093537279 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7d8005710 msgr2=0x7fb7d8132990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:50.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.466+0000 7fb7df7fe640 1 --2- 172.21.15.6:0/4093537279 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7d8005710 0x7fb7d8132990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:50.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.466+0000 7fb7df7fe640 1 -- 172.21.15.6:0/4093537279 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d8137400 con 0x7fb7d8004c80 2024-09-12T03:24:50.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.466+0000 7fb7dffff640 1 --2- 172.21.15.6:0/4093537279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7d809a4c0 0x7fb7d8136d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:50.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.467+0000 7fb7df7fe640 1 --2- 172.21.15.6:0/4093537279 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7d8004c80 0x7fb7d8132450 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400e9f0 tx=0x7fb7d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:50.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.468+0000 7fb7dcff9640 1 -- 172.21.15.6:0/4093537279 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d400cd80 con 0x7fb7d8004c80 2024-09-12T03:24:50.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.468+0000 7fb7dcff9640 1 -- 172.21.15.6:0/4093537279 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7d4010910 con 0x7fb7d8004c80 2024-09-12T03:24:50.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.468+0000 7fb7e4ab7640 1 -- 172.21.15.6:0/4093537279 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d81376f0 con 0x7fb7d8004c80 2024-09-12T03:24:50.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.469+0000 7fb7e4ab7640 1 -- 172.21.15.6:0/4093537279 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d8137b40 con 0x7fb7d8004c80 2024-09-12T03:24:50.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.469+0000 7fb7dcff9640 1 -- 172.21.15.6:0/4093537279 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4018990 con 0x7fb7d8004c80 2024-09-12T03:24:50.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.470+0000 7fb7e4ab7640 1 -- 172.21.15.6:0/4093537279 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7d8097fd0 con 0x7fb7d8004c80 2024-09-12T03:24:50.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.472+0000 7fb7dcff9640 1 -- 172.21.15.6:0/4093537279 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb7d4010ac0 con 0x7fb7d8004c80 2024-09-12T03:24:50.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.474+0000 7fb7dcff9640 1 --2- 172.21.15.6:0/4093537279 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb7d00a2d60 0x7fb7d00a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:50.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.474+0000 7fb7dcff9640 1 -- 172.21.15.6:0/4093537279 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fb7d40cf770 con 0x7fb7d8004c80 2024-09-12T03:24:50.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.475+0000 7fb7deffd640 1 --2- 172.21.15.6:0/4093537279 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb7d00a2d60 0x7fb7d00a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:50.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.476+0000 7fb7deffd640 1 --2- 172.21.15.6:0/4093537279 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb7d00a2d60 0x7fb7d00a5220 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc000c00 tx=0x7fb7cc005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:50.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.481+0000 7fb7dcff9640 1 -- 172.21.15.6:0/4093537279 <== mon.2 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb7d409c0d0 con 0x7fb7d8004c80 2024-09-12T03:24:50.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.934+0000 7fb7e4ab7640 1 -- 172.21.15.6:0/4093537279 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7d81337b0 con 0x7fb7d8004c80 2024-09-12T03:24:50.941 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:24:50.942 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"btime":"2024-09-12T03:24:26:920603+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:24:26.080047+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24463]},"id":1}]} 2024-09-12T03:24:50.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.937+0000 7fb7dcff9640 1 -- 172.21.15.6:0/4093537279 <== mon.2 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11908 (secure 0 0 0) 0x7fb7d40a03d0 con 0x7fb7d8004c80 2024-09-12T03:24:50.942 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 27 2024-09-12T03:24:50.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.943+0000 7fb7ca7fc640 1 -- 172.21.15.6:0/4093537279 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb7d00a2d60 msgr2=0x7fb7d00a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:50.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.943+0000 7fb7ca7fc640 1 --2- 172.21.15.6:0/4093537279 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb7d00a2d60 0x7fb7d00a5220 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc000c00 tx=0x7fb7cc005b50 comp rx=0 tx=0).stop 2024-09-12T03:24:50.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.943+0000 7fb7ca7fc640 1 -- 172.21.15.6:0/4093537279 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7d8004c80 msgr2=0x7fb7d8132450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:50.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.943+0000 7fb7ca7fc640 1 --2- 172.21.15.6:0/4093537279 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7d8004c80 0x7fb7d8132450 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400e9f0 tx=0x7fb7d400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:24:50.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.943+0000 7fb7ca7fc640 1 -- 172.21.15.6:0/4093537279 shutdown_connections 2024-09-12T03:24:50.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.944+0000 7fb7ca7fc640 1 --2- 172.21.15.6:0/4093537279 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb7d00a2d60 0x7fb7d00a5220 secure :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc000c00 tx=0x7fb7cc005b50 comp rx=0 tx=0).stop 2024-09-12T03:24:50.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.944+0000 7fb7ca7fc640 1 --2- 172.21.15.6:0/4093537279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7d809a4c0 0x7fb7d8136d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:50.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.944+0000 7fb7ca7fc640 1 --2- 172.21.15.6:0/4093537279 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7d8005710 0x7fb7d8132990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:50.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.944+0000 7fb7ca7fc640 1 --2- 172.21.15.6:0/4093537279 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7d8004c80 0x7fb7d8132450 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:50.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.944+0000 7fb7ca7fc640 1 -- 172.21.15.6:0/4093537279 >> 172.21.15.6:0/4093537279 conn(0x7fb7d800a1f0 msgr2=0x7fb7d8096fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:50.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.944+0000 7fb7ca7fc640 1 -- 172.21.15.6:0/4093537279 shutdown_connections 2024-09-12T03:24:50.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:50.944+0000 7fb7ca7fc640 1 -- 172.21.15.6:0/4093537279 wait complete. 2024-09-12T03:24:51.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:51 smithi006 ceph-mon[30991]: pgmap v447: 97 pgs: 97 active+clean; 13 GiB data, 84 GiB used, 988 GiB / 1.0 TiB avail; 913 KiB/s rd, 226 MiB/s wr, 22.70k op/s 2024-09-12T03:24:51.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:51 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/4093537279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:51.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:51 smithi161 ceph-mon[35485]: pgmap v447: 97 pgs: 97 active+clean; 13 GiB data, 84 GiB used, 988 GiB / 1.0 TiB avail; 913 KiB/s rd, 226 MiB/s wr, 22.70k op/s 2024-09-12T03:24:51.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:51 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/4093537279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:51.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:51 smithi118 ceph-mon[35048]: pgmap v447: 97 pgs: 97 active+clean; 13 GiB data, 84 GiB used, 988 GiB / 1.0 TiB avail; 913 KiB/s rd, 226 MiB/s wr, 22.70k op/s 2024-09-12T03:24:51.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:51 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/4093537279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:52.589 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-12T03:24:52.589 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-12T03:24:53.154 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:53.296 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:53 smithi006 ceph-mon[30991]: pgmap v448: 97 pgs: 97 active+clean; 13 GiB data, 85 GiB used, 988 GiB / 1.0 TiB avail; 912 KiB/s rd, 233 MiB/s wr, 23.26k op/s 2024-09-12T03:24:53.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:53 smithi118 ceph-mon[35048]: pgmap v448: 97 pgs: 97 active+clean; 13 GiB data, 85 GiB used, 988 GiB / 1.0 TiB avail; 912 KiB/s rd, 233 MiB/s wr, 23.26k op/s 2024-09-12T03:24:53.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:53 smithi161 ceph-mon[35485]: pgmap v448: 97 pgs: 97 active+clean; 13 GiB data, 85 GiB used, 988 GiB / 1.0 TiB avail; 912 KiB/s rd, 233 MiB/s wr, 23.26k op/s 2024-09-12T03:24:54.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.113+0000 7f3735ebe640 1 -- 172.21.15.6:0/878096051 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3730110190 msgr2=0x7f373006e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:54.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.113+0000 7f3735ebe640 1 --2- 172.21.15.6:0/878096051 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3730110190 0x7f373006e4f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3724009900 tx=0x7f372402a420 comp rx=0 tx=0).stop 2024-09-12T03:24:54.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.113+0000 7f3735ebe640 1 -- 172.21.15.6:0/878096051 shutdown_connections 2024-09-12T03:24:54.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.113+0000 7f3735ebe640 1 --2- 172.21.15.6:0/878096051 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f373011b140 0x7f373011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.113+0000 7f3735ebe640 1 --2- 172.21.15.6:0/878096051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f373006ea30 0x7f373006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.113+0000 7f3735ebe640 1 --2- 172.21.15.6:0/878096051 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3730110190 0x7f373006e4f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.113+0000 7f3735ebe640 1 -- 172.21.15.6:0/878096051 >> 172.21.15.6:0/878096051 conn(0x7f3730073180 msgr2=0x7f3730073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:54.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.114+0000 7f3735ebe640 1 -- 172.21.15.6:0/878096051 shutdown_connections 2024-09-12T03:24:54.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.115+0000 7f3735ebe640 1 -- 172.21.15.6:0/878096051 wait complete. 2024-09-12T03:24:54.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.116+0000 7f3735ebe640 1 Processor -- start 2024-09-12T03:24:54.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.116+0000 7f3735ebe640 1 -- start start 2024-09-12T03:24:54.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.116+0000 7f3735ebe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f373006ea30 0x7f37301a71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.116+0000 7f3735ebe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3730110190 0x7f37301a7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.116+0000 7f3735ebe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f373011b140 0x7f37301abab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.116+0000 7f3735ebe640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3730120c40 con 0x7f3730110190 2024-09-12T03:24:54.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.116+0000 7f3735ebe640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f3730120ac0 con 0x7f373011b140 2024-09-12T03:24:54.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.116+0000 7f3735ebe640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3730120dc0 con 0x7f373006ea30 2024-09-12T03:24:54.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.117+0000 7f3734ebc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f373006ea30 0x7f37301a71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.117+0000 7f3734ebc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f373006ea30 0x7f37301a71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:48924/0 (socket says 172.21.15.6:48924) 2024-09-12T03:24:54.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.117+0000 7f3734ebc640 1 -- 172.21.15.6:0/3035179611 learned_addr learned my addr 172.21.15.6:0/3035179611 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:54.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.118+0000 7f3734ebc640 1 -- 172.21.15.6:0/3035179611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f373011b140 msgr2=0x7f37301abab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:54.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.118+0000 7f3734ebc640 1 --2- 172.21.15.6:0/3035179611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f373011b140 0x7f37301abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.118+0000 7f3734ebc640 1 -- 172.21.15.6:0/3035179611 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3730110190 msgr2=0x7f37301a7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:54.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.118+0000 7f3734ebc640 1 --2- 172.21.15.6:0/3035179611 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3730110190 0x7f37301a7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.118+0000 7f3734ebc640 1 -- 172.21.15.6:0/3035179611 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37301ac1b0 con 0x7f373006ea30 2024-09-12T03:24:54.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.119+0000 7f3734ebc640 1 --2- 172.21.15.6:0/3035179611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f373006ea30 0x7f37301a71e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3724009230 tx=0x7f3724003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:54.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.121+0000 7f372dffb640 1 -- 172.21.15.6:0/3035179611 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372403f070 con 0x7f373006ea30 2024-09-12T03:24:54.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.121+0000 7f372dffb640 1 -- 172.21.15.6:0/3035179611 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3724031c90 con 0x7f373006ea30 2024-09-12T03:24:54.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.122+0000 7f372dffb640 1 -- 172.21.15.6:0/3035179611 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724035030 con 0x7f373006ea30 2024-09-12T03:24:54.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.122+0000 7f3735ebe640 1 -- 172.21.15.6:0/3035179611 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37301ac440 con 0x7f373006ea30 2024-09-12T03:24:54.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.122+0000 7f3735ebe640 1 -- 172.21.15.6:0/3035179611 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37301ac8a0 con 0x7f373006ea30 2024-09-12T03:24:54.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.128+0000 7f37394f9740 1 Processor -- start 2024-09-12T03:24:54.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.128+0000 7f37394f9740 1 -- start start 2024-09-12T03:24:54.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.128+0000 7f37394f9740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e42a20 0x555770e42e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.128+0000 7f37394f9740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e43340 0x555770e4d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.128+0000 7f37394f9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e4dc00 0x555770e4fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.128+0000 7f37394f9740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x555770da7160 con 0x555770e42a20 2024-09-12T03:24:54.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.128+0000 7f37394f9740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x555770d4e720 con 0x555770e43340 2024-09-12T03:24:54.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.128+0000 7f37394f9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555770da3120 con 0x555770e4dc00 2024-09-12T03:24:54.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.129+0000 7f372ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e4dc00 0x555770e4fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.129+0000 7f37356bd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e42a20 0x555770e42e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.129+0000 7f37356bd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e42a20 0x555770e42e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52712/0 (socket says 172.21.15.6:52712) 2024-09-12T03:24:54.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.129+0000 7f37356bd640 1 -- 172.21.15.6:0/8981992 learned_addr learned my addr 172.21.15.6:0/8981992 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:54.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.129+0000 7f372dffb640 1 -- 172.21.15.6:0/3035179611 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3724042070 con 0x7f373006ea30 2024-09-12T03:24:54.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.131+0000 7f372dffb640 1 --2- 172.21.15.6:0/3035179611 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f37200a2b80 0x7f37200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.132+0000 7f3734ebc640 1 --2- 172.21.15.6:0/8981992 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e43340 0x555770e4d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.132+0000 7f372ffff640 1 -- 172.21.15.6:0/8981992 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e43340 msgr2=0x555770e4d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:54.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.132+0000 7f372ffff640 1 --2- 172.21.15.6:0/8981992 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e43340 0x555770e4d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.132+0000 7f372ffff640 1 -- 172.21.15.6:0/8981992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e42a20 msgr2=0x555770e42e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:54.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.132+0000 7f372ffff640 1 --2- 172.21.15.6:0/8981992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e42a20 0x555770e42e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.132+0000 7f372ffff640 1 -- 172.21.15.6:0/8981992 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555770b18ba0 con 0x555770e4dc00 2024-09-12T03:24:54.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.133+0000 7f372ffff640 1 --2- 172.21.15.6:0/8981992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e4dc00 0x555770e4fff0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f371c00ba10 tx=0x7f371c00bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2f1b930cba71471 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:54.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.133+0000 7f372dffb640 1 -- 172.21.15.6:0/3035179611 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f37240ef8e0 con 0x7f373006ea30 2024-09-12T03:24:54.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.133+0000 7f3734ebc640 1 --2- 172.21.15.6:0/8981992 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e43340 0x555770e4d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:54.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.134+0000 7f371affd640 1 -- 172.21.15.6:0/8981992 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c00dbe0 con 0x555770e4dc00 2024-09-12T03:24:54.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.134+0000 7f371affd640 1 -- 172.21.15.6:0/8981992 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371c00dd80 con 0x555770e4dc00 2024-09-12T03:24:54.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.134+0000 7f371affd640 1 -- 172.21.15.6:0/8981992 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c004a00 con 0x555770e4dc00 2024-09-12T03:24:54.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.136+0000 7f37356bd640 1 --2- 172.21.15.6:0/3035179611 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f37200a2b80 0x7f37200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.138+0000 7f37394f9740 1 -- 172.21.15.6:0/8981992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e4dc00 msgr2=0x555770e4fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:54.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.138+0000 7f37394f9740 1 --2- 172.21.15.6:0/8981992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e4dc00 0x555770e4fff0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f371c00ba10 tx=0x7f371c00bed0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.138+0000 7f37394f9740 1 -- 172.21.15.6:0/8981992 shutdown_connections 2024-09-12T03:24:54.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.138+0000 7f37394f9740 1 --2- 172.21.15.6:0/8981992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e4dc00 0x555770e4fff0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.138+0000 7f37394f9740 1 --2- 172.21.15.6:0/8981992 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e43340 0x555770e4d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.138+0000 7f37394f9740 1 --2- 172.21.15.6:0/8981992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e42a20 0x555770e42e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.138+0000 7f37394f9740 1 -- 172.21.15.6:0/8981992 >> 172.21.15.6:0/8981992 conn(0x555770e3efb0 msgr2=0x555770e3f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:54.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.139+0000 7f37356bd640 1 --2- 172.21.15.6:0/3035179611 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f37200a2b80 0x7f37200a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f372802bfb0 tx=0x7f372802d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:54.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.139+0000 7f37394f9740 1 -- 172.21.15.6:0/8981992 shutdown_connections 2024-09-12T03:24:54.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.139+0000 7f37394f9740 1 -- 172.21.15.6:0/8981992 wait complete. 2024-09-12T03:24:54.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.140+0000 7f37394f9740 1 -- start start 2024-09-12T03:24:54.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.140+0000 7f37394f9740 1 Processor -- start 2024-09-12T03:24:54.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.140+0000 7f37394f9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e42a20 0x555770e4aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.140+0000 7f37394f9740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e43340 0x555770e4aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.140+0000 7f37394f9740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e4dc00 0x555770e4b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.140+0000 7f37394f9740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x555770e507d0 con 0x555770e43340 2024-09-12T03:24:54.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.140+0000 7f37394f9740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x555770d2f900 con 0x555770e4dc00 2024-09-12T03:24:54.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.140+0000 7f37394f9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555770e50950 con 0x555770e42a20 2024-09-12T03:24:54.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.140+0000 7f37394f9740 10 client.0 _ll_register_callbacks cb 0x555770d2d780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:54.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.141+0000 7f37356bd640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e4dc00 0x555770e4b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.141+0000 7f37356bd640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e4dc00 0x555770e4b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:39606/0 (socket says 172.21.15.6:39606) 2024-09-12T03:24:54.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.141+0000 7f37356bd640 1 -- 172.21.15.6:0/3013048394 learned_addr learned my addr 172.21.15.6:0/3013048394 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:54.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.143+0000 7f37356bd640 1 -- 172.21.15.6:0/3013048394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e42a20 msgr2=0x555770e4aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:54.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.143+0000 7f37356bd640 1 --2- 172.21.15.6:0/3013048394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e42a20 0x555770e4aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.143+0000 7f37356bd640 1 -- 172.21.15.6:0/3013048394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e43340 msgr2=0x555770e4aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:54.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.143+0000 7f37356bd640 1 --2- 172.21.15.6:0/3013048394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e43340 0x555770e4aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.143+0000 7f37356bd640 1 -- 172.21.15.6:0/3013048394 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555770d6fc50 con 0x555770e4dc00 2024-09-12T03:24:54.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.144+0000 7f37356bd640 1 --2- 172.21.15.6:0/3013048394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e4dc00 0x555770e4b530 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3728037dc0 tx=0x7f372802b5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62f85df633e1b8df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:54.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.144+0000 7f370ffff640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:24:54.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.145+0000 7f370ffff640 1 -- 172.21.15.6:0/3013048394 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728047060 con 0x555770e4dc00 2024-09-12T03:24:54.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.145+0000 7f37394f9740 1 -- 172.21.15.6:0/3013048394 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f371c00dd80 con 0x555770e4dc00 2024-09-12T03:24:54.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.145+0000 7f370ffff640 1 -- 172.21.15.6:0/3013048394 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f3728036da0 con 0x555770e4dc00 2024-09-12T03:24:54.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.146+0000 7f370ffff640 1 -- 172.21.15.6:0/3013048394 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3728042400 con 0x555770e4dc00 2024-09-12T03:24:54.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.149+0000 7f37394f9740 10 client.25388 fetch_fsmap learned FSMap version 27 2024-09-12T03:24:54.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.149+0000 7f37394f9740 1 -- 172.21.15.6:0/3013048394 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555770e5c4a0 con 0x555770e4dc00 2024-09-12T03:24:54.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.152+0000 7f370ffff640 1 -- 172.21.15.6:0/3013048394 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371c00dd80 con 0x555770e4dc00 2024-09-12T03:24:54.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.152+0000 7f370ffff640 1 -- 172.21.15.6:0/3013048394 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3728036070 con 0x555770e4dc00 2024-09-12T03:24:54.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.153+0000 7f37394f9740 10 client.25388 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:24:54.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.153+0000 7f37394f9740 10 client.25388 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:24:54.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.153+0000 7f37394f9740 20 client.25388 populate_metadata read hostname 'smithi006' 2024-09-12T03:24:54.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.153+0000 7f37394f9740 1 --2- 172.21.15.6:0/3013048394 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x555770e5de40 0x555770e60300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.153+0000 7f37394f9740 4 client.25388 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:24:54.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.153+0000 7f37394f9740 1 -- 172.21.15.6:0/3013048394 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555770db4890 con 0x555770e5de40 2024-09-12T03:24:54.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.153+0000 7f3734ebc640 1 --2- 172.21.15.6:0/3013048394 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x555770e5de40 0x555770e60300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.154+0000 7f370ffff640 10 client.25388 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:54.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.154+0000 7f3734ebc640 1 --2- 172.21.15.6:0/3013048394 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x555770e5de40 0x555770e60300 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b32c308e67a1b9ab server_cookie=22630d0fa0515b8d in_seq=0 out_seq=0 2024-09-12T03:24:54.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.155+0000 7f370ffff640 1 -- 172.21.15.6:0/3013048394 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555770db4890 con 0x555770e5de40 2024-09-12T03:24:54.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.155+0000 7f370ffff640 10 client.25388 handle_command_reply: tid=0 2024-09-12T03:24:54.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.155+0000 7f37394f9740 1 client.25388 shutdown 2024-09-12T03:24:54.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.155+0000 7f37394f9740 1 -- 172.21.15.6:0/3013048394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e4dc00 msgr2=0x555770e4b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:54.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.155+0000 7f37394f9740 1 --2- 172.21.15.6:0/3013048394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e4dc00 0x555770e4b530 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3728037dc0 tx=0x7f372802b5c0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.156+0000 7f37394f9740 1 -- 172.21.15.6:0/3013048394 shutdown_connections 2024-09-12T03:24:54.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.156+0000 7f37394f9740 1 --2- 172.21.15.6:0/3013048394 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x555770e5de40 0x555770e60300 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.156+0000 7f37394f9740 1 --2- 172.21.15.6:0/3013048394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e4dc00 0x555770e4b530 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.156+0000 7f37394f9740 1 --2- 172.21.15.6:0/3013048394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e43340 0x555770e4aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.156+0000 7f37394f9740 1 --2- 172.21.15.6:0/3013048394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e42a20 0x555770e4aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.156+0000 7f37394f9740 1 -- 172.21.15.6:0/3013048394 >> 172.21.15.6:0/3013048394 conn(0x555770e3efb0 msgr2=0x555770e44ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:54.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.156+0000 7f370ffff640 0 client.25388 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:54.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.161+0000 7f37394f9740 1 -- 172.21.15.6:0/3013048394 shutdown_connections 2024-09-12T03:24:54.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.161+0000 7f37394f9740 1 -- 172.21.15.6:0/3013048394 wait complete. 2024-09-12T03:24:54.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.162+0000 7f37394f9740 20 client.25388 trim_cache size 0 max 16384 2024-09-12T03:24:54.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.199+0000 7f37394f9740 1 Processor -- start 2024-09-12T03:24:54.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.200+0000 7f37394f9740 1 -- start start 2024-09-12T03:24:54.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.200+0000 7f37394f9740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e42a20 0x555770e454d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.200+0000 7f37394f9740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e43340 0x555770e4ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.200+0000 7f37394f9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e4dc00 0x555770e65c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.200+0000 7f37394f9740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3730076670 con 0x555770e42a20 2024-09-12T03:24:54.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.200+0000 7f37394f9740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f3730118c90 con 0x555770e43340 2024-09-12T03:24:54.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.200+0000 7f37394f9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f37301096f0 con 0x555770e4dc00 2024-09-12T03:24:54.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.201+0000 7f37356bd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e4dc00 0x555770e65c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.201+0000 7f3734ebc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e42a20 0x555770e454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.201+0000 7f3734ebc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e42a20 0x555770e454d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52724/0 (socket says 172.21.15.6:52724) 2024-09-12T03:24:54.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.201+0000 7f3734ebc640 1 -- 172.21.15.6:0/3580599008 learned_addr learned my addr 172.21.15.6:0/3580599008 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:54.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.202+0000 7f3734ebc640 1 -- 172.21.15.6:0/3580599008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e4dc00 msgr2=0x555770e65c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:54.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.202+0000 7f3734ebc640 1 --2- 172.21.15.6:0/3580599008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e4dc00 0x555770e65c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.202+0000 7f3734ebc640 1 -- 172.21.15.6:0/3580599008 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e43340 msgr2=0x555770e4ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:54.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.202+0000 7f3734ebc640 1 --2- 172.21.15.6:0/3580599008 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e43340 0x555770e4ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.202+0000 7f3734ebc640 1 -- 172.21.15.6:0/3580599008 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555770e49a60 con 0x555770e42a20 2024-09-12T03:24:54.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.202+0000 7f37356bd640 1 --2- 172.21.15.6:0/3580599008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e4dc00 0x555770e65c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:54.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.204+0000 7f3734ebc640 1 --2- 172.21.15.6:0/3580599008 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e42a20 0x555770e454d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3724009a20 tx=0x7f3724003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8cf65e82d55798 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:54.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.204+0000 7f371a7fc640 1 -- 172.21.15.6:0/3580599008 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37240c04c0 con 0x555770e42a20 2024-09-12T03:24:54.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.205+0000 7f371a7fc640 1 -- 172.21.15.6:0/3580599008 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37240c4d80 con 0x555770e42a20 2024-09-12T03:24:54.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.205+0000 7f371a7fc640 1 -- 172.21.15.6:0/3580599008 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37240b8780 con 0x555770e42a20 2024-09-12T03:24:54.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.206+0000 7f37394f9740 1 -- 172.21.15.6:0/3580599008 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e42a20 msgr2=0x555770e454d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:54.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.206+0000 7f37394f9740 1 --2- 172.21.15.6:0/3580599008 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e42a20 0x555770e454d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3724009a20 tx=0x7f3724003f10 comp rx=0 tx=0).stop 2024-09-12T03:24:54.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.206+0000 7f37394f9740 1 -- 172.21.15.6:0/3580599008 shutdown_connections 2024-09-12T03:24:54.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.206+0000 7f37394f9740 1 --2- 172.21.15.6:0/3580599008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e4dc00 0x555770e65c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.206+0000 7f37394f9740 1 --2- 172.21.15.6:0/3580599008 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e43340 0x555770e4ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.207+0000 7f37394f9740 1 --2- 172.21.15.6:0/3580599008 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e42a20 0x555770e454d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.207+0000 7f37394f9740 1 -- 172.21.15.6:0/3580599008 >> 172.21.15.6:0/3580599008 conn(0x555770e3efb0 msgr2=0x555770e4c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:54.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.207+0000 7f37394f9740 1 -- 172.21.15.6:0/3580599008 shutdown_connections 2024-09-12T03:24:54.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.207+0000 7f37394f9740 1 -- 172.21.15.6:0/3580599008 wait complete. 2024-09-12T03:24:54.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.207+0000 7f37394f9740 1 -- start start 2024-09-12T03:24:54.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.207+0000 7f37394f9740 1 Processor -- start 2024-09-12T03:24:54.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.208+0000 7f37394f9740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e42a20 0x555770e55ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.208+0000 7f37394f9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e43340 0x555770e563e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.208+0000 7f37394f9740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e4dc00 0x555770e51230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.209+0000 7f37394f9740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3730117150 con 0x555770e4dc00 2024-09-12T03:24:54.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.209+0000 7f37394f9740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f37301185c0 con 0x555770e42a20 2024-09-12T03:24:54.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.209+0000 7f37394f9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3730117820 con 0x555770e43340 2024-09-12T03:24:54.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.209+0000 7f37394f9740 10 client.0 _ll_register_callbacks cb 0x7f3728029f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:54.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.209+0000 7f3734ebc640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e42a20 0x555770e55ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.209+0000 7f372ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e43340 0x555770e563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.209+0000 7f3734ebc640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e42a20 0x555770e55ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:39634/0 (socket says 172.21.15.6:39634) 2024-09-12T03:24:54.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.209+0000 7f3734ebc640 1 -- 172.21.15.6:0/3773078362 learned_addr learned my addr 172.21.15.6:0/3773078362 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:24:54.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.210+0000 7f37356bd640 1 --2- 172.21.15.6:0/3773078362 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e4dc00 0x555770e51230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.210+0000 7f372ffff640 1 -- 172.21.15.6:0/3773078362 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e42a20 msgr2=0x555770e55ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:54.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.210+0000 7f372ffff640 1 --2- 172.21.15.6:0/3773078362 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e42a20 0x555770e55ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.210+0000 7f372ffff640 1 -- 172.21.15.6:0/3773078362 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e4dc00 msgr2=0x555770e51230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:54.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.210+0000 7f372ffff640 1 --2- 172.21.15.6:0/3773078362 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e4dc00 0x555770e51230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.210+0000 7f372ffff640 1 -- 172.21.15.6:0/3773078362 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555770e51a60 con 0x555770e43340 2024-09-12T03:24:54.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.210+0000 7f372ffff640 1 --2- 172.21.15.6:0/3773078362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e43340 0x555770e563e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x555770e4c0f0 tx=0x7f371c00dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e5114c24110ef11 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:54.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.210+0000 7f370ffff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:24:54.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.211+0000 7f370ffff640 1 -- 172.21.15.6:0/3773078362 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c004030 con 0x555770e43340 2024-09-12T03:24:54.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.211+0000 7f370ffff640 1 -- 172.21.15.6:0/3773078362 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f371c0041d0 con 0x555770e43340 2024-09-12T03:24:54.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.211+0000 7f37394f9740 1 -- 172.21.15.6:0/3773078362 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37240c4d80 con 0x555770e43340 2024-09-12T03:24:54.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.211+0000 7f370ffff640 1 -- 172.21.15.6:0/3773078362 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f371c02b050 con 0x555770e43340 2024-09-12T03:24:54.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.212+0000 7f37394f9740 10 client.25444 fetch_fsmap learned FSMap version 27 2024-09-12T03:24:54.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.212+0000 7f37394f9740 1 -- 172.21.15.6:0/3773078362 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555770ec8740 con 0x555770e43340 2024-09-12T03:24:54.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.214+0000 7f370ffff640 1 -- 172.21.15.6:0/3773078362 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37240c4d80 con 0x555770e43340 2024-09-12T03:24:54.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.214+0000 7f370ffff640 1 -- 172.21.15.6:0/3773078362 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f371c004b80 con 0x555770e43340 2024-09-12T03:24:54.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.214+0000 7f37394f9740 10 client.25444 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:24:54.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.214+0000 7f37394f9740 10 client.25444 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:24:54.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.214+0000 7f37394f9740 20 client.25444 populate_metadata read hostname 'smithi006' 2024-09-12T03:24:54.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.214+0000 7f37394f9740 1 --2- 172.21.15.6:0/3773078362 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x555770ec8ba0 0x555770ecaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.214+0000 7f37394f9740 4 client.25444 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-12T03:24:54.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.214+0000 7f37394f9740 1 -- 172.21.15.6:0/3773078362 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x555770daebe0 con 0x555770ec8ba0 2024-09-12T03:24:54.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.216+0000 7f3734ebc640 1 --2- 172.21.15.6:0/3773078362 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x555770ec8ba0 0x555770ecaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.220+0000 7f3734ebc640 1 --2- 172.21.15.6:0/3773078362 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x555770ec8ba0 0x555770ecaf90 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ab03d68265ef935 server_cookie=13844ad45688b499 in_seq=0 out_seq=0 2024-09-12T03:24:54.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:54.220+0000 7f370ffff640 10 client.25444 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:55.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:55 smithi118 ceph-mon[35048]: pgmap v449: 97 pgs: 97 active+clean; 13 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 684 KiB/s rd, 223 MiB/s wr, 22.36k op/s 2024-09-12T03:24:55.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:55 smithi006 ceph-mon[30991]: pgmap v449: 97 pgs: 97 active+clean; 13 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 684 KiB/s rd, 223 MiB/s wr, 22.36k op/s 2024-09-12T03:24:55.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:55 smithi161 ceph-mon[35485]: pgmap v449: 97 pgs: 97 active+clean; 13 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 684 KiB/s rd, 223 MiB/s wr, 22.36k op/s 2024-09-12T03:24:57.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:57 smithi118 ceph-mon[35048]: pgmap v450: 97 pgs: 97 active+clean; 13 GiB data, 87 GiB used, 986 GiB / 1.0 TiB avail; 864 KiB/s rd, 239 MiB/s wr, 24.26k op/s 2024-09-12T03:24:57.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:57 smithi006 ceph-mon[30991]: pgmap v450: 97 pgs: 97 active+clean; 13 GiB data, 87 GiB used, 986 GiB / 1.0 TiB avail; 864 KiB/s rd, 239 MiB/s wr, 24.26k op/s 2024-09-12T03:24:57.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:57 smithi161 ceph-mon[35485]: pgmap v450: 97 pgs: 97 active+clean; 13 GiB data, 87 GiB used, 986 GiB / 1.0 TiB avail; 864 KiB/s rd, 239 MiB/s wr, 24.26k op/s 2024-09-12T03:24:57.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.740+0000 7f370ffff640 1 -- 172.21.15.6:0/3773078362 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555770daebe0 con 0x555770ec8ba0 2024-09-12T03:24:57.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.740+0000 7f370ffff640 10 client.25444 handle_command_reply: tid=0 2024-09-12T03:24:57.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.740+0000 7f37394f9740 1 client.25444 shutdown 2024-09-12T03:24:57.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.740+0000 7f37394f9740 1 -- 172.21.15.6:0/3773078362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e43340 msgr2=0x555770e563e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:57.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.740+0000 7f37394f9740 1 --2- 172.21.15.6:0/3773078362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e43340 0x555770e563e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x555770e4c0f0 tx=0x7f371c00dc10 comp rx=0 tx=0).stop 2024-09-12T03:24:57.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.740+0000 7f37394f9740 1 -- 172.21.15.6:0/3773078362 shutdown_connections 2024-09-12T03:24:57.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.740+0000 7f37394f9740 1 --2- 172.21.15.6:0/3773078362 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x555770ec8ba0 0x555770ecaf90 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:57.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.740+0000 7f37394f9740 1 --2- 172.21.15.6:0/3773078362 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555770e4dc00 0x555770e51230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:57.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.740+0000 7f37394f9740 1 --2- 172.21.15.6:0/3773078362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555770e43340 0x555770e563e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:57.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.740+0000 7f37394f9740 1 --2- 172.21.15.6:0/3773078362 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555770e42a20 0x555770e55ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:57.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.740+0000 7f370ffff640 0 client.25444 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:24:57.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.740+0000 7f37394f9740 1 -- 172.21.15.6:0/3773078362 >> 172.21.15.6:0/3773078362 conn(0x555770e3efb0 msgr2=0x555770e4cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:57.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.740+0000 7f37394f9740 1 -- 172.21.15.6:0/3773078362 shutdown_connections 2024-09-12T03:24:57.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.740+0000 7f37394f9740 1 -- 172.21.15.6:0/3773078362 wait complete. 2024-09-12T03:24:57.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.741+0000 7f37394f9740 20 client.25444 trim_cache size 0 max 16384 2024-09-12T03:24:57.742 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:24:57.742 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 27, 2024-09-12T03:24:57.743 INFO:teuthology.orchestra.run.smithi006.stdout: "leader": 24494, 2024-09-12T03:24:57.743 INFO:teuthology.orchestra.run.smithi006.stdout: "set_version": 14, 2024-09-12T03:24:57.743 INFO:teuthology.orchestra.run.smithi006.stdout: "sets": { 2024-09-12T03:24:57.743 INFO:teuthology.orchestra.run.smithi006.stdout: "d89d56e5": { 2024-09-12T03:24:57.743 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 14, 2024-09-12T03:24:57.743 INFO:teuthology.orchestra.run.smithi006.stdout: "age_ref": 0.0, 2024-09-12T03:24:57.743 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:24:57.743 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "QUIESCED", 2024-09-12T03:24:57.743 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:24:57.743 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:24:57.743 INFO:teuthology.orchestra.run.smithi006.stdout: "timeout": 90.0, 2024-09-12T03:24:57.743 INFO:teuthology.orchestra.run.smithi006.stdout: "expiration": 140.2, 2024-09-12T03:24:57.743 INFO:teuthology.orchestra.run.smithi006.stdout: "members": { 2024-09-12T03:24:57.744 INFO:teuthology.orchestra.run.smithi006.stdout: "file:/": { 2024-09-12T03:24:57.744 INFO:teuthology.orchestra.run.smithi006.stdout: "excluded": false, 2024-09-12T03:24:57.744 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:24:57.744 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "QUIESCED", 2024-09-12T03:24:57.744 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:24:57.744 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:24:57.744 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:24:57.744 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:24:57.744 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:24:57.744 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:24:57.744 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:24:57.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.741+0000 7f3735ebe640 1 -- 172.21.15.6:0/3035179611 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f37200a2b80 msgr2=0x7f37200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:57.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.741+0000 7f3735ebe640 1 --2- 172.21.15.6:0/3035179611 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f37200a2b80 0x7f37200a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f372802bfb0 tx=0x7f372802d040 comp rx=0 tx=0).stop 2024-09-12T03:24:57.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.741+0000 7f3735ebe640 1 -- 172.21.15.6:0/3035179611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f373006ea30 msgr2=0x7f37301a71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:57.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.741+0000 7f3735ebe640 1 --2- 172.21.15.6:0/3035179611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f373006ea30 0x7f37301a71e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3724009230 tx=0x7f3724003b90 comp rx=0 tx=0).stop 2024-09-12T03:24:57.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.741+0000 7f3735ebe640 1 -- 172.21.15.6:0/3035179611 shutdown_connections 2024-09-12T03:24:57.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.741+0000 7f3735ebe640 1 --2- 172.21.15.6:0/3035179611 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f37200a2b80 0x7f37200a5040 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:57.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.741+0000 7f3735ebe640 1 --2- 172.21.15.6:0/3035179611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f373011b140 0x7f37301abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:57.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.741+0000 7f3735ebe640 1 --2- 172.21.15.6:0/3035179611 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3730110190 0x7f37301a7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:57.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.741+0000 7f3735ebe640 1 --2- 172.21.15.6:0/3035179611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f373006ea30 0x7f37301a71e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:57.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.741+0000 7f3735ebe640 1 -- 172.21.15.6:0/3035179611 >> 172.21.15.6:0/3035179611 conn(0x7f3730073180 msgr2=0x7f373011a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:57.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.741+0000 7f3735ebe640 1 -- 172.21.15.6:0/3035179611 shutdown_connections 2024-09-12T03:24:57.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:24:57.741+0000 7f3735ebe640 1 -- 172.21.15.6:0/3035179611 wait complete. 2024-09-12T03:24:58.572 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd89d56e5', quiesce took 9.5 seconds. Will release after: 10.649999999999999 2024-09-12T03:24:59.014 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:24:59.275 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:59.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:59 smithi006 ceph-mon[30991]: pgmap v451: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 538 KiB/s rd, 196 MiB/s wr, 19.76k op/s 2024-09-12T03:24:59.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:59 smithi006 ceph-mon[30991]: scrub summary: active paths [/] 2024-09-12T03:24:59.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:59 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:24:59.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:59 smithi006 ceph-mon[30991]: scrub summary: active paths [/] 2024-09-12T03:24:59.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:59 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:24:59.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:24:59 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:24:59.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:59 smithi118 ceph-mon[35048]: pgmap v451: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 538 KiB/s rd, 196 MiB/s wr, 19.76k op/s 2024-09-12T03:24:59.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:59 smithi118 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-12T03:24:59.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:59 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:24:59.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:59 smithi118 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-12T03:24:59.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:59 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:24:59.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:24:59 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:24:59.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:59 smithi161 ceph-mon[35485]: pgmap v451: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 538 KiB/s rd, 196 MiB/s wr, 19.76k op/s 2024-09-12T03:24:59.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:59 smithi161 ceph-mon[35485]: scrub summary: active paths [/] 2024-09-12T03:24:59.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:59 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:24:59.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:59 smithi161 ceph-mon[35485]: scrub summary: active paths [/] 2024-09-12T03:24:59.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:59 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:24:59.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:24:59 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:00.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.087+0000 7fd8406ff640 1 -- 172.21.15.6:0/2478214175 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd83811b140 msgr2=0x7fd83811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:00.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.087+0000 7fd8406ff640 1 --2- 172.21.15.6:0/2478214175 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd83811b140 0x7fd83811d530 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd830009be0 tx=0x7fd83002a710 comp rx=0 tx=0).stop 2024-09-12T03:25:00.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.089+0000 7fd8406ff640 1 -- 172.21.15.6:0/2478214175 shutdown_connections 2024-09-12T03:25:00.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.089+0000 7fd8406ff640 1 --2- 172.21.15.6:0/2478214175 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd83811b140 0x7fd83811d530 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.089+0000 7fd8406ff640 1 --2- 172.21.15.6:0/2478214175 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd838109830 0x7fd838109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.089+0000 7fd8406ff640 1 --2- 172.21.15.6:0/2478214175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd838109450 0x7fd838110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.089+0000 7fd8406ff640 1 -- 172.21.15.6:0/2478214175 >> 172.21.15.6:0/2478214175 conn(0x7fd83806e140 msgr2=0x7fd83806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:00.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.089+0000 7fd8406ff640 1 -- 172.21.15.6:0/2478214175 shutdown_connections 2024-09-12T03:25:00.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.089+0000 7fd8406ff640 1 -- 172.21.15.6:0/2478214175 wait complete. 2024-09-12T03:25:00.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.090+0000 7fd8406ff640 1 Processor -- start 2024-09-12T03:25:00.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.090+0000 7fd8406ff640 1 -- start start 2024-09-12T03:25:00.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.091+0000 7fd8406ff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd838109450 0x7fd8381a72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:00.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.091+0000 7fd8406ff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd838109830 0x7fd8381a7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:00.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.091+0000 7fd8406ff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd83811b140 0x7fd8381abb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:00.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.091+0000 7fd8406ff640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd838120da0 con 0x7fd838109830 2024-09-12T03:25:00.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.091+0000 7fd83dc9a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd838109830 0x7fd8381a7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:00.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.091+0000 7fd8406ff640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fd838120c20 con 0x7fd838109450 2024-09-12T03:25:00.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.091+0000 7fd8406ff640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd838120f20 con 0x7fd83811b140 2024-09-12T03:25:00.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.091+0000 7fd83dc9a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd838109830 0x7fd8381a7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54782/0 (socket says 172.21.15.6:54782) 2024-09-12T03:25:00.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.091+0000 7fd83dc9a640 1 -- 172.21.15.6:0/1889648608 learned_addr learned my addr 172.21.15.6:0/1889648608 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:00.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.091+0000 7fd83ec9c640 1 --2- 172.21.15.6:0/1889648608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd83811b140 0x7fd8381abb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:00.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.091+0000 7fd83e49b640 1 --2- 172.21.15.6:0/1889648608 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd838109450 0x7fd8381a72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:00.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.092+0000 7fd83dc9a640 1 -- 172.21.15.6:0/1889648608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd83811b140 msgr2=0x7fd8381abb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:00.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.092+0000 7fd83dc9a640 1 --2- 172.21.15.6:0/1889648608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd83811b140 0x7fd8381abb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.092+0000 7fd83dc9a640 1 -- 172.21.15.6:0/1889648608 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd838109450 msgr2=0x7fd8381a72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:00.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.092+0000 7fd83dc9a640 1 --2- 172.21.15.6:0/1889648608 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd838109450 0x7fd8381a72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.092+0000 7fd83dc9a640 1 -- 172.21.15.6:0/1889648608 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8381ac270 con 0x7fd838109830 2024-09-12T03:25:00.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.092+0000 7fd83dc9a640 1 --2- 172.21.15.6:0/1889648608 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd838109830 0x7fd8381a7800 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00e9f0 tx=0x7fd82c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:00.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.093+0000 7fd8377fe640 1 -- 172.21.15.6:0/1889648608 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c00cd80 con 0x7fd838109830 2024-09-12T03:25:00.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.093+0000 7fd8377fe640 1 -- 172.21.15.6:0/1889648608 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd82c010910 con 0x7fd838109830 2024-09-12T03:25:00.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.093+0000 7fd8406ff640 1 -- 172.21.15.6:0/1889648608 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8381ac560 con 0x7fd838109830 2024-09-12T03:25:00.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.093+0000 7fd8406ff640 1 -- 172.21.15.6:0/1889648608 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8381b3e40 con 0x7fd838109830 2024-09-12T03:25:00.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.095+0000 7fd8377fe640 1 -- 172.21.15.6:0/1889648608 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c018880 con 0x7fd838109830 2024-09-12T03:25:00.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.098+0000 7fd8377fe640 1 -- 172.21.15.6:0/1889648608 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd82c010ac0 con 0x7fd838109830 2024-09-12T03:25:00.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.099+0000 7fd842ad8740 1 Processor -- start 2024-09-12T03:25:00.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.100+0000 7fd8377fe640 1 --2- 172.21.15.6:0/1889648608 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd8200a2c40 0x7fd8200a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:00.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.100+0000 7fd8377fe640 1 -- 172.21.15.6:0/1889648608 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd82c0cf4c0 con 0x7fd838109830 2024-09-12T03:25:00.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.101+0000 7fd842ad8740 1 -- start start 2024-09-12T03:25:00.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.101+0000 7fd83ec9c640 1 --2- 172.21.15.6:0/1889648608 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd8200a2c40 0x7fd8200a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:00.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.101+0000 7fd842ad8740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c4310e4a0 0x559c4310e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:00.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.101+0000 7fd842ad8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c4310edc0 0x559c43119140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:00.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.101+0000 7fd842ad8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c43119680 0x559c4311ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:00.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.101+0000 7fd842ad8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x559c430742e0 con 0x559c43119680 2024-09-12T03:25:00.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.101+0000 7fd842ad8740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x559c4301bce0 con 0x559c4310e4a0 2024-09-12T03:25:00.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.101+0000 7fd842ad8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559c430702a0 con 0x559c4310edc0 2024-09-12T03:25:00.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.101+0000 7fd83ec9c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c43119680 0x559c4311ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:00.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.101+0000 7fd83ec9c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c43119680 0x559c4311ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54786/0 (socket says 172.21.15.6:54786) 2024-09-12T03:25:00.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.101+0000 7fd83ec9c640 1 -- 172.21.15.6:0/1221034432 learned_addr learned my addr 172.21.15.6:0/1221034432 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:00.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.101+0000 7fd83dc9a640 1 --2- 172.21.15.6:0/1221034432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c4310edc0 0x559c43119140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:00.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.102+0000 7fd83ec9c640 1 -- 172.21.15.6:0/1221034432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c4310edc0 msgr2=0x559c43119140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:00.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.102+0000 7fd83ec9c640 1 --2- 172.21.15.6:0/1221034432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c4310edc0 0x559c43119140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.102+0000 7fd83ec9c640 1 -- 172.21.15.6:0/1221034432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c4310e4a0 msgr2=0x559c4310e880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:25:00.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.102+0000 7fd83ec9c640 1 --2- 172.21.15.6:0/1221034432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c4310e4a0 0x559c4310e880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.102+0000 7fd83ec9c640 1 -- 172.21.15.6:0/1221034432 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c43000440 con 0x559c43119680 2024-09-12T03:25:00.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.102+0000 7fd83ec9c640 1 --2- 172.21.15.6:0/1221034432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c43119680 0x559c4311ba70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd830033000 tx=0x7fd8300306c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b819b864e0cdc63a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:00.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.103+0000 7fd83ec9c640 1 --2- 172.21.15.6:0/1889648608 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd8200a2c40 0x7fd8200a5100 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd83003afd0 tx=0x7fd83003f430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:00.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.105+0000 7fd834ff9640 1 -- 172.21.15.6:0/1221034432 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830036040 con 0x559c43119680 2024-09-12T03:25:00.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.105+0000 7fd834ff9640 1 -- 172.21.15.6:0/1221034432 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd830031ac0 con 0x559c43119680 2024-09-12T03:25:00.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.105+0000 7fd834ff9640 1 -- 172.21.15.6:0/1221034432 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83003adf0 con 0x559c43119680 2024-09-12T03:25:00.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.106+0000 7fd842ad8740 1 -- 172.21.15.6:0/1221034432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c43119680 msgr2=0x559c4311ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:00.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.106+0000 7fd842ad8740 1 --2- 172.21.15.6:0/1221034432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c43119680 0x559c4311ba70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd830033000 tx=0x7fd8300306c0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.106+0000 7fd842ad8740 1 -- 172.21.15.6:0/1221034432 shutdown_connections 2024-09-12T03:25:00.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.106+0000 7fd842ad8740 1 --2- 172.21.15.6:0/1221034432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c43119680 0x559c4311ba70 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.106+0000 7fd842ad8740 1 --2- 172.21.15.6:0/1221034432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c4310edc0 0x559c43119140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.106+0000 7fd842ad8740 1 --2- 172.21.15.6:0/1221034432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c4310e4a0 0x559c4310e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.106+0000 7fd842ad8740 1 -- 172.21.15.6:0/1221034432 >> 172.21.15.6:0/1221034432 conn(0x559c4310aa30 msgr2=0x559c4310ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:00.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.109+0000 7fd842ad8740 1 -- 172.21.15.6:0/1221034432 shutdown_connections 2024-09-12T03:25:00.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.109+0000 7fd842ad8740 1 -- 172.21.15.6:0/1221034432 wait complete. 2024-09-12T03:25:00.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.110+0000 7fd842ad8740 1 -- start start 2024-09-12T03:25:00.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.110+0000 7fd842ad8740 1 Processor -- start 2024-09-12T03:25:00.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.111+0000 7fd842ad8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c4310e4a0 0x559c43118fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:00.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.111+0000 7fd842ad8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c4310edc0 0x559c43116280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:00.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.111+0000 7fd83e49b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c4310e4a0 0x559c43118fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:00.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.111+0000 7fd83e49b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c4310e4a0 0x559c43118fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:54254/0 (socket says 172.21.15.6:54254) 2024-09-12T03:25:00.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.111+0000 7fd83e49b640 1 -- 172.21.15.6:0/3536871368 learned_addr learned my addr 172.21.15.6:0/3536871368 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:00.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.112+0000 7fd842ad8740 1 --2- 172.21.15.6:0/3536871368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c43119680 0x559c431167e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:00.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.112+0000 7fd842ad8740 1 -- 172.21.15.6:0/3536871368 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x559c4311c250 con 0x559c4310edc0 2024-09-12T03:25:00.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.112+0000 7fd842ad8740 1 -- 172.21.15.6:0/3536871368 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x559c4302f050 con 0x559c43119680 2024-09-12T03:25:00.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.112+0000 7fd842ad8740 1 -- 172.21.15.6:0/3536871368 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559c4311c3d0 con 0x559c4310e4a0 2024-09-12T03:25:00.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.112+0000 7fd842ad8740 10 client.0 _ll_register_callbacks cb 0x559c4306f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:00.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.113+0000 7fd83dc9a640 1 --2- 172.21.15.6:0/3536871368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c4310edc0 0x559c43116280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:00.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.113+0000 7fd83ec9c640 1 --2- 172.21.15.6:0/3536871368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c43119680 0x559c431167e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:00.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.113+0000 7fd83e49b640 1 -- 172.21.15.6:0/3536871368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c43119680 msgr2=0x559c431167e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:00.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.113+0000 7fd83e49b640 1 --2- 172.21.15.6:0/3536871368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c43119680 0x559c431167e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.113+0000 7fd83e49b640 1 -- 172.21.15.6:0/3536871368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c4310edc0 msgr2=0x559c43116280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:00.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.113+0000 7fd83e49b640 1 --2- 172.21.15.6:0/3536871368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c4310edc0 0x559c43116280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.113+0000 7fd83e49b640 1 -- 172.21.15.6:0/3536871368 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c42de53c0 con 0x559c4310e4a0 2024-09-12T03:25:00.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.114+0000 7fd83e49b640 1 --2- 172.21.15.6:0/3536871368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c4310e4a0 0x559c43118fd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd828011690 tx=0x7fd828011b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fc8e21cfbc8a2d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:00.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.114+0000 7fd81dffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:25:00.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.115+0000 7fd81dffb640 1 -- 172.21.15.6:0/3536871368 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82801f060 con 0x559c4310e4a0 2024-09-12T03:25:00.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.115+0000 7fd81dffb640 1 -- 172.21.15.6:0/3536871368 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd82800fac0 con 0x559c4310e4a0 2024-09-12T03:25:00.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.115+0000 7fd842ad8740 1 -- 172.21.15.6:0/3536871368 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd830031ac0 con 0x559c4310e4a0 2024-09-12T03:25:00.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.116+0000 7fd81dffb640 1 -- 172.21.15.6:0/3536871368 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd828011e90 con 0x559c4310e4a0 2024-09-12T03:25:00.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.116+0000 7fd842ad8740 10 client.25456 fetch_fsmap learned FSMap version 27 2024-09-12T03:25:00.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.116+0000 7fd842ad8740 1 -- 172.21.15.6:0/3536871368 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c43127c60 con 0x559c4310e4a0 2024-09-12T03:25:00.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.117+0000 7fd81dffb640 1 -- 172.21.15.6:0/3536871368 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd82800fcc0 con 0x559c4310e4a0 2024-09-12T03:25:00.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.118+0000 7fd81dffb640 1 -- 172.21.15.6:0/3536871368 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd828012450 con 0x559c4310e4a0 2024-09-12T03:25:00.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.118+0000 7fd842ad8740 10 client.25456 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:25:00.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.118+0000 7fd842ad8740 10 client.25456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:25:00.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.118+0000 7fd842ad8740 20 client.25456 populate_metadata read hostname 'smithi006' 2024-09-12T03:25:00.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.118+0000 7fd842ad8740 1 --2- 172.21.15.6:0/3536871368 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x559c43129070 0x559c4312b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:00.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.118+0000 7fd842ad8740 4 client.25456 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:25:00.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.118+0000 7fd842ad8740 1 -- 172.21.15.6:0/3536871368 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd830031ac0 con 0x559c43129070 2024-09-12T03:25:00.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.120+0000 7fd83dc9a640 1 --2- 172.21.15.6:0/3536871368 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x559c43129070 0x559c4312b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:00.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.121+0000 7fd83dc9a640 1 --2- 172.21.15.6:0/3536871368 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x559c43129070 0x559c4312b460 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8ffc2295a4c2773 server_cookie=abd71f85c1532a8b in_seq=0 out_seq=0 2024-09-12T03:25:00.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.121+0000 7fd81dffb640 10 client.25456 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:00.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.122+0000 7fd81dffb640 1 -- 172.21.15.6:0/3536871368 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fd830031ac0 con 0x559c43129070 2024-09-12T03:25:00.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.122+0000 7fd81dffb640 10 client.25456 handle_command_reply: tid=0 2024-09-12T03:25:00.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.122+0000 7fd842ad8740 1 client.25456 shutdown 2024-09-12T03:25:00.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.123+0000 7fd842ad8740 1 -- 172.21.15.6:0/3536871368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c4310e4a0 msgr2=0x559c43118fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:00.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.123+0000 7fd842ad8740 1 --2- 172.21.15.6:0/3536871368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c4310e4a0 0x559c43118fd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd828011690 tx=0x7fd828011b50 comp rx=0 tx=0).stop 2024-09-12T03:25:00.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.123+0000 7fd842ad8740 1 -- 172.21.15.6:0/3536871368 shutdown_connections 2024-09-12T03:25:00.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.123+0000 7fd842ad8740 1 --2- 172.21.15.6:0/3536871368 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x559c43129070 0x559c4312b460 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.124+0000 7fd842ad8740 1 --2- 172.21.15.6:0/3536871368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c43119680 0x559c431167e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.124+0000 7fd81dffb640 0 client.25456 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:00.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.124+0000 7fd842ad8740 1 --2- 172.21.15.6:0/3536871368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c4310edc0 0x559c43116280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.124+0000 7fd842ad8740 1 --2- 172.21.15.6:0/3536871368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c4310e4a0 0x559c43118fd0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.124+0000 7fd842ad8740 1 -- 172.21.15.6:0/3536871368 >> 172.21.15.6:0/3536871368 conn(0x559c4310aa30 msgr2=0x559c43110710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:00.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.124+0000 7fd842ad8740 1 -- 172.21.15.6:0/3536871368 shutdown_connections 2024-09-12T03:25:00.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.124+0000 7fd842ad8740 1 -- 172.21.15.6:0/3536871368 wait complete. 2024-09-12T03:25:00.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.125+0000 7fd842ad8740 20 client.25456 trim_cache size 0 max 16384 2024-09-12T03:25:00.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.160+0000 7fd842ad8740 1 Processor -- start 2024-09-12T03:25:00.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.160+0000 7fd842ad8740 1 -- start start 2024-09-12T03:25:00.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.161+0000 7fd842ad8740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c4310e4a0 0x559c431254b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:00.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.161+0000 7fd842ad8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c4310edc0 0x559c43114b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:00.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.161+0000 7fd842ad8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c43119680 0x559c43130ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:00.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.161+0000 7fd83e49b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c4310e4a0 0x559c431254b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:00.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.161+0000 7fd83e49b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c4310e4a0 0x559c431254b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:56218/0 (socket says 172.21.15.6:56218) 2024-09-12T03:25:00.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.161+0000 7fd83dc9a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c4310edc0 0x559c43114b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:00.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.161+0000 7fd83dc9a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c4310edc0 0x559c43114b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54816/0 (socket says 172.21.15.6:54816) 2024-09-12T03:25:00.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.161+0000 7fd83e49b640 1 -- 172.21.15.6:0/3200486758 learned_addr learned my addr 172.21.15.6:0/3200486758 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:00.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.161+0000 7fd842ad8740 1 -- 172.21.15.6:0/3200486758 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd8381becd0 con 0x559c4310edc0 2024-09-12T03:25:00.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.162+0000 7fd83ec9c640 1 --2- 172.21.15.6:0/3200486758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c43119680 0x559c43130ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:00.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.162+0000 7fd842ad8740 1 -- 172.21.15.6:0/3200486758 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fd8381bc5a0 con 0x559c4310e4a0 2024-09-12T03:25:00.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.162+0000 7fd842ad8740 1 -- 172.21.15.6:0/3200486758 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd8381ba410 con 0x559c43119680 2024-09-12T03:25:00.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.162+0000 7fd83dc9a640 1 -- 172.21.15.6:0/3200486758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c43119680 msgr2=0x559c43130ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:00.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.162+0000 7fd83dc9a640 1 --2- 172.21.15.6:0/3200486758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c43119680 0x559c43130ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.162+0000 7fd83dc9a640 1 -- 172.21.15.6:0/3200486758 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c4310e4a0 msgr2=0x559c431254b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:00.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.162+0000 7fd83dc9a640 1 --2- 172.21.15.6:0/3200486758 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c4310e4a0 0x559c431254b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.162+0000 7fd83dc9a640 1 -- 172.21.15.6:0/3200486758 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c43115230 con 0x559c4310edc0 2024-09-12T03:25:00.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.163+0000 7fd83dc9a640 1 --2- 172.21.15.6:0/3200486758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c4310edc0 0x559c43114b50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0ccf60 tx=0x7fd82c0cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bd4d31a93bfd0d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:00.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.163+0000 7fd81ffff640 1 -- 172.21.15.6:0/3200486758 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c099ff0 con 0x559c4310edc0 2024-09-12T03:25:00.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.163+0000 7fd81ffff640 1 -- 172.21.15.6:0/3200486758 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd82c0cd0a0 con 0x559c4310edc0 2024-09-12T03:25:00.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.163+0000 7fd81ffff640 1 -- 172.21.15.6:0/3200486758 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c09b3a0 con 0x559c4310edc0 2024-09-12T03:25:00.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.164+0000 7fd842ad8740 1 -- 172.21.15.6:0/3200486758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c4310edc0 msgr2=0x559c43114b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:00.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.164+0000 7fd842ad8740 1 --2- 172.21.15.6:0/3200486758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c4310edc0 0x559c43114b50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0ccf60 tx=0x7fd82c0cf040 comp rx=0 tx=0).stop 2024-09-12T03:25:00.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.165+0000 7fd842ad8740 1 -- 172.21.15.6:0/3200486758 shutdown_connections 2024-09-12T03:25:00.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.165+0000 7fd842ad8740 1 --2- 172.21.15.6:0/3200486758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c43119680 0x559c43130ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.165+0000 7fd842ad8740 1 --2- 172.21.15.6:0/3200486758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c4310edc0 0x559c43114b50 secure :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0ccf60 tx=0x7fd82c0cf040 comp rx=0 tx=0).stop 2024-09-12T03:25:00.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.165+0000 7fd842ad8740 1 --2- 172.21.15.6:0/3200486758 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c4310e4a0 0x559c431254b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.165+0000 7fd842ad8740 1 -- 172.21.15.6:0/3200486758 >> 172.21.15.6:0/3200486758 conn(0x559c4310aa30 msgr2=0x559c43111890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:00.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.165+0000 7fd842ad8740 1 -- 172.21.15.6:0/3200486758 shutdown_connections 2024-09-12T03:25:00.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.165+0000 7fd842ad8740 1 -- 172.21.15.6:0/3200486758 wait complete. 2024-09-12T03:25:00.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.166+0000 7fd842ad8740 1 -- start start 2024-09-12T03:25:00.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.166+0000 7fd842ad8740 1 Processor -- start 2024-09-12T03:25:00.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.167+0000 7fd842ad8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c4310e4a0 0x559c4311dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:00.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.167+0000 7fd842ad8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c43119680 0x559c4311e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:00.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.167+0000 7fd842ad8740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c43129070 0x559c4318cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:00.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.167+0000 7fd842ad8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd8381baa60 con 0x559c43119680 2024-09-12T03:25:00.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.167+0000 7fd842ad8740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fd8381bbed0 con 0x559c43129070 2024-09-12T03:25:00.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.167+0000 7fd842ad8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd8381bb130 con 0x559c4310e4a0 2024-09-12T03:25:00.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.167+0000 7fd83dc9a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c43119680 0x559c4311e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:00.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.167+0000 7fd83dc9a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c43119680 0x559c4311e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54822/0 (socket says 172.21.15.6:54822) 2024-09-12T03:25:00.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.167+0000 7fd83dc9a640 1 -- 172.21.15.6:0/2965788366 learned_addr learned my addr 172.21.15.6:0/2965788366 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:00.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.168+0000 7fd842ad8740 10 client.0 _ll_register_callbacks cb 0x7fd828009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:00.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.168+0000 7fd83ec9c640 1 --2- 172.21.15.6:0/2965788366 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c43129070 0x559c4318cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:00.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.168+0000 7fd83e49b640 1 --2- 172.21.15.6:0/2965788366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c4310e4a0 0x559c4311dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:00.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.168+0000 7fd83ec9c640 1 -- 172.21.15.6:0/2965788366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c4310e4a0 msgr2=0x559c4311dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:00.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.170+0000 7fd83ec9c640 1 --2- 172.21.15.6:0/2965788366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c4310e4a0 0x559c4311dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.170+0000 7fd83ec9c640 1 -- 172.21.15.6:0/2965788366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c43119680 msgr2=0x559c4311e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:00.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.170+0000 7fd83ec9c640 1 --2- 172.21.15.6:0/2965788366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c43119680 0x559c4311e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.170+0000 7fd83ec9c640 1 -- 172.21.15.6:0/2965788366 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c43116b50 con 0x559c43129070 2024-09-12T03:25:00.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.170+0000 7fd83e49b640 1 --2- 172.21.15.6:0/2965788366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c4310e4a0 0x559c4311dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:25:00.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.170+0000 7fd83ec9c640 1 --2- 172.21.15.6:0/2965788366 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c43129070 0x559c4318cc80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x559c4310e260 tx=0x7fd83002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9e85d7244ffb88c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:00.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.170+0000 7fd81dffb640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:25:00.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.171+0000 7fd83dc9a640 1 --2- 172.21.15.6:0/2965788366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c43119680 0x559c4311e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:00.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.172+0000 7fd81dffb640 1 -- 172.21.15.6:0/2965788366 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830036040 con 0x559c43129070 2024-09-12T03:25:00.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.172+0000 7fd81dffb640 1 -- 172.21.15.6:0/2965788366 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd830030b80 con 0x559c43129070 2024-09-12T03:25:00.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.172+0000 7fd842ad8740 1 -- 172.21.15.6:0/2965788366 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd82c0cd0a0 con 0x559c43129070 2024-09-12T03:25:00.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.172+0000 7fd81dffb640 1 -- 172.21.15.6:0/2965788366 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd830050070 con 0x559c43129070 2024-09-12T03:25:00.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.172+0000 7fd842ad8740 10 client.25403 fetch_fsmap learned FSMap version 27 2024-09-12T03:25:00.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.172+0000 7fd842ad8740 1 -- 172.21.15.6:0/2965788366 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c43194460 con 0x559c43129070 2024-09-12T03:25:00.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.173+0000 7fd81dffb640 1 -- 172.21.15.6:0/2965788366 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd830031440 con 0x559c43129070 2024-09-12T03:25:00.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.173+0000 7fd81dffb640 1 -- 172.21.15.6:0/2965788366 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd83003c040 con 0x559c43129070 2024-09-12T03:25:00.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.173+0000 7fd842ad8740 10 client.25403 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:25:00.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.173+0000 7fd842ad8740 10 client.25403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:25:00.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.173+0000 7fd842ad8740 20 client.25403 populate_metadata read hostname 'smithi006' 2024-09-12T03:25:00.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.173+0000 7fd842ad8740 1 --2- 172.21.15.6:0/2965788366 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x559c431947a0 0x559c43196b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:00.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.173+0000 7fd842ad8740 4 client.25403 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:25:00.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.173+0000 7fd842ad8740 1 -- 172.21.15.6:0/2965788366 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fd82c0cd0a0 con 0x559c431947a0 2024-09-12T03:25:00.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.173+0000 7fd83e49b640 1 --2- 172.21.15.6:0/2965788366 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x559c431947a0 0x559c43196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:00.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.174+0000 7fd81dffb640 10 client.25403 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:00.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.174+0000 7fd83e49b640 1 --2- 172.21.15.6:0/2965788366 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x559c431947a0 0x559c43196b90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0a006bb91b73ac6 server_cookie=4fac8ace52e92384 in_seq=0 out_seq=0 2024-09-12T03:25:00.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.174+0000 7fd81dffb640 1 -- 172.21.15.6:0/2965788366 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7fd82c0cd0a0 con 0x559c431947a0 2024-09-12T03:25:00.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.174+0000 7fd81dffb640 10 client.25403 handle_command_reply: tid=0 2024-09-12T03:25:00.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.174+0000 7fd842ad8740 1 client.25403 shutdown 2024-09-12T03:25:00.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.176+0000 7fd842ad8740 1 -- 172.21.15.6:0/2965788366 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c43129070 msgr2=0x559c4318cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:00.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.176+0000 7fd842ad8740 1 --2- 172.21.15.6:0/2965788366 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c43129070 0x559c4318cc80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x559c4310e260 tx=0x7fd83002a710 comp rx=0 tx=0).stop 2024-09-12T03:25:00.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.177+0000 7fd842ad8740 1 -- 172.21.15.6:0/2965788366 shutdown_connections 2024-09-12T03:25:00.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.177+0000 7fd842ad8740 1 --2- 172.21.15.6:0/2965788366 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x559c431947a0 0x559c43196b90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.177+0000 7fd842ad8740 1 --2- 172.21.15.6:0/2965788366 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c43129070 0x559c4318cc80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.177+0000 7fd81dffb640 0 client.25403 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:00.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.177+0000 7fd842ad8740 1 --2- 172.21.15.6:0/2965788366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559c43119680 0x559c4311e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.177+0000 7fd842ad8740 1 --2- 172.21.15.6:0/2965788366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c4310e4a0 0x559c4311dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.177+0000 7fd842ad8740 1 -- 172.21.15.6:0/2965788366 >> 172.21.15.6:0/2965788366 conn(0x559c4310aa30 msgr2=0x559c43109350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:00.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.178+0000 7fd842ad8740 1 -- 172.21.15.6:0/2965788366 shutdown_connections 2024-09-12T03:25:00.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.178+0000 7fd842ad8740 1 -- 172.21.15.6:0/2965788366 wait complete. 2024-09-12T03:25:00.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.178+0000 7fd842ad8740 20 client.25403 trim_cache size 0 max 16384 2024-09-12T03:25:00.179 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:25:00.179 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:25:00.179 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2024-09-12T03:25:00.179 INFO:teuthology.orchestra.run.smithi006.stdout: "9cc9dee5-d9ac-4c99-b9a3-da95838d23be": { 2024-09-12T03:25:00.179 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-09-12T03:25:00.179 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "9cc9dee5-d9ac-4c99-b9a3-da95838d23be", 2024-09-12T03:25:00.180 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2024-09-12T03:25:00.180 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:25:00.180 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:25:00.180 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:25:00.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.179+0000 7fd8406ff640 1 -- 172.21.15.6:0/1889648608 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd8200a2c40 msgr2=0x7fd8200a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:00.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.179+0000 7fd8406ff640 1 --2- 172.21.15.6:0/1889648608 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd8200a2c40 0x7fd8200a5100 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd83003afd0 tx=0x7fd83003f430 comp rx=0 tx=0).stop 2024-09-12T03:25:00.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.180+0000 7fd8406ff640 1 -- 172.21.15.6:0/1889648608 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd838109830 msgr2=0x7fd8381a7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:00.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.180+0000 7fd8406ff640 1 --2- 172.21.15.6:0/1889648608 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd838109830 0x7fd8381a7800 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00e9f0 tx=0x7fd82c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.180+0000 7fd8406ff640 1 -- 172.21.15.6:0/1889648608 shutdown_connections 2024-09-12T03:25:00.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.180+0000 7fd8406ff640 1 --2- 172.21.15.6:0/1889648608 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fd8200a2c40 0x7fd8200a5100 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.180+0000 7fd8406ff640 1 --2- 172.21.15.6:0/1889648608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd83811b140 0x7fd8381abb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.180+0000 7fd8406ff640 1 --2- 172.21.15.6:0/1889648608 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd838109830 0x7fd8381a7800 secure :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00e9f0 tx=0x7fd82c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.180+0000 7fd8406ff640 1 --2- 172.21.15.6:0/1889648608 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd838109450 0x7fd8381a72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:00.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.180+0000 7fd8406ff640 1 -- 172.21.15.6:0/1889648608 >> 172.21.15.6:0/1889648608 conn(0x7fd83806e140 msgr2=0x7fd83811be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:00.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.180+0000 7fd8406ff640 1 -- 172.21.15.6:0/1889648608 shutdown_connections 2024-09-12T03:25:00.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:00.180+0000 7fd8406ff640 1 -- 172.21.15.6:0/1889648608 wait complete. 2024-09-12T03:25:01.132 INFO:tasks.cephfs.filesystem:scrub status for tag:9cc9dee5-d9ac-4c99-b9a3-da95838d23be - {'path': '/', 'tag': '9cc9dee5-d9ac-4c99-b9a3-da95838d23be', 'options': 'recursive,force'} 2024-09-12T03:25:01.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:01 smithi006 ceph-mon[30991]: pgmap v452: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 984 GiB / 1.0 TiB avail; 700 KiB/s rd, 206 MiB/s wr, 21.28k op/s 2024-09-12T03:25:01.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:01 smithi118 ceph-mon[35048]: pgmap v452: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 984 GiB / 1.0 TiB avail; 700 KiB/s rd, 206 MiB/s wr, 21.28k op/s 2024-09-12T03:25:01.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:01 smithi161 ceph-mon[35485]: pgmap v452: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 984 GiB / 1.0 TiB avail; 700 KiB/s rd, 206 MiB/s wr, 21.28k op/s 2024-09-12T03:25:03.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:03 smithi118 ceph-mon[35048]: pgmap v453: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 359 KiB/s rd, 197 MiB/s wr, 21.12k op/s 2024-09-12T03:25:03.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:03 smithi006 ceph-mon[30991]: pgmap v453: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 359 KiB/s rd, 197 MiB/s wr, 21.12k op/s 2024-09-12T03:25:03.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:03 smithi161 ceph-mon[35485]: pgmap v453: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 359 KiB/s rd, 197 MiB/s wr, 21.12k op/s 2024-09-12T03:25:05.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:05 smithi118 ceph-mon[35048]: pgmap v454: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 366 KiB/s rd, 156 MiB/s wr, 17.12k op/s 2024-09-12T03:25:05.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:05 smithi006 ceph-mon[30991]: pgmap v454: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 366 KiB/s rd, 156 MiB/s wr, 17.12k op/s 2024-09-12T03:25:05.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:05 smithi161 ceph-mon[35485]: pgmap v454: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 366 KiB/s rd, 156 MiB/s wr, 17.12k op/s 2024-09-12T03:25:06.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:06 smithi118 ceph-mon[35048]: scrub summary: idle 2024-09-12T03:25:06.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:06 smithi118 ceph-mon[35048]: scrub summary: idle 2024-09-12T03:25:06.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:06 smithi006 ceph-mon[30991]: scrub summary: idle 2024-09-12T03:25:06.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:06 smithi006 ceph-mon[30991]: scrub summary: idle 2024-09-12T03:25:06.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:06 smithi161 ceph-mon[35485]: scrub summary: idle 2024-09-12T03:25:06.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:06 smithi161 ceph-mon[35485]: scrub summary: idle 2024-09-12T03:25:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:07 smithi118 ceph-mon[35048]: pgmap v455: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 366 KiB/s rd, 133 MiB/s wr, 14.80k op/s 2024-09-12T03:25:07.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:07 smithi118 ceph-mon[35048]: scrub summary: idle 2024-09-12T03:25:07.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:07 smithi006 ceph-mon[30991]: pgmap v455: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 366 KiB/s rd, 133 MiB/s wr, 14.80k op/s 2024-09-12T03:25:07.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:07 smithi006 ceph-mon[30991]: scrub summary: idle 2024-09-12T03:25:07.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:07 smithi161 ceph-mon[35485]: pgmap v455: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 366 KiB/s rd, 133 MiB/s wr, 14.80k op/s 2024-09-12T03:25:07.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:07 smithi161 ceph-mon[35485]: scrub summary: idle 2024-09-12T03:25:09.222 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:25:09.464 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:09 smithi118 ceph-mon[35048]: pgmap v456: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 188 KiB/s rd, 88 MiB/s wr, 9.88k op/s 2024-09-12T03:25:09.484 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:09.606 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:09 smithi006 ceph-mon[30991]: pgmap v456: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 188 KiB/s rd, 88 MiB/s wr, 9.88k op/s 2024-09-12T03:25:09.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:09 smithi161 ceph-mon[35485]: pgmap v456: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 188 KiB/s rd, 88 MiB/s wr, 9.88k op/s 2024-09-12T03:25:10.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.007+0000 7f7ee2223640 1 -- 172.21.15.6:0/1101955579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7edc100e20 msgr2=0x7f7edc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:10.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.007+0000 7f7ee2223640 1 --2- 172.21.15.6:0/1101955579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7edc100e20 0x7f7edc101200 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009900 tx=0x7f7ecc02a480 comp rx=0 tx=0).stop 2024-09-12T03:25:10.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.007+0000 7f7ee2223640 1 -- 172.21.15.6:0/1101955579 shutdown_connections 2024-09-12T03:25:10.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.007+0000 7f7ee2223640 1 --2- 172.21.15.6:0/1101955579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7edc102ef0 0x7f7edc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:10.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.007+0000 7f7ee2223640 1 --2- 172.21.15.6:0/1101955579 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7edc102570 0x7f7edc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:10.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.007+0000 7f7ee2223640 1 --2- 172.21.15.6:0/1101955579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7edc100e20 0x7f7edc101200 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:10.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.007+0000 7f7ee2223640 1 -- 172.21.15.6:0/1101955579 >> 172.21.15.6:0/1101955579 conn(0x7f7edc0fc7d0 msgr2=0x7f7edc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:10.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.007+0000 7f7ee2223640 1 -- 172.21.15.6:0/1101955579 shutdown_connections 2024-09-12T03:25:10.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.007+0000 7f7ee2223640 1 -- 172.21.15.6:0/1101955579 wait complete. 2024-09-12T03:25:10.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.007+0000 7f7ee2223640 1 Processor -- start 2024-09-12T03:25:10.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.008+0000 7f7ee2223640 1 -- start start 2024-09-12T03:25:10.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.008+0000 7f7ee2223640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7edc100e20 0x7f7edc1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:10.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.008+0000 7f7ee2223640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7edc102570 0x7f7edc199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:10.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.008+0000 7f7ee2223640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7edc102ef0 0x7f7edc1a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:10.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.008+0000 7f7ee2223640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc110f80 con 0x7f7edc102ef0 2024-09-12T03:25:10.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.008+0000 7f7ee2223640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc110e00 con 0x7f7edc102570 2024-09-12T03:25:10.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.008+0000 7f7ee2223640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc111100 con 0x7f7edc100e20 2024-09-12T03:25:10.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.008+0000 7f7edbfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7edc102ef0 0x7f7edc1a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:10.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.008+0000 7f7edb7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7edc100e20 0x7f7edc1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:10.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.008+0000 7f7edbfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7edc102ef0 0x7f7edc1a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57282/0 (socket says 172.21.15.6:57282) 2024-09-12T03:25:10.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.008+0000 7f7edbfff640 1 -- 172.21.15.6:0/1629631220 learned_addr learned my addr 172.21.15.6:0/1629631220 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:10.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.008+0000 7f7edaffd640 1 --2- 172.21.15.6:0/1629631220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7edc102570 0x7f7edc199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:10.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.008+0000 7f7edbfff640 1 -- 172.21.15.6:0/1629631220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7edc100e20 msgr2=0x7f7edc1992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:10.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.008+0000 7f7edbfff640 1 --2- 172.21.15.6:0/1629631220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7edc100e20 0x7f7edc1992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:10.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.008+0000 7f7edbfff640 1 -- 172.21.15.6:0/1629631220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7edc102570 msgr2=0x7f7edc199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:10.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.008+0000 7f7edbfff640 1 --2- 172.21.15.6:0/1629631220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7edc102570 0x7f7edc199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:10.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.008+0000 7f7edbfff640 1 -- 172.21.15.6:0/1629631220 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7edc1a4890 con 0x7f7edc102ef0 2024-09-12T03:25:10.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.008+0000 7f7edb7fe640 1 --2- 172.21.15.6:0/1629631220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7edc100e20 0x7f7edc1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:10.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.009+0000 7f7edbfff640 1 --2- 172.21.15.6:0/1629631220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7edc102ef0 0x7f7edc1a4350 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400cce0 tx=0x7f7ed400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:10.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.009+0000 7f7ed8ff9640 1 -- 172.21.15.6:0/1629631220 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed400f450 con 0x7f7edc102ef0 2024-09-12T03:25:10.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.009+0000 7f7ee2223640 1 -- 172.21.15.6:0/1629631220 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7edc1a4a60 con 0x7f7edc102ef0 2024-09-12T03:25:10.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.009+0000 7f7ed8ff9640 1 -- 172.21.15.6:0/1629631220 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ed400fad0 con 0x7f7edc102ef0 2024-09-12T03:25:10.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.009+0000 7f7ed8ff9640 1 -- 172.21.15.6:0/1629631220 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4018c10 con 0x7f7edc102ef0 2024-09-12T03:25:10.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.009+0000 7f7ee2223640 1 -- 172.21.15.6:0/1629631220 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7edc1a4f20 con 0x7f7edc102ef0 2024-09-12T03:25:10.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.010+0000 7f7ee2223640 1 -- 172.21.15.6:0/1629631220 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7edc101200 con 0x7f7edc102ef0 2024-09-12T03:25:10.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.012+0000 7f7ed8ff9640 1 -- 172.21.15.6:0/1629631220 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7ed400f5f0 con 0x7f7edc102ef0 2024-09-12T03:25:10.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.013+0000 7f7ed8ff9640 1 --2- 172.21.15.6:0/1629631220 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7ec80a2b80 0x7f7ec80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:10.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.013+0000 7f7ed8ff9640 1 -- 172.21.15.6:0/1629631220 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7ed40cf5e0 con 0x7f7edc102ef0 2024-09-12T03:25:10.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.013+0000 7f7edb7fe640 1 --2- 172.21.15.6:0/1629631220 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7ec80a2b80 0x7f7ec80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:10.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.013+0000 7f7edb7fe640 1 --2- 172.21.15.6:0/1629631220 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7ec80a2b80 0x7f7ec80a5040 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc000c00 tx=0x7f7ecc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:10.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.016+0000 7f7ed8ff9640 1 -- 172.21.15.6:0/1629631220 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7ed4014030 con 0x7f7edc102ef0 2024-09-12T03:25:10.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.227+0000 7f7ee2223640 1 -- 172.21.15.6:0/1629631220 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7edc1a51b0 con 0x7f7edc102ef0 2024-09-12T03:25:10.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.228+0000 7f7ed8ff9640 1 -- 172.21.15.6:0/1629631220 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11908 (secure 0 0 0) 0x7f7ed409c600 con 0x7f7edc102ef0 2024-09-12T03:25:10.229 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:25:10.229 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"btime":"2024-09-12T03:24:26:920603+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:24:26.080047+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24463]},"id":1}]} 2024-09-12T03:25:10.229 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 27 2024-09-12T03:25:10.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.231+0000 7f7ee2223640 1 -- 172.21.15.6:0/1629631220 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7ec80a2b80 msgr2=0x7f7ec80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:10.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.231+0000 7f7ee2223640 1 --2- 172.21.15.6:0/1629631220 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7ec80a2b80 0x7f7ec80a5040 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc000c00 tx=0x7f7ecc0058d0 comp rx=0 tx=0).stop 2024-09-12T03:25:10.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.231+0000 7f7ee2223640 1 -- 172.21.15.6:0/1629631220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7edc102ef0 msgr2=0x7f7edc1a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:10.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.231+0000 7f7ee2223640 1 --2- 172.21.15.6:0/1629631220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7edc102ef0 0x7f7edc1a4350 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400cce0 tx=0x7f7ed400a620 comp rx=0 tx=0).stop 2024-09-12T03:25:10.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.231+0000 7f7ee2223640 1 -- 172.21.15.6:0/1629631220 shutdown_connections 2024-09-12T03:25:10.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.231+0000 7f7ee2223640 1 --2- 172.21.15.6:0/1629631220 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f7ec80a2b80 0x7f7ec80a5040 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:10.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.231+0000 7f7ee2223640 1 --2- 172.21.15.6:0/1629631220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7edc102ef0 0x7f7edc1a4350 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:10.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.231+0000 7f7ee2223640 1 --2- 172.21.15.6:0/1629631220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7edc102570 0x7f7edc199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:10.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.231+0000 7f7ee2223640 1 --2- 172.21.15.6:0/1629631220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7edc100e20 0x7f7edc1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:10.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.231+0000 7f7ee2223640 1 -- 172.21.15.6:0/1629631220 >> 172.21.15.6:0/1629631220 conn(0x7f7edc0fc7d0 msgr2=0x7f7edc0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:10.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.231+0000 7f7ee2223640 1 -- 172.21.15.6:0/1629631220 shutdown_connections 2024-09-12T03:25:10.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:10.232+0000 7f7ee2223640 1 -- 172.21.15.6:0/1629631220 wait complete. 2024-09-12T03:25:11.325 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id d89d56e5 --release --await' 2024-09-12T03:25:11.325 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id d89d56e5 --release --await 2024-09-12T03:25:11.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:11 smithi118 ceph-mon[35048]: pgmap v457: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 984 GiB / 1.0 TiB avail; 173 KiB/s rd, 76 MiB/s wr, 8.66k op/s 2024-09-12T03:25:11.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:11 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1629631220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:25:11.582 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:11.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:11 smithi006 ceph-mon[30991]: pgmap v457: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 984 GiB / 1.0 TiB avail; 173 KiB/s rd, 76 MiB/s wr, 8.66k op/s 2024-09-12T03:25:11.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:11 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1629631220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:25:11.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:11 smithi161 ceph-mon[35485]: pgmap v457: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 984 GiB / 1.0 TiB avail; 173 KiB/s rd, 76 MiB/s wr, 8.66k op/s 2024-09-12T03:25:11.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:11 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1629631220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:25:12.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.092+0000 7f6742205640 1 -- 172.21.15.6:0/2981098446 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f673c10c170 msgr2=0x7f673c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:12.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.092+0000 7f6742205640 1 --2- 172.21.15.6:0/2981098446 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f673c10c170 0x7f673c10e610 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6734009ca0 tx=0x7f673402a950 comp rx=0 tx=0).stop 2024-09-12T03:25:12.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.092+0000 7f6742205640 1 -- 172.21.15.6:0/2981098446 shutdown_connections 2024-09-12T03:25:12.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.092+0000 7f6742205640 1 --2- 172.21.15.6:0/2981098446 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f673c10c170 0x7f673c10e610 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.092+0000 7f6742205640 1 --2- 172.21.15.6:0/2981098446 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f673c103660 0x7f673c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.092+0000 7f6742205640 1 --2- 172.21.15.6:0/2981098446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f673c102d40 0x7f673c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.092+0000 7f6742205640 1 -- 172.21.15.6:0/2981098446 >> 172.21.15.6:0/2981098446 conn(0x7f673c0fc7a0 msgr2=0x7f673c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:12.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.092+0000 7f6742205640 1 -- 172.21.15.6:0/2981098446 shutdown_connections 2024-09-12T03:25:12.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.092+0000 7f6742205640 1 -- 172.21.15.6:0/2981098446 wait complete. 2024-09-12T03:25:12.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.093+0000 7f6742205640 1 Processor -- start 2024-09-12T03:25:12.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.093+0000 7f6742205640 1 -- start start 2024-09-12T03:25:12.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.093+0000 7f6742205640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f673c102d40 0x7f673c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:12.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.093+0000 7f6742205640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f673c103660 0x7f673c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:12.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.093+0000 7f6742205640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f673c10c170 0x7f673c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:12.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.093+0000 7f6742205640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f673c111f70 con 0x7f673c103660 2024-09-12T03:25:12.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.093+0000 7f6742205640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f673c111df0 con 0x7f673c102d40 2024-09-12T03:25:12.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.093+0000 7f6742205640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f673c1120f0 con 0x7f673c10c170 2024-09-12T03:25:12.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.093+0000 7f673affd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f673c103660 0x7f673c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:12.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.094+0000 7f673b7fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f673c102d40 0x7f673c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:12.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.094+0000 7f673affd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f673c103660 0x7f673c100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57286/0 (socket says 172.21.15.6:57286) 2024-09-12T03:25:12.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.094+0000 7f673affd640 1 -- 172.21.15.6:0/1112669051 learned_addr learned my addr 172.21.15.6:0/1112669051 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:12.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.094+0000 7f673bfff640 1 --2- 172.21.15.6:0/1112669051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f673c10c170 0x7f673c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:12.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.094+0000 7f673affd640 1 -- 172.21.15.6:0/1112669051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f673c10c170 msgr2=0x7f673c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:12.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.094+0000 7f673affd640 1 --2- 172.21.15.6:0/1112669051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f673c10c170 0x7f673c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.094+0000 7f673affd640 1 -- 172.21.15.6:0/1112669051 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f673c102d40 msgr2=0x7f673c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:12.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.094+0000 7f673affd640 1 --2- 172.21.15.6:0/1112669051 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f673c102d40 0x7f673c1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.094+0000 7f673affd640 1 -- 172.21.15.6:0/1112669051 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673c1a68b0 con 0x7f673c103660 2024-09-12T03:25:12.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.094+0000 7f673bfff640 1 --2- 172.21.15.6:0/1112669051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f673c10c170 0x7f673c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:12.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.094+0000 7f673affd640 1 --2- 172.21.15.6:0/1112669051 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f673c103660 0x7f673c100ee0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f672400d740 tx=0x7f672400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:12.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.094+0000 7f6738ff9640 1 -- 172.21.15.6:0/1112669051 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724017070 con 0x7f673c103660 2024-09-12T03:25:12.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.094+0000 7f6742205640 1 -- 172.21.15.6:0/1112669051 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673c1a6ba0 con 0x7f673c103660 2024-09-12T03:25:12.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.094+0000 7f6738ff9640 1 -- 172.21.15.6:0/1112669051 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f672400de50 con 0x7f673c103660 2024-09-12T03:25:12.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.094+0000 7f6738ff9640 1 -- 172.21.15.6:0/1112669051 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672400f620 con 0x7f673c103660 2024-09-12T03:25:12.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.094+0000 7f6742205640 1 -- 172.21.15.6:0/1112669051 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f673c1a70e0 con 0x7f673c103660 2024-09-12T03:25:12.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.097+0000 7f6738ff9640 1 -- 172.21.15.6:0/1112669051 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6724009c30 con 0x7f673c103660 2024-09-12T03:25:12.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.097+0000 7f67445de740 1 Processor -- start 2024-09-12T03:25:12.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.097+0000 7f67445de740 1 -- start start 2024-09-12T03:25:12.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.097+0000 7f67445de740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a7c3a3b060 0x55a7c3a3b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:12.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.097+0000 7f67445de740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a3b980 0x55a7c3a45d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:12.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.097+0000 7f67445de740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a46240 0x55a7c3a48630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:12.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.097+0000 7f67445de740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a7c39a0620 con 0x55a7c3a3b980 2024-09-12T03:25:12.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.097+0000 7f67445de740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55a7c3947e50 con 0x55a7c3a3b060 2024-09-12T03:25:12.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.097+0000 7f67445de740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a7c399c5e0 con 0x55a7c3a46240 2024-09-12T03:25:12.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.097+0000 7f673b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a3b980 0x55a7c3a45d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:12.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.097+0000 7f673b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a3b980 0x55a7c3a45d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57292/0 (socket says 172.21.15.6:57292) 2024-09-12T03:25:12.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.097+0000 7f673b7fe640 1 -- 172.21.15.6:0/3003381795 learned_addr learned my addr 172.21.15.6:0/3003381795 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:12.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.098+0000 7f673affd640 1 --2- 172.21.15.6:0/3003381795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a46240 0x55a7c3a48630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:12.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.098+0000 7f6738ff9640 1 --2- 172.21.15.6:0/1112669051 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f67280a2b80 0x7f67280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:12.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.098+0000 7f673bfff640 1 --2- 172.21.15.6:0/3003381795 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a7c3a3b060 0x55a7c3a3b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:12.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.098+0000 7f673b7fe640 1 -- 172.21.15.6:0/3003381795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a46240 msgr2=0x55a7c3a48630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:12.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.098+0000 7f673bfff640 1 --2- 172.21.15.6:0/1112669051 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f67280a2b80 0x7f67280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:12.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.098+0000 7f673b7fe640 1 --2- 172.21.15.6:0/3003381795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a46240 0x55a7c3a48630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.098+0000 7f673b7fe640 1 -- 172.21.15.6:0/3003381795 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a7c3a3b060 msgr2=0x55a7c3a3b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:12.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.098+0000 7f673b7fe640 1 --2- 172.21.15.6:0/3003381795 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a7c3a3b060 0x55a7c3a3b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.098+0000 7f673b7fe640 1 -- 172.21.15.6:0/3003381795 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7c3938d40 con 0x55a7c3a3b980 2024-09-12T03:25:12.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.098+0000 7f673affd640 1 --2- 172.21.15.6:0/3003381795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a46240 0x55a7c3a48630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:25:12.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.098+0000 7f6738ff9640 1 -- 172.21.15.6:0/1112669051 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f67240ceab0 con 0x7f673c103660 2024-09-12T03:25:12.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.098+0000 7f673b7fe640 1 --2- 172.21.15.6:0/3003381795 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a3b980 0x55a7c3a45d00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f672c009d20 tx=0x7f672c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65ceec0e50ce11a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:12.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.098+0000 7f6731ffb640 1 -- 172.21.15.6:0/3003381795 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c01e070 con 0x55a7c3a3b980 2024-09-12T03:25:12.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.098+0000 7f673bfff640 1 --2- 172.21.15.6:0/1112669051 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f67280a2b80 0x7f67280a5040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6734009430 tx=0x7f6734005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:12.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.098+0000 7f6731ffb640 1 -- 172.21.15.6:0/3003381795 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f672c019b10 con 0x55a7c3a3b980 2024-09-12T03:25:12.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.098+0000 7f6731ffb640 1 -- 172.21.15.6:0/3003381795 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c00ebb0 con 0x55a7c3a3b980 2024-09-12T03:25:12.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.099+0000 7f67445de740 1 -- 172.21.15.6:0/3003381795 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a3b980 msgr2=0x55a7c3a45d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:12.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.099+0000 7f67445de740 1 --2- 172.21.15.6:0/3003381795 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a3b980 0x55a7c3a45d00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f672c009d20 tx=0x7f672c011810 comp rx=0 tx=0).stop 2024-09-12T03:25:12.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.099+0000 7f67445de740 1 -- 172.21.15.6:0/3003381795 shutdown_connections 2024-09-12T03:25:12.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.099+0000 7f67445de740 1 --2- 172.21.15.6:0/3003381795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a46240 0x55a7c3a48630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.099+0000 7f67445de740 1 --2- 172.21.15.6:0/3003381795 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a3b980 0x55a7c3a45d00 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.099+0000 7f67445de740 1 --2- 172.21.15.6:0/3003381795 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a7c3a3b060 0x55a7c3a3b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.099+0000 7f67445de740 1 -- 172.21.15.6:0/3003381795 >> 172.21.15.6:0/3003381795 conn(0x55a7c3a375f0 msgr2=0x55a7c3a379d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:12.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.099+0000 7f67445de740 1 -- 172.21.15.6:0/3003381795 shutdown_connections 2024-09-12T03:25:12.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.099+0000 7f67445de740 1 -- 172.21.15.6:0/3003381795 wait complete. 2024-09-12T03:25:12.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.100+0000 7f67445de740 1 -- start start 2024-09-12T03:25:12.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.100+0000 7f67445de740 1 Processor -- start 2024-09-12T03:25:12.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.100+0000 7f67445de740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a3b060 0x55a7c3a430a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:12.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.100+0000 7f67445de740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a7c3a3b980 0x55a7c3a435e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:12.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.100+0000 7f67445de740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a46240 0x55a7c3a43b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:12.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.100+0000 7f67445de740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a7c3a48e10 con 0x55a7c3a3b060 2024-09-12T03:25:12.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.100+0000 7f67445de740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55a7c3928f10 con 0x55a7c3a3b980 2024-09-12T03:25:12.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.100+0000 7f67445de740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a7c3a48f90 con 0x55a7c3a46240 2024-09-12T03:25:12.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.100+0000 7f67445de740 10 client.0 _ll_register_callbacks cb 0x55a7c394cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:12.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.100+0000 7f673b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a3b060 0x55a7c3a430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:12.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.100+0000 7f673b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a3b060 0x55a7c3a430a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57306/0 (socket says 172.21.15.6:57306) 2024-09-12T03:25:12.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.100+0000 7f673b7fe640 1 -- 172.21.15.6:0/707851451 learned_addr learned my addr 172.21.15.6:0/707851451 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:12.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.100+0000 7f673bfff640 1 --2- 172.21.15.6:0/707851451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a46240 0x55a7c3a43b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:12.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.100+0000 7f673affd640 1 --2- 172.21.15.6:0/707851451 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a7c3a3b980 0x55a7c3a435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:12.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.101+0000 7f673b7fe640 1 -- 172.21.15.6:0/707851451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a46240 msgr2=0x55a7c3a43b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:12.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.101+0000 7f673b7fe640 1 --2- 172.21.15.6:0/707851451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a46240 0x55a7c3a43b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.101+0000 7f673b7fe640 1 -- 172.21.15.6:0/707851451 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a7c3a3b980 msgr2=0x55a7c3a435e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:12.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.101+0000 7f673b7fe640 1 --2- 172.21.15.6:0/707851451 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a7c3a3b980 0x55a7c3a435e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.101+0000 7f673b7fe640 1 -- 172.21.15.6:0/707851451 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7c36af580 con 0x55a7c3a3b060 2024-09-12T03:25:12.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.101+0000 7f673affd640 1 --2- 172.21.15.6:0/707851451 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a7c3a3b980 0x55a7c3a435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:25:12.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.101+0000 7f673b7fe640 1 --2- 172.21.15.6:0/707851451 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a3b060 0x55a7c3a430a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f672c019660 tx=0x7f672c019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e840435e0524128 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:12.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.101+0000 7f671affd640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:25:12.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.101+0000 7f671affd640 1 -- 172.21.15.6:0/707851451 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c027020 con 0x55a7c3a3b060 2024-09-12T03:25:12.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.101+0000 7f67445de740 1 -- 172.21.15.6:0/707851451 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f672c019b10 con 0x55a7c3a3b060 2024-09-12T03:25:12.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.101+0000 7f671affd640 1 -- 172.21.15.6:0/707851451 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f672c00eaf0 con 0x55a7c3a3b060 2024-09-12T03:25:12.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.101+0000 7f671affd640 1 -- 172.21.15.6:0/707851451 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f672c01e070 con 0x55a7c3a3b060 2024-09-12T03:25:12.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.101+0000 7f67445de740 10 client.15864 fetch_fsmap learned FSMap version 27 2024-09-12T03:25:12.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.101+0000 7f67445de740 1 -- 172.21.15.6:0/707851451 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a7c3779960 con 0x55a7c3a3b060 2024-09-12T03:25:12.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.101+0000 7f671affd640 1 -- 172.21.15.6:0/707851451 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f672c019b10 con 0x55a7c3a3b060 2024-09-12T03:25:12.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.102+0000 7f671affd640 1 -- 172.21.15.6:0/707851451 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f672c004750 con 0x55a7c3a3b060 2024-09-12T03:25:12.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.102+0000 7f67445de740 10 client.15864 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:25:12.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.102+0000 7f67445de740 10 client.15864 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:25:12.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.102+0000 7f67445de740 20 client.15864 populate_metadata read hostname 'smithi006' 2024-09-12T03:25:12.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.102+0000 7f67445de740 1 --2- 172.21.15.6:0/707851451 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a7c3a55c80 0x55a7c3a58070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:12.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.102+0000 7f67445de740 4 client.15864 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:25:12.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.102+0000 7f67445de740 1 -- 172.21.15.6:0/707851451 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7c39adb90 con 0x55a7c3a55c80 2024-09-12T03:25:12.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.102+0000 7f673affd640 1 --2- 172.21.15.6:0/707851451 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a7c3a55c80 0x55a7c3a58070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:12.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.102+0000 7f671affd640 10 client.15864 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:12.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.102+0000 7f673affd640 1 --2- 172.21.15.6:0/707851451 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a7c3a55c80 0x55a7c3a58070 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38c47eca75af0390 server_cookie=eb34a55f6ddb6f81 in_seq=0 out_seq=0 2024-09-12T03:25:12.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.103+0000 7f671affd640 1 -- 172.21.15.6:0/707851451 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a7c39adb90 con 0x55a7c3a55c80 2024-09-12T03:25:12.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.103+0000 7f671affd640 10 client.15864 handle_command_reply: tid=0 2024-09-12T03:25:12.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.103+0000 7f67445de740 1 client.15864 shutdown 2024-09-12T03:25:12.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.103+0000 7f67445de740 1 -- 172.21.15.6:0/707851451 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a3b060 msgr2=0x55a7c3a430a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:12.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.103+0000 7f67445de740 1 --2- 172.21.15.6:0/707851451 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a3b060 0x55a7c3a430a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f672c019660 tx=0x7f672c019690 comp rx=0 tx=0).stop 2024-09-12T03:25:12.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.103+0000 7f67445de740 1 -- 172.21.15.6:0/707851451 shutdown_connections 2024-09-12T03:25:12.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.103+0000 7f67445de740 1 --2- 172.21.15.6:0/707851451 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a7c3a55c80 0x55a7c3a58070 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.103+0000 7f67445de740 1 --2- 172.21.15.6:0/707851451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a46240 0x55a7c3a43b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.103+0000 7f67445de740 1 --2- 172.21.15.6:0/707851451 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a7c3a3b980 0x55a7c3a435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.103+0000 7f671affd640 0 client.15864 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:12.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.103+0000 7f67445de740 1 --2- 172.21.15.6:0/707851451 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a3b060 0x55a7c3a430a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.103+0000 7f67445de740 1 -- 172.21.15.6:0/707851451 >> 172.21.15.6:0/707851451 conn(0x55a7c3a375f0 msgr2=0x55a7c3a3d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:12.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.103+0000 7f67445de740 1 -- 172.21.15.6:0/707851451 shutdown_connections 2024-09-12T03:25:12.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.103+0000 7f67445de740 1 -- 172.21.15.6:0/707851451 wait complete. 2024-09-12T03:25:12.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.104+0000 7f67445de740 20 client.15864 trim_cache size 0 max 16384 2024-09-12T03:25:12.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.121+0000 7f67445de740 1 Processor -- start 2024-09-12T03:25:12.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.121+0000 7f67445de740 1 -- start start 2024-09-12T03:25:12.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.121+0000 7f67445de740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a3b060 0x55a7c3a42320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:12.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.121+0000 7f67445de740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a3b980 0x55a7c3a36640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:12.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.121+0000 7f67445de740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a7c3a46240 0x55a7c3a5d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:12.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.121+0000 7f67445de740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f673c1b1f20 con 0x55a7c3a3b060 2024-09-12T03:25:12.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.121+0000 7f67445de740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f673c1af7f0 con 0x55a7c3a46240 2024-09-12T03:25:12.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.121+0000 7f67445de740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f673c1ad660 con 0x55a7c3a3b980 2024-09-12T03:25:12.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.121+0000 7f673b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a3b060 0x55a7c3a42320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:12.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.121+0000 7f673b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a3b060 0x55a7c3a42320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57308/0 (socket says 172.21.15.6:57308) 2024-09-12T03:25:12.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.121+0000 7f673b7fe640 1 -- 172.21.15.6:0/4029125121 learned_addr learned my addr 172.21.15.6:0/4029125121 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:12.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.121+0000 7f673affd640 1 --2- 172.21.15.6:0/4029125121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a3b980 0x55a7c3a36640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:12.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.121+0000 7f673b7fe640 1 -- 172.21.15.6:0/4029125121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a3b980 msgr2=0x55a7c3a36640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:12.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.121+0000 7f673b7fe640 1 --2- 172.21.15.6:0/4029125121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a3b980 0x55a7c3a36640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.121+0000 7f673b7fe640 1 -- 172.21.15.6:0/4029125121 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a7c3a46240 msgr2=0x55a7c3a5d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:12.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.121+0000 7f673b7fe640 1 --2- 172.21.15.6:0/4029125121 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a7c3a46240 0x55a7c3a5d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.121+0000 7f673b7fe640 1 -- 172.21.15.6:0/4029125121 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7c3a37420 con 0x55a7c3a3b060 2024-09-12T03:25:12.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.122+0000 7f673b7fe640 1 --2- 172.21.15.6:0/4029125121 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a3b060 0x55a7c3a42320 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f672c011850 tx=0x7f672c00e460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8163f9f20fcf6638 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:12.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.122+0000 7f67317fa640 1 -- 172.21.15.6:0/4029125121 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c015070 con 0x55a7c3a3b060 2024-09-12T03:25:12.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.122+0000 7f67317fa640 1 -- 172.21.15.6:0/4029125121 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f672c01f040 con 0x55a7c3a3b060 2024-09-12T03:25:12.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.122+0000 7f67317fa640 1 -- 172.21.15.6:0/4029125121 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c027040 con 0x55a7c3a3b060 2024-09-12T03:25:12.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.122+0000 7f67445de740 1 -- 172.21.15.6:0/4029125121 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a3b060 msgr2=0x55a7c3a42320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:12.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.122+0000 7f67445de740 1 --2- 172.21.15.6:0/4029125121 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a3b060 0x55a7c3a42320 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f672c011850 tx=0x7f672c00e460 comp rx=0 tx=0).stop 2024-09-12T03:25:12.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.122+0000 7f67445de740 1 -- 172.21.15.6:0/4029125121 shutdown_connections 2024-09-12T03:25:12.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.122+0000 7f67445de740 1 --2- 172.21.15.6:0/4029125121 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a7c3a46240 0x55a7c3a5d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.122+0000 7f67445de740 1 --2- 172.21.15.6:0/4029125121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a3b980 0x55a7c3a36640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.122+0000 7f67445de740 1 --2- 172.21.15.6:0/4029125121 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a3b060 0x55a7c3a42320 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.122+0000 7f67445de740 1 -- 172.21.15.6:0/4029125121 >> 172.21.15.6:0/4029125121 conn(0x55a7c3a375f0 msgr2=0x55a7c3a35f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:12.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.123+0000 7f67445de740 1 -- 172.21.15.6:0/4029125121 shutdown_connections 2024-09-12T03:25:12.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.123+0000 7f67445de740 1 -- 172.21.15.6:0/4029125121 wait complete. 2024-09-12T03:25:12.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.123+0000 7f67445de740 1 -- start start 2024-09-12T03:25:12.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.123+0000 7f67445de740 1 Processor -- start 2024-09-12T03:25:12.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.123+0000 7f67445de740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a3b060 0x55a7c3a4e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:12.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.123+0000 7f67445de740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a7c3a3b980 0x55a7c3a4e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:12.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.123+0000 7f67445de740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a46240 0x55a7c3a4b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:12.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.123+0000 7f67445de740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f673c1adcb0 con 0x55a7c3a46240 2024-09-12T03:25:12.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.123+0000 7f67445de740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f673c1af120 con 0x55a7c3a3b980 2024-09-12T03:25:12.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.123+0000 7f67445de740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f673c1ae380 con 0x55a7c3a3b060 2024-09-12T03:25:12.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.123+0000 7f67445de740 10 client.0 _ll_register_callbacks cb 0x7f672c00e530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:12.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.123+0000 7f673bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a46240 0x55a7c3a4b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:12.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.123+0000 7f673bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a46240 0x55a7c3a4b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57316/0 (socket says 172.21.15.6:57316) 2024-09-12T03:25:12.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.124+0000 7f673b7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a3b060 0x55a7c3a4e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:12.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.124+0000 7f673bfff640 1 -- 172.21.15.6:0/2239186419 learned_addr learned my addr 172.21.15.6:0/2239186419 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:12.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.124+0000 7f673affd640 1 --2- 172.21.15.6:0/2239186419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a7c3a3b980 0x55a7c3a4e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:12.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.124+0000 7f673b7fe640 1 -- 172.21.15.6:0/2239186419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a7c3a3b980 msgr2=0x55a7c3a4e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:12.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.124+0000 7f673b7fe640 1 --2- 172.21.15.6:0/2239186419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a7c3a3b980 0x55a7c3a4e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.124+0000 7f673b7fe640 1 -- 172.21.15.6:0/2239186419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a46240 msgr2=0x55a7c3a4b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:12.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.124+0000 7f673b7fe640 1 --2- 172.21.15.6:0/2239186419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a46240 0x55a7c3a4b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.124+0000 7f673b7fe640 1 -- 172.21.15.6:0/2239186419 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7c3a44070 con 0x55a7c3a3b060 2024-09-12T03:25:12.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.124+0000 7f673affd640 1 --2- 172.21.15.6:0/2239186419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a7c3a3b980 0x55a7c3a4e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:25:12.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.124+0000 7f673bfff640 1 --2- 172.21.15.6:0/2239186419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a46240 0x55a7c3a4b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:12.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.124+0000 7f673b7fe640 1 --2- 172.21.15.6:0/2239186419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a3b060 0x55a7c3a4e3c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f672c011850 tx=0x7f672c00ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6dd5fce040a296 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:12.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.124+0000 7f671affd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:25:12.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.124+0000 7f671affd640 1 -- 172.21.15.6:0/2239186419 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c015070 con 0x55a7c3a3b060 2024-09-12T03:25:12.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.124+0000 7f671affd640 1 -- 172.21.15.6:0/2239186419 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f672c019cc0 con 0x55a7c3a3b060 2024-09-12T03:25:12.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.124+0000 7f67445de740 1 -- 172.21.15.6:0/2239186419 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f672c01f040 con 0x55a7c3a3b060 2024-09-12T03:25:12.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.124+0000 7f671affd640 1 -- 172.21.15.6:0/2239186419 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f672c027070 con 0x55a7c3a3b060 2024-09-12T03:25:12.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.124+0000 7f67445de740 10 client.25480 fetch_fsmap learned FSMap version 27 2024-09-12T03:25:12.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.124+0000 7f67445de740 1 -- 172.21.15.6:0/2239186419 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a7c3ac1080 con 0x55a7c3a3b060 2024-09-12T03:25:12.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.125+0000 7f671affd640 1 -- 172.21.15.6:0/2239186419 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f672c01f040 con 0x55a7c3a3b060 2024-09-12T03:25:12.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.125+0000 7f671affd640 1 -- 172.21.15.6:0/2239186419 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f672c004040 con 0x55a7c3a3b060 2024-09-12T03:25:12.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.125+0000 7f67445de740 10 client.25480 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:25:12.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.125+0000 7f67445de740 10 client.25480 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:25:12.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.125+0000 7f67445de740 20 client.25480 populate_metadata read hostname 'smithi006' 2024-09-12T03:25:12.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.125+0000 7f67445de740 1 --2- 172.21.15.6:0/2239186419 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a7c3ac16d0 0x55a7c3ac3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:12.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.125+0000 7f67445de740 4 client.25480 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d89d56e5", "release": true, "await": true}] 2024-09-12T03:25:12.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.125+0000 7f67445de740 1 -- 172.21.15.6:0/2239186419 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d89d56e5", "release": true, "await": true}) -- 0x55a7c39a80a0 con 0x55a7c3ac16d0 2024-09-12T03:25:12.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.125+0000 7f673affd640 1 --2- 172.21.15.6:0/2239186419 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a7c3ac16d0 0x55a7c3ac3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:12.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.125+0000 7f673affd640 1 --2- 172.21.15.6:0/2239186419 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a7c3ac16d0 0x55a7c3ac3af0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb9dcbc8fae790bd server_cookie=bac0d8b6aaa9a73 in_seq=0 out_seq=0 2024-09-12T03:25:12.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.125+0000 7f671affd640 10 client.25480 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:12.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.159+0000 7f671affd640 1 -- 172.21.15.6:0/2239186419 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a7c39a80a0 con 0x55a7c3ac16d0 2024-09-12T03:25:12.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.160+0000 7f671affd640 10 client.25480 handle_command_reply: tid=0 2024-09-12T03:25:12.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.160+0000 7f67445de740 1 client.25480 shutdown 2024-09-12T03:25:12.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.179+0000 7f67445de740 1 -- 172.21.15.6:0/2239186419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a3b060 msgr2=0x55a7c3a4e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:12.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.179+0000 7f67445de740 1 --2- 172.21.15.6:0/2239186419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a3b060 0x55a7c3a4e3c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f672c011850 tx=0x7f672c00ec10 comp rx=0 tx=0).stop 2024-09-12T03:25:12.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.185+0000 7f67445de740 1 -- 172.21.15.6:0/2239186419 shutdown_connections 2024-09-12T03:25:12.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.185+0000 7f67445de740 1 --2- 172.21.15.6:0/2239186419 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55a7c3ac16d0 0x55a7c3ac3af0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.185+0000 7f67445de740 1 --2- 172.21.15.6:0/2239186419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a7c3a46240 0x55a7c3a4b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.185+0000 7f67445de740 1 --2- 172.21.15.6:0/2239186419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a7c3a3b980 0x55a7c3a4e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.185+0000 7f67445de740 1 --2- 172.21.15.6:0/2239186419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c3a3b060 0x55a7c3a4e3c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.185+0000 7f67445de740 1 -- 172.21.15.6:0/2239186419 >> 172.21.15.6:0/2239186419 conn(0x55a7c3a375f0 msgr2=0x55a7c3a43840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:12.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.185+0000 7f671affd640 0 client.25480 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:12.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.185+0000 7f67445de740 1 -- 172.21.15.6:0/2239186419 shutdown_connections 2024-09-12T03:25:12.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.185+0000 7f67445de740 1 -- 172.21.15.6:0/2239186419 wait complete. 2024-09-12T03:25:12.186 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:25:12.187 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 27, 2024-09-12T03:25:12.187 INFO:teuthology.orchestra.run.smithi006.stdout: "leader": 24494, 2024-09-12T03:25:12.187 INFO:teuthology.orchestra.run.smithi006.stdout: "set_version": 16, 2024-09-12T03:25:12.187 INFO:teuthology.orchestra.run.smithi006.stdout: "sets": { 2024-09-12T03:25:12.187 INFO:teuthology.orchestra.run.smithi006.stdout: "d89d56e5": { 2024-09-12T03:25:12.187 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 16, 2024-09-12T03:25:12.187 INFO:teuthology.orchestra.run.smithi006.stdout: "age_ref": 0.0, 2024-09-12T03:25:12.187 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:25:12.187 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RELEASED", 2024-09-12T03:25:12.187 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:25:12.187 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:25:12.187 INFO:teuthology.orchestra.run.smithi006.stdout: "timeout": 90.0, 2024-09-12T03:25:12.187 INFO:teuthology.orchestra.run.smithi006.stdout: "expiration": 140.2, 2024-09-12T03:25:12.187 INFO:teuthology.orchestra.run.smithi006.stdout: "members": { 2024-09-12T03:25:12.188 INFO:teuthology.orchestra.run.smithi006.stdout: "file:/": { 2024-09-12T03:25:12.188 INFO:teuthology.orchestra.run.smithi006.stdout: "excluded": false, 2024-09-12T03:25:12.188 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:25:12.188 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RELEASED", 2024-09-12T03:25:12.188 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:25:12.188 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:25:12.188 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:25:12.188 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:25:12.188 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:25:12.188 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:25:12.188 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:25:12.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.185+0000 7f67445de740 20 client.25480 trim_cache size 0 max 16384 2024-09-12T03:25:12.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.186+0000 7f6742205640 1 -- 172.21.15.6:0/1112669051 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f67280a2b80 msgr2=0x7f67280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:12.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.186+0000 7f6742205640 1 --2- 172.21.15.6:0/1112669051 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f67280a2b80 0x7f67280a5040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6734009430 tx=0x7f6734005c50 comp rx=0 tx=0).stop 2024-09-12T03:25:12.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.186+0000 7f6742205640 1 -- 172.21.15.6:0/1112669051 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f673c103660 msgr2=0x7f673c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:12.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.186+0000 7f6742205640 1 --2- 172.21.15.6:0/1112669051 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f673c103660 0x7f673c100ee0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f672400d740 tx=0x7f672400dc00 comp rx=0 tx=0).stop 2024-09-12T03:25:12.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.190+0000 7f6742205640 1 -- 172.21.15.6:0/1112669051 shutdown_connections 2024-09-12T03:25:12.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.190+0000 7f6742205640 1 --2- 172.21.15.6:0/1112669051 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f67280a2b80 0x7f67280a5040 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.190+0000 7f6742205640 1 --2- 172.21.15.6:0/1112669051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f673c10c170 0x7f673c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.190+0000 7f6742205640 1 --2- 172.21.15.6:0/1112669051 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f673c103660 0x7f673c100ee0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.190+0000 7f6742205640 1 --2- 172.21.15.6:0/1112669051 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f673c102d40 0x7f673c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:12.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.190+0000 7f6742205640 1 -- 172.21.15.6:0/1112669051 >> 172.21.15.6:0/1112669051 conn(0x7f673c0fc7a0 msgr2=0x7f673c0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:12.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.190+0000 7f6742205640 1 -- 172.21.15.6:0/1112669051 shutdown_connections 2024-09-12T03:25:12.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:12.190+0000 7f6742205640 1 -- 172.21.15.6:0/1112669051 wait complete. 2024-09-12T03:25:13.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:13 smithi118 ceph-mon[35048]: pgmap v458: 97 pgs: 97 active+clean; 13 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 13 KiB/s rd, 58 MiB/s wr, 6.56k op/s 2024-09-12T03:25:13.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:13 smithi006 ceph-mon[30991]: pgmap v458: 97 pgs: 97 active+clean; 13 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 13 KiB/s rd, 58 MiB/s wr, 6.56k op/s 2024-09-12T03:25:13.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:13 smithi161 ceph-mon[35485]: pgmap v458: 97 pgs: 97 active+clean; 13 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 13 KiB/s rd, 58 MiB/s wr, 6.56k op/s 2024-09-12T03:25:13.686 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd89d56e5', total seconds elapsed: 24.6 2024-09-12T03:25:13.686 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-12T03:25:13.686 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:25:14.318 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:15.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.344+0000 7f37d0463640 1 -- 172.21.15.6:0/2547917426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37c00051c0 msgr2=0x7f37c0005600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:15.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.344+0000 7f37d0463640 1 --2- 172.21.15.6:0/2547917426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37c00051c0 0x7f37c0005600 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009900 tx=0x7f37c402a410 comp rx=0 tx=0).stop 2024-09-12T03:25:15.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.352+0000 7f37d0463640 1 -- 172.21.15.6:0/2547917426 shutdown_connections 2024-09-12T03:25:15.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.352+0000 7f37d0463640 1 --2- 172.21.15.6:0/2547917426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37c00a1fd0 0x7f37c00a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:15.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.352+0000 7f37d0463640 1 --2- 172.21.15.6:0/2547917426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37c00051c0 0x7f37c0005600 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:15.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.352+0000 7f37d0463640 1 --2- 172.21.15.6:0/2547917426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37c00964b0 0x7f37c0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:15.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.352+0000 7f37d0463640 1 -- 172.21.15.6:0/2547917426 >> 172.21.15.6:0/2547917426 conn(0x7f37c000a220 msgr2=0x7f37c000a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:15.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.358+0000 7f37d0463640 1 -- 172.21.15.6:0/2547917426 shutdown_connections 2024-09-12T03:25:15.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.359+0000 7f37d0463640 1 -- 172.21.15.6:0/2547917426 wait complete. 2024-09-12T03:25:15.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.360+0000 7f37d0463640 1 Processor -- start 2024-09-12T03:25:15.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.360+0000 7f37d0463640 1 -- start start 2024-09-12T03:25:15.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.361+0000 7f37d0463640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37c00051c0 0x7f37c012e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:15.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.361+0000 7f37d0463640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37c00964b0 0x7f37c012e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:15.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.361+0000 7f37d0463640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37c00a1fd0 0x7f37c0132920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:15.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.361+0000 7f37ce1ff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37c00051c0 0x7f37c012e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:15.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.361+0000 7f37ce1ff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37c00051c0 0x7f37c012e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:39152/0 (socket says 172.21.15.6:39152) 2024-09-12T03:25:15.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.361+0000 7f37ce1ff640 1 -- 172.21.15.6:0/1093895047 learned_addr learned my addr 172.21.15.6:0/1093895047 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:15.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.362+0000 7f37d0463640 1 -- 172.21.15.6:0/1093895047 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f37c00a7c60 con 0x7f37c00a1fd0 2024-09-12T03:25:15.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.362+0000 7f37d0463640 1 -- 172.21.15.6:0/1093895047 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f37c00a7ae0 con 0x7f37c00964b0 2024-09-12T03:25:15.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.362+0000 7f37d0463640 1 -- 172.21.15.6:0/1093895047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f37c00a7de0 con 0x7f37c00051c0 2024-09-12T03:25:15.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.366+0000 7f37cd9fe640 1 --2- 172.21.15.6:0/1093895047 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37c00964b0 0x7f37c012e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:15.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.366+0000 7f37ce1ff640 1 -- 172.21.15.6:0/1093895047 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37c00964b0 msgr2=0x7f37c012e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:15.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.366+0000 7f37ce1ff640 1 --2- 172.21.15.6:0/1093895047 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37c00964b0 0x7f37c012e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:15.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.366+0000 7f37ce1ff640 1 -- 172.21.15.6:0/1093895047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37c00a1fd0 msgr2=0x7f37c0132920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:15.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.366+0000 7f37ce1ff640 1 --2- 172.21.15.6:0/1093895047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37c00a1fd0 0x7f37c0132920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:15.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.367+0000 7f37ce1ff640 1 -- 172.21.15.6:0/1093895047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c0133000 con 0x7f37c00051c0 2024-09-12T03:25:15.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.367+0000 7f37cd9fe640 1 --2- 172.21.15.6:0/1093895047 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37c00964b0 0x7f37c012e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:25:15.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.367+0000 7f37ce1ff640 1 --2- 172.21.15.6:0/1093895047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37c00051c0 0x7f37c012e050 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f37b800c970 tx=0x7f37b800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:15.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.369+0000 7f37bf7fe640 1 -- 172.21.15.6:0/1093895047 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8007bb0 con 0x7f37c00051c0 2024-09-12T03:25:15.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.370+0000 7f37bf7fe640 1 -- 172.21.15.6:0/1093895047 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b8007d50 con 0x7f37c00051c0 2024-09-12T03:25:15.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.370+0000 7f37d0463640 1 -- 172.21.15.6:0/1093895047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c01332f0 con 0x7f37c00051c0 2024-09-12T03:25:15.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.371+0000 7f37bf7fe640 1 -- 172.21.15.6:0/1093895047 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b800f450 con 0x7f37c00051c0 2024-09-12T03:25:15.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.371+0000 7f37d0463640 1 -- 172.21.15.6:0/1093895047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c013abd0 con 0x7f37c00051c0 2024-09-12T03:25:15.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.375+0000 7f37d0463640 1 -- 172.21.15.6:0/1093895047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37c0094b70 con 0x7f37c00051c0 2024-09-12T03:25:15.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.378+0000 7f37bf7fe640 1 -- 172.21.15.6:0/1093895047 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f37b8004660 con 0x7f37c00051c0 2024-09-12T03:25:15.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.380+0000 7f37bf7fe640 1 --2- 172.21.15.6:0/1093895047 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f37b40a2c70 0x7f37b40a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:15.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.380+0000 7f37bf7fe640 1 -- 172.21.15.6:0/1093895047 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f37b80ce9a0 con 0x7f37c00051c0 2024-09-12T03:25:15.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.384+0000 7f37bf7fe640 1 -- 172.21.15.6:0/1093895047 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f37b8014030 con 0x7f37c00051c0 2024-09-12T03:25:15.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.385+0000 7f37cd9fe640 1 --2- 172.21.15.6:0/1093895047 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f37b40a2c70 0x7f37b40a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:15.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.386+0000 7f37cd9fe640 1 --2- 172.21.15.6:0/1093895047 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f37b40a2c70 0x7f37b40a5130 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f37c4005ab0 tx=0x7f37c4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:15.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:15 smithi006 ceph-mon[30991]: pgmap v459: 97 pgs: 97 active+clean; 13 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 13 KiB/s rd, 36 MiB/s wr, 3.87k op/s 2024-09-12T03:25:15.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:15 smithi161 ceph-mon[35485]: pgmap v459: 97 pgs: 97 active+clean; 13 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 13 KiB/s rd, 36 MiB/s wr, 3.87k op/s 2024-09-12T03:25:15.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:15 smithi118 ceph-mon[35048]: pgmap v459: 97 pgs: 97 active+clean; 13 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 13 KiB/s rd, 36 MiB/s wr, 3.87k op/s 2024-09-12T03:25:15.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.873+0000 7f37d0463640 1 -- 172.21.15.6:0/1093895047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37c013ada0 con 0x7f37c00051c0 2024-09-12T03:25:15.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.877+0000 7f37bf7fe640 1 -- 172.21.15.6:0/1093895047 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11908 (secure 0 0 0) 0x7f37b809b950 con 0x7f37c00051c0 2024-09-12T03:25:15.878 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:25:15.878 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"btime":"2024-09-12T03:24:26:920603+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:24:26.080047+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24463]},"id":1}]} 2024-09-12T03:25:15.878 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 27 2024-09-12T03:25:15.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.882+0000 7f37d0463640 1 -- 172.21.15.6:0/1093895047 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f37b40a2c70 msgr2=0x7f37b40a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:15.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.882+0000 7f37d0463640 1 --2- 172.21.15.6:0/1093895047 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f37b40a2c70 0x7f37b40a5130 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f37c4005ab0 tx=0x7f37c4005a20 comp rx=0 tx=0).stop 2024-09-12T03:25:15.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.882+0000 7f37d0463640 1 -- 172.21.15.6:0/1093895047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37c00051c0 msgr2=0x7f37c012e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:15.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.882+0000 7f37d0463640 1 --2- 172.21.15.6:0/1093895047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37c00051c0 0x7f37c012e050 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f37b800c970 tx=0x7f37b800ce30 comp rx=0 tx=0).stop 2024-09-12T03:25:15.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.882+0000 7f37d0463640 1 -- 172.21.15.6:0/1093895047 shutdown_connections 2024-09-12T03:25:15.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.882+0000 7f37d0463640 1 --2- 172.21.15.6:0/1093895047 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f37b40a2c70 0x7f37b40a5130 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:15.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.882+0000 7f37d0463640 1 --2- 172.21.15.6:0/1093895047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37c00a1fd0 0x7f37c0132920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:15.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.882+0000 7f37d0463640 1 --2- 172.21.15.6:0/1093895047 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37c00964b0 0x7f37c012e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:15.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.882+0000 7f37d0463640 1 --2- 172.21.15.6:0/1093895047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37c00051c0 0x7f37c012e050 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:15.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.882+0000 7f37d0463640 1 -- 172.21.15.6:0/1093895047 >> 172.21.15.6:0/1093895047 conn(0x7f37c000a220 msgr2=0x7f37c0093950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:15.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.883+0000 7f37d0463640 1 -- 172.21.15.6:0/1093895047 shutdown_connections 2024-09-12T03:25:15.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:15.883+0000 7f37d0463640 1 -- 172.21.15.6:0/1093895047 wait complete. 2024-09-12T03:25:16.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:16 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1093895047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:25:16.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:16 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1093895047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:25:16.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:16 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1093895047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:25:17.016 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-12T03:25:17.017 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-12T03:25:17.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:17 smithi006 ceph-mon[30991]: pgmap v460: 97 pgs: 97 active+clean; 13 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 4.8 KiB/s rd, 60 MiB/s wr, 6.53k op/s 2024-09-12T03:25:17.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:17 smithi161 ceph-mon[35485]: pgmap v460: 97 pgs: 97 active+clean; 13 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 4.8 KiB/s rd, 60 MiB/s wr, 6.53k op/s 2024-09-12T03:25:17.660 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:17.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:17 smithi118 ceph-mon[35048]: pgmap v460: 97 pgs: 97 active+clean; 13 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 4.8 KiB/s rd, 60 MiB/s wr, 6.53k op/s 2024-09-12T03:25:18.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.540+0000 7f25222b3640 1 -- 172.21.15.6:0/145988427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251c101270 msgr2=0x7f251c10d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:18.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.540+0000 7f25222b3640 1 --2- 172.21.15.6:0/145988427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251c101270 0x7f251c10d7a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f251400bda0 tx=0x7f251402c4b0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.542+0000 7f25222b3640 1 -- 172.21.15.6:0/145988427 shutdown_connections 2024-09-12T03:25:18.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.542+0000 7f25222b3640 1 --2- 172.21.15.6:0/145988427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251c101270 0x7f251c10d7a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.542+0000 7f25222b3640 1 --2- 172.21.15.6:0/145988427 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f251c1008d0 0x7f251c100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.542+0000 7f25222b3640 1 --2- 172.21.15.6:0/145988427 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f251c1068d0 0x7f251c106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.542+0000 7f25222b3640 1 -- 172.21.15.6:0/145988427 >> 172.21.15.6:0/145988427 conn(0x7f251c0fc610 msgr2=0x7f251c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:18.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.542+0000 7f25222b3640 1 -- 172.21.15.6:0/145988427 shutdown_connections 2024-09-12T03:25:18.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.542+0000 7f25222b3640 1 -- 172.21.15.6:0/145988427 wait complete. 2024-09-12T03:25:18.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.543+0000 7f25222b3640 1 Processor -- start 2024-09-12T03:25:18.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.543+0000 7f25222b3640 1 -- start start 2024-09-12T03:25:18.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.544+0000 7f25222b3640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f251c1008d0 0x7f251c1a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:18.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.544+0000 7f25222b3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f251c101270 0x7f251c1a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:18.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.544+0000 7f25222b3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251c1068d0 0x7f251c19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:18.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.544+0000 7f2520ab0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f251c101270 0x7f251c1a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:18.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.544+0000 7f2520ab0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f251c101270 0x7f251c1a4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57352/0 (socket says 172.21.15.6:57352) 2024-09-12T03:25:18.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.544+0000 7f2520ab0640 1 -- 172.21.15.6:0/1951784421 learned_addr learned my addr 172.21.15.6:0/1951784421 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:18.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.544+0000 7f25222b3640 1 -- 172.21.15.6:0/1951784421 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f251c10fbc0 con 0x7f251c101270 2024-09-12T03:25:18.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.544+0000 7f25222b3640 1 -- 172.21.15.6:0/1951784421 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f251c10fa40 con 0x7f251c1008d0 2024-09-12T03:25:18.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.544+0000 7f25212b1640 1 --2- 172.21.15.6:0/1951784421 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f251c1008d0 0x7f251c1a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:18.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.544+0000 7f25222b3640 1 -- 172.21.15.6:0/1951784421 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f251c10fd40 con 0x7f251c1068d0 2024-09-12T03:25:18.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.545+0000 7f2521ab2640 1 --2- 172.21.15.6:0/1951784421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251c1068d0 0x7f251c19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:18.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.549+0000 7f2520ab0640 1 -- 172.21.15.6:0/1951784421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251c1068d0 msgr2=0x7f251c19e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:18.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.549+0000 7f2520ab0640 1 --2- 172.21.15.6:0/1951784421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251c1068d0 0x7f251c19e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.549+0000 7f2520ab0640 1 -- 172.21.15.6:0/1951784421 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f251c1008d0 msgr2=0x7f251c1a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:18.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.549+0000 7f2520ab0640 1 --2- 172.21.15.6:0/1951784421 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f251c1008d0 0x7f251c1a4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.549+0000 7f2520ab0640 1 -- 172.21.15.6:0/1951784421 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251c19ef90 con 0x7f251c101270 2024-09-12T03:25:18.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.551+0000 7f2520ab0640 1 --2- 172.21.15.6:0/1951784421 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f251c101270 0x7f251c1a4b90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f250c00d950 tx=0x7f250c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:18.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.553+0000 7f25127fc640 1 -- 172.21.15.6:0/1951784421 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c017070 con 0x7f251c101270 2024-09-12T03:25:18.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.553+0000 7f25127fc640 1 -- 172.21.15.6:0/1951784421 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f250c009c30 con 0x7f251c101270 2024-09-12T03:25:18.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.553+0000 7f25222b3640 1 -- 172.21.15.6:0/1951784421 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f251c19f280 con 0x7f251c101270 2024-09-12T03:25:18.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.554+0000 7f25222b3640 1 -- 172.21.15.6:0/1951784421 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f251c1ab5d0 con 0x7f251c101270 2024-09-12T03:25:18.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.558+0000 7f25258ee740 1 Processor -- start 2024-09-12T03:25:18.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.558+0000 7f25127fc640 1 -- 172.21.15.6:0/1951784421 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c00f690 con 0x7f251c101270 2024-09-12T03:25:18.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.565+0000 7f25127fc640 1 -- 172.21.15.6:0/1951784421 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f250c00f830 con 0x7f251c101270 2024-09-12T03:25:18.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.566+0000 7f25127fc640 1 --2- 172.21.15.6:0/1951784421 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f25080a2d50 0x7f25080a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:18.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.567+0000 7f25127fc640 1 -- 172.21.15.6:0/1951784421 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f250c0ce9f0 con 0x7f251c101270 2024-09-12T03:25:18.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.573+0000 7f25258ee740 1 -- start start 2024-09-12T03:25:18.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.573+0000 7f25258ee740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b58f8b7320 0x55b58f8b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:18.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.573+0000 7f25258ee740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8b6030 0x55b58f8b6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:18.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.573+0000 7f25258ee740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b58f8c0990 0x55b58f8c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:18.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.573+0000 7f25258ee740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b58f81a160 con 0x55b58f8c0990 2024-09-12T03:25:18.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.573+0000 7f25258ee740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55b58f7c1720 con 0x55b58f8b6030 2024-09-12T03:25:18.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.573+0000 7f25258ee740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b58f816120 con 0x55b58f8b7320 2024-09-12T03:25:18.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.573+0000 7f2521ab2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b58f8c0990 0x55b58f8c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:18.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.574+0000 7f2520ab0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8b6030 0x55b58f8b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:18.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.574+0000 7f2520ab0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8b6030 0x55b58f8b6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:42590/0 (socket says 172.21.15.6:42590) 2024-09-12T03:25:18.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.574+0000 7f2520ab0640 1 -- 172.21.15.6:0/2426494995 learned_addr learned my addr 172.21.15.6:0/2426494995 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:18.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.574+0000 7f2521ab2640 1 --2- 172.21.15.6:0/1951784421 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f25080a2d50 0x7f25080a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:18.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.574+0000 7f2520ab0640 1 -- 172.21.15.6:0/2426494995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b58f8b7320 msgr2=0x55b58f8b5af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:25:18.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.574+0000 7f2520ab0640 1 --2- 172.21.15.6:0/2426494995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b58f8b7320 0x55b58f8b5af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.574+0000 7f2520ab0640 1 -- 172.21.15.6:0/2426494995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b58f8c0990 msgr2=0x55b58f8c2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:18.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.574+0000 7f2520ab0640 1 --2- 172.21.15.6:0/2426494995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b58f8c0990 0x55b58f8c2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.575+0000 7f2520ab0640 1 -- 172.21.15.6:0/2426494995 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b58f58bba0 con 0x55b58f8b6030 2024-09-12T03:25:18.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.575+0000 7f2520ab0640 1 --2- 172.21.15.6:0/2426494995 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8b6030 0x55b58f8b6470 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f250c09afd0 tx=0x7f250c09b2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7e2329a074acc09 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:18.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.576+0000 7f2521ab2640 1 --2- 172.21.15.6:0/1951784421 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f25080a2d50 0x7f25080a5210 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2514004ab0 tx=0x7f2514005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:18.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.576+0000 7f25222b3640 1 -- 172.21.15.6:0/2426494995 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c0b9e60 con 0x55b58f8b6030 2024-09-12T03:25:18.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.576+0000 7f25222b3640 1 -- 172.21.15.6:0/2426494995 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f250c0a3e90 con 0x55b58f8b6030 2024-09-12T03:25:18.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.577+0000 7f25258ee740 1 -- 172.21.15.6:0/2426494995 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8b6030 msgr2=0x55b58f8b6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:18.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.577+0000 7f25258ee740 1 --2- 172.21.15.6:0/2426494995 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8b6030 0x55b58f8b6470 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f250c09afd0 tx=0x7f250c09b2d0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.578+0000 7f25258ee740 1 -- 172.21.15.6:0/2426494995 shutdown_connections 2024-09-12T03:25:18.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.578+0000 7f25258ee740 1 --2- 172.21.15.6:0/2426494995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b58f8c0990 0x55b58f8c2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.578+0000 7f25258ee740 1 --2- 172.21.15.6:0/2426494995 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8b6030 0x55b58f8b6470 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f250c09afd0 tx=0x7f250c09b2d0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.578+0000 7f25258ee740 1 --2- 172.21.15.6:0/2426494995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b58f8b7320 0x55b58f8b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.578+0000 7f25258ee740 1 -- 172.21.15.6:0/2426494995 >> 172.21.15.6:0/2426494995 conn(0x55b58f8b2280 msgr2=0x55b58f8b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:18.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.581+0000 7f25258ee740 1 -- 172.21.15.6:0/2426494995 shutdown_connections 2024-09-12T03:25:18.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.582+0000 7f25258ee740 1 -- 172.21.15.6:0/2426494995 wait complete. 2024-09-12T03:25:18.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.582+0000 7f25258ee740 1 -- start start 2024-09-12T03:25:18.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.582+0000 7f25258ee740 1 Processor -- start 2024-09-12T03:25:18.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.583+0000 7f25258ee740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8b7320 0x55b58f8bda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:18.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.583+0000 7f25258ee740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b58f8c0990 0x55b58f8bdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:18.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.583+0000 7f25258ee740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b58f8be480 0x55b58f8ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:18.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.583+0000 7f25258ee740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b58f8c3840 con 0x55b58f8c0990 2024-09-12T03:25:18.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.583+0000 7f25258ee740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55b58f7a2900 con 0x55b58f8b7320 2024-09-12T03:25:18.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.583+0000 7f25258ee740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b58f8c39c0 con 0x55b58f8be480 2024-09-12T03:25:18.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.583+0000 7f25258ee740 10 client.0 _ll_register_callbacks cb 0x55b58f7a0780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:18.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.584+0000 7f25212b1640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8b7320 0x55b58f8bda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:18.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.584+0000 7f25212b1640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8b7320 0x55b58f8bda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:42598/0 (socket says 172.21.15.6:42598) 2024-09-12T03:25:18.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.584+0000 7f25212b1640 1 -- 172.21.15.6:0/2126867585 learned_addr learned my addr 172.21.15.6:0/2126867585 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:18.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.584+0000 7f2520ab0640 1 --2- 172.21.15.6:0/2126867585 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b58f8c0990 0x55b58f8bdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:18.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.584+0000 7f2521ab2640 1 --2- 172.21.15.6:0/2126867585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b58f8be480 0x55b58f8ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:18.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.588+0000 7f2520ab0640 1 -- 172.21.15.6:0/2126867585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b58f8be480 msgr2=0x55b58f8ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:18.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.588+0000 7f2520ab0640 1 --2- 172.21.15.6:0/2126867585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b58f8be480 0x55b58f8ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.588+0000 7f2520ab0640 1 -- 172.21.15.6:0/2126867585 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8b7320 msgr2=0x55b58f8bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:18.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.588+0000 7f2520ab0640 1 --2- 172.21.15.6:0/2126867585 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8b7320 0x55b58f8bda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.588+0000 7f2520ab0640 1 -- 172.21.15.6:0/2126867585 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b58f7e2c50 con 0x55b58f8c0990 2024-09-12T03:25:18.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.595+0000 7f2520ab0640 1 --2- 172.21.15.6:0/2126867585 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b58f8c0990 0x55b58f8bdf40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x55b58f8b31e0 tx=0x7f250c00b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751218aa19ddab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:18.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.595+0000 7f25212b1640 1 --2- 172.21.15.6:0/2126867585 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8b7320 0x55b58f8bda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:25:18.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.597+0000 7f2504ff9640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:25:18.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.597+0000 7f2504ff9640 1 -- 172.21.15.6:0/2126867585 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c09f3a0 con 0x55b58f8c0990 2024-09-12T03:25:18.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.597+0000 7f2504ff9640 1 -- 172.21.15.6:0/2126867585 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f250c09a720 con 0x55b58f8c0990 2024-09-12T03:25:18.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.601+0000 7f25258ee740 1 -- 172.21.15.6:0/2126867585 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f250c0a3e90 con 0x55b58f8c0990 2024-09-12T03:25:18.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.601+0000 7f2504ff9640 1 -- 172.21.15.6:0/2126867585 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f250c0a6030 con 0x55b58f8c0990 2024-09-12T03:25:18.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.602+0000 7f25258ee740 10 client.15882 fetch_fsmap learned FSMap version 27 2024-09-12T03:25:18.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.602+0000 7f25258ee740 1 -- 172.21.15.6:0/2126867585 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b58f8e0b40 con 0x55b58f8c0990 2024-09-12T03:25:18.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.602+0000 7f2504ff9640 1 -- 172.21.15.6:0/2126867585 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f250c0a3e90 con 0x55b58f8c0990 2024-09-12T03:25:18.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.603+0000 7f2504ff9640 1 -- 172.21.15.6:0/2126867585 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f250c00fb20 con 0x55b58f8c0990 2024-09-12T03:25:18.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.603+0000 7f25258ee740 10 client.15882 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:25:18.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.603+0000 7f25258ee740 10 client.15882 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:25:18.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.603+0000 7f25258ee740 20 client.15882 populate_metadata read hostname 'smithi006' 2024-09-12T03:25:18.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.603+0000 7f25258ee740 1 --2- 172.21.15.6:0/2126867585 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b58f8e1e90 0x55b58f8e4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:18.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.603+0000 7f25258ee740 4 client.15882 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:25:18.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.603+0000 7f25258ee740 1 -- 172.21.15.6:0/2126867585 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b58f827890 con 0x55b58f8e1e90 2024-09-12T03:25:18.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.603+0000 7f25212b1640 1 --2- 172.21.15.6:0/2126867585 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b58f8e1e90 0x55b58f8e4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:18.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.606+0000 7f25212b1640 1 --2- 172.21.15.6:0/2126867585 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b58f8e1e90 0x55b58f8e4280 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80c12a371072a883 server_cookie=35d75591c1092cbb in_seq=0 out_seq=0 2024-09-12T03:25:18.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.606+0000 7f2504ff9640 10 client.15882 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:18.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.607+0000 7f2504ff9640 1 -- 172.21.15.6:0/2126867585 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b58f827890 con 0x55b58f8e1e90 2024-09-12T03:25:18.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.607+0000 7f2504ff9640 10 client.15882 handle_command_reply: tid=0 2024-09-12T03:25:18.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.607+0000 7f25258ee740 1 client.15882 shutdown 2024-09-12T03:25:18.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.612+0000 7f25258ee740 1 -- 172.21.15.6:0/2126867585 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b58f8c0990 msgr2=0x55b58f8bdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:18.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.612+0000 7f25258ee740 1 --2- 172.21.15.6:0/2126867585 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b58f8c0990 0x55b58f8bdf40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x55b58f8b31e0 tx=0x7f250c00b530 comp rx=0 tx=0).stop 2024-09-12T03:25:18.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.614+0000 7f25258ee740 1 -- 172.21.15.6:0/2126867585 shutdown_connections 2024-09-12T03:25:18.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.614+0000 7f25258ee740 1 --2- 172.21.15.6:0/2126867585 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b58f8e1e90 0x55b58f8e4280 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.614+0000 7f25258ee740 1 --2- 172.21.15.6:0/2126867585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b58f8be480 0x55b58f8ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.614+0000 7f25258ee740 1 --2- 172.21.15.6:0/2126867585 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b58f8c0990 0x55b58f8bdf40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.614+0000 7f25258ee740 1 --2- 172.21.15.6:0/2126867585 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8b7320 0x55b58f8bda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.614+0000 7f25258ee740 1 -- 172.21.15.6:0/2126867585 >> 172.21.15.6:0/2126867585 conn(0x55b58f8b2280 msgr2=0x55b58f8b7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:18.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.614+0000 7f2504ff9640 0 client.15882 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:18.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.615+0000 7f25258ee740 1 -- 172.21.15.6:0/2126867585 shutdown_connections 2024-09-12T03:25:18.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.617+0000 7f25258ee740 1 -- 172.21.15.6:0/2126867585 wait complete. 2024-09-12T03:25:18.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.617+0000 7f25258ee740 20 client.15882 trim_cache size 0 max 16384 2024-09-12T03:25:18.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.655+0000 7f25258ee740 1 Processor -- start 2024-09-12T03:25:18.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.656+0000 7f25258ee740 1 -- start start 2024-09-12T03:25:18.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.657+0000 7f25258ee740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b58f8b7320 0x55b58f8ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:18.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.657+0000 7f25258ee740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8be480 0x55b58f8bc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:18.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.657+0000 7f25258ee740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b58f8c0990 0x55b58f8bc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:18.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.657+0000 7f25258ee740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2500010d00 con 0x55b58f8c0990 2024-09-12T03:25:18.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.657+0000 7f25258ee740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f250000e5d0 con 0x55b58f8be480 2024-09-12T03:25:18.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.657+0000 7f25258ee740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f250000c420 con 0x55b58f8b7320 2024-09-12T03:25:18.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.658+0000 7f25212b1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b58f8b7320 0x55b58f8ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:18.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.658+0000 7f25212b1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b58f8b7320 0x55b58f8ccd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:39210/0 (socket says 172.21.15.6:39210) 2024-09-12T03:25:18.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.658+0000 7f25212b1640 1 -- 172.21.15.6:0/870457975 learned_addr learned my addr 172.21.15.6:0/870457975 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:18.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.659+0000 7f2520ab0640 1 --2- 172.21.15.6:0/870457975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8be480 0x55b58f8bc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:18.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.660+0000 7f25212b1640 1 -- 172.21.15.6:0/870457975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8be480 msgr2=0x55b58f8bc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:18.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.660+0000 7f25212b1640 1 --2- 172.21.15.6:0/870457975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8be480 0x55b58f8bc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.660+0000 7f25212b1640 1 -- 172.21.15.6:0/870457975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b58f8c0990 msgr2=0x55b58f8bc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:18.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.660+0000 7f25212b1640 1 --2- 172.21.15.6:0/870457975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b58f8c0990 0x55b58f8bc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.660+0000 7f25212b1640 1 -- 172.21.15.6:0/870457975 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b58f8bedd0 con 0x55b58f8b7320 2024-09-12T03:25:18.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.660+0000 7f25212b1640 1 --2- 172.21.15.6:0/870457975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b58f8b7320 0x55b58f8ccd10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2518000c00 tx=0x7f2518004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee0a08d3f16e0ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:18.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.661+0000 7f2506ffd640 1 -- 172.21.15.6:0/870457975 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251800be60 con 0x55b58f8b7320 2024-09-12T03:25:18.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.661+0000 7f2506ffd640 1 -- 172.21.15.6:0/870457975 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2518003cc0 con 0x55b58f8b7320 2024-09-12T03:25:18.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.661+0000 7f2506ffd640 1 -- 172.21.15.6:0/870457975 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25180117c0 con 0x55b58f8b7320 2024-09-12T03:25:18.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.661+0000 7f2520ab0640 1 --2- 172.21.15.6:0/870457975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8be480 0x55b58f8bc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:25:18.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.662+0000 7f25258ee740 1 -- 172.21.15.6:0/870457975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b58f8b7320 msgr2=0x55b58f8ccd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:18.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.662+0000 7f25258ee740 1 --2- 172.21.15.6:0/870457975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b58f8b7320 0x55b58f8ccd10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2518000c00 tx=0x7f2518004610 comp rx=0 tx=0).stop 2024-09-12T03:25:18.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.666+0000 7f25258ee740 1 -- 172.21.15.6:0/870457975 shutdown_connections 2024-09-12T03:25:18.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.666+0000 7f25258ee740 1 --2- 172.21.15.6:0/870457975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b58f8c0990 0x55b58f8bc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.666+0000 7f25258ee740 1 --2- 172.21.15.6:0/870457975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8be480 0x55b58f8bc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.666+0000 7f25258ee740 1 --2- 172.21.15.6:0/870457975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b58f8b7320 0x55b58f8ccd10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.666+0000 7f25258ee740 1 -- 172.21.15.6:0/870457975 >> 172.21.15.6:0/870457975 conn(0x55b58f8b2280 msgr2=0x55b58f8bd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:18.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.666+0000 7f25258ee740 1 -- 172.21.15.6:0/870457975 shutdown_connections 2024-09-12T03:25:18.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.666+0000 7f25258ee740 1 -- 172.21.15.6:0/870457975 wait complete. 2024-09-12T03:25:18.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.667+0000 7f25258ee740 1 -- start start 2024-09-12T03:25:18.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.667+0000 7f25258ee740 1 Processor -- start 2024-09-12T03:25:18.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.668+0000 7f25258ee740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b58f8b7320 0x55b58f8e7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:18.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.668+0000 7f25258ee740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8be480 0x55b58f8e4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:18.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.668+0000 7f25258ee740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b58f8c0990 0x55b58f8e4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:18.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.669+0000 7f2520ab0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8be480 0x55b58f8e4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:18.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.669+0000 7f2520ab0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8be480 0x55b58f8e4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:42606/0 (socket says 172.21.15.6:42606) 2024-09-12T03:25:18.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.669+0000 7f2520ab0640 1 -- 172.21.15.6:0/1499302173 learned_addr learned my addr 172.21.15.6:0/1499302173 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:18.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.669+0000 7f25258ee740 1 -- 172.21.15.6:0/1499302173 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f250000ca90 con 0x55b58f8b7320 2024-09-12T03:25:18.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.669+0000 7f25258ee740 1 -- 172.21.15.6:0/1499302173 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f250000df00 con 0x55b58f8be480 2024-09-12T03:25:18.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.669+0000 7f25258ee740 1 -- 172.21.15.6:0/1499302173 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f250000d160 con 0x55b58f8c0990 2024-09-12T03:25:18.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.669+0000 7f2521ab2640 1 --2- 172.21.15.6:0/1499302173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b58f8c0990 0x55b58f8e4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:18.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.670+0000 7f25258ee740 10 client.0 _ll_register_callbacks cb 0x7f250c09acc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:18.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.670+0000 7f2520ab0640 1 -- 172.21.15.6:0/1499302173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b58f8c0990 msgr2=0x55b58f8e4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:18.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.670+0000 7f2520ab0640 1 --2- 172.21.15.6:0/1499302173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b58f8c0990 0x55b58f8e4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.670+0000 7f2520ab0640 1 -- 172.21.15.6:0/1499302173 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b58f8b7320 msgr2=0x55b58f8e7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:18.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.670+0000 7f2520ab0640 1 --2- 172.21.15.6:0/1499302173 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b58f8b7320 0x55b58f8e7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.670+0000 7f2520ab0640 1 -- 172.21.15.6:0/1499302173 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b58f8e5490 con 0x55b58f8be480 2024-09-12T03:25:18.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.671+0000 7f2520ab0640 1 --2- 172.21.15.6:0/1499302173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8be480 0x55b58f8e4860 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f250c0058c0 tx=0x7f250c097b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cceb63afb023370 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:18.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.671+0000 7f2504ff9640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:25:18.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.671+0000 7f2504ff9640 1 -- 172.21.15.6:0/1499302173 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c097570 con 0x55b58f8be480 2024-09-12T03:25:18.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.671+0000 7f25258ee740 1 -- 172.21.15.6:0/1499302173 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2518003cc0 con 0x55b58f8be480 2024-09-12T03:25:18.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.671+0000 7f2504ff9640 1 -- 172.21.15.6:0/1499302173 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f250c09f120 con 0x55b58f8be480 2024-09-12T03:25:18.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.675+0000 7f2504ff9640 1 -- 172.21.15.6:0/1499302173 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f250c0a0b30 con 0x55b58f8be480 2024-09-12T03:25:18.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.676+0000 7f25258ee740 10 client.25439 fetch_fsmap learned FSMap version 27 2024-09-12T03:25:18.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.676+0000 7f25258ee740 1 -- 172.21.15.6:0/1499302173 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b58f94c690 con 0x55b58f8be480 2024-09-12T03:25:18.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.677+0000 7f2504ff9640 1 -- 172.21.15.6:0/1499302173 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f250c0a3e90 con 0x55b58f8be480 2024-09-12T03:25:18.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.677+0000 7f2504ff9640 1 -- 172.21.15.6:0/1499302173 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f250c097020 con 0x55b58f8be480 2024-09-12T03:25:18.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.677+0000 7f25258ee740 10 client.25439 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:25:18.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.677+0000 7f25258ee740 10 client.25439 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:25:18.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.677+0000 7f25258ee740 20 client.25439 populate_metadata read hostname 'smithi006' 2024-09-12T03:25:18.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.677+0000 7f25258ee740 1 --2- 172.21.15.6:0/1499302173 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b58f94cc00 0x55b58f94f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:18.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.677+0000 7f25258ee740 4 client.25439 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-12T03:25:18.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.677+0000 7f25258ee740 1 -- 172.21.15.6:0/1499302173 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x7f2518003cc0 con 0x55b58f94cc00 2024-09-12T03:25:18.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.678+0000 7f25212b1640 1 --2- 172.21.15.6:0/1499302173 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b58f94cc00 0x55b58f94f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:18.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.679+0000 7f25212b1640 1 --2- 172.21.15.6:0/1499302173 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b58f94cc00 0x55b58f94f020 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79d94af46a2c8d68 server_cookie=4f57d8b8e34ca876 in_seq=0 out_seq=0 2024-09-12T03:25:18.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:18.679+0000 7f2504ff9640 10 client.25439 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:19.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:19 smithi006 ceph-mon[30991]: pgmap v461: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 323 KiB/s rd, 77 MiB/s wr, 8.23k op/s 2024-09-12T03:25:19.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:19 smithi161 ceph-mon[35485]: pgmap v461: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 323 KiB/s rd, 77 MiB/s wr, 8.23k op/s 2024-09-12T03:25:19.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:19 smithi118 ceph-mon[35048]: pgmap v461: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 323 KiB/s rd, 77 MiB/s wr, 8.23k op/s 2024-09-12T03:25:20.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.216+0000 7f2504ff9640 1 -- 172.21.15.6:0/1499302173 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f2518003cc0 con 0x55b58f94cc00 2024-09-12T03:25:20.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.216+0000 7f2504ff9640 10 client.25439 handle_command_reply: tid=0 2024-09-12T03:25:20.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.216+0000 7f25258ee740 1 client.25439 shutdown 2024-09-12T03:25:20.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.216+0000 7f25258ee740 1 -- 172.21.15.6:0/1499302173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8be480 msgr2=0x55b58f8e4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:20.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.216+0000 7f25258ee740 1 --2- 172.21.15.6:0/1499302173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8be480 0x55b58f8e4860 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f250c0058c0 tx=0x7f250c097b70 comp rx=0 tx=0).stop 2024-09-12T03:25:20.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.216+0000 7f25258ee740 1 -- 172.21.15.6:0/1499302173 shutdown_connections 2024-09-12T03:25:20.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.216+0000 7f25258ee740 1 --2- 172.21.15.6:0/1499302173 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b58f94cc00 0x55b58f94f020 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:20.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.216+0000 7f25258ee740 1 --2- 172.21.15.6:0/1499302173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b58f8c0990 0x55b58f8e4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:20.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.216+0000 7f25258ee740 1 --2- 172.21.15.6:0/1499302173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b58f8be480 0x55b58f8e4860 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:20.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.216+0000 7f25258ee740 1 --2- 172.21.15.6:0/1499302173 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b58f8b7320 0x55b58f8e7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:20.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.216+0000 7f25258ee740 1 -- 172.21.15.6:0/1499302173 >> 172.21.15.6:0/1499302173 conn(0x55b58f8b2280 msgr2=0x55b58f8bc250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:20.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.217+0000 7f2504ff9640 0 client.25439 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:20.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.217+0000 7f25258ee740 1 -- 172.21.15.6:0/1499302173 shutdown_connections 2024-09-12T03:25:20.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.217+0000 7f25258ee740 1 -- 172.21.15.6:0/1499302173 wait complete. 2024-09-12T03:25:20.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.217+0000 7f25258ee740 20 client.25439 trim_cache size 0 max 16384 2024-09-12T03:25:20.219 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:25:20.219 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 27, 2024-09-12T03:25:20.219 INFO:teuthology.orchestra.run.smithi006.stdout: "leader": 24494, 2024-09-12T03:25:20.219 INFO:teuthology.orchestra.run.smithi006.stdout: "set_version": 18, 2024-09-12T03:25:20.219 INFO:teuthology.orchestra.run.smithi006.stdout: "sets": { 2024-09-12T03:25:20.219 INFO:teuthology.orchestra.run.smithi006.stdout: "6c171048": { 2024-09-12T03:25:20.220 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 18, 2024-09-12T03:25:20.220 INFO:teuthology.orchestra.run.smithi006.stdout: "age_ref": 0.0, 2024-09-12T03:25:20.220 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:25:20.220 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "QUIESCED", 2024-09-12T03:25:20.220 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:25:20.220 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:25:20.220 INFO:teuthology.orchestra.run.smithi006.stdout: "timeout": 90.0, 2024-09-12T03:25:20.220 INFO:teuthology.orchestra.run.smithi006.stdout: "expiration": 140.2, 2024-09-12T03:25:20.220 INFO:teuthology.orchestra.run.smithi006.stdout: "members": { 2024-09-12T03:25:20.220 INFO:teuthology.orchestra.run.smithi006.stdout: "file:/": { 2024-09-12T03:25:20.220 INFO:teuthology.orchestra.run.smithi006.stdout: "excluded": false, 2024-09-12T03:25:20.220 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:25:20.220 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "QUIESCED", 2024-09-12T03:25:20.221 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:25:20.221 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:25:20.221 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:25:20.221 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:25:20.221 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:25:20.221 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:25:20.221 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:25:20.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.218+0000 7f2507fff640 1 -- 172.21.15.6:0/1951784421 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f25080a2d50 msgr2=0x7f25080a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:20.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.218+0000 7f2507fff640 1 --2- 172.21.15.6:0/1951784421 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f25080a2d50 0x7f25080a5210 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2514004ab0 tx=0x7f2514005fd0 comp rx=0 tx=0).stop 2024-09-12T03:25:20.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.218+0000 7f2507fff640 1 -- 172.21.15.6:0/1951784421 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f251c101270 msgr2=0x7f251c1a4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:20.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.218+0000 7f2507fff640 1 --2- 172.21.15.6:0/1951784421 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f251c101270 0x7f251c1a4b90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f250c00d950 tx=0x7f250c00de10 comp rx=0 tx=0).stop 2024-09-12T03:25:20.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.218+0000 7f2507fff640 1 -- 172.21.15.6:0/1951784421 shutdown_connections 2024-09-12T03:25:20.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.218+0000 7f2507fff640 1 --2- 172.21.15.6:0/1951784421 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f25080a2d50 0x7f25080a5210 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:20.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.218+0000 7f2507fff640 1 --2- 172.21.15.6:0/1951784421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251c1068d0 0x7f251c19e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:20.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.218+0000 7f2507fff640 1 --2- 172.21.15.6:0/1951784421 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f251c101270 0x7f251c1a4b90 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:20.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.218+0000 7f2507fff640 1 --2- 172.21.15.6:0/1951784421 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f251c1008d0 0x7f251c1a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:20.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.218+0000 7f2507fff640 1 -- 172.21.15.6:0/1951784421 >> 172.21.15.6:0/1951784421 conn(0x7f251c0fc610 msgr2=0x7f251c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:20.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.218+0000 7f2507fff640 1 -- 172.21.15.6:0/1951784421 shutdown_connections 2024-09-12T03:25:20.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:20.218+0000 7f2507fff640 1 -- 172.21.15.6:0/1951784421 wait complete. 2024-09-12T03:25:20.981 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6c171048', quiesce took 7.3 seconds. Will release after: 12.849999999999998 2024-09-12T03:25:21.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:21 smithi006 ceph-mon[30991]: pgmap v462: 97 pgs: 97 active+clean; 13 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 320 KiB/s rd, 98 MiB/s wr, 10.25k op/s 2024-09-12T03:25:21.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:21 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:21.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:21 smithi161 ceph-mon[35485]: pgmap v462: 97 pgs: 97 active+clean; 13 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 320 KiB/s rd, 98 MiB/s wr, 10.25k op/s 2024-09-12T03:25:21.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:21 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:21.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:21 smithi118 ceph-mon[35048]: pgmap v462: 97 pgs: 97 active+clean; 13 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 320 KiB/s rd, 98 MiB/s wr, 10.25k op/s 2024-09-12T03:25:21.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:21 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:22.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:22 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:22.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:22 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:25:22.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:22 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:25:22.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:22 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:22.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:22 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:25:22.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:22 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:25:22.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:22 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:22.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:22 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:25:22.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:22 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:25:23.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:23 smithi006 ceph-mon[30991]: pgmap v463: 97 pgs: 97 active+clean; 13 GiB data, 90 GiB used, 983 GiB / 1.0 TiB avail; 551 KiB/s rd, 143 MiB/s wr, 14.73k op/s 2024-09-12T03:25:23.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:23 smithi161 ceph-mon[35485]: pgmap v463: 97 pgs: 97 active+clean; 13 GiB data, 90 GiB used, 983 GiB / 1.0 TiB avail; 551 KiB/s rd, 143 MiB/s wr, 14.73k op/s 2024-09-12T03:25:23.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:23 smithi118 ceph-mon[35048]: pgmap v463: 97 pgs: 97 active+clean; 13 GiB data, 90 GiB used, 983 GiB / 1.0 TiB avail; 551 KiB/s rd, 143 MiB/s wr, 14.73k op/s 2024-09-12T03:25:25.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:25 smithi006 ceph-mon[30991]: pgmap v464: 97 pgs: 97 active+clean; 13 GiB data, 90 GiB used, 982 GiB / 1.0 TiB avail; 550 KiB/s rd, 145 MiB/s wr, 14.70k op/s 2024-09-12T03:25:25.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:25 smithi161 ceph-mon[35485]: pgmap v464: 97 pgs: 97 active+clean; 13 GiB data, 90 GiB used, 982 GiB / 1.0 TiB avail; 550 KiB/s rd, 145 MiB/s wr, 14.70k op/s 2024-09-12T03:25:25.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:25 smithi118 ceph-mon[35048]: pgmap v464: 97 pgs: 97 active+clean; 13 GiB data, 90 GiB used, 982 GiB / 1.0 TiB avail; 550 KiB/s rd, 145 MiB/s wr, 14.70k op/s 2024-09-12T03:25:27.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:27 smithi006 ceph-mon[30991]: pgmap v465: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 549 KiB/s rd, 154 MiB/s wr, 15.78k op/s 2024-09-12T03:25:27.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:27 smithi161 ceph-mon[35485]: pgmap v465: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 549 KiB/s rd, 154 MiB/s wr, 15.78k op/s 2024-09-12T03:25:27.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:27 smithi118 ceph-mon[35048]: pgmap v465: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 549 KiB/s rd, 154 MiB/s wr, 15.78k op/s 2024-09-12T03:25:29.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:29 smithi006 ceph-mon[30991]: pgmap v466: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 552 KiB/s rd, 127 MiB/s wr, 12.79k op/s 2024-09-12T03:25:29.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:29 smithi161 ceph-mon[35485]: pgmap v466: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 552 KiB/s rd, 127 MiB/s wr, 12.79k op/s 2024-09-12T03:25:29.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:29 smithi118 ceph-mon[35048]: pgmap v466: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 552 KiB/s rd, 127 MiB/s wr, 12.79k op/s 2024-09-12T03:25:31.133 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:25:31.395 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:31.516 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:31 smithi006 ceph-mon[30991]: pgmap v467: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 233 KiB/s rd, 106 MiB/s wr, 10.63k op/s 2024-09-12T03:25:31.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:31 smithi161 ceph-mon[35485]: pgmap v467: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 233 KiB/s rd, 106 MiB/s wr, 10.63k op/s 2024-09-12T03:25:31.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:31 smithi118 ceph-mon[35048]: pgmap v467: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 233 KiB/s rd, 106 MiB/s wr, 10.63k op/s 2024-09-12T03:25:31.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.918+0000 7f614e3c9640 1 -- 172.21.15.6:0/1241866343 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61481082a0 msgr2=0x7f614810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:31.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.918+0000 7f614e3c9640 1 --2- 172.21.15.6:0/1241866343 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61481082a0 0x7f614810ec40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6140009cd0 tx=0x7f614002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.918+0000 7f614e3c9640 1 -- 172.21.15.6:0/1241866343 shutdown_connections 2024-09-12T03:25:31.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.918+0000 7f614e3c9640 1 --2- 172.21.15.6:0/1241866343 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61481082a0 0x7f614810ec40 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.918+0000 7f614e3c9640 1 --2- 172.21.15.6:0/1241866343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61481036e0 0x7f6148107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.918+0000 7f614e3c9640 1 --2- 172.21.15.6:0/1241866343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6148102d30 0x7f6148103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.918+0000 7f614e3c9640 1 -- 172.21.15.6:0/1241866343 >> 172.21.15.6:0/1241866343 conn(0x7f61480fc7d0 msgr2=0x7f61480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:31.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.918+0000 7f614e3c9640 1 -- 172.21.15.6:0/1241866343 shutdown_connections 2024-09-12T03:25:31.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.918+0000 7f614e3c9640 1 -- 172.21.15.6:0/1241866343 wait complete. 2024-09-12T03:25:31.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.919+0000 7f614e3c9640 1 Processor -- start 2024-09-12T03:25:31.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.919+0000 7f614e3c9640 1 -- start start 2024-09-12T03:25:31.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.919+0000 7f614e3c9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6148102d30 0x7f6148100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.919+0000 7f614e3c9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61481036e0 0x7f6148100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.919+0000 7f614e3c9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61481082a0 0x7f6148101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.919+0000 7f614e3c9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f61481120a0 con 0x7f61481082a0 2024-09-12T03:25:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.919+0000 7f614e3c9640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f6148111f20 con 0x7f6148102d30 2024-09-12T03:25:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.919+0000 7f614e3c9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6148112220 con 0x7f61481036e0 2024-09-12T03:25:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.919+0000 7f614c966640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61481082a0 0x7f6148101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.919+0000 7f614c966640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61481082a0 0x7f6148101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56596/0 (socket says 172.21.15.6:56596) 2024-09-12T03:25:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.919+0000 7f614c966640 1 -- 172.21.15.6:0/1009483373 learned_addr learned my addr 172.21.15.6:0/1009483373 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.919+0000 7f6147fff640 1 --2- 172.21.15.6:0/1009483373 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6148102d30 0x7f6148100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.919+0000 7f613ffff640 1 --2- 172.21.15.6:0/1009483373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61481036e0 0x7f6148100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.920+0000 7f614c966640 1 -- 172.21.15.6:0/1009483373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61481036e0 msgr2=0x7f6148100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.920+0000 7f614c966640 1 --2- 172.21.15.6:0/1009483373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61481036e0 0x7f6148100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.920+0000 7f614c966640 1 -- 172.21.15.6:0/1009483373 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6148102d30 msgr2=0x7f6148100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.920+0000 7f614c966640 1 --2- 172.21.15.6:0/1009483373 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6148102d30 0x7f6148100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.920+0000 7f614c966640 1 -- 172.21.15.6:0/1009483373 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61481a6a00 con 0x7f61481082a0 2024-09-12T03:25:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.920+0000 7f6147fff640 1 --2- 172.21.15.6:0/1009483373 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6148102d30 0x7f6148100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:25:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.920+0000 7f614c966640 1 --2- 172.21.15.6:0/1009483373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61481082a0 0x7f6148101410 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6140005cc0 tx=0x7f6140032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.920+0000 7f6145ffb640 1 -- 172.21.15.6:0/1009483373 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6140004040 con 0x7f61481082a0 2024-09-12T03:25:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.920+0000 7f614e3c9640 1 -- 172.21.15.6:0/1009483373 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61481a6c90 con 0x7f61481082a0 2024-09-12T03:25:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.920+0000 7f6145ffb640 1 -- 172.21.15.6:0/1009483373 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61400046c0 con 0x7f61481082a0 2024-09-12T03:25:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.920+0000 7f6145ffb640 1 -- 172.21.15.6:0/1009483373 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6140030e00 con 0x7f61481082a0 2024-09-12T03:25:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.920+0000 7f614e3c9640 1 -- 172.21.15.6:0/1009483373 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61481a7220 con 0x7f61481082a0 2024-09-12T03:25:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.922+0000 7f6145ffb640 1 -- 172.21.15.6:0/1009483373 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f61400305a0 con 0x7f61481082a0 2024-09-12T03:25:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.923+0000 7f61507a2740 1 Processor -- start 2024-09-12T03:25:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.923+0000 7f61507a2740 1 -- start start 2024-09-12T03:25:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.923+0000 7f61507a2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc035c84a0 0x55fc035c8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.923+0000 7f61507a2740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c8dc0 0x55fc035d3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.923+0000 7f61507a2740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fc035d3680 0x55fc035d5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.923+0000 7f61507a2740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55fc0352e2e0 con 0x55fc035c8dc0 2024-09-12T03:25:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.923+0000 7f61507a2740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55fc034d5ce0 con 0x55fc035d3680 2024-09-12T03:25:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.923+0000 7f61507a2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fc0352a2a0 con 0x55fc035c84a0 2024-09-12T03:25:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.923+0000 7f6145ffb640 1 --2- 172.21.15.6:0/1009483373 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f61340a2b80 0x7f61340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.923+0000 7f6147fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c8dc0 0x55fc035d3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.923+0000 7f613ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc035c84a0 0x55fc035c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.923+0000 7f6147fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c8dc0 0x55fc035d3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56598/0 (socket says 172.21.15.6:56598) 2024-09-12T03:25:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.923+0000 7f6147fff640 1 -- 172.21.15.6:0/2495744809 learned_addr learned my addr 172.21.15.6:0/2495744809 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.923+0000 7f613ffff640 1 --2- 172.21.15.6:0/2495744809 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fc035d3680 0x55fc035d5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.923+0000 7f614c966640 1 --2- 172.21.15.6:0/1009483373 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f61340a2b80 0x7f61340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:31.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.924+0000 7f6145ffb640 1 -- 172.21.15.6:0/1009483373 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f614002e030 con 0x7f61481082a0 2024-09-12T03:25:31.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.924+0000 7f613ffff640 1 -- 172.21.15.6:0/2495744809 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fc035d3680 msgr2=0x55fc035d5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:31.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.924+0000 7f613ffff640 1 --2- 172.21.15.6:0/2495744809 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fc035d3680 0x55fc035d5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.924+0000 7f613ffff640 1 -- 172.21.15.6:0/2495744809 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c8dc0 msgr2=0x55fc035d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:31.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.924+0000 7f613ffff640 1 --2- 172.21.15.6:0/2495744809 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c8dc0 0x55fc035d3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.924+0000 7f613ffff640 1 -- 172.21.15.6:0/2495744809 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc034ba440 con 0x55fc035c84a0 2024-09-12T03:25:31.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.924+0000 7f613ffff640 1 --2- 172.21.15.6:0/2495744809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc035c84a0 0x55fc035c8880 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6148102420 tx=0x7f6130009900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caeb666b1d77bc90 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:31.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.924+0000 7f614c966640 1 --2- 172.21.15.6:0/1009483373 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f61340a2b80 0x7f61340a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f61400ee360 tx=0x7f61400dd040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:31.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.924+0000 7f613e7fc640 1 -- 172.21.15.6:0/2495744809 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130005020 con 0x55fc035c84a0 2024-09-12T03:25:31.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.924+0000 7f613e7fc640 1 -- 172.21.15.6:0/2495744809 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61300047a0 con 0x55fc035c84a0 2024-09-12T03:25:31.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.924+0000 7f613e7fc640 1 -- 172.21.15.6:0/2495744809 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613000f6e0 con 0x55fc035c84a0 2024-09-12T03:25:31.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.925+0000 7f61507a2740 1 -- 172.21.15.6:0/2495744809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc035c84a0 msgr2=0x55fc035c8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:31.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.925+0000 7f61507a2740 1 --2- 172.21.15.6:0/2495744809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc035c84a0 0x55fc035c8880 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6148102420 tx=0x7f6130009900 comp rx=0 tx=0).stop 2024-09-12T03:25:31.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.925+0000 7f61507a2740 1 -- 172.21.15.6:0/2495744809 shutdown_connections 2024-09-12T03:25:31.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.925+0000 7f61507a2740 1 --2- 172.21.15.6:0/2495744809 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fc035d3680 0x55fc035d5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.925+0000 7f61507a2740 1 --2- 172.21.15.6:0/2495744809 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c8dc0 0x55fc035d3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.925+0000 7f61507a2740 1 --2- 172.21.15.6:0/2495744809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc035c84a0 0x55fc035c8880 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.925+0000 7f61507a2740 1 -- 172.21.15.6:0/2495744809 >> 172.21.15.6:0/2495744809 conn(0x55fc035c4a30 msgr2=0x55fc035c4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:31.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.925+0000 7f61507a2740 1 -- 172.21.15.6:0/2495744809 shutdown_connections 2024-09-12T03:25:31.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.925+0000 7f61507a2740 1 -- 172.21.15.6:0/2495744809 wait complete. 2024-09-12T03:25:31.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.925+0000 7f61507a2740 1 -- start start 2024-09-12T03:25:31.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.925+0000 7f61507a2740 1 Processor -- start 2024-09-12T03:25:31.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.926+0000 7f61507a2740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c84a0 0x55fc035d2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.926+0000 7f61507a2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc035c8dc0 0x55fc035d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.926+0000 7f61507a2740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fc035d3680 0x55fc035d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.926+0000 7f61507a2740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55fc035d6250 con 0x55fc035c84a0 2024-09-12T03:25:31.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.926+0000 7f61507a2740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55fc034e9050 con 0x55fc035d3680 2024-09-12T03:25:31.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.926+0000 7f61507a2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fc035d63d0 con 0x55fc035c8dc0 2024-09-12T03:25:31.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.926+0000 7f61507a2740 10 client.0 _ll_register_callbacks cb 0x55fc03529160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:31.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.926+0000 7f613ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c84a0 0x55fc035d2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:31.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.926+0000 7f613ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c84a0 0x55fc035d2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56612/0 (socket says 172.21.15.6:56612) 2024-09-12T03:25:31.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.926+0000 7f613ffff640 1 -- 172.21.15.6:0/4171598995 learned_addr learned my addr 172.21.15.6:0/4171598995 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:31.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.926+0000 7f6147fff640 1 --2- 172.21.15.6:0/4171598995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc035c8dc0 0x55fc035d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:31.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.926+0000 7f613ffff640 1 -- 172.21.15.6:0/4171598995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc035c8dc0 msgr2=0x55fc035d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:31.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.926+0000 7f613ffff640 1 --2- 172.21.15.6:0/4171598995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc035c8dc0 0x55fc035d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.926+0000 7f613ffff640 1 -- 172.21.15.6:0/4171598995 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fc035d3680 msgr2=0x55fc035d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:31.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.926+0000 7f613ffff640 1 --2- 172.21.15.6:0/4171598995 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fc035d3680 0x55fc035d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.926+0000 7f613ffff640 1 -- 172.21.15.6:0/4171598995 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc0329f3c0 con 0x55fc035c84a0 2024-09-12T03:25:31.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.926+0000 7f613ffff640 1 --2- 172.21.15.6:0/4171598995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c84a0 0x55fc035d2fd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6130009250 tx=0x7f6130009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5031c3c52591becd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:31.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.926+0000 7f61277fe640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:25:31.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.926+0000 7f61277fe640 1 -- 172.21.15.6:0/4171598995 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130005570 con 0x55fc035c84a0 2024-09-12T03:25:31.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.926+0000 7f61277fe640 1 -- 172.21.15.6:0/4171598995 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f613000f480 con 0x55fc035c84a0 2024-09-12T03:25:31.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.927+0000 7f61507a2740 1 -- 172.21.15.6:0/4171598995 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61300047a0 con 0x55fc035c84a0 2024-09-12T03:25:31.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.927+0000 7f61277fe640 1 -- 172.21.15.6:0/4171598995 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f613000f680 con 0x55fc035c84a0 2024-09-12T03:25:31.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.927+0000 7f61507a2740 10 client.15894 fetch_fsmap learned FSMap version 27 2024-09-12T03:25:31.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.927+0000 7f61507a2740 1 -- 172.21.15.6:0/4171598995 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fc035e1c50 con 0x55fc035c84a0 2024-09-12T03:25:31.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.927+0000 7f61277fe640 1 -- 172.21.15.6:0/4171598995 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61300047a0 con 0x55fc035c84a0 2024-09-12T03:25:31.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.927+0000 7f61277fe640 1 -- 172.21.15.6:0/4171598995 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f613001a050 con 0x55fc035c84a0 2024-09-12T03:25:31.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.927+0000 7f61507a2740 10 client.15894 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:25:31.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.927+0000 7f61507a2740 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:25:31.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.927+0000 7f61507a2740 20 client.15894 populate_metadata read hostname 'smithi006' 2024-09-12T03:25:31.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.927+0000 7f61507a2740 1 --2- 172.21.15.6:0/4171598995 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55fc035e3110 0x55fc035e5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.927+0000 7f61507a2740 4 client.15894 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:25:31.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.927+0000 7f61507a2740 1 -- 172.21.15.6:0/4171598995 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc0353b850 con 0x55fc035e3110 2024-09-12T03:25:31.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.927+0000 7f6147fff640 1 --2- 172.21.15.6:0/4171598995 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55fc035e3110 0x55fc035e5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:31.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.928+0000 7f61277fe640 10 client.15894 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:31.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.928+0000 7f6147fff640 1 --2- 172.21.15.6:0/4171598995 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55fc035e3110 0x55fc035e5500 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca886fd5dcae3619 server_cookie=a89a3875a77b2c99 in_seq=0 out_seq=0 2024-09-12T03:25:31.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.929+0000 7f61277fe640 1 -- 172.21.15.6:0/4171598995 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc0353b850 con 0x55fc035e3110 2024-09-12T03:25:31.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.929+0000 7f61277fe640 10 client.15894 handle_command_reply: tid=0 2024-09-12T03:25:31.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.929+0000 7f61507a2740 1 client.15894 shutdown 2024-09-12T03:25:31.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.929+0000 7f61507a2740 1 -- 172.21.15.6:0/4171598995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c84a0 msgr2=0x55fc035d2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:31.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.929+0000 7f61507a2740 1 --2- 172.21.15.6:0/4171598995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c84a0 0x55fc035d2fd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6130009250 tx=0x7f6130009370 comp rx=0 tx=0).stop 2024-09-12T03:25:31.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.929+0000 7f61507a2740 1 -- 172.21.15.6:0/4171598995 shutdown_connections 2024-09-12T03:25:31.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.929+0000 7f61507a2740 1 --2- 172.21.15.6:0/4171598995 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55fc035e3110 0x55fc035e5500 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.929+0000 7f61507a2740 1 --2- 172.21.15.6:0/4171598995 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fc035d3680 0x55fc035d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.929+0000 7f61507a2740 1 --2- 172.21.15.6:0/4171598995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc035c8dc0 0x55fc035d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.929+0000 7f61507a2740 1 --2- 172.21.15.6:0/4171598995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c84a0 0x55fc035d2fd0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.929+0000 7f61507a2740 1 -- 172.21.15.6:0/4171598995 >> 172.21.15.6:0/4171598995 conn(0x55fc035c4a30 msgr2=0x55fc035ca760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:31.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.929+0000 7f61277fe640 0 client.15894 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:31.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.929+0000 7f61507a2740 1 -- 172.21.15.6:0/4171598995 shutdown_connections 2024-09-12T03:25:31.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.929+0000 7f61507a2740 1 -- 172.21.15.6:0/4171598995 wait complete. 2024-09-12T03:25:31.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.929+0000 7f61507a2740 20 client.15894 trim_cache size 0 max 16384 2024-09-12T03:25:31.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.946+0000 7f61507a2740 1 Processor -- start 2024-09-12T03:25:31.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.946+0000 7f61507a2740 1 -- start start 2024-09-12T03:25:31.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.947+0000 7f61507a2740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c84a0 0x55fc035ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.947+0000 7f61507a2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc035c8dc0 0x55fc035eacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.947+0000 7f61507a2740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fc035d3680 0x55fc035eb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.947+0000 7f61507a2740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f61481b1f70 con 0x55fc035c84a0 2024-09-12T03:25:31.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.947+0000 7f61507a2740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f61481af840 con 0x55fc035d3680 2024-09-12T03:25:31.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.947+0000 7f61507a2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f61481ad690 con 0x55fc035c8dc0 2024-09-12T03:25:31.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.947+0000 7f613ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c84a0 0x55fc035ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:31.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.947+0000 7f613ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c84a0 0x55fc035ceb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56626/0 (socket says 172.21.15.6:56626) 2024-09-12T03:25:31.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.947+0000 7f613ffff640 1 -- 172.21.15.6:0/1279399140 learned_addr learned my addr 172.21.15.6:0/1279399140 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:31.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.947+0000 7f6147fff640 1 --2- 172.21.15.6:0/1279399140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc035c8dc0 0x55fc035eacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:31.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.947+0000 7f613ffff640 1 -- 172.21.15.6:0/1279399140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc035c8dc0 msgr2=0x55fc035eacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:31.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.947+0000 7f613ffff640 1 --2- 172.21.15.6:0/1279399140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc035c8dc0 0x55fc035eacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.947+0000 7f613ffff640 1 -- 172.21.15.6:0/1279399140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fc035d3680 msgr2=0x55fc035eb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:31.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.947+0000 7f613ffff640 1 --2- 172.21.15.6:0/1279399140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fc035d3680 0x55fc035eb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.947+0000 7f613ffff640 1 -- 172.21.15.6:0/1279399140 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc035c5ae0 con 0x55fc035c84a0 2024-09-12T03:25:31.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.947+0000 7f613ffff640 1 --2- 172.21.15.6:0/1279399140 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c84a0 0x55fc035ceb50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f61300097e0 tx=0x7f6130009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d197b13b366643 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:31.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.947+0000 7f613dffb640 1 -- 172.21.15.6:0/1279399140 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130009d10 con 0x55fc035c84a0 2024-09-12T03:25:31.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.947+0000 7f613dffb640 1 -- 172.21.15.6:0/1279399140 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6130036070 con 0x55fc035c84a0 2024-09-12T03:25:31.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.947+0000 7f613dffb640 1 -- 172.21.15.6:0/1279399140 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613000fa80 con 0x55fc035c84a0 2024-09-12T03:25:31.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.948+0000 7f61507a2740 1 -- 172.21.15.6:0/1279399140 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c84a0 msgr2=0x55fc035ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:31.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.948+0000 7f61507a2740 1 --2- 172.21.15.6:0/1279399140 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c84a0 0x55fc035ceb50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f61300097e0 tx=0x7f6130009850 comp rx=0 tx=0).stop 2024-09-12T03:25:31.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.948+0000 7f61507a2740 1 -- 172.21.15.6:0/1279399140 shutdown_connections 2024-09-12T03:25:31.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.948+0000 7f61507a2740 1 --2- 172.21.15.6:0/1279399140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fc035d3680 0x55fc035eb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.948+0000 7f61507a2740 1 --2- 172.21.15.6:0/1279399140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc035c8dc0 0x55fc035eacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.948+0000 7f61507a2740 1 --2- 172.21.15.6:0/1279399140 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c84a0 0x55fc035ceb50 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.948+0000 7f61507a2740 1 -- 172.21.15.6:0/1279399140 >> 172.21.15.6:0/1279399140 conn(0x55fc035c4a30 msgr2=0x55fc035c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:31.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.948+0000 7f61507a2740 1 -- 172.21.15.6:0/1279399140 shutdown_connections 2024-09-12T03:25:31.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.948+0000 7f61507a2740 1 -- 172.21.15.6:0/1279399140 wait complete. 2024-09-12T03:25:31.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.948+0000 7f61507a2740 1 -- start start 2024-09-12T03:25:31.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.949+0000 7f61507a2740 1 Processor -- start 2024-09-12T03:25:31.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.949+0000 7f61507a2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc035c84a0 0x55fc035db670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.949+0000 7f61507a2740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c8dc0 0x55fc035dbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.949+0000 7f61507a2740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fc035d3680 0x55fc035d8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.949+0000 7f61507a2740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f61481add00 con 0x55fc035c8dc0 2024-09-12T03:25:31.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.949+0000 7f61507a2740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f61481af170 con 0x55fc035d3680 2024-09-12T03:25:31.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.949+0000 7f61507a2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f61481ae3d0 con 0x55fc035c84a0 2024-09-12T03:25:31.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.949+0000 7f61507a2740 10 client.0 _ll_register_callbacks cb 0x7f613800b500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:31.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.949+0000 7f6147fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c8dc0 0x55fc035dbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:31.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.949+0000 7f6147fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c8dc0 0x55fc035dbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56632/0 (socket says 172.21.15.6:56632) 2024-09-12T03:25:31.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.949+0000 7f6147fff640 1 -- 172.21.15.6:0/2424363992 learned_addr learned my addr 172.21.15.6:0/2424363992 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:31.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.949+0000 7f613ffff640 1 --2- 172.21.15.6:0/2424363992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc035c84a0 0x55fc035db670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:31.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.949+0000 7f613ffff640 1 --2- 172.21.15.6:0/2424363992 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fc035d3680 0x55fc035d8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:31.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.950+0000 7f6147fff640 1 -- 172.21.15.6:0/2424363992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc035c84a0 msgr2=0x55fc035db670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:31.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.950+0000 7f6147fff640 1 --2- 172.21.15.6:0/2424363992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc035c84a0 0x55fc035db670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.950+0000 7f6147fff640 1 -- 172.21.15.6:0/2424363992 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fc035d3680 msgr2=0x55fc035d8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.950+0000 7f6147fff640 1 --2- 172.21.15.6:0/2424363992 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fc035d3680 0x55fc035d8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.950+0000 7f6147fff640 1 -- 172.21.15.6:0/2424363992 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc035d92d0 con 0x55fc035c8dc0 2024-09-12T03:25:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.950+0000 7f613ffff640 1 --2- 172.21.15.6:0/2424363992 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fc035d3680 0x55fc035d8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:25:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.950+0000 7f6147fff640 1 --2- 172.21.15.6:0/2424363992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c8dc0 0x55fc035dbbb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6138005fd0 tx=0x7f6138004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a952921d32a5ce41 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.950+0000 7f61277fe640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:25:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.950+0000 7f61277fe640 1 -- 172.21.15.6:0/2424363992 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138020020 con 0x55fc035c8dc0 2024-09-12T03:25:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.950+0000 7f61507a2740 1 -- 172.21.15.6:0/2424363992 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6130036070 con 0x55fc035c8dc0 2024-09-12T03:25:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.950+0000 7f61277fe640 1 -- 172.21.15.6:0/2424363992 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f613801c340 con 0x55fc035c8dc0 2024-09-12T03:25:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.950+0000 7f61277fe640 1 -- 172.21.15.6:0/2424363992 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f613801c540 con 0x55fc035c8dc0 2024-09-12T03:25:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.950+0000 7f61507a2740 10 client.15900 fetch_fsmap learned FSMap version 27 2024-09-12T03:25:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.950+0000 7f61507a2740 1 -- 172.21.15.6:0/2424363992 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fc0364d7f0 con 0x55fc035c8dc0 2024-09-12T03:25:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.951+0000 7f61277fe640 1 -- 172.21.15.6:0/2424363992 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6130036070 con 0x55fc035c8dc0 2024-09-12T03:25:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.951+0000 7f61277fe640 1 -- 172.21.15.6:0/2424363992 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f61380119d0 con 0x55fc035c8dc0 2024-09-12T03:25:31.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.951+0000 7f61507a2740 10 client.15900 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:25:31.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.951+0000 7f61507a2740 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:25:31.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.951+0000 7f61507a2740 20 client.15900 populate_metadata read hostname 'smithi006' 2024-09-12T03:25:31.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.951+0000 7f61507a2740 1 --2- 172.21.15.6:0/2424363992 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55fc0364df80 0x55fc036503a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.951+0000 7f61507a2740 4 client.15900 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:25:31.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.951+0000 7f61507a2740 1 -- 172.21.15.6:0/2424363992 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fc03535d60 con 0x55fc0364df80 2024-09-12T03:25:31.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.951+0000 7f613ffff640 1 --2- 172.21.15.6:0/2424363992 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55fc0364df80 0x55fc036503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:31.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.952+0000 7f61277fe640 10 client.15900 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:31.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.952+0000 7f613ffff640 1 --2- 172.21.15.6:0/2424363992 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55fc0364df80 0x55fc036503a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d67424016c029966 server_cookie=76eb9b7ff0af353a in_seq=0 out_seq=0 2024-09-12T03:25:31.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.952+0000 7f61277fe640 1 -- 172.21.15.6:0/2424363992 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fc03535d60 con 0x55fc0364df80 2024-09-12T03:25:31.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.952+0000 7f61277fe640 10 client.15900 handle_command_reply: tid=0 2024-09-12T03:25:31.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.952+0000 7f61507a2740 1 client.15900 shutdown 2024-09-12T03:25:31.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.952+0000 7f61507a2740 1 -- 172.21.15.6:0/2424363992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c8dc0 msgr2=0x55fc035dbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:31.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.952+0000 7f61507a2740 1 --2- 172.21.15.6:0/2424363992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c8dc0 0x55fc035dbbb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6138005fd0 tx=0x7f6138004410 comp rx=0 tx=0).stop 2024-09-12T03:25:31.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.952+0000 7f61507a2740 1 -- 172.21.15.6:0/2424363992 shutdown_connections 2024-09-12T03:25:31.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.952+0000 7f61507a2740 1 --2- 172.21.15.6:0/2424363992 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55fc0364df80 0x55fc036503a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.952+0000 7f61507a2740 1 --2- 172.21.15.6:0/2424363992 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fc035d3680 0x55fc035d8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.952+0000 7f61507a2740 1 --2- 172.21.15.6:0/2424363992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fc035c8dc0 0x55fc035dbbb0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.952+0000 7f61507a2740 1 --2- 172.21.15.6:0/2424363992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc035c84a0 0x55fc035db670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.952+0000 7f61507a2740 1 -- 172.21.15.6:0/2424363992 >> 172.21.15.6:0/2424363992 conn(0x55fc035c4a30 msgr2=0x55fc035d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:31.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.952+0000 7f61277fe640 0 client.15900 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:31.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.953+0000 7f61507a2740 1 -- 172.21.15.6:0/2424363992 shutdown_connections 2024-09-12T03:25:31.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.953+0000 7f61507a2740 1 -- 172.21.15.6:0/2424363992 wait complete. 2024-09-12T03:25:31.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.953+0000 7f61507a2740 20 client.15900 trim_cache size 0 max 16384 2024-09-12T03:25:31.959 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:25:31.959 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2024-09-12T03:25:31.959 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2024-09-12T03:25:31.960 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:25:31.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.953+0000 7f614e3c9640 1 -- 172.21.15.6:0/1009483373 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f61340a2b80 msgr2=0x7f61340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:31.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.953+0000 7f614e3c9640 1 --2- 172.21.15.6:0/1009483373 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f61340a2b80 0x7f61340a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f61400ee360 tx=0x7f61400dd040 comp rx=0 tx=0).stop 2024-09-12T03:25:31.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.953+0000 7f614e3c9640 1 -- 172.21.15.6:0/1009483373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61481082a0 msgr2=0x7f6148101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:31.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.953+0000 7f614e3c9640 1 --2- 172.21.15.6:0/1009483373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61481082a0 0x7f6148101410 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6140005cc0 tx=0x7f6140032f40 comp rx=0 tx=0).stop 2024-09-12T03:25:31.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.953+0000 7f614e3c9640 1 -- 172.21.15.6:0/1009483373 shutdown_connections 2024-09-12T03:25:31.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.953+0000 7f614e3c9640 1 --2- 172.21.15.6:0/1009483373 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f61340a2b80 0x7f61340a5040 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.953+0000 7f614e3c9640 1 --2- 172.21.15.6:0/1009483373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61481082a0 0x7f6148101410 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.953+0000 7f614e3c9640 1 --2- 172.21.15.6:0/1009483373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61481036e0 0x7f6148100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.953+0000 7f614e3c9640 1 --2- 172.21.15.6:0/1009483373 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6148102d30 0x7f6148100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.953+0000 7f614e3c9640 1 -- 172.21.15.6:0/1009483373 >> 172.21.15.6:0/1009483373 conn(0x7f61480fc7d0 msgr2=0x7f6148105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:31.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.953+0000 7f614e3c9640 1 -- 172.21.15.6:0/1009483373 shutdown_connections 2024-09-12T03:25:31.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:31.953+0000 7f614e3c9640 1 -- 172.21.15.6:0/1009483373 wait complete. 2024-09-12T03:25:32.715 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:25:32.715 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:25:32.976 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:33.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:33 smithi006 ceph-mon[30991]: pgmap v468: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 236 KiB/s rd, 93 MiB/s wr, 9.35k op/s 2024-09-12T03:25:33.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.499+0000 7f6549bc3640 1 -- 172.21.15.6:0/665232124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6544100e20 msgr2=0x7f6544101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:33.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.499+0000 7f6549bc3640 1 --2- 172.21.15.6:0/665232124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6544100e20 0x7f6544101200 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6534009900 tx=0x7f653402a480 comp rx=0 tx=0).stop 2024-09-12T03:25:33.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.499+0000 7f6549bc3640 1 -- 172.21.15.6:0/665232124 shutdown_connections 2024-09-12T03:25:33.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.499+0000 7f6549bc3640 1 --2- 172.21.15.6:0/665232124 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6544102ef0 0x7f654410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:33.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.499+0000 7f6549bc3640 1 --2- 172.21.15.6:0/665232124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6544102570 0x7f65441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:33.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.499+0000 7f6549bc3640 1 --2- 172.21.15.6:0/665232124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6544100e20 0x7f6544101200 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:33.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.499+0000 7f6549bc3640 1 -- 172.21.15.6:0/665232124 >> 172.21.15.6:0/665232124 conn(0x7f65440fc7d0 msgr2=0x7f65440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:33.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.499+0000 7f6549bc3640 1 -- 172.21.15.6:0/665232124 shutdown_connections 2024-09-12T03:25:33.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.500+0000 7f6549bc3640 1 -- 172.21.15.6:0/665232124 wait complete. 2024-09-12T03:25:33.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.500+0000 7f6549bc3640 1 Processor -- start 2024-09-12T03:25:33.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.500+0000 7f6549bc3640 1 -- start start 2024-09-12T03:25:33.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.501+0000 7f6549bc3640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6544100e20 0x7f65440fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:33.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.501+0000 7f6549bc3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6544102570 0x7f65440fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:33.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.501+0000 7f6549bc3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6544102ef0 0x7f65440fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:33.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.501+0000 7f6549bc3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6544110f50 con 0x7f6544102ef0 2024-09-12T03:25:33.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.501+0000 7f6549bc3640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f6544110dd0 con 0x7f6544100e20 2024-09-12T03:25:33.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.501+0000 7f6549bc3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f65441110d0 con 0x7f6544102570 2024-09-12T03:25:33.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.501+0000 7f6543fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6544102ef0 0x7f65440fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:33.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.501+0000 7f653bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6544102570 0x7f65440fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:33.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.501+0000 7f653bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6544102570 0x7f65440fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:50000/0 (socket says 172.21.15.6:50000) 2024-09-12T03:25:33.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.501+0000 7f6543fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6544102ef0 0x7f65440fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56650/0 (socket says 172.21.15.6:56650) 2024-09-12T03:25:33.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.501+0000 7f653bfff640 1 -- 172.21.15.6:0/3889161151 learned_addr learned my addr 172.21.15.6:0/3889161151 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:33.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.501+0000 7f65437fe640 1 --2- 172.21.15.6:0/3889161151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6544100e20 0x7f65440fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:33.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.501+0000 7f653bfff640 1 -- 172.21.15.6:0/3889161151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6544100e20 msgr2=0x7f65440fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:33.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.501+0000 7f653bfff640 1 --2- 172.21.15.6:0/3889161151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6544100e20 0x7f65440fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:33.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.501+0000 7f653bfff640 1 -- 172.21.15.6:0/3889161151 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6544102ef0 msgr2=0x7f65440fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:33.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.501+0000 7f653bfff640 1 --2- 172.21.15.6:0/3889161151 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6544102ef0 0x7f65440fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:33.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.501+0000 7f653bfff640 1 -- 172.21.15.6:0/3889161151 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65441a69e0 con 0x7f6544102570 2024-09-12T03:25:33.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.502+0000 7f653bfff640 1 --2- 172.21.15.6:0/3889161151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6544102570 0x7f65440fd1c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f652c00e9f0 tx=0x7f652c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:33.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.502+0000 7f65417fa640 1 -- 172.21.15.6:0/3889161151 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c00cd80 con 0x7f6544102570 2024-09-12T03:25:33.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.502+0000 7f6549bc3640 1 -- 172.21.15.6:0/3889161151 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65441a6cd0 con 0x7f6544102570 2024-09-12T03:25:33.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.502+0000 7f6549bc3640 1 -- 172.21.15.6:0/3889161151 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65441a7200 con 0x7f6544102570 2024-09-12T03:25:33.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.502+0000 7f65417fa640 1 -- 172.21.15.6:0/3889161151 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f652c010910 con 0x7f6544102570 2024-09-12T03:25:33.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.502+0000 7f65417fa640 1 -- 172.21.15.6:0/3889161151 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c018980 con 0x7f6544102570 2024-09-12T03:25:33.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.503+0000 7f6549bc3640 1 -- 172.21.15.6:0/3889161151 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6524005290 con 0x7f6544102570 2024-09-12T03:25:33.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.504+0000 7f65417fa640 1 -- 172.21.15.6:0/3889161151 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f652c010ac0 con 0x7f6544102570 2024-09-12T03:25:33.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.505+0000 7f65417fa640 1 --2- 172.21.15.6:0/3889161151 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f65300a2b80 0x7f65300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:33.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.505+0000 7f65437fe640 1 --2- 172.21.15.6:0/3889161151 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f65300a2b80 0x7f65300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:33.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.505+0000 7f65417fa640 1 -- 172.21.15.6:0/3889161151 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f652c0cf2c0 con 0x7f6544102570 2024-09-12T03:25:33.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.506+0000 7f65437fe640 1 --2- 172.21.15.6:0/3889161151 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f65300a2b80 0x7f65300a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6534005ab0 tx=0x7f6534005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:33.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.508+0000 7f65417fa640 1 -- 172.21.15.6:0/3889161151 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f652c09c1f0 con 0x7f6544102570 2024-09-12T03:25:33.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:33 smithi161 ceph-mon[35485]: pgmap v468: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 236 KiB/s rd, 93 MiB/s wr, 9.35k op/s 2024-09-12T03:25:33.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:33 smithi118 ceph-mon[35048]: pgmap v468: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 236 KiB/s rd, 93 MiB/s wr, 9.35k op/s 2024-09-12T03:25:33.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.727+0000 7f6549bc3640 1 -- 172.21.15.6:0/3889161151 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6524005580 con 0x7f6544102570 2024-09-12T03:25:33.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.728+0000 7f65417fa640 1 -- 172.21.15.6:0/3889161151 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11908 (secure 0 0 0) 0x7f652c09baf0 con 0x7f6544102570 2024-09-12T03:25:33.728 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:25:33.728 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"btime":"2024-09-12T03:24:26:920603+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:24:26.080047+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24463]},"id":1}]} 2024-09-12T03:25:33.729 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 27 2024-09-12T03:25:33.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.730+0000 7f6549bc3640 1 -- 172.21.15.6:0/3889161151 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f65300a2b80 msgr2=0x7f65300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:33.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.730+0000 7f6549bc3640 1 --2- 172.21.15.6:0/3889161151 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f65300a2b80 0x7f65300a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6534005ab0 tx=0x7f6534005a20 comp rx=0 tx=0).stop 2024-09-12T03:25:33.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.730+0000 7f6549bc3640 1 -- 172.21.15.6:0/3889161151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6544102570 msgr2=0x7f65440fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:33.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.730+0000 7f6549bc3640 1 --2- 172.21.15.6:0/3889161151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6544102570 0x7f65440fd1c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f652c00e9f0 tx=0x7f652c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:25:33.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.730+0000 7f6549bc3640 1 -- 172.21.15.6:0/3889161151 shutdown_connections 2024-09-12T03:25:33.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.730+0000 7f6549bc3640 1 --2- 172.21.15.6:0/3889161151 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f65300a2b80 0x7f65300a5040 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:33.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.730+0000 7f6549bc3640 1 --2- 172.21.15.6:0/3889161151 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6544102ef0 0x7f65440fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:33.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.730+0000 7f6549bc3640 1 --2- 172.21.15.6:0/3889161151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6544102570 0x7f65440fd1c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:33.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.730+0000 7f6549bc3640 1 --2- 172.21.15.6:0/3889161151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6544100e20 0x7f65440fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:33.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.730+0000 7f6549bc3640 1 -- 172.21.15.6:0/3889161151 >> 172.21.15.6:0/3889161151 conn(0x7f65440fc7d0 msgr2=0x7f654406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:33.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.730+0000 7f6549bc3640 1 -- 172.21.15.6:0/3889161151 shutdown_connections 2024-09-12T03:25:33.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:33.730+0000 7f6549bc3640 1 -- 172.21.15.6:0/3889161151 wait complete. 2024-09-12T03:25:33.832 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:25:34.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:34 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3889161151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:25:34.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:34 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3889161151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:25:34.701 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:25:34.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:34 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3889161151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:25:34.843 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:35.509 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:35 smithi006 ceph-mon[30991]: pgmap v469: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 48 MiB/s wr, 4.88k op/s 2024-09-12T03:25:35.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.509+0000 7fab34d91640 1 -- 172.21.15.6:0/2801894110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab30102570 msgr2=0x7fab301029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:35.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.509+0000 7fab34d91640 1 --2- 172.21.15.6:0/2801894110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab30102570 0x7fab301029b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fab24009900 tx=0x7fab2402a480 comp rx=0 tx=0).stop 2024-09-12T03:25:35.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.509+0000 7fab34d91640 1 -- 172.21.15.6:0/2801894110 shutdown_connections 2024-09-12T03:25:35.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.509+0000 7fab34d91640 1 --2- 172.21.15.6:0/2801894110 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab30102ef0 0x7fab3010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:35.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.509+0000 7fab34d91640 1 --2- 172.21.15.6:0/2801894110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab30102570 0x7fab301029b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:35.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.509+0000 7fab34d91640 1 --2- 172.21.15.6:0/2801894110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab30100e20 0x7fab30101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:35.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.509+0000 7fab34d91640 1 -- 172.21.15.6:0/2801894110 >> 172.21.15.6:0/2801894110 conn(0x7fab300fc7d0 msgr2=0x7fab300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:35.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.509+0000 7fab34d91640 1 -- 172.21.15.6:0/2801894110 shutdown_connections 2024-09-12T03:25:35.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.509+0000 7fab34d91640 1 -- 172.21.15.6:0/2801894110 wait complete. 2024-09-12T03:25:35.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.510+0000 7fab34d91640 1 Processor -- start 2024-09-12T03:25:35.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.510+0000 7fab34d91640 1 -- start start 2024-09-12T03:25:35.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.510+0000 7fab34d91640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab30100e20 0x7fab301984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:35.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.510+0000 7fab34d91640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab30102570 0x7fab301989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:35.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.510+0000 7fab34d91640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab30102ef0 0x7fab3019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:35.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.510+0000 7fab34d91640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fab30110f80 con 0x7fab30100e20 2024-09-12T03:25:35.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.510+0000 7fab34d91640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fab30110e00 con 0x7fab30102ef0 2024-09-12T03:25:35.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.510+0000 7fab34d91640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fab30111100 con 0x7fab30102570 2024-09-12T03:25:35.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.510+0000 7fab2dd9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab30102570 0x7fab301989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:35.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.510+0000 7fab2dd9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab30102570 0x7fab301989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:50014/0 (socket says 172.21.15.6:50014) 2024-09-12T03:25:35.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.510+0000 7fab2dd9b640 1 -- 172.21.15.6:0/3025452359 learned_addr learned my addr 172.21.15.6:0/3025452359 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:35.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.510+0000 7fab2e59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab30100e20 0x7fab301984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:35.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.510+0000 7fab2ed9d640 1 --2- 172.21.15.6:0/3025452359 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab30102ef0 0x7fab3019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:35.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.511+0000 7fab2dd9b640 1 -- 172.21.15.6:0/3025452359 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab30102ef0 msgr2=0x7fab3019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:35.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.511+0000 7fab2dd9b640 1 --2- 172.21.15.6:0/3025452359 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab30102ef0 0x7fab3019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:35.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.511+0000 7fab2dd9b640 1 -- 172.21.15.6:0/3025452359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab30100e20 msgr2=0x7fab301984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:35.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.511+0000 7fab2dd9b640 1 --2- 172.21.15.6:0/3025452359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab30100e20 0x7fab301984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:35.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.511+0000 7fab2dd9b640 1 -- 172.21.15.6:0/3025452359 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3019d460 con 0x7fab30102570 2024-09-12T03:25:35.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.511+0000 7fab2dd9b640 1 --2- 172.21.15.6:0/3025452359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab30102570 0x7fab301989f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fab24006010 tx=0x7fab24003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:35.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.511+0000 7fab1f7fe640 1 -- 172.21.15.6:0/3025452359 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2403f070 con 0x7fab30102570 2024-09-12T03:25:35.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.511+0000 7fab1f7fe640 1 -- 172.21.15.6:0/3025452359 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab24031c30 con 0x7fab30102570 2024-09-12T03:25:35.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.511+0000 7fab34d91640 1 -- 172.21.15.6:0/3025452359 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3019d6f0 con 0x7fab30102570 2024-09-12T03:25:35.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.511+0000 7fab1f7fe640 1 -- 172.21.15.6:0/3025452359 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24035030 con 0x7fab30102570 2024-09-12T03:25:35.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.511+0000 7fab34d91640 1 -- 172.21.15.6:0/3025452359 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab301a5030 con 0x7fab30102570 2024-09-12T03:25:35.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.512+0000 7fab34d91640 1 -- 172.21.15.6:0/3025452359 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab30101200 con 0x7fab30102570 2024-09-12T03:25:35.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.514+0000 7fab1f7fe640 1 -- 172.21.15.6:0/3025452359 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fab24042070 con 0x7fab30102570 2024-09-12T03:25:35.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.514+0000 7fab1f7fe640 1 --2- 172.21.15.6:0/3025452359 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fab180a2b80 0x7fab180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:35.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.514+0000 7fab2e59c640 1 --2- 172.21.15.6:0/3025452359 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fab180a2b80 0x7fab180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:35.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.514+0000 7fab1f7fe640 1 -- 172.21.15.6:0/3025452359 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fab240efb90 con 0x7fab30102570 2024-09-12T03:25:35.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.515+0000 7fab2e59c640 1 --2- 172.21.15.6:0/3025452359 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fab180a2b80 0x7fab180a5040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fab20009bd0 tx=0x7fab20005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:35.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.517+0000 7fab1f7fe640 1 -- 172.21.15.6:0/3025452359 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab240bc470 con 0x7fab30102570 2024-09-12T03:25:35.583 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:35.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:35 smithi161 ceph-mon[35485]: pgmap v469: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 48 MiB/s wr, 4.88k op/s 2024-09-12T03:25:35.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:35 smithi118 ceph-mon[35048]: pgmap v469: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 48 MiB/s wr, 4.88k op/s 2024-09-12T03:25:35.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.738+0000 7fab34d91640 1 -- 172.21.15.6:0/3025452359 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab3010a0a0 con 0x7fab30102570 2024-09-12T03:25:35.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.739+0000 7fab1f7fe640 1 -- 172.21.15.6:0/3025452359 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11908 (secure 0 0 0) 0x7fab240c0770 con 0x7fab30102570 2024-09-12T03:25:35.739 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:25:35.739 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"btime":"2024-09-12T03:24:26:920603+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:24:26.080047+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24463]},"id":1}]} 2024-09-12T03:25:35.740 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 27 2024-09-12T03:25:35.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.741+0000 7fab34d91640 1 -- 172.21.15.6:0/3025452359 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fab180a2b80 msgr2=0x7fab180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:35.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.741+0000 7fab34d91640 1 --2- 172.21.15.6:0/3025452359 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fab180a2b80 0x7fab180a5040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fab20009bd0 tx=0x7fab20005f90 comp rx=0 tx=0).stop 2024-09-12T03:25:35.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.741+0000 7fab34d91640 1 -- 172.21.15.6:0/3025452359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab30102570 msgr2=0x7fab301989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:35.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.741+0000 7fab34d91640 1 --2- 172.21.15.6:0/3025452359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab30102570 0x7fab301989f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fab24006010 tx=0x7fab24003c10 comp rx=0 tx=0).stop 2024-09-12T03:25:35.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.741+0000 7fab34d91640 1 -- 172.21.15.6:0/3025452359 shutdown_connections 2024-09-12T03:25:35.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.741+0000 7fab34d91640 1 --2- 172.21.15.6:0/3025452359 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fab180a2b80 0x7fab180a5040 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:35.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.741+0000 7fab34d91640 1 --2- 172.21.15.6:0/3025452359 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab30102ef0 0x7fab3019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:35.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.741+0000 7fab34d91640 1 --2- 172.21.15.6:0/3025452359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab30102570 0x7fab301989f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:35.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.741+0000 7fab34d91640 1 --2- 172.21.15.6:0/3025452359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab30100e20 0x7fab301984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:35.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.741+0000 7fab34d91640 1 -- 172.21.15.6:0/3025452359 >> 172.21.15.6:0/3025452359 conn(0x7fab300fc7d0 msgr2=0x7fab3006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:35.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.741+0000 7fab34d91640 1 -- 172.21.15.6:0/3025452359 shutdown_connections 2024-09-12T03:25:35.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:35.741+0000 7fab34d91640 1 -- 172.21.15.6:0/3025452359 wait complete. 2024-09-12T03:25:36.608 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:36 smithi006 ceph-mon[30991]: pgmap v470: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 981 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 48 MiB/s wr, 4.99k op/s 2024-09-12T03:25:36.608 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:36 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3025452359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:25:36.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:36 smithi161 ceph-mon[35485]: pgmap v470: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 981 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 48 MiB/s wr, 4.99k op/s 2024-09-12T03:25:36.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:36 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3025452359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:25:36.635 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 6c171048 --release --await' 2024-09-12T03:25:36.635 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 6c171048 --release --await 2024-09-12T03:25:36.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:36 smithi118 ceph-mon[35048]: pgmap v470: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 981 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 48 MiB/s wr, 4.99k op/s 2024-09-12T03:25:36.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:36 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3025452359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:25:37.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.991+0000 7fcb55e23640 1 -- 172.21.15.6:0/2880701654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb50102ef0 msgr2=0x7fcb5010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:37.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.991+0000 7fcb55e23640 1 --2- 172.21.15.6:0/2880701654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb50102ef0 0x7fcb5010da40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009900 tx=0x7fcb4802a480 comp rx=0 tx=0).stop 2024-09-12T03:25:37.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.991+0000 7fcb55e23640 1 -- 172.21.15.6:0/2880701654 shutdown_connections 2024-09-12T03:25:37.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.991+0000 7fcb55e23640 1 --2- 172.21.15.6:0/2880701654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb50102ef0 0x7fcb5010da40 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.991+0000 7fcb55e23640 1 --2- 172.21.15.6:0/2880701654 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb50102570 0x7fcb501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.991+0000 7fcb55e23640 1 --2- 172.21.15.6:0/2880701654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb50100e20 0x7fcb50101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.991+0000 7fcb55e23640 1 -- 172.21.15.6:0/2880701654 >> 172.21.15.6:0/2880701654 conn(0x7fcb500fc7d0 msgr2=0x7fcb500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:37.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.991+0000 7fcb55e23640 1 -- 172.21.15.6:0/2880701654 shutdown_connections 2024-09-12T03:25:37.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.991+0000 7fcb55e23640 1 -- 172.21.15.6:0/2880701654 wait complete. 2024-09-12T03:25:37.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.992+0000 7fcb55e23640 1 Processor -- start 2024-09-12T03:25:37.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.992+0000 7fcb55e23640 1 -- start start 2024-09-12T03:25:37.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.992+0000 7fcb55e23640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb50100e20 0x7fcb50197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:37.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.992+0000 7fcb55e23640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb50102570 0x7fcb50197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:37.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.992+0000 7fcb55e23640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb50102ef0 0x7fcb5019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:37.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.992+0000 7fcb55e23640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcb50110f80 con 0x7fcb50102ef0 2024-09-12T03:25:37.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.992+0000 7fcb55e23640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fcb50110e00 con 0x7fcb50102570 2024-09-12T03:25:37.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.992+0000 7fcb55e23640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcb50111100 con 0x7fcb50100e20 2024-09-12T03:25:37.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.992+0000 7fcb4ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb50102ef0 0x7fcb5019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:37.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.992+0000 7fcb4ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb50102ef0 0x7fcb5019bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56684/0 (socket says 172.21.15.6:56684) 2024-09-12T03:25:37.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.992+0000 7fcb4ffff640 1 -- 172.21.15.6:0/325599676 learned_addr learned my addr 172.21.15.6:0/325599676 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:37.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.992+0000 7fcb4f7fe640 1 --2- 172.21.15.6:0/325599676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb50100e20 0x7fcb50197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:37.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.992+0000 7fcb46ffd640 1 --2- 172.21.15.6:0/325599676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb50102570 0x7fcb50197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:37.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.993+0000 7fcb4ffff640 1 -- 172.21.15.6:0/325599676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb50100e20 msgr2=0x7fcb50197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:37.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.993+0000 7fcb4ffff640 1 --2- 172.21.15.6:0/325599676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb50100e20 0x7fcb50197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.993+0000 7fcb4ffff640 1 -- 172.21.15.6:0/325599676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb50102570 msgr2=0x7fcb50197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:37.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.993+0000 7fcb4ffff640 1 --2- 172.21.15.6:0/325599676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb50102570 0x7fcb50197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.993+0000 7fcb4ffff640 1 -- 172.21.15.6:0/325599676 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5019c3f0 con 0x7fcb50102ef0 2024-09-12T03:25:37.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.993+0000 7fcb4ffff640 1 --2- 172.21.15.6:0/325599676 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb50102ef0 0x7fcb5019bd10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcb48004770 tx=0x7fcb48003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:37.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.993+0000 7fcb4d7fa640 1 -- 172.21.15.6:0/325599676 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4803f070 con 0x7fcb50102ef0 2024-09-12T03:25:37.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.993+0000 7fcb4d7fa640 1 -- 172.21.15.6:0/325599676 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb48031c30 con 0x7fcb50102ef0 2024-09-12T03:25:37.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.993+0000 7fcb55e23640 1 -- 172.21.15.6:0/325599676 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb5019c680 con 0x7fcb50102ef0 2024-09-12T03:25:37.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.993+0000 7fcb4d7fa640 1 -- 172.21.15.6:0/325599676 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48035030 con 0x7fcb50102ef0 2024-09-12T03:25:37.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.993+0000 7fcb55e23640 1 -- 172.21.15.6:0/325599676 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb501a3fc0 con 0x7fcb50102ef0 2024-09-12T03:25:37.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.996+0000 7fcb4d7fa640 1 -- 172.21.15.6:0/325599676 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcb48042070 con 0x7fcb50102ef0 2024-09-12T03:25:37.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.996+0000 7fcb581fc740 1 Processor -- start 2024-09-12T03:25:37.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.996+0000 7fcb581fc740 1 -- start start 2024-09-12T03:25:37.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.996+0000 7fcb581fc740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c3647b40 0x5647c3646090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:37.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.996+0000 7fcb581fc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647c36465d0 0x5647c3646a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:37.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.996+0000 7fcb581fc740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c3655970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:37.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.996+0000 7fcb581fc740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5647c35ac2f0 con 0x5647c3653580 2024-09-12T03:25:37.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.996+0000 7fcb581fc740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5647c3553cf0 con 0x5647c3647b40 2024-09-12T03:25:37.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.996+0000 7fcb581fc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5647c35a82b0 con 0x5647c36465d0 2024-09-12T03:25:37.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.996+0000 7fcb46ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c3655970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:37.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.996+0000 7fcb46ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c3655970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56690/0 (socket says 172.21.15.6:56690) 2024-09-12T03:25:37.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.996+0000 7fcb46ffd640 1 -- 172.21.15.6:0/2558614162 learned_addr learned my addr 172.21.15.6:0/2558614162 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:37.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.996+0000 7fcb46ffd640 1 --2- 172.21.15.6:0/2558614162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c3647b40 0x5647c3646090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:37.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.996+0000 7fcb4f7fe640 1 --2- 172.21.15.6:0/2558614162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647c36465d0 0x5647c3646a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:37.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.997+0000 7fcb46ffd640 1 -- 172.21.15.6:0/2558614162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647c36465d0 msgr2=0x5647c3646a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:37.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.997+0000 7fcb46ffd640 1 --2- 172.21.15.6:0/2558614162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647c36465d0 0x5647c3646a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.997+0000 7fcb46ffd640 1 -- 172.21.15.6:0/2558614162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c3647b40 msgr2=0x5647c3646090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:37.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.997+0000 7fcb46ffd640 1 --2- 172.21.15.6:0/2558614162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c3647b40 0x5647c3646090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.997+0000 7fcb46ffd640 1 -- 172.21.15.6:0/2558614162 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647c35383a0 con 0x5647c3653580 2024-09-12T03:25:37.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.997+0000 7fcb4d7fa640 1 --2- 172.21.15.6:0/325599676 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcb3c0a2b80 0x7fcb3c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:37.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.997+0000 7fcb46ffd640 1 --2- 172.21.15.6:0/2558614162 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c3655970 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcb38011690 tx=0x7fcb3800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab09dd22fd0591c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:37.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.997+0000 7fcb4f7fe640 1 --2- 172.21.15.6:0/325599676 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcb3c0a2b80 0x7fcb3c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:37.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.997+0000 7fcb4d7fa640 1 -- 172.21.15.6:0/325599676 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fcb480efa50 con 0x7fcb50102ef0 2024-09-12T03:25:37.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.997+0000 7fcb45ffb640 1 -- 172.21.15.6:0/2558614162 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38004020 con 0x5647c3653580 2024-09-12T03:25:37.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.997+0000 7fcb45ffb640 1 -- 172.21.15.6:0/2558614162 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb38005500 con 0x5647c3653580 2024-09-12T03:25:37.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.997+0000 7fcb45ffb640 1 -- 172.21.15.6:0/2558614162 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38004d70 con 0x5647c3653580 2024-09-12T03:25:37.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.998+0000 7fcb581fc740 1 -- 172.21.15.6:0/2558614162 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 msgr2=0x5647c3655970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:37.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.998+0000 7fcb4f7fe640 1 --2- 172.21.15.6:0/325599676 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcb3c0a2b80 0x7fcb3c0a5040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x5647c3643ac0 tx=0x7fcb40009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:37.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.998+0000 7fcb581fc740 1 --2- 172.21.15.6:0/2558614162 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c3655970 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcb38011690 tx=0x7fcb3800a620 comp rx=0 tx=0).stop 2024-09-12T03:25:37.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.998+0000 7fcb581fc740 1 -- 172.21.15.6:0/2558614162 shutdown_connections 2024-09-12T03:25:37.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.998+0000 7fcb581fc740 1 --2- 172.21.15.6:0/2558614162 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c3655970 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.998+0000 7fcb581fc740 1 --2- 172.21.15.6:0/2558614162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647c36465d0 0x5647c3646a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.998+0000 7fcb581fc740 1 --2- 172.21.15.6:0/2558614162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c3647b40 0x5647c3646090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.998+0000 7fcb581fc740 1 -- 172.21.15.6:0/2558614162 >> 172.21.15.6:0/2558614162 conn(0x5647c360f310 msgr2=0x5647c3642c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:37.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.998+0000 7fcb581fc740 1 -- 172.21.15.6:0/2558614162 shutdown_connections 2024-09-12T03:25:37.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.998+0000 7fcb581fc740 1 -- 172.21.15.6:0/2558614162 wait complete. 2024-09-12T03:25:37.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.998+0000 7fcb581fc740 1 -- start start 2024-09-12T03:25:37.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.999+0000 7fcb581fc740 1 Processor -- start 2024-09-12T03:25:37.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.999+0000 7fcb581fc740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c36465d0 0x5647c3652f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:37.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.999+0000 7fcb581fc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647c3647b40 0x5647c3650230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:37.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.999+0000 7fcb581fc740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c3650820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:37.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.999+0000 7fcb581fc740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5647c3656280 con 0x5647c3653580 2024-09-12T03:25:37.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.999+0000 7fcb581fc740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5647c3567060 con 0x5647c36465d0 2024-09-12T03:25:37.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.999+0000 7fcb581fc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5647c3656400 con 0x5647c3647b40 2024-09-12T03:25:37.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.999+0000 7fcb581fc740 10 client.0 _ll_register_callbacks cb 0x5647c32ba8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:37.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.999+0000 7fcb4ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c3650820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:37.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.999+0000 7fcb46ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647c3647b40 0x5647c3650230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:37.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.999+0000 7fcb4ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c3650820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56696/0 (socket says 172.21.15.6:56696) 2024-09-12T03:25:37.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.999+0000 7fcb4ffff640 1 -- 172.21.15.6:0/2541592156 learned_addr learned my addr 172.21.15.6:0/2541592156 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.999+0000 7fcb4f7fe640 1 --2- 172.21.15.6:0/2541592156 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c36465d0 0x5647c3652f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.999+0000 7fcb4ffff640 1 -- 172.21.15.6:0/2541592156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647c3647b40 msgr2=0x5647c3650230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.999+0000 7fcb4ffff640 1 --2- 172.21.15.6:0/2541592156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647c3647b40 0x5647c3650230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.999+0000 7fcb4ffff640 1 -- 172.21.15.6:0/2541592156 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c36465d0 msgr2=0x5647c3652f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.999+0000 7fcb4ffff640 1 --2- 172.21.15.6:0/2541592156 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c36465d0 0x5647c3652f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.999+0000 7fcb4ffff640 1 -- 172.21.15.6:0/2541592156 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647c331d3c0 con 0x5647c3653580 2024-09-12T03:25:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.999+0000 7fcb46ffd640 1 --2- 172.21.15.6:0/2541592156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647c3647b40 0x5647c3650230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:36.999+0000 7fcb4f7fe640 1 --2- 172.21.15.6:0/2541592156 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c36465d0 0x5647c3652f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:25:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.000+0000 7fcb4ffff640 1 --2- 172.21.15.6:0/2541592156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c3650820 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcb48032fd0 tx=0x7fcb480bbed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2114468c6ff6adb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.000+0000 7fcb2effd640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:25:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.000+0000 7fcb2effd640 1 -- 172.21.15.6:0/2541592156 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48049050 con 0x5647c3653580 2024-09-12T03:25:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.000+0000 7fcb2effd640 1 -- 172.21.15.6:0/2541592156 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fcb480b8a00 con 0x5647c3653580 2024-09-12T03:25:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.000+0000 7fcb581fc740 1 -- 172.21.15.6:0/2541592156 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb38005500 con 0x5647c3653580 2024-09-12T03:25:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.000+0000 7fcb2effd640 1 -- 172.21.15.6:0/2541592156 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcb480bc390 con 0x5647c3653580 2024-09-12T03:25:37.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.000+0000 7fcb581fc740 10 client.15924 fetch_fsmap learned FSMap version 27 2024-09-12T03:25:37.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.000+0000 7fcb581fc740 1 -- 172.21.15.6:0/2541592156 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5647c3661d20 con 0x5647c3653580 2024-09-12T03:25:37.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.000+0000 7fcb2effd640 1 -- 172.21.15.6:0/2541592156 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb38005500 con 0x5647c3653580 2024-09-12T03:25:37.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.000+0000 7fcb2effd640 1 -- 172.21.15.6:0/2541592156 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fcb480bc5f0 con 0x5647c3653580 2024-09-12T03:25:37.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.000+0000 7fcb581fc740 10 client.15924 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:25:37.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.000+0000 7fcb581fc740 10 client.15924 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-12T03:25:37.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.000+0000 7fcb581fc740 20 client.15924 populate_metadata read hostname 'smithi006' 2024-09-12T03:25:37.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.000+0000 7fcb581fc740 1 --2- 172.21.15.6:0/2541592156 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5647c3663110 0x5647c3665520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:37.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.000+0000 7fcb581fc740 4 client.15924 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:25:37.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.000+0000 7fcb581fc740 1 -- 172.21.15.6:0/2541592156 --> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647c35b9860 con 0x5647c3663110 2024-09-12T03:25:37.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.001+0000 7fcb4f7fe640 1 --2- 172.21.15.6:0/2541592156 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5647c3663110 0x5647c3665520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:37.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.001+0000 7fcb4f7fe640 1 --2- 172.21.15.6:0/2541592156 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5647c3663110 0x5647c3665520 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1380e20a16ecca8c server_cookie=54325600189038f6 in_seq=0 out_seq=0 2024-09-12T03:25:37.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.001+0000 7fcb2effd640 10 client.15924 ms_handle_connect on v2:172.21.15.161:6838/3645555254 2024-09-12T03:25:37.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.002+0000 7fcb2effd640 1 -- 172.21.15.6:0/2541592156 <== mds.2 v2:172.21.15.161:6838/3645555254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5647c35b9860 con 0x5647c3663110 2024-09-12T03:25:37.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.002+0000 7fcb2effd640 10 client.15924 handle_command_reply: tid=0 2024-09-12T03:25:37.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.002+0000 7fcb581fc740 1 client.15924 shutdown 2024-09-12T03:25:37.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.002+0000 7fcb581fc740 1 -- 172.21.15.6:0/2541592156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 msgr2=0x5647c3650820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:37.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.002+0000 7fcb581fc740 1 --2- 172.21.15.6:0/2541592156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c3650820 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcb48032fd0 tx=0x7fcb480bbed0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.002+0000 7fcb581fc740 1 -- 172.21.15.6:0/2541592156 shutdown_connections 2024-09-12T03:25:37.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.002+0000 7fcb581fc740 1 --2- 172.21.15.6:0/2541592156 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5647c3663110 0x5647c3665520 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.002+0000 7fcb581fc740 1 --2- 172.21.15.6:0/2541592156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c3650820 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.002+0000 7fcb581fc740 1 --2- 172.21.15.6:0/2541592156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647c3647b40 0x5647c3650230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.002+0000 7fcb581fc740 1 --2- 172.21.15.6:0/2541592156 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c36465d0 0x5647c3652f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.002+0000 7fcb2effd640 0 client.15924 ms_handle_reset on v2:172.21.15.161:6838/3645555254 2024-09-12T03:25:37.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.002+0000 7fcb581fc740 1 -- 172.21.15.6:0/2541592156 >> 172.21.15.6:0/2541592156 conn(0x5647c360f310 msgr2=0x5647c3648960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:37.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.002+0000 7fcb581fc740 1 -- 172.21.15.6:0/2541592156 shutdown_connections 2024-09-12T03:25:37.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.002+0000 7fcb581fc740 1 -- 172.21.15.6:0/2541592156 wait complete. 2024-09-12T03:25:37.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.002+0000 7fcb581fc740 20 client.15924 trim_cache size 0 max 16384 2024-09-12T03:25:37.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.015+0000 7fcb581fc740 1 Processor -- start 2024-09-12T03:25:37.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.015+0000 7fcb581fc740 1 -- start start 2024-09-12T03:25:37.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.016+0000 7fcb581fc740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c36465d0 0x5647c363dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:37.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.016+0000 7fcb581fc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647c3647b40 0x5647c3642e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:37.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.016+0000 7fcb581fc740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c365f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:37.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.016+0000 7fcb581fc740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcb501adac0 con 0x5647c3653580 2024-09-12T03:25:37.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.016+0000 7fcb581fc740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fcb501ab390 con 0x5647c36465d0 2024-09-12T03:25:37.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.016+0000 7fcb581fc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcb501a9200 con 0x5647c3647b40 2024-09-12T03:25:37.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.016+0000 7fcb4ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c365f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:37.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.016+0000 7fcb4ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c365f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56700/0 (socket says 172.21.15.6:56700) 2024-09-12T03:25:37.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.016+0000 7fcb4ffff640 1 -- 172.21.15.6:0/883772218 learned_addr learned my addr 172.21.15.6:0/883772218 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:37.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.016+0000 7fcb46ffd640 1 --2- 172.21.15.6:0/883772218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647c3647b40 0x5647c3642e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:37.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.016+0000 7fcb4f7fe640 1 --2- 172.21.15.6:0/883772218 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c36465d0 0x5647c363dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:37.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.016+0000 7fcb4ffff640 1 -- 172.21.15.6:0/883772218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647c3647b40 msgr2=0x5647c3642e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:37.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.016+0000 7fcb4ffff640 1 --2- 172.21.15.6:0/883772218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647c3647b40 0x5647c3642e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.016+0000 7fcb4ffff640 1 -- 172.21.15.6:0/883772218 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c36465d0 msgr2=0x5647c363dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:37.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.016+0000 7fcb4ffff640 1 --2- 172.21.15.6:0/883772218 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c36465d0 0x5647c363dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.016+0000 7fcb4ffff640 1 -- 172.21.15.6:0/883772218 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647c3665ba0 con 0x5647c3653580 2024-09-12T03:25:37.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.016+0000 7fcb4f7fe640 1 --2- 172.21.15.6:0/883772218 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c36465d0 0x5647c363dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:25:37.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.016+0000 7fcb4ffff640 1 --2- 172.21.15.6:0/883772218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c365f520 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fcb480ed8f0 tx=0x7fcb480ba8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76776510d73cabad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:37.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.016+0000 7fcb457fa640 1 -- 172.21.15.6:0/883772218 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48049070 con 0x5647c3653580 2024-09-12T03:25:37.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.016+0000 7fcb457fa640 1 -- 172.21.15.6:0/883772218 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb480c4f50 con 0x5647c3653580 2024-09-12T03:25:37.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.016+0000 7fcb457fa640 1 -- 172.21.15.6:0/883772218 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb480bc5f0 con 0x5647c3653580 2024-09-12T03:25:37.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.017+0000 7fcb581fc740 1 -- 172.21.15.6:0/883772218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 msgr2=0x5647c365f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:37.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.017+0000 7fcb581fc740 1 --2- 172.21.15.6:0/883772218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c365f520 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fcb480ed8f0 tx=0x7fcb480ba8a0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.017+0000 7fcb581fc740 1 -- 172.21.15.6:0/883772218 shutdown_connections 2024-09-12T03:25:37.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.017+0000 7fcb581fc740 1 --2- 172.21.15.6:0/883772218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c365f520 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.017+0000 7fcb581fc740 1 --2- 172.21.15.6:0/883772218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647c3647b40 0x5647c3642e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.017+0000 7fcb581fc740 1 --2- 172.21.15.6:0/883772218 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c36465d0 0x5647c363dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.017+0000 7fcb581fc740 1 -- 172.21.15.6:0/883772218 >> 172.21.15.6:0/883772218 conn(0x5647c360f310 msgr2=0x5647c3642700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:37.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.017+0000 7fcb581fc740 1 -- 172.21.15.6:0/883772218 shutdown_connections 2024-09-12T03:25:37.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.017+0000 7fcb581fc740 1 -- 172.21.15.6:0/883772218 wait complete. 2024-09-12T03:25:37.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.018+0000 7fcb581fc740 1 -- start start 2024-09-12T03:25:37.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.018+0000 7fcb581fc740 1 Processor -- start 2024-09-12T03:25:37.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.018+0000 7fcb581fc740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c36465d0 0x5647c364d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:37.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.018+0000 7fcb581fc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647c3647b40 0x5647c364d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:37.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.018+0000 7fcb581fc740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c364dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:37.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.018+0000 7fcb581fc740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcb501a9850 con 0x5647c3653580 2024-09-12T03:25:37.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.018+0000 7fcb581fc740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fcb501aacc0 con 0x5647c36465d0 2024-09-12T03:25:37.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.018+0000 7fcb581fc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcb501a9f20 con 0x5647c3647b40 2024-09-12T03:25:37.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.018+0000 7fcb581fc740 10 client.0 _ll_register_callbacks cb 0x7fcb48005dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:37.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.018+0000 7fcb4ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c364dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:37.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.018+0000 7fcb4ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c364dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56706/0 (socket says 172.21.15.6:56706) 2024-09-12T03:25:37.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.018+0000 7fcb4ffff640 1 -- 172.21.15.6:0/2190327883 learned_addr learned my addr 172.21.15.6:0/2190327883 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:37.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.018+0000 7fcb4f7fe640 1 --2- 172.21.15.6:0/2190327883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c36465d0 0x5647c364d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:37.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.018+0000 7fcb46ffd640 1 --2- 172.21.15.6:0/2190327883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647c3647b40 0x5647c364d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:37.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.019+0000 7fcb4ffff640 1 -- 172.21.15.6:0/2190327883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647c3647b40 msgr2=0x5647c364d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:37.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.019+0000 7fcb4ffff640 1 --2- 172.21.15.6:0/2190327883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647c3647b40 0x5647c364d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.019+0000 7fcb4ffff640 1 -- 172.21.15.6:0/2190327883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c36465d0 msgr2=0x5647c364d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:37.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.019+0000 7fcb4ffff640 1 --2- 172.21.15.6:0/2190327883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c36465d0 0x5647c364d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.019+0000 7fcb4ffff640 1 -- 172.21.15.6:0/2190327883 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647c365fa60 con 0x5647c3653580 2024-09-12T03:25:37.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.019+0000 7fcb4ffff640 1 --2- 172.21.15.6:0/2190327883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c364dca0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009a20 tx=0x7fcb480bbed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13bf086cf6888c6a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:37.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.019+0000 7fcb2effd640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:25:37.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.019+0000 7fcb2effd640 1 -- 172.21.15.6:0/2190327883 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48049070 con 0x5647c3653580 2024-09-12T03:25:37.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.019+0000 7fcb2effd640 1 -- 172.21.15.6:0/2190327883 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fcb480c03f0 con 0x5647c3653580 2024-09-12T03:25:37.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.019+0000 7fcb581fc740 1 -- 172.21.15.6:0/2190327883 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb480c4f50 con 0x5647c3653580 2024-09-12T03:25:37.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.019+0000 7fcb2effd640 1 -- 172.21.15.6:0/2190327883 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcb480dc0b0 con 0x5647c3653580 2024-09-12T03:25:37.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.019+0000 7fcb581fc740 10 client.15930 fetch_fsmap learned FSMap version 27 2024-09-12T03:25:37.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.019+0000 7fcb581fc740 1 -- 172.21.15.6:0/2190327883 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5647c365fa60 con 0x5647c3653580 2024-09-12T03:25:37.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.019+0000 7fcb2effd640 1 -- 172.21.15.6:0/2190327883 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb480c4f50 con 0x5647c3653580 2024-09-12T03:25:37.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.020+0000 7fcb2effd640 1 -- 172.21.15.6:0/2190327883 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fcb480dc250 con 0x5647c3653580 2024-09-12T03:25:37.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.020+0000 7fcb581fc740 10 client.15930 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:25:37.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.020+0000 7fcb581fc740 10 client.15930 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-12T03:25:37.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.020+0000 7fcb581fc740 20 client.15930 populate_metadata read hostname 'smithi006' 2024-09-12T03:25:37.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.020+0000 7fcb581fc740 1 --2- 172.21.15.6:0/2190327883 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5647c36ce110 0x5647c36d0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:37.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.020+0000 7fcb581fc740 4 client.15930 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:25:37.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.020+0000 7fcb581fc740 1 -- 172.21.15.6:0/2190327883 --> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5647c35b3d70 con 0x5647c36ce110 2024-09-12T03:25:37.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.020+0000 7fcb4f7fe640 1 --2- 172.21.15.6:0/2190327883 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5647c36ce110 0x5647c36d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:37.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.020+0000 7fcb2effd640 10 client.15930 ms_handle_connect on v2:172.21.15.161:6838/3645555254 2024-09-12T03:25:37.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.020+0000 7fcb4f7fe640 1 --2- 172.21.15.6:0/2190327883 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5647c36ce110 0x5647c36d0500 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b3b1c60169b8c672 server_cookie=68488cacc4f6a293 in_seq=0 out_seq=0 2024-09-12T03:25:37.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.020+0000 7fcb2effd640 1 -- 172.21.15.6:0/2190327883 <== mds.2 v2:172.21.15.161:6838/3645555254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5647c35b3d70 con 0x5647c36ce110 2024-09-12T03:25:37.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.020+0000 7fcb2effd640 10 client.15930 handle_command_reply: tid=0 2024-09-12T03:25:37.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.020+0000 7fcb581fc740 1 client.15930 shutdown 2024-09-12T03:25:37.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.021+0000 7fcb581fc740 1 -- 172.21.15.6:0/2190327883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 msgr2=0x5647c364dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:37.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.021+0000 7fcb581fc740 1 --2- 172.21.15.6:0/2190327883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c364dca0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009a20 tx=0x7fcb480bbed0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.021+0000 7fcb581fc740 1 -- 172.21.15.6:0/2190327883 shutdown_connections 2024-09-12T03:25:37.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.021+0000 7fcb581fc740 1 --2- 172.21.15.6:0/2190327883 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x5647c36ce110 0x5647c36d0500 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.021+0000 7fcb581fc740 1 --2- 172.21.15.6:0/2190327883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5647c3653580 0x5647c364dca0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.021+0000 7fcb581fc740 1 --2- 172.21.15.6:0/2190327883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647c3647b40 0x5647c364d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.021+0000 7fcb2effd640 0 client.15930 ms_handle_reset on v2:172.21.15.161:6838/3645555254 2024-09-12T03:25:37.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.021+0000 7fcb581fc740 1 --2- 172.21.15.6:0/2190327883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647c36465d0 0x5647c364d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.021+0000 7fcb581fc740 1 -- 172.21.15.6:0/2190327883 >> 172.21.15.6:0/2190327883 conn(0x5647c360f310 msgr2=0x5647c3651a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:37.120 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-09-12T03:25:37.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.021+0000 7fcb581fc740 1 -- 172.21.15.6:0/2190327883 shutdown_connections 2024-09-12T03:25:37.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.021+0000 7fcb581fc740 1 -- 172.21.15.6:0/2190327883 wait complete. 2024-09-12T03:25:37.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.021+0000 7fcb581fc740 20 client.15930 trim_cache size 0 max 16384 2024-09-12T03:25:37.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.021+0000 7fcb55e23640 1 -- 172.21.15.6:0/325599676 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcb3c0a2b80 msgr2=0x7fcb3c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:37.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.021+0000 7fcb55e23640 1 --2- 172.21.15.6:0/325599676 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcb3c0a2b80 0x7fcb3c0a5040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x5647c3643ac0 tx=0x7fcb40009500 comp rx=0 tx=0).stop 2024-09-12T03:25:37.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.021+0000 7fcb55e23640 1 -- 172.21.15.6:0/325599676 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb50102ef0 msgr2=0x7fcb5019bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:37.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.021+0000 7fcb55e23640 1 --2- 172.21.15.6:0/325599676 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb50102ef0 0x7fcb5019bd10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcb48004770 tx=0x7fcb48003c10 comp rx=0 tx=0).stop 2024-09-12T03:25:37.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.022+0000 7fcb55e23640 1 -- 172.21.15.6:0/325599676 shutdown_connections 2024-09-12T03:25:37.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.022+0000 7fcb55e23640 1 --2- 172.21.15.6:0/325599676 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcb3c0a2b80 0x7fcb3c0a5040 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.022+0000 7fcb55e23640 1 --2- 172.21.15.6:0/325599676 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb50102ef0 0x7fcb5019bd10 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.022+0000 7fcb55e23640 1 --2- 172.21.15.6:0/325599676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb50102570 0x7fcb50197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.022+0000 7fcb55e23640 1 --2- 172.21.15.6:0/325599676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb50100e20 0x7fcb50197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:37.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.022+0000 7fcb55e23640 1 -- 172.21.15.6:0/325599676 >> 172.21.15.6:0/325599676 conn(0x7fcb500fc7d0 msgr2=0x7fcb501068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:37.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.022+0000 7fcb55e23640 1 -- 172.21.15.6:0/325599676 shutdown_connections 2024-09-12T03:25:37.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:37.022+0000 7fcb55e23640 1 -- 172.21.15.6:0/325599676 wait complete. 2024-09-12T03:25:37.833 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:25:37.938 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:38.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.454+0000 7f32b71f6640 1 -- 172.21.15.6:0/1891122722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32b0100a80 msgr2=0x7f32b0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:38.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.454+0000 7f32b71f6640 1 --2- 172.21.15.6:0/1891122722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32b0100a80 0x7f32b0100ee0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f329c009930 tx=0x7f329c02a420 comp rx=0 tx=0).stop 2024-09-12T03:25:38.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.454+0000 7f32b71f6640 1 -- 172.21.15.6:0/1891122722 shutdown_connections 2024-09-12T03:25:38.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.454+0000 7f32b71f6640 1 --2- 172.21.15.6:0/1891122722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32b0101420 0x7f32b010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.454+0000 7f32b71f6640 1 --2- 172.21.15.6:0/1891122722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32b0100a80 0x7f32b0100ee0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.454+0000 7f32b71f6640 1 --2- 172.21.15.6:0/1891122722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b0106a80 0x7f32b0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.454+0000 7f32b71f6640 1 -- 172.21.15.6:0/1891122722 >> 172.21.15.6:0/1891122722 conn(0x7f32b00fc7a0 msgr2=0x7f32b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:38.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.454+0000 7f32b71f6640 1 -- 172.21.15.6:0/1891122722 shutdown_connections 2024-09-12T03:25:38.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.454+0000 7f32b71f6640 1 -- 172.21.15.6:0/1891122722 wait complete. 2024-09-12T03:25:38.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.455+0000 7f32b71f6640 1 Processor -- start 2024-09-12T03:25:38.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.455+0000 7f32b71f6640 1 -- start start 2024-09-12T03:25:38.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.455+0000 7f32b71f6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b0100a80 0x7f32b01a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.455+0000 7f32b71f6640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32b0101420 0x7f32b01a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.455+0000 7f32b71f6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32b0106a80 0x7f32b019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.455+0000 7f32b71f6640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f32b0110e80 con 0x7f32b0106a80 2024-09-12T03:25:38.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.455+0000 7f32b71f6640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f32b0110d00 con 0x7f32b0101420 2024-09-12T03:25:38.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.455+0000 7f32b71f6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f32b0111000 con 0x7f32b0100a80 2024-09-12T03:25:38.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.455+0000 7f32b4f92640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b0100a80 0x7f32b01a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.455+0000 7f32b4f92640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b0100a80 0x7f32b01a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:50092/0 (socket says 172.21.15.6:50092) 2024-09-12T03:25:38.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.455+0000 7f32b4f92640 1 -- 172.21.15.6:0/190446892 learned_addr learned my addr 172.21.15.6:0/190446892 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:38.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.455+0000 7f32b5793640 1 --2- 172.21.15.6:0/190446892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32b0106a80 0x7f32b019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.456+0000 7f32affff640 1 --2- 172.21.15.6:0/190446892 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32b0101420 0x7f32b01a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.456+0000 7f32b4f92640 1 -- 172.21.15.6:0/190446892 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32b0101420 msgr2=0x7f32b01a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:38.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.456+0000 7f32b4f92640 1 --2- 172.21.15.6:0/190446892 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32b0101420 0x7f32b01a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.456+0000 7f32b4f92640 1 -- 172.21.15.6:0/190446892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32b0106a80 msgr2=0x7f32b019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:38.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.456+0000 7f32b4f92640 1 --2- 172.21.15.6:0/190446892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32b0106a80 0x7f32b019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.456+0000 7f32b4f92640 1 -- 172.21.15.6:0/190446892 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b019f230 con 0x7f32b0100a80 2024-09-12T03:25:38.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.456+0000 7f32b4f92640 1 --2- 172.21.15.6:0/190446892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b0100a80 0x7f32b01a46a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f32a400e9f0 tx=0x7f32a400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:38.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.456+0000 7f32adffb640 1 -- 172.21.15.6:0/190446892 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a400cd80 con 0x7f32b0100a80 2024-09-12T03:25:38.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.456+0000 7f32b71f6640 1 -- 172.21.15.6:0/190446892 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b019f520 con 0x7f32b0100a80 2024-09-12T03:25:38.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.456+0000 7f32adffb640 1 -- 172.21.15.6:0/190446892 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32a4010910 con 0x7f32b0100a80 2024-09-12T03:25:38.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.456+0000 7f32adffb640 1 -- 172.21.15.6:0/190446892 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a40188f0 con 0x7f32b0100a80 2024-09-12T03:25:38.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.456+0000 7f32b71f6640 1 -- 172.21.15.6:0/190446892 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b01ab570 con 0x7f32b0100a80 2024-09-12T03:25:38.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.459+0000 7f32adffb640 1 -- 172.21.15.6:0/190446892 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f32a4018bb0 con 0x7f32b0100a80 2024-09-12T03:25:38.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.460+0000 7f32adffb640 1 --2- 172.21.15.6:0/190446892 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f32a00a2c00 0x7f32a00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.460+0000 7f32adffb640 1 -- 172.21.15.6:0/190446892 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f32a40cf3a0 con 0x7f32b0100a80 2024-09-12T03:25:38.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.460+0000 7f32affff640 1 --2- 172.21.15.6:0/190446892 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f32a00a2c00 0x7f32a00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.461+0000 7f32affff640 1 --2- 172.21.15.6:0/190446892 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f32a00a2c00 0x7f32a00a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f329c000c00 tx=0x7f329c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:38.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.461+0000 7f32b95cf740 1 Processor -- start 2024-09-12T03:25:38.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.461+0000 7f32b95cf740 1 -- start start 2024-09-12T03:25:38.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.461+0000 7f32b95cf740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412d6060 0x55b8412d6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.461+0000 7f32b95cf740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412d6980 0x55b8412e0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.461+0000 7f32affff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412d6980 0x55b8412e0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.461+0000 7f32affff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412d6980 0x55b8412e0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56734/0 (socket says 172.21.15.6:56734) 2024-09-12T03:25:38.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.461+0000 7f32affff640 1 -- 172.21.15.6:0/1359693809 learned_addr learned my addr 172.21.15.6:0/1359693809 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:38.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.461+0000 7f32b95cf740 1 --2- 172.21.15.6:0/1359693809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412e1240 0x55b8412e3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.462+0000 7f32b95cf740 1 -- 172.21.15.6:0/1359693809 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b84123b620 con 0x55b8412d6980 2024-09-12T03:25:38.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.462+0000 7f32b95cf740 1 -- 172.21.15.6:0/1359693809 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55b8411e2e50 con 0x55b8412d6060 2024-09-12T03:25:38.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.462+0000 7f32b95cf740 1 -- 172.21.15.6:0/1359693809 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b8412375e0 con 0x55b8412e1240 2024-09-12T03:25:38.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.462+0000 7f32b4f92640 1 --2- 172.21.15.6:0/1359693809 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412d6060 0x55b8412d6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.462+0000 7f32b5793640 1 --2- 172.21.15.6:0/1359693809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412e1240 0x55b8412e3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.462+0000 7f32affff640 1 -- 172.21.15.6:0/1359693809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412e1240 msgr2=0x55b8412e3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:38.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.462+0000 7f32affff640 1 --2- 172.21.15.6:0/1359693809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412e1240 0x55b8412e3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.462+0000 7f32affff640 1 -- 172.21.15.6:0/1359693809 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412d6060 msgr2=0x55b8412d6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:38.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.462+0000 7f32affff640 1 --2- 172.21.15.6:0/1359693809 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412d6060 0x55b8412d6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.462+0000 7f32affff640 1 -- 172.21.15.6:0/1359693809 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8411d3d40 con 0x55b8412d6980 2024-09-12T03:25:38.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.462+0000 7f32b5793640 1 --2- 172.21.15.6:0/1359693809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412e1240 0x55b8412e3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:38.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.462+0000 7f32affff640 1 --2- 172.21.15.6:0/1359693809 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412d6980 0x55b8412e0d00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f329c02a940 tx=0x7f329c03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c05b19463ac1e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:38.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.462+0000 7f329affd640 1 -- 172.21.15.6:0/1359693809 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c035050 con 0x55b8412d6980 2024-09-12T03:25:38.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.463+0000 7f329affd640 1 -- 172.21.15.6:0/1359693809 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329c04c070 con 0x55b8412d6980 2024-09-12T03:25:38.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.463+0000 7f329affd640 1 -- 172.21.15.6:0/1359693809 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c004b80 con 0x55b8412d6980 2024-09-12T03:25:38.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.463+0000 7f32b95cf740 1 -- 172.21.15.6:0/1359693809 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412d6980 msgr2=0x55b8412e0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:38.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.463+0000 7f32b95cf740 1 --2- 172.21.15.6:0/1359693809 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412d6980 0x55b8412e0d00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f329c02a940 tx=0x7f329c03c000 comp rx=0 tx=0).stop 2024-09-12T03:25:38.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.463+0000 7f32b95cf740 1 -- 172.21.15.6:0/1359693809 shutdown_connections 2024-09-12T03:25:38.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.463+0000 7f32b95cf740 1 --2- 172.21.15.6:0/1359693809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412e1240 0x55b8412e3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.463+0000 7f32b95cf740 1 --2- 172.21.15.6:0/1359693809 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412d6980 0x55b8412e0d00 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.463+0000 7f32b95cf740 1 --2- 172.21.15.6:0/1359693809 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412d6060 0x55b8412d6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.463+0000 7f32b95cf740 1 -- 172.21.15.6:0/1359693809 >> 172.21.15.6:0/1359693809 conn(0x55b8412d25f0 msgr2=0x55b8412d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:38.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.463+0000 7f32b95cf740 1 -- 172.21.15.6:0/1359693809 shutdown_connections 2024-09-12T03:25:38.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.463+0000 7f32b95cf740 1 -- 172.21.15.6:0/1359693809 wait complete. 2024-09-12T03:25:38.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.464+0000 7f32b95cf740 1 -- start start 2024-09-12T03:25:38.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.464+0000 7f32b95cf740 1 Processor -- start 2024-09-12T03:25:38.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.465+0000 7f32b95cf740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412d6060 0x55b8412de0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.465+0000 7f32b4f92640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412d6060 0x55b8412de0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.465+0000 7f32b4f92640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412d6060 0x55b8412de0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56750/0 (socket says 172.21.15.6:56750) 2024-09-12T03:25:38.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.465+0000 7f32b4f92640 1 -- 172.21.15.6:0/648448015 learned_addr learned my addr 172.21.15.6:0/648448015 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:38.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.465+0000 7f32b95cf740 1 --2- 172.21.15.6:0/648448015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412d6980 0x55b8412de5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.465+0000 7f32b95cf740 1 --2- 172.21.15.6:0/648448015 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412e1240 0x55b8412deb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.465+0000 7f32b95cf740 1 -- 172.21.15.6:0/648448015 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b8412e3e10 con 0x55b8412d6060 2024-09-12T03:25:38.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.465+0000 7f32b95cf740 1 -- 172.21.15.6:0/648448015 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55b8411c3f10 con 0x55b8412e1240 2024-09-12T03:25:38.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.465+0000 7f32b95cf740 1 -- 172.21.15.6:0/648448015 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b8412e3f90 con 0x55b8412d6980 2024-09-12T03:25:38.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.465+0000 7f32affff640 1 --2- 172.21.15.6:0/648448015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412d6980 0x55b8412de5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.466+0000 7f32b5793640 1 --2- 172.21.15.6:0/648448015 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412e1240 0x55b8412deb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.466+0000 7f32b4f92640 1 -- 172.21.15.6:0/648448015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412d6980 msgr2=0x55b8412de5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:38.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.466+0000 7f32b4f92640 1 --2- 172.21.15.6:0/648448015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412d6980 0x55b8412de5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.466+0000 7f32b4f92640 1 -- 172.21.15.6:0/648448015 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412e1240 msgr2=0x55b8412deb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:38.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.466+0000 7f32b4f92640 1 --2- 172.21.15.6:0/648448015 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412e1240 0x55b8412deb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.466+0000 7f32b4f92640 1 -- 172.21.15.6:0/648448015 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b840f4a580 con 0x55b8412d6060 2024-09-12T03:25:38.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.466+0000 7f32b5793640 1 --2- 172.21.15.6:0/648448015 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412e1240 0x55b8412deb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:25:38.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.466+0000 7f32b95cf740 10 client.0 _ll_register_callbacks cb 0x55b8411e7db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:38.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.466+0000 7f32b95cf740 1 -- 172.21.15.6:0/648448015 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f329c04c070 con 0x55b8412d6060 2024-09-12T03:25:38.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.466+0000 7f32b4f92640 1 --2- 172.21.15.6:0/648448015 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412d6060 0x55b8412de0b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f32a40ccf20 tx=0x7f32a409c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c285d60837394e55 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:38.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.466+0000 7f328ffff640 10 client.15942 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:25:38.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.467+0000 7f328ffff640 1 -- 172.21.15.6:0/648448015 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a409b080 con 0x55b8412d6060 2024-09-12T03:25:38.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.467+0000 7f328ffff640 1 -- 172.21.15.6:0/648448015 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f32a409b220 con 0x55b8412d6060 2024-09-12T03:25:38.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.467+0000 7f328ffff640 1 -- 172.21.15.6:0/648448015 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f32a4010430 con 0x55b8412d6060 2024-09-12T03:25:38.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.467+0000 7f32b95cf740 10 client.15942 fetch_fsmap learned FSMap version 27 2024-09-12T03:25:38.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.467+0000 7f32b95cf740 1 -- 172.21.15.6:0/648448015 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55b841014960 con 0x55b8412d6060 2024-09-12T03:25:38.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.467+0000 7f328ffff640 1 -- 172.21.15.6:0/648448015 <== mon.0 v2:172.21.15.6:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f32a40105d0 con 0x55b8412d6060 2024-09-12T03:25:38.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.467+0000 7f32b95cf740 10 client.15942 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:25:38.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.467+0000 7f32b95cf740 10 client.15942 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:25:38.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.467+0000 7f32b95cf740 20 client.15942 populate_metadata read hostname 'smithi006' 2024-09-12T03:25:38.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.467+0000 7f32b95cf740 1 --2- 172.21.15.6:0/648448015 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b8412f0ad0 0x55b8412f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.467+0000 7f32b95cf740 4 client.15942 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:25:38.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.467+0000 7f32b95cf740 1 -- 172.21.15.6:0/648448015 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b841248b90 con 0x55b8412f0ad0 2024-09-12T03:25:38.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.468+0000 7f32affff640 1 --2- 172.21.15.6:0/648448015 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b8412f0ad0 0x55b8412f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.468+0000 7f328ffff640 10 client.15942 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:38.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.468+0000 7f32affff640 1 --2- 172.21.15.6:0/648448015 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b8412f0ad0 0x55b8412f2ee0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4343ac6b92cc252b server_cookie=e5759aed824c9f3 in_seq=0 out_seq=0 2024-09-12T03:25:38.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.469+0000 7f328ffff640 1 -- 172.21.15.6:0/648448015 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b841248b90 con 0x55b8412f0ad0 2024-09-12T03:25:38.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.469+0000 7f328ffff640 10 client.15942 handle_command_reply: tid=0 2024-09-12T03:25:38.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.469+0000 7f32b95cf740 1 client.15942 shutdown 2024-09-12T03:25:38.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.469+0000 7f32b95cf740 1 -- 172.21.15.6:0/648448015 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412d6060 msgr2=0x55b8412de0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:38.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.469+0000 7f32b95cf740 1 --2- 172.21.15.6:0/648448015 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412d6060 0x55b8412de0b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f32a40ccf20 tx=0x7f32a409c470 comp rx=0 tx=0).stop 2024-09-12T03:25:38.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.469+0000 7f32b95cf740 1 -- 172.21.15.6:0/648448015 shutdown_connections 2024-09-12T03:25:38.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.470+0000 7f32b95cf740 1 --2- 172.21.15.6:0/648448015 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b8412f0ad0 0x55b8412f2ee0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.470+0000 7f32b95cf740 1 --2- 172.21.15.6:0/648448015 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412e1240 0x55b8412deb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.470+0000 7f328ffff640 0 client.15942 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:38.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.470+0000 7f32b95cf740 1 --2- 172.21.15.6:0/648448015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412d6980 0x55b8412de5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.470+0000 7f32b95cf740 1 --2- 172.21.15.6:0/648448015 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412d6060 0x55b8412de0b0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.470+0000 7f32b95cf740 1 -- 172.21.15.6:0/648448015 >> 172.21.15.6:0/648448015 conn(0x55b8412d25f0 msgr2=0x55b8412d1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:38.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.470+0000 7f32b95cf740 1 -- 172.21.15.6:0/648448015 shutdown_connections 2024-09-12T03:25:38.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.470+0000 7f32b95cf740 1 -- 172.21.15.6:0/648448015 wait complete. 2024-09-12T03:25:38.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.470+0000 7f32b95cf740 20 client.15942 trim_cache size 0 max 16384 2024-09-12T03:25:38.477 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:38.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.503+0000 7f32b95cf740 1 Processor -- start 2024-09-12T03:25:38.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.503+0000 7f32b95cf740 1 -- start start 2024-09-12T03:25:38.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.504+0000 7f32b95cf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412d6060 0x55b8412dff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.504+0000 7f32b95cf740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412d6980 0x55b8412f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.504+0000 7f32b4f92640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412d6060 0x55b8412dff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.504+0000 7f32b4f92640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412d6060 0x55b8412dff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:50138/0 (socket says 172.21.15.6:50138) 2024-09-12T03:25:38.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.505+0000 7f32affff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412d6980 0x55b8412f87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.505+0000 7f32affff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412d6980 0x55b8412f87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56758/0 (socket says 172.21.15.6:56758) 2024-09-12T03:25:38.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.505+0000 7f32b4f92640 1 -- 172.21.15.6:0/3843124057 learned_addr learned my addr 172.21.15.6:0/3843124057 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:38.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.505+0000 7f32b95cf740 1 --2- 172.21.15.6:0/3843124057 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412e1240 0x55b8412f8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.505+0000 7f32b95cf740 1 -- 172.21.15.6:0/3843124057 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01b6360 con 0x55b8412d6980 2024-09-12T03:25:38.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.505+0000 7f32b95cf740 1 -- 172.21.15.6:0/3843124057 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01b3c30 con 0x55b8412e1240 2024-09-12T03:25:38.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.505+0000 7f32b95cf740 1 -- 172.21.15.6:0/3843124057 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01b1a80 con 0x55b8412d6060 2024-09-12T03:25:38.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.505+0000 7f32b5793640 1 --2- 172.21.15.6:0/3843124057 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412e1240 0x55b8412f8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.505+0000 7f32b4f92640 1 -- 172.21.15.6:0/3843124057 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412e1240 msgr2=0x55b8412f8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:38.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.505+0000 7f32b4f92640 1 --2- 172.21.15.6:0/3843124057 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412e1240 0x55b8412f8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.505+0000 7f32b4f92640 1 -- 172.21.15.6:0/3843124057 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412d6980 msgr2=0x55b8412f87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:38.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.505+0000 7f32b4f92640 1 --2- 172.21.15.6:0/3843124057 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412d6980 0x55b8412f87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.505+0000 7f32b4f92640 1 -- 172.21.15.6:0/3843124057 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8412d2090 con 0x55b8412d6060 2024-09-12T03:25:38.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.506+0000 7f32b4f92640 1 --2- 172.21.15.6:0/3843124057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412d6060 0x55b8412dff80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f32a400c6a0 tx=0x7f32a4022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbbfdbc972f1bad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:38.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.506+0000 7f329a7fc640 1 -- 172.21.15.6:0/3843124057 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a40ba120 con 0x55b8412d6060 2024-09-12T03:25:38.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.506+0000 7f329a7fc640 1 -- 172.21.15.6:0/3843124057 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329c04c070 con 0x55b8412d6060 2024-09-12T03:25:38.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.506+0000 7f329a7fc640 1 -- 172.21.15.6:0/3843124057 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a40ba430 con 0x55b8412d6060 2024-09-12T03:25:38.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.507+0000 7f32b95cf740 1 -- 172.21.15.6:0/3843124057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412d6060 msgr2=0x55b8412dff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:38.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.507+0000 7f32b95cf740 1 --2- 172.21.15.6:0/3843124057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412d6060 0x55b8412dff80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f32a400c6a0 tx=0x7f32a4022000 comp rx=0 tx=0).stop 2024-09-12T03:25:38.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.507+0000 7f32b95cf740 1 -- 172.21.15.6:0/3843124057 shutdown_connections 2024-09-12T03:25:38.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.507+0000 7f32b95cf740 1 --2- 172.21.15.6:0/3843124057 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412e1240 0x55b8412f8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.507+0000 7f32b95cf740 1 --2- 172.21.15.6:0/3843124057 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412d6980 0x55b8412f87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.507+0000 7f32b95cf740 1 --2- 172.21.15.6:0/3843124057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412d6060 0x55b8412dff80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.507+0000 7f32b95cf740 1 -- 172.21.15.6:0/3843124057 >> 172.21.15.6:0/3843124057 conn(0x55b8412d25f0 msgr2=0x55b8412d82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:38.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.507+0000 7f32b95cf740 1 -- 172.21.15.6:0/3843124057 shutdown_connections 2024-09-12T03:25:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.507+0000 7f32b95cf740 1 -- 172.21.15.6:0/3843124057 wait complete. 2024-09-12T03:25:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.508+0000 7f32b95cf740 1 -- start start 2024-09-12T03:25:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.508+0000 7f32b95cf740 1 Processor -- start 2024-09-12T03:25:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.508+0000 7f32b95cf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412d6060 0x55b8412e9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.508+0000 7f32b95cf740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412d6980 0x55b8412e41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.508+0000 7f32b95cf740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412e1240 0x55b8412e46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.508+0000 7f32b95cf740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01b20f0 con 0x55b8412e1240 2024-09-12T03:25:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.508+0000 7f32b95cf740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01b3560 con 0x55b8412d6980 2024-09-12T03:25:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.508+0000 7f32b95cf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01b27c0 con 0x55b8412d6060 2024-09-12T03:25:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.509+0000 7f32b4f92640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412d6060 0x55b8412e9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.509+0000 7f32b95cf740 10 client.0 _ll_register_callbacks cb 0x7f32a409b590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:38.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.509+0000 7f32affff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412d6980 0x55b8412e41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.509+0000 7f32b4f92640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412d6060 0x55b8412e9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:50150/0 (socket says 172.21.15.6:50150) 2024-09-12T03:25:38.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.509+0000 7f32b4f92640 1 -- 172.21.15.6:0/649714635 learned_addr learned my addr 172.21.15.6:0/649714635 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:38.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.509+0000 7f32b5793640 1 --2- 172.21.15.6:0/649714635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412e1240 0x55b8412e46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.509+0000 7f32b4f92640 1 -- 172.21.15.6:0/649714635 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412d6980 msgr2=0x55b8412e41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:38.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.509+0000 7f32b4f92640 1 --2- 172.21.15.6:0/649714635 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412d6980 0x55b8412e41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.509+0000 7f32b4f92640 1 -- 172.21.15.6:0/649714635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412e1240 msgr2=0x55b8412e46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:38.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.509+0000 7f32b4f92640 1 --2- 172.21.15.6:0/649714635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412e1240 0x55b8412e46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.509+0000 7f32b4f92640 1 -- 172.21.15.6:0/649714635 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8412e4f10 con 0x55b8412d6060 2024-09-12T03:25:38.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.509+0000 7f32affff640 1 --2- 172.21.15.6:0/649714635 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412d6980 0x55b8412e41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:25:38.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.509+0000 7f32b4f92640 1 --2- 172.21.15.6:0/649714635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412d6060 0x55b8412e9940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f32a409b930 tx=0x7f32a409b640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1f0ee029972c786 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:38.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.509+0000 7f328ffff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:25:38.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.510+0000 7f328ffff640 1 -- 172.21.15.6:0/649714635 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a408a070 con 0x55b8412d6060 2024-09-12T03:25:38.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.510+0000 7f328ffff640 1 -- 172.21.15.6:0/649714635 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f32a40982a0 con 0x55b8412d6060 2024-09-12T03:25:38.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.510+0000 7f32b95cf740 1 -- 172.21.15.6:0/649714635 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f329c04c070 con 0x55b8412d6060 2024-09-12T03:25:38.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.510+0000 7f328ffff640 1 -- 172.21.15.6:0/649714635 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f32a4097ed0 con 0x55b8412d6060 2024-09-12T03:25:38.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.510+0000 7f32b95cf740 10 client.25564 fetch_fsmap learned FSMap version 27 2024-09-12T03:25:38.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.510+0000 7f32b95cf740 1 -- 172.21.15.6:0/649714635 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b8412e4f10 con 0x55b8412d6060 2024-09-12T03:25:38.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.510+0000 7f328ffff640 1 -- 172.21.15.6:0/649714635 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329c04c070 con 0x55b8412d6060 2024-09-12T03:25:38.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.510+0000 7f328ffff640 1 -- 172.21.15.6:0/649714635 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f32a409aaf0 con 0x55b8412d6060 2024-09-12T03:25:38.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.510+0000 7f32b95cf740 10 client.25564 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:25:38.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.510+0000 7f32b95cf740 10 client.25564 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:25:38.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.510+0000 7f32b95cf740 20 client.25564 populate_metadata read hostname 'smithi006' 2024-09-12T03:25:38.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.510+0000 7f32b95cf740 1 --2- 172.21.15.6:0/649714635 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b8412f0ad0 0x55b84135eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.511+0000 7f32b95cf740 4 client.25564 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6c171048", "release": true, "await": true}] 2024-09-12T03:25:38.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.511+0000 7f32b95cf740 1 -- 172.21.15.6:0/649714635 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6c171048", "release": true, "await": true}) -- 0x55b8412430a0 con 0x55b8412f0ad0 2024-09-12T03:25:38.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.511+0000 7f32affff640 1 --2- 172.21.15.6:0/649714635 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b8412f0ad0 0x55b84135eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.511+0000 7f32affff640 1 --2- 172.21.15.6:0/649714635 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b8412f0ad0 0x55b84135eae0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff5c68761a30bc server_cookie=89b3793fa4eee3a5 in_seq=0 out_seq=0 2024-09-12T03:25:38.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.511+0000 7f328ffff640 10 client.25564 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:38.585 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:25:38.585 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 27, 2024-09-12T03:25:38.585 INFO:teuthology.orchestra.run.smithi006.stdout: "leader": 24494, 2024-09-12T03:25:38.585 INFO:teuthology.orchestra.run.smithi006.stdout: "set_version": 20, 2024-09-12T03:25:38.585 INFO:teuthology.orchestra.run.smithi006.stdout: "sets": { 2024-09-12T03:25:38.586 INFO:teuthology.orchestra.run.smithi006.stdout: "6c171048": { 2024-09-12T03:25:38.586 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 20, 2024-09-12T03:25:38.586 INFO:teuthology.orchestra.run.smithi006.stdout: "age_ref": 0.0, 2024-09-12T03:25:38.586 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:25:38.586 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RELEASED", 2024-09-12T03:25:38.586 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:25:38.586 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:25:38.586 INFO:teuthology.orchestra.run.smithi006.stdout: "timeout": 90.0, 2024-09-12T03:25:38.586 INFO:teuthology.orchestra.run.smithi006.stdout: "expiration": 140.2, 2024-09-12T03:25:38.586 INFO:teuthology.orchestra.run.smithi006.stdout: "members": { 2024-09-12T03:25:38.586 INFO:teuthology.orchestra.run.smithi006.stdout: "file:/": { 2024-09-12T03:25:38.586 INFO:teuthology.orchestra.run.smithi006.stdout: "excluded": false, 2024-09-12T03:25:38.586 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:25:38.587 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RELEASED", 2024-09-12T03:25:38.587 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:25:38.587 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:25:38.587 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:25:38.587 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:25:38.587 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:25:38.587 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:25:38.587 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:25:38.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.540+0000 7f328ffff640 1 -- 172.21.15.6:0/649714635 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b8412430a0 con 0x55b8412f0ad0 2024-09-12T03:25:38.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.540+0000 7f328ffff640 10 client.25564 handle_command_reply: tid=0 2024-09-12T03:25:38.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.540+0000 7f32b95cf740 1 client.25564 shutdown 2024-09-12T03:25:38.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.541+0000 7f32b95cf740 1 -- 172.21.15.6:0/649714635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412d6060 msgr2=0x55b8412e9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:38.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.541+0000 7f32b95cf740 1 --2- 172.21.15.6:0/649714635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412d6060 0x55b8412e9940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f32a409b930 tx=0x7f32a409b640 comp rx=0 tx=0).stop 2024-09-12T03:25:38.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.541+0000 7f32b95cf740 1 -- 172.21.15.6:0/649714635 shutdown_connections 2024-09-12T03:25:38.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.541+0000 7f32b95cf740 1 --2- 172.21.15.6:0/649714635 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55b8412f0ad0 0x55b84135eae0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.541+0000 7f328ffff640 0 client.25564 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:38.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.541+0000 7f32b95cf740 1 --2- 172.21.15.6:0/649714635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b8412e1240 0x55b8412e46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.541+0000 7f32b95cf740 1 --2- 172.21.15.6:0/649714635 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b8412d6980 0x55b8412e41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.541+0000 7f32b95cf740 1 --2- 172.21.15.6:0/649714635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b8412d6060 0x55b8412e9940 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.541+0000 7f32b95cf740 1 -- 172.21.15.6:0/649714635 >> 172.21.15.6:0/649714635 conn(0x55b8412d25f0 msgr2=0x55b8412d7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:38.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.542+0000 7f32b95cf740 1 -- 172.21.15.6:0/649714635 shutdown_connections 2024-09-12T03:25:38.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.542+0000 7f32b95cf740 1 -- 172.21.15.6:0/649714635 wait complete. 2024-09-12T03:25:38.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.549+0000 7f32b95cf740 20 client.25564 trim_cache size 0 max 16384 2024-09-12T03:25:38.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.552+0000 7f32b71f6640 1 -- 172.21.15.6:0/190446892 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f32a00a2c00 msgr2=0x7f32a00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:38.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.552+0000 7f32b71f6640 1 --2- 172.21.15.6:0/190446892 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f32a00a2c00 0x7f32a00a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f329c000c00 tx=0x7f329c005a20 comp rx=0 tx=0).stop 2024-09-12T03:25:38.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.552+0000 7f32b71f6640 1 -- 172.21.15.6:0/190446892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b0100a80 msgr2=0x7f32b01a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:38.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.552+0000 7f32b71f6640 1 --2- 172.21.15.6:0/190446892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b0100a80 0x7f32b01a46a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f32a400e9f0 tx=0x7f32a400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.552+0000 7f32b71f6640 1 -- 172.21.15.6:0/190446892 shutdown_connections 2024-09-12T03:25:38.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.552+0000 7f32b71f6640 1 --2- 172.21.15.6:0/190446892 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f32a00a2c00 0x7f32a00a50c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.552+0000 7f32b71f6640 1 --2- 172.21.15.6:0/190446892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32b0106a80 0x7f32b019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.552+0000 7f32b71f6640 1 --2- 172.21.15.6:0/190446892 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32b0101420 0x7f32b01a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.552+0000 7f32b71f6640 1 --2- 172.21.15.6:0/190446892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b0100a80 0x7f32b01a46a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.552+0000 7f32b71f6640 1 -- 172.21.15.6:0/190446892 >> 172.21.15.6:0/190446892 conn(0x7f32b00fc7a0 msgr2=0x7f32b006a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:38.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.553+0000 7f32b71f6640 1 -- 172.21.15.6:0/190446892 shutdown_connections 2024-09-12T03:25:38.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:38.553+0000 7f32b71f6640 1 -- 172.21.15.6:0/190446892 wait complete. 2024-09-12T03:25:38.614 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:38 smithi006 ceph-mon[30991]: pgmap v471: 97 pgs: 97 active+clean; 13 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 7.8 KiB/s rd, 38 MiB/s wr, 3.85k op/s 2024-09-12T03:25:38.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:38 smithi118 ceph-mon[35048]: pgmap v471: 97 pgs: 97 active+clean; 13 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 7.8 KiB/s rd, 38 MiB/s wr, 3.85k op/s 2024-09-12T03:25:38.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:38 smithi161 ceph-mon[35485]: pgmap v471: 97 pgs: 97 active+clean; 13 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 7.8 KiB/s rd, 38 MiB/s wr, 3.85k op/s 2024-09-12T03:25:39.887 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6c171048', total seconds elapsed: 26.2 2024-09-12T03:25:40.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.244+0000 7fc337c88640 1 -- 172.21.15.6:0/1102181208 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3301008c0 msgr2=0x7fc330100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:40.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.244+0000 7fc337c88640 1 --2- 172.21.15.6:0/1102181208 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3301008c0 0x7fc330100ca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009890 tx=0x7fc32c02a390 comp rx=0 tx=0).stop 2024-09-12T03:25:40.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.246+0000 7fc337c88640 1 -- 172.21.15.6:0/1102181208 shutdown_connections 2024-09-12T03:25:40.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.246+0000 7fc337c88640 1 --2- 172.21.15.6:0/1102181208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc330105ec0 0x7fc330106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.246+0000 7fc337c88640 1 --2- 172.21.15.6:0/1102181208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3301011e0 0x7fc330105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.246+0000 7fc337c88640 1 --2- 172.21.15.6:0/1102181208 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3301008c0 0x7fc330100ca0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.246+0000 7fc337c88640 1 -- 172.21.15.6:0/1102181208 >> 172.21.15.6:0/1102181208 conn(0x7fc3300fc7d0 msgr2=0x7fc3300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:40.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.247+0000 7fc337c88640 1 -- 172.21.15.6:0/1102181208 shutdown_connections 2024-09-12T03:25:40.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.248+0000 7fc337c88640 1 -- 172.21.15.6:0/1102181208 wait complete. 2024-09-12T03:25:40.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.249+0000 7fc337c88640 1 Processor -- start 2024-09-12T03:25:40.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.249+0000 7fc337c88640 1 -- start start 2024-09-12T03:25:40.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.249+0000 7fc337c88640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3301008c0 0x7fc330073760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.249+0000 7fc337c88640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3301011e0 0x7fc330073ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.249+0000 7fc337c88640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc330105ec0 0x7fc33006e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.249+0000 7fc337c88640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc33006c220 con 0x7fc3301011e0 2024-09-12T03:25:40.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.249+0000 7fc337c88640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fc33006c0a0 con 0x7fc3301008c0 2024-09-12T03:25:40.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.249+0000 7fc337c88640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc33006c3a0 con 0x7fc330105ec0 2024-09-12T03:25:40.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.250+0000 7fc335a24640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3301008c0 0x7fc330073760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:40.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.250+0000 7fc335a24640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3301008c0 0x7fc330073760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:57920/0 (socket says 172.21.15.6:57920) 2024-09-12T03:25:40.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.250+0000 7fc335a24640 1 -- 172.21.15.6:0/2694059042 learned_addr learned my addr 172.21.15.6:0/2694059042 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:40.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.251+0000 7fc335a24640 1 -- 172.21.15.6:0/2694059042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc330105ec0 msgr2=0x7fc33006e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:40.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.251+0000 7fc335a24640 1 --2- 172.21.15.6:0/2694059042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc330105ec0 0x7fc33006e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.251+0000 7fc335a24640 1 -- 172.21.15.6:0/2694059042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3301011e0 msgr2=0x7fc330073ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:40.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.251+0000 7fc335a24640 1 --2- 172.21.15.6:0/2694059042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3301011e0 0x7fc330073ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.251+0000 7fc335a24640 1 -- 172.21.15.6:0/2694059042 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33006f140 con 0x7fc3301008c0 2024-09-12T03:25:40.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.251+0000 7fc335a24640 1 --2- 172.21.15.6:0/2694059042 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3301008c0 0x7fc330073760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc32c035040 tx=0x7fc32c033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:40.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.252+0000 7fc326ffd640 1 -- 172.21.15.6:0/2694059042 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c03f020 con 0x7fc3301008c0 2024-09-12T03:25:40.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.252+0000 7fc326ffd640 1 -- 172.21.15.6:0/2694059042 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc32c030d10 con 0x7fc3301008c0 2024-09-12T03:25:40.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.252+0000 7fc326ffd640 1 -- 172.21.15.6:0/2694059042 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c037040 con 0x7fc3301008c0 2024-09-12T03:25:40.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.253+0000 7fc337c88640 1 -- 172.21.15.6:0/2694059042 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc33006f3d0 con 0x7fc3301008c0 2024-09-12T03:25:40.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.253+0000 7fc337c88640 1 -- 172.21.15.6:0/2694059042 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc33010dfb0 con 0x7fc3301008c0 2024-09-12T03:25:40.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.269+0000 7fc33a061740 1 Processor -- start 2024-09-12T03:25:40.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.270+0000 7fc326ffd640 1 -- 172.21.15.6:0/2694059042 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc32c042020 con 0x7fc3301008c0 2024-09-12T03:25:40.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.270+0000 7fc33a061740 1 -- start start 2024-09-12T03:25:40.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.270+0000 7fc33a061740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c46f0bb40 0x555c46f0a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.270+0000 7fc33a061740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f0a5d0 0x555c46f0aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.270+0000 7fc33a061740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555c46f15370 0x555c46f17760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.270+0000 7fc33a061740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x555c46e702f0 con 0x555c46f0a5d0 2024-09-12T03:25:40.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.270+0000 7fc33a061740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x555c46e17cf0 con 0x555c46f15370 2024-09-12T03:25:40.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.270+0000 7fc33a061740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555c46e6c2b0 con 0x555c46f0bb40 2024-09-12T03:25:40.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.271+0000 7fc335a24640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f0a5d0 0x555c46f0aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:40.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.271+0000 7fc335a24640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f0a5d0 0x555c46f0aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55000/0 (socket says 172.21.15.6:55000) 2024-09-12T03:25:40.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.271+0000 7fc335a24640 1 -- 172.21.15.6:0/744844922 learned_addr learned my addr 172.21.15.6:0/744844922 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:40.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.271+0000 7fc326ffd640 1 --2- 172.21.15.6:0/2694059042 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc31c0a2da0 0x7fc31c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.271+0000 7fc335a24640 1 -- 172.21.15.6:0/744844922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c46f0bb40 msgr2=0x555c46f0a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:40.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.271+0000 7fc335a24640 1 --2- 172.21.15.6:0/744844922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c46f0bb40 0x555c46f0a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.271+0000 7fc335a24640 1 -- 172.21.15.6:0/744844922 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555c46f15370 msgr2=0x555c46f17760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:25:40.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.271+0000 7fc335a24640 1 --2- 172.21.15.6:0/744844922 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555c46f15370 0x555c46f17760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.271+0000 7fc335a24640 1 -- 172.21.15.6:0/744844922 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c46dfc3a0 con 0x555c46f0a5d0 2024-09-12T03:25:40.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.272+0000 7fc336225640 1 --2- 172.21.15.6:0/2694059042 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc31c0a2da0 0x7fc31c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:40.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.272+0000 7fc326ffd640 1 -- 172.21.15.6:0/2694059042 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fc32c0eee70 con 0x7fc3301008c0 2024-09-12T03:25:40.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.272+0000 7fc335a24640 1 --2- 172.21.15.6:0/744844922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f0a5d0 0x555c46f0aa10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0a8c30 tx=0x7fc32c0862f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa429e154e2ed13 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:40.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.272+0000 7fc336225640 1 --2- 172.21.15.6:0/2694059042 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc31c0a2da0 0x7fc31c0a5260 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc3300743f0 tx=0x7fc32000b250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:40.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.273+0000 7fc31bfff640 1 -- 172.21.15.6:0/744844922 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0bb470 con 0x555c46f0a5d0 2024-09-12T03:25:40.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.273+0000 7fc31bfff640 1 -- 172.21.15.6:0/744844922 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc32c009460 con 0x555c46f0a5d0 2024-09-12T03:25:40.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.273+0000 7fc31bfff640 1 -- 172.21.15.6:0/744844922 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0f2540 con 0x555c46f0a5d0 2024-09-12T03:25:40.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.274+0000 7fc33a061740 1 -- 172.21.15.6:0/744844922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f0a5d0 msgr2=0x555c46f0aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:40.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.274+0000 7fc33a061740 1 --2- 172.21.15.6:0/744844922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f0a5d0 0x555c46f0aa10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0a8c30 tx=0x7fc32c0862f0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.274+0000 7fc33a061740 1 -- 172.21.15.6:0/744844922 shutdown_connections 2024-09-12T03:25:40.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.274+0000 7fc33a061740 1 --2- 172.21.15.6:0/744844922 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555c46f15370 0x555c46f17760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.274+0000 7fc33a061740 1 --2- 172.21.15.6:0/744844922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f0a5d0 0x555c46f0aa10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.274+0000 7fc33a061740 1 --2- 172.21.15.6:0/744844922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c46f0bb40 0x555c46f0a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.274+0000 7fc33a061740 1 -- 172.21.15.6:0/744844922 >> 172.21.15.6:0/744844922 conn(0x555c46ed3310 msgr2=0x555c46f06c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:40.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.279+0000 7fc33a061740 1 -- 172.21.15.6:0/744844922 shutdown_connections 2024-09-12T03:25:40.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.279+0000 7fc33a061740 1 -- 172.21.15.6:0/744844922 wait complete. 2024-09-12T03:25:40.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.280+0000 7fc33a061740 1 -- start start 2024-09-12T03:25:40.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.280+0000 7fc33a061740 1 Processor -- start 2024-09-12T03:25:40.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.281+0000 7fc33a061740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f0a5d0 0x555c46f12130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.281+0000 7fc33a061740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555c46f0bb40 0x555c46f12670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.281+0000 7fc33a061740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c46f12bb0 0x555c46f210e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.281+0000 7fc33a061740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x555c46e702f0 con 0x555c46f0a5d0 2024-09-12T03:25:40.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.281+0000 7fc33a061740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x555c46e2b060 con 0x555c46f0bb40 2024-09-12T03:25:40.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.281+0000 7fc33a061740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555c46f17e30 con 0x555c46f12bb0 2024-09-12T03:25:40.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.281+0000 7fc33a061740 10 client.0 _ll_register_callbacks cb 0x555c46b7e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:40.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.281+0000 7fc336225640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c46f12bb0 0x555c46f210e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:40.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.282+0000 7fc335a24640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f0a5d0 0x555c46f12130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:40.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.282+0000 7fc335a24640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f0a5d0 0x555c46f12130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55016/0 (socket says 172.21.15.6:55016) 2024-09-12T03:25:40.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.282+0000 7fc335a24640 1 -- 172.21.15.6:0/1539941572 learned_addr learned my addr 172.21.15.6:0/1539941572 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:40.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.282+0000 7fc336225640 1 -- 172.21.15.6:0/1539941572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555c46f0bb40 msgr2=0x555c46f12670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:25:40.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.282+0000 7fc336225640 1 --2- 172.21.15.6:0/1539941572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555c46f0bb40 0x555c46f12670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.282+0000 7fc336225640 1 -- 172.21.15.6:0/1539941572 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f0a5d0 msgr2=0x555c46f12130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:40.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.282+0000 7fc336225640 1 --2- 172.21.15.6:0/1539941572 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f0a5d0 0x555c46f12130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.282+0000 7fc336225640 1 -- 172.21.15.6:0/1539941572 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c46be13c0 con 0x555c46f12bb0 2024-09-12T03:25:40.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.283+0000 7fc336225640 1 --2- 172.21.15.6:0/1539941572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c46f12bb0 0x555c46f210e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc32000cf70 tx=0x7fc320006960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b923a2b0c0991d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:40.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.283+0000 7fc3197fa640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:25:40.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.283+0000 7fc3197fa640 1 -- 172.21.15.6:0/1539941572 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32001a620 con 0x555c46f12bb0 2024-09-12T03:25:40.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.283+0000 7fc3197fa640 1 -- 172.21.15.6:0/1539941572 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fc32001a7c0 con 0x555c46f12bb0 2024-09-12T03:25:40.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.284+0000 7fc33a061740 1 -- 172.21.15.6:0/1539941572 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555c46e7d860 con 0x555c46f12bb0 2024-09-12T03:25:40.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.286+0000 7fc3197fa640 1 -- 172.21.15.6:0/1539941572 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc32001a9c0 con 0x555c46f12bb0 2024-09-12T03:25:40.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.286+0000 7fc33a061740 10 client.25570 fetch_fsmap learned FSMap version 27 2024-09-12T03:25:40.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.286+0000 7fc33a061740 1 -- 172.21.15.6:0/1539941572 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555c46f23c50 con 0x555c46f12bb0 2024-09-12T03:25:40.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.287+0000 7fc3197fa640 1 -- 172.21.15.6:0/1539941572 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc320012070 con 0x555c46f12bb0 2024-09-12T03:25:40.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.287+0000 7fc3197fa640 1 -- 172.21.15.6:0/1539941572 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc3200223e0 con 0x555c46f12bb0 2024-09-12T03:25:40.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.288+0000 7fc33a061740 10 client.25570 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:25:40.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.288+0000 7fc33a061740 10 client.25570 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-12T03:25:40.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.288+0000 7fc33a061740 20 client.25570 populate_metadata read hostname 'smithi006' 2024-09-12T03:25:40.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.288+0000 7fc33a061740 1 --2- 172.21.15.6:0/1539941572 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x555c46f25130 0x555c46f275b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.288+0000 7fc33a061740 4 client.25570 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:25:40.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.288+0000 7fc33a061740 1 -- 172.21.15.6:0/1539941572 --> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c46e7d860 con 0x555c46f25130 2024-09-12T03:25:40.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.288+0000 7fc335a24640 1 --2- 172.21.15.6:0/1539941572 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x555c46f25130 0x555c46f275b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:40.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.289+0000 7fc335a24640 1 --2- 172.21.15.6:0/1539941572 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x555c46f25130 0x555c46f275b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d47719605c6d21d2 server_cookie=2688577ae82de561 in_seq=0 out_seq=0 2024-09-12T03:25:40.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.289+0000 7fc3197fa640 10 client.25570 ms_handle_connect on v2:172.21.15.118:6836/3879191727 2024-09-12T03:25:40.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.290+0000 7fc3197fa640 1 -- 172.21.15.6:0/1539941572 <== mds.1 v2:172.21.15.118:6836/3879191727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c46e7d860 con 0x555c46f25130 2024-09-12T03:25:40.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.290+0000 7fc3197fa640 10 client.25570 handle_command_reply: tid=0 2024-09-12T03:25:40.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.290+0000 7fc33a061740 1 client.25570 shutdown 2024-09-12T03:25:40.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.290+0000 7fc33a061740 1 -- 172.21.15.6:0/1539941572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c46f12bb0 msgr2=0x555c46f210e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:40.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.290+0000 7fc33a061740 1 --2- 172.21.15.6:0/1539941572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c46f12bb0 0x555c46f210e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc32000cf70 tx=0x7fc320006960 comp rx=0 tx=0).stop 2024-09-12T03:25:40.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.291+0000 7fc33a061740 1 -- 172.21.15.6:0/1539941572 shutdown_connections 2024-09-12T03:25:40.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.291+0000 7fc33a061740 1 --2- 172.21.15.6:0/1539941572 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x555c46f25130 0x555c46f275b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.291+0000 7fc33a061740 1 --2- 172.21.15.6:0/1539941572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c46f12bb0 0x555c46f210e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.291+0000 7fc33a061740 1 --2- 172.21.15.6:0/1539941572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555c46f0bb40 0x555c46f12670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.291+0000 7fc33a061740 1 --2- 172.21.15.6:0/1539941572 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f0a5d0 0x555c46f12130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.291+0000 7fc33a061740 1 -- 172.21.15.6:0/1539941572 >> 172.21.15.6:0/1539941572 conn(0x555c46ed3310 msgr2=0x555c46f0c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:40.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.291+0000 7fc3197fa640 0 client.25570 ms_handle_reset on v2:172.21.15.118:6836/3879191727 2024-09-12T03:25:40.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.291+0000 7fc33a061740 1 -- 172.21.15.6:0/1539941572 shutdown_connections 2024-09-12T03:25:40.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.291+0000 7fc33a061740 1 -- 172.21.15.6:0/1539941572 wait complete. 2024-09-12T03:25:40.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.291+0000 7fc33a061740 20 client.25570 trim_cache size 0 max 16384 2024-09-12T03:25:40.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.321+0000 7fc33a061740 1 Processor -- start 2024-09-12T03:25:40.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.321+0000 7fc33a061740 1 -- start start 2024-09-12T03:25:40.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.321+0000 7fc33a061740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555c46f0a5d0 0x555c46f20f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.321+0000 7fc33a061740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c46f0bb40 0x555c46f21450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.321+0000 7fc33a061740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f12bb0 0x555c46f2ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.322+0000 7fc33a061740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc314010c30 con 0x555c46f12bb0 2024-09-12T03:25:40.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.322+0000 7fc33a061740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fc31400e500 con 0x555c46f0a5d0 2024-09-12T03:25:40.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.322+0000 7fc33a061740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc31400c350 con 0x555c46f0bb40 2024-09-12T03:25:40.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.322+0000 7fc335223640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c46f0bb40 0x555c46f21450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:40.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.322+0000 7fc335223640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c46f0bb40 0x555c46f21450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:44160/0 (socket says 172.21.15.6:44160) 2024-09-12T03:25:40.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.322+0000 7fc335223640 1 -- 172.21.15.6:0/1872902221 learned_addr learned my addr 172.21.15.6:0/1872902221 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:40.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.322+0000 7fc336225640 1 --2- 172.21.15.6:0/1872902221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f12bb0 0x555c46f2ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:40.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.323+0000 7fc335a24640 1 --2- 172.21.15.6:0/1872902221 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555c46f0a5d0 0x555c46f20f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:40.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.325+0000 7fc336225640 1 -- 172.21.15.6:0/1872902221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c46f0bb40 msgr2=0x555c46f21450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:40.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.325+0000 7fc336225640 1 --2- 172.21.15.6:0/1872902221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c46f0bb40 0x555c46f21450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.325+0000 7fc336225640 1 -- 172.21.15.6:0/1872902221 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555c46f0a5d0 msgr2=0x555c46f20f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:40.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.325+0000 7fc336225640 1 --2- 172.21.15.6:0/1872902221 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555c46f0a5d0 0x555c46f20f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.325+0000 7fc336225640 1 -- 172.21.15.6:0/1872902221 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c46f0d120 con 0x555c46f12bb0 2024-09-12T03:25:40.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.328+0000 7fc336225640 1 --2- 172.21.15.6:0/1872902221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f12bb0 0x555c46f2ce80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc32000e010 tx=0x7fc320022fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ee1afccca5086c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:40.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.331+0000 7fc31b7fe640 1 -- 172.21.15.6:0/1872902221 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc320030070 con 0x555c46f12bb0 2024-09-12T03:25:40.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.331+0000 7fc31b7fe640 1 -- 172.21.15.6:0/1872902221 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc320012070 con 0x555c46f12bb0 2024-09-12T03:25:40.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.331+0000 7fc31b7fe640 1 -- 172.21.15.6:0/1872902221 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc320004030 con 0x555c46f12bb0 2024-09-12T03:25:40.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.332+0000 7fc33a061740 1 -- 172.21.15.6:0/1872902221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f12bb0 msgr2=0x555c46f2ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:40.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.332+0000 7fc33a061740 1 --2- 172.21.15.6:0/1872902221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f12bb0 0x555c46f2ce80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc32000e010 tx=0x7fc320022fb0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.332+0000 7fc33a061740 1 -- 172.21.15.6:0/1872902221 shutdown_connections 2024-09-12T03:25:40.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.332+0000 7fc33a061740 1 --2- 172.21.15.6:0/1872902221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f12bb0 0x555c46f2ce80 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.332+0000 7fc33a061740 1 --2- 172.21.15.6:0/1872902221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c46f0bb40 0x555c46f21450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.332+0000 7fc33a061740 1 --2- 172.21.15.6:0/1872902221 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555c46f0a5d0 0x555c46f20f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.332+0000 7fc33a061740 1 -- 172.21.15.6:0/1872902221 >> 172.21.15.6:0/1872902221 conn(0x555c46ed3310 msgr2=0x555c46f10a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:40.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.333+0000 7fc33a061740 1 -- 172.21.15.6:0/1872902221 shutdown_connections 2024-09-12T03:25:40.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.333+0000 7fc33a061740 1 -- 172.21.15.6:0/1872902221 wait complete. 2024-09-12T03:25:40.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.333+0000 7fc33a061740 1 -- start start 2024-09-12T03:25:40.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.333+0000 7fc33a061740 1 Processor -- start 2024-09-12T03:25:40.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.334+0000 7fc33a061740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f0a5d0 0x555c46f1d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.334+0000 7fc33a061740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555c46f0bb40 0x555c46f0eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.334+0000 7fc33a061740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c46f12bb0 0x555c46f0f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.334+0000 7fc33a061740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc31400c9c0 con 0x555c46f0a5d0 2024-09-12T03:25:40.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.334+0000 7fc33a061740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fc31400de30 con 0x555c46f0bb40 2024-09-12T03:25:40.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.334+0000 7fc33a061740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc31400d090 con 0x555c46f12bb0 2024-09-12T03:25:40.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.334+0000 7fc33a061740 10 client.0 _ll_register_callbacks cb 0x7fc32000c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:40.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.334+0000 7fc335223640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555c46f0bb40 0x555c46f0eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:40.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.334+0000 7fc335223640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555c46f0bb40 0x555c46f0eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:57954/0 (socket says 172.21.15.6:57954) 2024-09-12T03:25:40.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.334+0000 7fc335223640 1 -- 172.21.15.6:0/1347090044 learned_addr learned my addr 172.21.15.6:0/1347090044 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:40.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.334+0000 7fc336225640 1 --2- 172.21.15.6:0/1347090044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c46f12bb0 0x555c46f0f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:40.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.335+0000 7fc335a24640 1 --2- 172.21.15.6:0/1347090044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f0a5d0 0x555c46f1d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:40.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.336+0000 7fc335a24640 1 -- 172.21.15.6:0/1347090044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c46f12bb0 msgr2=0x555c46f0f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:40.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.336+0000 7fc335a24640 1 --2- 172.21.15.6:0/1347090044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c46f12bb0 0x555c46f0f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.336+0000 7fc335a24640 1 -- 172.21.15.6:0/1347090044 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555c46f0bb40 msgr2=0x555c46f0eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:40.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.336+0000 7fc335a24640 1 --2- 172.21.15.6:0/1347090044 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555c46f0bb40 0x555c46f0eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.336+0000 7fc335a24640 1 -- 172.21.15.6:0/1347090044 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c46f0f590 con 0x555c46f0a5d0 2024-09-12T03:25:40.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.336+0000 7fc335a24640 1 --2- 172.21.15.6:0/1347090044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f0a5d0 0x555c46f1d480 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0c3de0 tx=0x7fc32c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992b49656d140d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:40.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.338+0000 7fc3197fa640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:25:40.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.338+0000 7fc3197fa640 1 -- 172.21.15.6:0/1347090044 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0a8a60 con 0x555c46f0a5d0 2024-09-12T03:25:40.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.338+0000 7fc33a061740 1 -- 172.21.15.6:0/1347090044 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc320012070 con 0x555c46f0a5d0 2024-09-12T03:25:40.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.338+0000 7fc3197fa640 1 -- 172.21.15.6:0/1347090044 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fc32c0b7570 con 0x555c46f0a5d0 2024-09-12T03:25:40.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.338+0000 7fc3197fa640 1 -- 172.21.15.6:0/1347090044 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc32c0bb000 con 0x555c46f0a5d0 2024-09-12T03:25:40.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.339+0000 7fc33a061740 10 client.15960 fetch_fsmap learned FSMap version 27 2024-09-12T03:25:40.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.339+0000 7fc33a061740 1 -- 172.21.15.6:0/1347090044 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555c46f8fab0 con 0x555c46f0a5d0 2024-09-12T03:25:40.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.339+0000 7fc3197fa640 1 -- 172.21.15.6:0/1347090044 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc320012070 con 0x555c46f0a5d0 2024-09-12T03:25:40.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.340+0000 7fc3197fa640 1 -- 172.21.15.6:0/1347090044 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc32c04c060 con 0x555c46f0a5d0 2024-09-12T03:25:40.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.340+0000 7fc33a061740 10 client.15960 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:25:40.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.340+0000 7fc33a061740 10 client.15960 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-12T03:25:40.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.340+0000 7fc33a061740 20 client.15960 populate_metadata read hostname 'smithi006' 2024-09-12T03:25:40.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.340+0000 7fc33a061740 1 --2- 172.21.15.6:0/1347090044 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x555c46f8ff90 0x555c46f923b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.340+0000 7fc33a061740 4 client.15960 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:25:40.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.340+0000 7fc33a061740 1 -- 172.21.15.6:0/1347090044 --> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c46f12a00 con 0x555c46f8ff90 2024-09-12T03:25:40.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.341+0000 7fc335223640 1 --2- 172.21.15.6:0/1347090044 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x555c46f8ff90 0x555c46f923b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:40.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.346+0000 7fc335223640 1 --2- 172.21.15.6:0/1347090044 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x555c46f8ff90 0x555c46f923b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=282b8c03e53d31a9 server_cookie=6b1b04abc934d22e in_seq=0 out_seq=0 2024-09-12T03:25:40.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.346+0000 7fc3197fa640 10 client.15960 ms_handle_connect on v2:172.21.15.118:6836/3879191727 2024-09-12T03:25:40.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.346+0000 7fc3197fa640 1 -- 172.21.15.6:0/1347090044 <== mds.1 v2:172.21.15.118:6836/3879191727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c46f12a00 con 0x555c46f8ff90 2024-09-12T03:25:40.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.346+0000 7fc3197fa640 10 client.15960 handle_command_reply: tid=0 2024-09-12T03:25:40.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.346+0000 7fc33a061740 1 client.15960 shutdown 2024-09-12T03:25:40.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.347+0000 7fc33a061740 1 -- 172.21.15.6:0/1347090044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f0a5d0 msgr2=0x555c46f1d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:40.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.347+0000 7fc33a061740 1 --2- 172.21.15.6:0/1347090044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f0a5d0 0x555c46f1d480 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0c3de0 tx=0x7fc32c005f90 comp rx=0 tx=0).stop 2024-09-12T03:25:40.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.348+0000 7fc33a061740 1 -- 172.21.15.6:0/1347090044 shutdown_connections 2024-09-12T03:25:40.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.348+0000 7fc33a061740 1 --2- 172.21.15.6:0/1347090044 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x555c46f8ff90 0x555c46f923b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.348+0000 7fc33a061740 1 --2- 172.21.15.6:0/1347090044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c46f12bb0 0x555c46f0f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.348+0000 7fc33a061740 1 --2- 172.21.15.6:0/1347090044 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555c46f0bb40 0x555c46f0eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.348+0000 7fc33a061740 1 --2- 172.21.15.6:0/1347090044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c46f0a5d0 0x555c46f1d480 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.348+0000 7fc3197fa640 0 client.15960 ms_handle_reset on v2:172.21.15.118:6836/3879191727 2024-09-12T03:25:40.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.348+0000 7fc33a061740 1 -- 172.21.15.6:0/1347090044 >> 172.21.15.6:0/1347090044 conn(0x555c46ed3310 msgr2=0x555c46f054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:40.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.351+0000 7fc33a061740 1 -- 172.21.15.6:0/1347090044 shutdown_connections 2024-09-12T03:25:40.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.351+0000 7fc33a061740 1 -- 172.21.15.6:0/1347090044 wait complete. 2024-09-12T03:25:40.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.352+0000 7fc33a061740 20 client.15960 trim_cache size 0 max 16384 2024-09-12T03:25:40.352 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-09-12T03:25:40.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.353+0000 7fc337c88640 1 -- 172.21.15.6:0/2694059042 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc31c0a2da0 msgr2=0x7fc31c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:40.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.353+0000 7fc337c88640 1 --2- 172.21.15.6:0/2694059042 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc31c0a2da0 0x7fc31c0a5260 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc3300743f0 tx=0x7fc32000b250 comp rx=0 tx=0).stop 2024-09-12T03:25:40.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.353+0000 7fc337c88640 1 -- 172.21.15.6:0/2694059042 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3301008c0 msgr2=0x7fc330073760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:40.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.353+0000 7fc337c88640 1 --2- 172.21.15.6:0/2694059042 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3301008c0 0x7fc330073760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc32c035040 tx=0x7fc32c033d90 comp rx=0 tx=0).stop 2024-09-12T03:25:40.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.353+0000 7fc337c88640 1 -- 172.21.15.6:0/2694059042 shutdown_connections 2024-09-12T03:25:40.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.353+0000 7fc337c88640 1 --2- 172.21.15.6:0/2694059042 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc31c0a2da0 0x7fc31c0a5260 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.353+0000 7fc337c88640 1 --2- 172.21.15.6:0/2694059042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc330105ec0 0x7fc33006e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.353+0000 7fc337c88640 1 --2- 172.21.15.6:0/2694059042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3301011e0 0x7fc330073ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.353+0000 7fc337c88640 1 --2- 172.21.15.6:0/2694059042 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3301008c0 0x7fc330073760 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.353+0000 7fc337c88640 1 -- 172.21.15.6:0/2694059042 >> 172.21.15.6:0/2694059042 conn(0x7fc3300fc7d0 msgr2=0x7fc330103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:40.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.355+0000 7fc337c88640 1 -- 172.21.15.6:0/2694059042 shutdown_connections 2024-09-12T03:25:40.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:40.355+0000 7fc337c88640 1 -- 172.21.15.6:0/2694059042 wait complete. 2024-09-12T03:25:40.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:40 smithi006 ceph-mon[30991]: pgmap v472: 97 pgs: 97 active+clean; 13 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 40 MiB/s wr, 4.03k op/s 2024-09-12T03:25:40.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:40 smithi118 ceph-mon[35048]: pgmap v472: 97 pgs: 97 active+clean; 13 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 40 MiB/s wr, 4.03k op/s 2024-09-12T03:25:40.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:40 smithi161 ceph-mon[35485]: pgmap v472: 97 pgs: 97 active+clean; 13 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 40 MiB/s wr, 4.03k op/s 2024-09-12T03:25:41.560 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:25:42.110 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:42.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:42 smithi118 ceph-mon[35048]: pgmap v473: 97 pgs: 97 active+clean; 13 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 114 KiB/s rd, 50 MiB/s wr, 5.15k op/s 2024-09-12T03:25:42.823 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:42 smithi006 ceph-mon[30991]: pgmap v473: 97 pgs: 97 active+clean; 13 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 114 KiB/s rd, 50 MiB/s wr, 5.15k op/s 2024-09-12T03:25:42.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:42 smithi161 ceph-mon[35485]: pgmap v473: 97 pgs: 97 active+clean; 13 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 114 KiB/s rd, 50 MiB/s wr, 5.15k op/s 2024-09-12T03:25:42.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:42.991+0000 7f695e7cf640 1 -- 172.21.15.6:0/3754206448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f695806ea30 msgr2=0x7f695806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:42.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:42.991+0000 7f695e7cf640 1 --2- 172.21.15.6:0/3754206448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f695806ea30 0x7f695806ee70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6954009860 tx=0x7f695402a380 comp rx=0 tx=0).stop 2024-09-12T03:25:42.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:42.993+0000 7f695e7cf640 1 -- 172.21.15.6:0/3754206448 shutdown_connections 2024-09-12T03:25:42.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:42.993+0000 7f695e7cf640 1 --2- 172.21.15.6:0/3754206448 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f695811b140 0x7f695811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:42.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:42.993+0000 7f695e7cf640 1 --2- 172.21.15.6:0/3754206448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f695806ea30 0x7f695806ee70 secure :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6954009860 tx=0x7f695402a380 comp rx=0 tx=0).stop 2024-09-12T03:25:42.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:42.993+0000 7f695e7cf640 1 --2- 172.21.15.6:0/3754206448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6958110190 0x7f695806e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:42.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:42.993+0000 7f695e7cf640 1 -- 172.21.15.6:0/3754206448 >> 172.21.15.6:0/3754206448 conn(0x7f6958073180 msgr2=0x7f6958073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:42.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:42.998+0000 7f695e7cf640 1 -- 172.21.15.6:0/3754206448 shutdown_connections 2024-09-12T03:25:42.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:42.998+0000 7f695e7cf640 1 -- 172.21.15.6:0/3754206448 wait complete. 2024-09-12T03:25:43.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:42.999+0000 7f695e7cf640 1 Processor -- start 2024-09-12T03:25:43.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:42.999+0000 7f695e7cf640 1 -- start start 2024-09-12T03:25:43.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:42.999+0000 7f695e7cf640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6958110190 0x7f69581a7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:43.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:42.999+0000 7f695e7cf640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f695811b140 0x7f69581a7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:43.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:42.999+0000 7f695e7cf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69581abbd0 0x7f69581ac080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:43.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:42.999+0000 7f695e7cf640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6958120be0 con 0x7f6958110190 2024-09-12T03:25:43.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:42.999+0000 7f695e7cf640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f6958120a60 con 0x7f695811b140 2024-09-12T03:25:43.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:42.999+0000 7f695e7cf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6958120d60 con 0x7f69581abbd0 2024-09-12T03:25:43.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.001+0000 7f695dfce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69581abbd0 0x7f69581ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:43.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.001+0000 7f695dfce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69581abbd0 0x7f69581ac080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:44186/0 (socket says 172.21.15.6:44186) 2024-09-12T03:25:43.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.001+0000 7f695dfce640 1 -- 172.21.15.6:0/416782180 learned_addr learned my addr 172.21.15.6:0/416782180 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:43.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.001+0000 7f695cfcc640 1 --2- 172.21.15.6:0/416782180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f695811b140 0x7f69581a7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:43.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.001+0000 7f695d7cd640 1 --2- 172.21.15.6:0/416782180 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6958110190 0x7f69581a7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:43.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.002+0000 7f695cfcc640 1 -- 172.21.15.6:0/416782180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69581abbd0 msgr2=0x7f69581ac080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:43.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.002+0000 7f695cfcc640 1 --2- 172.21.15.6:0/416782180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69581abbd0 0x7f69581ac080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.002+0000 7f695cfcc640 1 -- 172.21.15.6:0/416782180 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6958110190 msgr2=0x7f69581a7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:43.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.002+0000 7f695cfcc640 1 --2- 172.21.15.6:0/416782180 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6958110190 0x7f69581a7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.002+0000 7f695cfcc640 1 -- 172.21.15.6:0/416782180 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69581ac690 con 0x7f695811b140 2024-09-12T03:25:43.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.002+0000 7f695cfcc640 1 --2- 172.21.15.6:0/416782180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f695811b140 0x7f69581a7840 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6954008000 tx=0x7f6954033d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:43.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.003+0000 7f694e7fc640 1 -- 172.21.15.6:0/416782180 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954040070 con 0x7f695811b140 2024-09-12T03:25:43.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.003+0000 7f695e7cf640 1 -- 172.21.15.6:0/416782180 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69581ac920 con 0x7f695811b140 2024-09-12T03:25:43.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.003+0000 7f694e7fc640 1 -- 172.21.15.6:0/416782180 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6954030e40 con 0x7f695811b140 2024-09-12T03:25:43.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.003+0000 7f694e7fc640 1 -- 172.21.15.6:0/416782180 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954037040 con 0x7f695811b140 2024-09-12T03:25:43.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.003+0000 7f695e7cf640 1 -- 172.21.15.6:0/416782180 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f695807a620 con 0x7f695811b140 2024-09-12T03:25:43.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.007+0000 7f694e7fc640 1 -- 172.21.15.6:0/416782180 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6954043070 con 0x7f695811b140 2024-09-12T03:25:43.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.008+0000 7f694e7fc640 1 --2- 172.21.15.6:0/416782180 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f69440a2cc0 0x7f69440a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:43.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.008+0000 7f694e7fc640 1 -- 172.21.15.6:0/416782180 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f69540ee3a0 con 0x7f695811b140 2024-09-12T03:25:43.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.009+0000 7f695d7cd640 1 --2- 172.21.15.6:0/416782180 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f69440a2cc0 0x7f69440a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:43.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.010+0000 7f695d7cd640 1 --2- 172.21.15.6:0/416782180 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f69440a2cc0 0x7f69440a5180 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f694800bbe0 tx=0x7f694800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:43.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.011+0000 7f6961e0a740 1 Processor -- start 2024-09-12T03:25:43.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.011+0000 7f6961e0a740 1 -- start start 2024-09-12T03:25:43.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.011+0000 7f6961e0a740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c835d20 0x556c1c834270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:43.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.012+0000 7f6961e0a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556c1c8347b0 0x556c1c834bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:43.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.012+0000 7f6961e0a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c83f550 0x556c1c841940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:43.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.012+0000 7f6961e0a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x556c1c79a2f0 con 0x556c1c8347b0 2024-09-12T03:25:43.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.012+0000 7f6961e0a740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x556c1c741cf0 con 0x556c1c835d20 2024-09-12T03:25:43.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.012+0000 7f6961e0a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556c1c7962b0 con 0x556c1c83f550 2024-09-12T03:25:43.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.012+0000 7f695d7cd640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c835d20 0x556c1c834270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:43.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.012+0000 7f695d7cd640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c835d20 0x556c1c834270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:57984/0 (socket says 172.21.15.6:57984) 2024-09-12T03:25:43.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.012+0000 7f695d7cd640 1 -- 172.21.15.6:0/4207488217 learned_addr learned my addr 172.21.15.6:0/4207488217 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:43.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.013+0000 7f695dfce640 1 --2- 172.21.15.6:0/4207488217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c83f550 0x556c1c841940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:43.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.014+0000 7f695d7cd640 1 -- 172.21.15.6:0/4207488217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c83f550 msgr2=0x556c1c841940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:43.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.014+0000 7f695d7cd640 1 --2- 172.21.15.6:0/4207488217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c83f550 0x556c1c841940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.014+0000 7f695d7cd640 1 -- 172.21.15.6:0/4207488217 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556c1c8347b0 msgr2=0x556c1c834bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:43.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.014+0000 7f695d7cd640 1 --2- 172.21.15.6:0/4207488217 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556c1c8347b0 0x556c1c834bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.014+0000 7f695d7cd640 1 -- 172.21.15.6:0/4207488217 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c1c7263a0 con 0x556c1c835d20 2024-09-12T03:25:43.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.014+0000 7f695d7cd640 1 --2- 172.21.15.6:0/4207488217 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c835d20 0x556c1c834270 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6948005d00 tx=0x7f6948013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b2438fac1a3022d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:43.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.014+0000 7f69437fe640 1 -- 172.21.15.6:0/4207488217 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948005220 con 0x556c1c835d20 2024-09-12T03:25:43.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.015+0000 7f69437fe640 1 -- 172.21.15.6:0/4207488217 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6948004c80 con 0x556c1c835d20 2024-09-12T03:25:43.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.015+0000 7f69437fe640 1 -- 172.21.15.6:0/4207488217 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694801ea30 con 0x556c1c835d20 2024-09-12T03:25:43.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.018+0000 7f6961e0a740 1 -- 172.21.15.6:0/4207488217 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c835d20 msgr2=0x556c1c834270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:43.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.018+0000 7f6961e0a740 1 --2- 172.21.15.6:0/4207488217 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c835d20 0x556c1c834270 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6948005d00 tx=0x7f6948013000 comp rx=0 tx=0).stop 2024-09-12T03:25:43.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.019+0000 7f6961e0a740 1 -- 172.21.15.6:0/4207488217 shutdown_connections 2024-09-12T03:25:43.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.019+0000 7f6961e0a740 1 --2- 172.21.15.6:0/4207488217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c83f550 0x556c1c841940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.019+0000 7f6961e0a740 1 --2- 172.21.15.6:0/4207488217 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556c1c8347b0 0x556c1c834bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.019+0000 7f6961e0a740 1 --2- 172.21.15.6:0/4207488217 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c835d20 0x556c1c834270 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.019+0000 7f6961e0a740 1 -- 172.21.15.6:0/4207488217 >> 172.21.15.6:0/4207488217 conn(0x556c1c7fd310 msgr2=0x556c1c830e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:43.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.019+0000 7f6961e0a740 1 -- 172.21.15.6:0/4207488217 shutdown_connections 2024-09-12T03:25:43.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.020+0000 7f6961e0a740 1 -- 172.21.15.6:0/4207488217 wait complete. 2024-09-12T03:25:43.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.020+0000 7f6961e0a740 1 -- start start 2024-09-12T03:25:43.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.020+0000 7f6961e0a740 1 Processor -- start 2024-09-12T03:25:43.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.021+0000 7f6961e0a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556c1c8347b0 0x556c1c83ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:43.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.021+0000 7f6961e0a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c835d20 0x556c1c83c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:43.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.021+0000 7f6961e0a740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c83f550 0x556c1c83c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:43.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.021+0000 7f6961e0a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x556c1c842250 con 0x556c1c8347b0 2024-09-12T03:25:43.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.021+0000 7f6961e0a740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x556c1c755060 con 0x556c1c83f550 2024-09-12T03:25:43.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.021+0000 7f6961e0a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556c1c8423d0 con 0x556c1c835d20 2024-09-12T03:25:43.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.021+0000 7f6961e0a740 10 client.0 _ll_register_callbacks cb 0x556c1c4a88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:43.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.022+0000 7f695dfce640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c83f550 0x556c1c83c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:43.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.022+0000 7f695dfce640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c83f550 0x556c1c83c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:57988/0 (socket says 172.21.15.6:57988) 2024-09-12T03:25:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.022+0000 7f695dfce640 1 -- 172.21.15.6:0/1483503461 learned_addr learned my addr 172.21.15.6:0/1483503461 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.022+0000 7f695cfcc640 1 --2- 172.21.15.6:0/1483503461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c835d20 0x556c1c83c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:43.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.023+0000 7f695dfce640 1 -- 172.21.15.6:0/1483503461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c835d20 msgr2=0x556c1c83c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:43.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.023+0000 7f695dfce640 1 --2- 172.21.15.6:0/1483503461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c835d20 0x556c1c83c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.023+0000 7f695dfce640 1 -- 172.21.15.6:0/1483503461 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556c1c8347b0 msgr2=0x556c1c83ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:43.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.023+0000 7f695dfce640 1 --2- 172.21.15.6:0/1483503461 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556c1c8347b0 0x556c1c83ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.023+0000 7f695dfce640 1 -- 172.21.15.6:0/1483503461 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c1c50b3c0 con 0x556c1c83f550 2024-09-12T03:25:43.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.024+0000 7f695dfce640 1 --2- 172.21.15.6:0/1483503461 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c83f550 0x556c1c83c830 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6950010c40 tx=0x7f695000d7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bbc7abef74c9ce8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:43.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.024+0000 7f6940ff9640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:25:43.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.025+0000 7f6940ff9640 1 -- 172.21.15.6:0/1483503461 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695000e450 con 0x556c1c83f550 2024-09-12T03:25:43.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.025+0000 7f6961e0a740 1 -- 172.21.15.6:0/1483503461 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6948004c80 con 0x556c1c83f550 2024-09-12T03:25:43.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.025+0000 7f6940ff9640 1 -- 172.21.15.6:0/1483503461 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f695000e5f0 con 0x556c1c83f550 2024-09-12T03:25:43.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.025+0000 7f6940ff9640 1 -- 172.21.15.6:0/1483503461 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f695000e7f0 con 0x556c1c83f550 2024-09-12T03:25:43.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.027+0000 7f6961e0a740 10 client.25514 fetch_fsmap learned FSMap version 27 2024-09-12T03:25:43.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.027+0000 7f6961e0a740 1 -- 172.21.15.6:0/1483503461 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556c1c84ddd0 con 0x556c1c83f550 2024-09-12T03:25:43.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.029+0000 7f6940ff9640 1 -- 172.21.15.6:0/1483503461 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6948004c80 con 0x556c1c83f550 2024-09-12T03:25:43.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.031+0000 7f6940ff9640 1 -- 172.21.15.6:0/1483503461 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f695000b040 con 0x556c1c83f550 2024-09-12T03:25:43.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.031+0000 7f6961e0a740 10 client.25514 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:25:43.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.031+0000 7f6961e0a740 10 client.25514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:25:43.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.031+0000 7f6961e0a740 20 client.25514 populate_metadata read hostname 'smithi006' 2024-09-12T03:25:43.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.031+0000 7f6961e0a740 1 --2- 172.21.15.6:0/1483503461 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x556c1c84f290 0x556c1c8516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:43.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.031+0000 7f6961e0a740 4 client.25514 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:25:43.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.031+0000 7f6961e0a740 1 -- 172.21.15.6:0/1483503461 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c1c7a7860 con 0x556c1c84f290 2024-09-12T03:25:43.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.032+0000 7f695d7cd640 1 --2- 172.21.15.6:0/1483503461 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x556c1c84f290 0x556c1c8516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:43.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.035+0000 7f695d7cd640 1 --2- 172.21.15.6:0/1483503461 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x556c1c84f290 0x556c1c8516e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eb3cbba0139c4f8 server_cookie=c58c0141bf203c8e in_seq=0 out_seq=0 2024-09-12T03:25:43.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.035+0000 7f6940ff9640 10 client.25514 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:43.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.038+0000 7f6940ff9640 1 -- 172.21.15.6:0/1483503461 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c1c7a7860 con 0x556c1c84f290 2024-09-12T03:25:43.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.038+0000 7f6940ff9640 10 client.25514 handle_command_reply: tid=0 2024-09-12T03:25:43.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.038+0000 7f6961e0a740 1 client.25514 shutdown 2024-09-12T03:25:43.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.042+0000 7f6961e0a740 1 -- 172.21.15.6:0/1483503461 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c83f550 msgr2=0x556c1c83c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:43.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.042+0000 7f6961e0a740 1 --2- 172.21.15.6:0/1483503461 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c83f550 0x556c1c83c830 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6950010c40 tx=0x7f695000d7b0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.044+0000 7f6961e0a740 1 -- 172.21.15.6:0/1483503461 shutdown_connections 2024-09-12T03:25:43.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.045+0000 7f6961e0a740 1 --2- 172.21.15.6:0/1483503461 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x556c1c84f290 0x556c1c8516e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.045+0000 7f6961e0a740 1 --2- 172.21.15.6:0/1483503461 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c83f550 0x556c1c83c830 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.045+0000 7f6961e0a740 1 --2- 172.21.15.6:0/1483503461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c835d20 0x556c1c83c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.045+0000 7f6961e0a740 1 --2- 172.21.15.6:0/1483503461 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556c1c8347b0 0x556c1c83ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.045+0000 7f6961e0a740 1 -- 172.21.15.6:0/1483503461 >> 172.21.15.6:0/1483503461 conn(0x556c1c7fd310 msgr2=0x556c1c836b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:43.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.045+0000 7f6940ff9640 0 client.25514 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:43.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.045+0000 7f6961e0a740 1 -- 172.21.15.6:0/1483503461 shutdown_connections 2024-09-12T03:25:43.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.045+0000 7f6961e0a740 1 -- 172.21.15.6:0/1483503461 wait complete. 2024-09-12T03:25:43.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.045+0000 7f6961e0a740 20 client.25514 trim_cache size 0 max 16384 2024-09-12T03:25:43.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.075+0000 7f6961e0a740 1 Processor -- start 2024-09-12T03:25:43.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.076+0000 7f6961e0a740 1 -- start start 2024-09-12T03:25:43.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.076+0000 7f6961e0a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556c1c8347b0 0x556c1c84b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:43.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.077+0000 7f6961e0a740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c835d20 0x556c1c83b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:43.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.077+0000 7f6961e0a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c83f550 0x556c1c856fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:43.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.077+0000 7f695cfcc640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c835d20 0x556c1c83b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:43.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.077+0000 7f695cfcc640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c835d20 0x556c1c83b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:58002/0 (socket says 172.21.15.6:58002) 2024-09-12T03:25:43.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.077+0000 7f695dfce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c83f550 0x556c1c856fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:43.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.078+0000 7f695dfce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c83f550 0x556c1c856fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:44216/0 (socket says 172.21.15.6:44216) 2024-09-12T03:25:43.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.078+0000 7f695cfcc640 1 -- 172.21.15.6:0/247263989 learned_addr learned my addr 172.21.15.6:0/247263989 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:43.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.078+0000 7f695d7cd640 1 --2- 172.21.15.6:0/247263989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556c1c8347b0 0x556c1c84b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:43.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.078+0000 7f6961e0a740 1 -- 172.21.15.6:0/247263989 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f69581cffa0 con 0x556c1c8347b0 2024-09-12T03:25:43.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.078+0000 7f6961e0a740 1 -- 172.21.15.6:0/247263989 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f6958082e30 con 0x556c1c835d20 2024-09-12T03:25:43.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.079+0000 7f6961e0a740 1 -- 172.21.15.6:0/247263989 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6958080c80 con 0x556c1c83f550 2024-09-12T03:25:43.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.079+0000 7f695cfcc640 1 -- 172.21.15.6:0/247263989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c83f550 msgr2=0x556c1c856fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:43.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.079+0000 7f695cfcc640 1 --2- 172.21.15.6:0/247263989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c83f550 0x556c1c856fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.079+0000 7f695cfcc640 1 -- 172.21.15.6:0/247263989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556c1c8347b0 msgr2=0x556c1c84b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:43.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.079+0000 7f695cfcc640 1 --2- 172.21.15.6:0/247263989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556c1c8347b0 0x556c1c84b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.079+0000 7f695cfcc640 1 -- 172.21.15.6:0/247263989 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c1c834f60 con 0x556c1c835d20 2024-09-12T03:25:43.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.084+0000 7f695cfcc640 1 --2- 172.21.15.6:0/247263989 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c835d20 0x556c1c83b200 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f69540b7030 tx=0x7f69540b6fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b609b2039529f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:43.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.084+0000 7f6942ffd640 1 -- 172.21.15.6:0/247263989 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69540bb2d0 con 0x556c1c835d20 2024-09-12T03:25:43.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.085+0000 7f6942ffd640 1 -- 172.21.15.6:0/247263989 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69540babd0 con 0x556c1c835d20 2024-09-12T03:25:43.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.085+0000 7f6942ffd640 1 -- 172.21.15.6:0/247263989 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69540b7240 con 0x556c1c835d20 2024-09-12T03:25:43.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.086+0000 7f6961e0a740 1 -- 172.21.15.6:0/247263989 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c835d20 msgr2=0x556c1c83b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:43.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.086+0000 7f6961e0a740 1 --2- 172.21.15.6:0/247263989 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c835d20 0x556c1c83b200 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f69540b7030 tx=0x7f69540b6fa0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.090+0000 7f6961e0a740 1 -- 172.21.15.6:0/247263989 shutdown_connections 2024-09-12T03:25:43.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.090+0000 7f6961e0a740 1 --2- 172.21.15.6:0/247263989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c83f550 0x556c1c856fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.090+0000 7f6961e0a740 1 --2- 172.21.15.6:0/247263989 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c835d20 0x556c1c83b200 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.090+0000 7f6961e0a740 1 --2- 172.21.15.6:0/247263989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556c1c8347b0 0x556c1c84b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.090+0000 7f6961e0a740 1 -- 172.21.15.6:0/247263989 >> 172.21.15.6:0/247263989 conn(0x556c1c7fd310 msgr2=0x556c1c83aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:43.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.093+0000 7f6961e0a740 1 -- 172.21.15.6:0/247263989 shutdown_connections 2024-09-12T03:25:43.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.093+0000 7f6961e0a740 1 -- 172.21.15.6:0/247263989 wait complete. 2024-09-12T03:25:43.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.094+0000 7f6961e0a740 1 -- start start 2024-09-12T03:25:43.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.094+0000 7f6961e0a740 1 Processor -- start 2024-09-12T03:25:43.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.095+0000 7f6961e0a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556c1c8347b0 0x556c1c847aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:43.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.095+0000 7f6961e0a740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c835d20 0x556c1c838c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:43.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.095+0000 7f6961e0a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c83f550 0x556c1c839150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:43.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.095+0000 7f695d7cd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556c1c8347b0 0x556c1c847aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:43.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.095+0000 7f6961e0a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f69580812f0 con 0x556c1c8347b0 2024-09-12T03:25:43.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.095+0000 7f695d7cd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556c1c8347b0 0x556c1c847aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55084/0 (socket says 172.21.15.6:55084) 2024-09-12T03:25:43.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.095+0000 7f695d7cd640 1 -- 172.21.15.6:0/2786462131 learned_addr learned my addr 172.21.15.6:0/2786462131 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:43.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.095+0000 7f695cfcc640 1 --2- 172.21.15.6:0/2786462131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c835d20 0x556c1c838c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:43.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.096+0000 7f695dfce640 1 --2- 172.21.15.6:0/2786462131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c83f550 0x556c1c839150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:43.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.096+0000 7f6961e0a740 1 -- 172.21.15.6:0/2786462131 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f6958082760 con 0x556c1c835d20 2024-09-12T03:25:43.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.096+0000 7f6961e0a740 1 -- 172.21.15.6:0/2786462131 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f69580819c0 con 0x556c1c83f550 2024-09-12T03:25:43.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.096+0000 7f6961e0a740 10 client.0 _ll_register_callbacks cb 0x7f69580688b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:43.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.096+0000 7f695dfce640 1 -- 172.21.15.6:0/2786462131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c835d20 msgr2=0x556c1c838c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:43.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.096+0000 7f695dfce640 1 --2- 172.21.15.6:0/2786462131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c835d20 0x556c1c838c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.096+0000 7f695dfce640 1 -- 172.21.15.6:0/2786462131 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556c1c8347b0 msgr2=0x556c1c847aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:43.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.096+0000 7f695dfce640 1 --2- 172.21.15.6:0/2786462131 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556c1c8347b0 0x556c1c847aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.096+0000 7f695dfce640 1 -- 172.21.15.6:0/2786462131 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c1c839690 con 0x556c1c83f550 2024-09-12T03:25:43.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.101+0000 7f695dfce640 1 --2- 172.21.15.6:0/2786462131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c83f550 0x556c1c839150 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6950000f80 tx=0x7f6950025f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36617bac1a777ecc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:43.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.101+0000 7f6940ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:25:43.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.102+0000 7f6940ff9640 1 -- 172.21.15.6:0/2786462131 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950009b20 con 0x556c1c83f550 2024-09-12T03:25:43.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.102+0000 7f6940ff9640 1 -- 172.21.15.6:0/2786462131 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f6950002030 con 0x556c1c83f550 2024-09-12T03:25:43.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.102+0000 7f6961e0a740 1 -- 172.21.15.6:0/2786462131 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69540babd0 con 0x556c1c83f550 2024-09-12T03:25:43.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.106+0000 7f6940ff9640 1 -- 172.21.15.6:0/2786462131 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6950004c10 con 0x556c1c83f550 2024-09-12T03:25:43.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.106+0000 7f6961e0a740 10 client.25591 fetch_fsmap learned FSMap version 27 2024-09-12T03:25:43.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.106+0000 7f6961e0a740 1 -- 172.21.15.6:0/2786462131 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556c1c8b9c10 con 0x556c1c83f550 2024-09-12T03:25:43.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.110+0000 7f6940ff9640 1 -- 172.21.15.6:0/2786462131 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6950029020 con 0x556c1c83f550 2024-09-12T03:25:43.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.110+0000 7f6940ff9640 1 -- 172.21.15.6:0/2786462131 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f695000b040 con 0x556c1c83f550 2024-09-12T03:25:43.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.111+0000 7f6961e0a740 10 client.25591 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:25:43.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.111+0000 7f6961e0a740 10 client.25591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:25:43.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.111+0000 7f6961e0a740 20 client.25591 populate_metadata read hostname 'smithi006' 2024-09-12T03:25:43.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.111+0000 7f6961e0a740 1 --2- 172.21.15.6:0/2786462131 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x556c1c8ba280 0x556c1c8bc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:43.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.111+0000 7f6961e0a740 4 client.25591 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:25:43.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.111+0000 7f6961e0a740 1 -- 172.21.15.6:0/2786462131 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f69540babd0 con 0x556c1c8ba280 2024-09-12T03:25:43.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.112+0000 7f695d7cd640 1 --2- 172.21.15.6:0/2786462131 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x556c1c8ba280 0x556c1c8bc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:43.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.112+0000 7f695d7cd640 1 --2- 172.21.15.6:0/2786462131 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x556c1c8ba280 0x556c1c8bc6a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=419cc9f7b2ad51f1 server_cookie=33326ca41a2c0b38 in_seq=0 out_seq=0 2024-09-12T03:25:43.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.113+0000 7f6940ff9640 10 client.25591 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:43.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.113+0000 7f6940ff9640 1 -- 172.21.15.6:0/2786462131 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f69540babd0 con 0x556c1c8ba280 2024-09-12T03:25:43.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.113+0000 7f6940ff9640 10 client.25591 handle_command_reply: tid=0 2024-09-12T03:25:43.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.113+0000 7f6961e0a740 1 client.25591 shutdown 2024-09-12T03:25:43.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.115+0000 7f6961e0a740 1 -- 172.21.15.6:0/2786462131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c83f550 msgr2=0x556c1c839150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:43.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.115+0000 7f6961e0a740 1 --2- 172.21.15.6:0/2786462131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c83f550 0x556c1c839150 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6950000f80 tx=0x7f6950025f40 comp rx=0 tx=0).stop 2024-09-12T03:25:43.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.117+0000 7f6961e0a740 1 -- 172.21.15.6:0/2786462131 shutdown_connections 2024-09-12T03:25:43.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.117+0000 7f6961e0a740 1 --2- 172.21.15.6:0/2786462131 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x556c1c8ba280 0x556c1c8bc6a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.117+0000 7f6940ff9640 0 client.25591 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:43.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.117+0000 7f6961e0a740 1 --2- 172.21.15.6:0/2786462131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c1c83f550 0x556c1c839150 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.117+0000 7f6961e0a740 1 --2- 172.21.15.6:0/2786462131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c1c835d20 0x556c1c838c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.117+0000 7f6961e0a740 1 --2- 172.21.15.6:0/2786462131 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556c1c8347b0 0x556c1c847aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.117+0000 7f6961e0a740 1 -- 172.21.15.6:0/2786462131 >> 172.21.15.6:0/2786462131 conn(0x556c1c7fd310 msgr2=0x556c1c83b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:43.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.117+0000 7f6961e0a740 1 -- 172.21.15.6:0/2786462131 shutdown_connections 2024-09-12T03:25:43.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.117+0000 7f6961e0a740 1 -- 172.21.15.6:0/2786462131 wait complete. 2024-09-12T03:25:43.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.118+0000 7f6961e0a740 20 client.25591 trim_cache size 0 max 16384 2024-09-12T03:25:43.120 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-09-12T03:25:43.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.121+0000 7f695e7cf640 1 -- 172.21.15.6:0/416782180 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f69440a2cc0 msgr2=0x7f69440a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:43.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.121+0000 7f695e7cf640 1 --2- 172.21.15.6:0/416782180 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f69440a2cc0 0x7f69440a5180 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f694800bbe0 tx=0x7f694800b390 comp rx=0 tx=0).stop 2024-09-12T03:25:43.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.121+0000 7f695e7cf640 1 -- 172.21.15.6:0/416782180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f695811b140 msgr2=0x7f69581a7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:43.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.121+0000 7f695e7cf640 1 --2- 172.21.15.6:0/416782180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f695811b140 0x7f69581a7840 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6954008000 tx=0x7f6954033d10 comp rx=0 tx=0).stop 2024-09-12T03:25:43.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.121+0000 7f695e7cf640 1 -- 172.21.15.6:0/416782180 shutdown_connections 2024-09-12T03:25:43.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.121+0000 7f695e7cf640 1 --2- 172.21.15.6:0/416782180 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f69440a2cc0 0x7f69440a5180 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.121+0000 7f695e7cf640 1 --2- 172.21.15.6:0/416782180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69581abbd0 0x7f69581ac080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.121+0000 7f695e7cf640 1 --2- 172.21.15.6:0/416782180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f695811b140 0x7f69581a7840 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.121+0000 7f695e7cf640 1 --2- 172.21.15.6:0/416782180 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6958110190 0x7f69581a7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:43.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.121+0000 7f695e7cf640 1 -- 172.21.15.6:0/416782180 >> 172.21.15.6:0/416782180 conn(0x7f6958073180 msgr2=0x7f69581193b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:43.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.122+0000 7f695e7cf640 1 -- 172.21.15.6:0/416782180 shutdown_connections 2024-09-12T03:25:43.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:43.122+0000 7f695e7cf640 1 -- 172.21.15.6:0/416782180 wait complete. 2024-09-12T03:25:44.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:44 smithi118 ceph-mon[35048]: pgmap v474: 97 pgs: 97 active+clean; 13 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 112 KiB/s rd, 64 MiB/s wr, 6.21k op/s 2024-09-12T03:25:44.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:44 smithi161 ceph-mon[35485]: pgmap v474: 97 pgs: 97 active+clean; 13 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 112 KiB/s rd, 64 MiB/s wr, 6.21k op/s 2024-09-12T03:25:44.865 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:44 smithi006 ceph-mon[30991]: pgmap v474: 97 pgs: 97 active+clean; 13 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 112 KiB/s rd, 64 MiB/s wr, 6.21k op/s 2024-09-12T03:25:45.198 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:25:45.198 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:25:45.855 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:46.767 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:46 smithi006 ceph-mon[30991]: pgmap v475: 97 pgs: 97 active+clean; 13 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 320 KiB/s rd, 79 MiB/s wr, 7.55k op/s 2024-09-12T03:25:46.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:46 smithi161 ceph-mon[35485]: pgmap v475: 97 pgs: 97 active+clean; 13 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 320 KiB/s rd, 79 MiB/s wr, 7.55k op/s 2024-09-12T03:25:46.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.933+0000 7f8b13a0b640 1 -- 172.21.15.6:0/2167746658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b0c102ef0 msgr2=0x7f8b0c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:46.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.934+0000 7f8b13a0b640 1 --2- 172.21.15.6:0/2167746658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b0c102ef0 0x7f8b0c10da40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8afc01a940 tx=0x7f8afc03b520 comp rx=0 tx=0).stop 2024-09-12T03:25:46.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.934+0000 7f8b13a0b640 1 -- 172.21.15.6:0/2167746658 shutdown_connections 2024-09-12T03:25:46.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.934+0000 7f8b13a0b640 1 --2- 172.21.15.6:0/2167746658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b0c102ef0 0x7f8b0c10da40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.934+0000 7f8b13a0b640 1 --2- 172.21.15.6:0/2167746658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b0c102570 0x7f8b0c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.934+0000 7f8b13a0b640 1 --2- 172.21.15.6:0/2167746658 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b0c100e20 0x7f8b0c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.934+0000 7f8b13a0b640 1 -- 172.21.15.6:0/2167746658 >> 172.21.15.6:0/2167746658 conn(0x7f8b0c0fc7d0 msgr2=0x7f8b0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:46.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.935+0000 7f8b13a0b640 1 -- 172.21.15.6:0/2167746658 shutdown_connections 2024-09-12T03:25:46.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.935+0000 7f8b13a0b640 1 -- 172.21.15.6:0/2167746658 wait complete. 2024-09-12T03:25:46.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.936+0000 7f8b13a0b640 1 Processor -- start 2024-09-12T03:25:46.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.936+0000 7f8b13a0b640 1 -- start start 2024-09-12T03:25:46.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.936+0000 7f8b13a0b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b0c100e20 0x7f8b0c073c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:46.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.936+0000 7f8b13a0b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b0c102570 0x7f8b0c06dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:46.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.936+0000 7f8b13a0b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b0c102ef0 0x7f8b0c06e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:46.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.936+0000 7f8b13a0b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c06c7a0 con 0x7f8b0c102ef0 2024-09-12T03:25:46.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.936+0000 7f8b13a0b640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c06c620 con 0x7f8b0c102570 2024-09-12T03:25:46.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.937+0000 7f8b13a0b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c06c920 con 0x7f8b0c100e20 2024-09-12T03:25:46.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.937+0000 7f8b10fa6640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b0c102570 0x7f8b0c06dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:46.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.937+0000 7f8b117a7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b0c100e20 0x7f8b0c073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:46.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.937+0000 7f8b117a7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b0c100e20 0x7f8b0c073c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:44236/0 (socket says 172.21.15.6:44236) 2024-09-12T03:25:46.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.937+0000 7f8b117a7640 1 -- 172.21.15.6:0/138405378 learned_addr learned my addr 172.21.15.6:0/138405378 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:46.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.937+0000 7f8b11fa8640 1 --2- 172.21.15.6:0/138405378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b0c102ef0 0x7f8b0c06e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:46.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.937+0000 7f8b117a7640 1 -- 172.21.15.6:0/138405378 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b0c102570 msgr2=0x7f8b0c06dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:46.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.938+0000 7f8b117a7640 1 --2- 172.21.15.6:0/138405378 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b0c102570 0x7f8b0c06dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.938+0000 7f8b117a7640 1 -- 172.21.15.6:0/138405378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b0c102ef0 msgr2=0x7f8b0c06e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:46.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.938+0000 7f8b117a7640 1 --2- 172.21.15.6:0/138405378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b0c102ef0 0x7f8b0c06e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.938+0000 7f8b117a7640 1 -- 172.21.15.6:0/138405378 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0c06eac0 con 0x7f8b0c100e20 2024-09-12T03:25:46.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.938+0000 7f8b11fa8640 1 --2- 172.21.15.6:0/138405378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b0c102ef0 0x7f8b0c06e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:25:46.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.940+0000 7f8b10fa6640 1 --2- 172.21.15.6:0/138405378 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b0c102570 0x7f8b0c06dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:46.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.940+0000 7f8b117a7640 1 --2- 172.21.15.6:0/138405378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b0c100e20 0x7f8b0c073c40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8b0400ba50 tx=0x7f8b0400be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:46.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.941+0000 7f8b027fc640 1 -- 172.21.15.6:0/138405378 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b04019070 con 0x7f8b0c100e20 2024-09-12T03:25:46.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.941+0000 7f8b13a0b640 1 -- 172.21.15.6:0/138405378 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b0c115480 con 0x7f8b0c100e20 2024-09-12T03:25:46.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.941+0000 7f8b027fc640 1 -- 172.21.15.6:0/138405378 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b0400cb50 con 0x7f8b0c100e20 2024-09-12T03:25:46.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.941+0000 7f8b13a0b640 1 -- 172.21.15.6:0/138405378 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b0c115a10 con 0x7f8b0c100e20 2024-09-12T03:25:46.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.942+0000 7f8b027fc640 1 -- 172.21.15.6:0/138405378 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b040075d0 con 0x7f8b0c100e20 2024-09-12T03:25:46.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.946+0000 7f8b15de4740 1 Processor -- start 2024-09-12T03:25:46.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.946+0000 7f8b15de4740 1 -- start start 2024-09-12T03:25:46.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.946+0000 7f8b15de4740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a470e4b60 0x560a470e4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:46.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.946+0000 7f8b15de4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470e5480 0x560a470ef800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:46.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.948+0000 7f8b15de4740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560a470efd40 0x560a470f2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:46.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.948+0000 7f8b15de4740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x560a4704a280 con 0x560a470e4b60 2024-09-12T03:25:46.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.948+0000 7f8b15de4740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x560a46ff1b10 con 0x560a470efd40 2024-09-12T03:25:46.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.948+0000 7f8b15de4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560a47046240 con 0x560a470e5480 2024-09-12T03:25:46.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.949+0000 7f8b10fa6640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560a470efd40 0x560a470f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:46.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.949+0000 7f8b10fa6640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560a470efd40 0x560a470f2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:58038/0 (socket says 172.21.15.6:58038) 2024-09-12T03:25:46.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.949+0000 7f8b10fa6640 1 -- 172.21.15.6:0/750272102 learned_addr learned my addr 172.21.15.6:0/750272102 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:46.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.949+0000 7f8b027fc640 1 -- 172.21.15.6:0/138405378 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8b0401c070 con 0x7f8b0c100e20 2024-09-12T03:25:46.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.950+0000 7f8b11fa8640 1 --2- 172.21.15.6:0/750272102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a470e4b60 0x560a470e4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:46.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.950+0000 7f8b117a7640 1 --2- 172.21.15.6:0/750272102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470e5480 0x560a470ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:46.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.951+0000 7f8b10fa6640 1 -- 172.21.15.6:0/750272102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470e5480 msgr2=0x560a470ef800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:46.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.951+0000 7f8b10fa6640 1 --2- 172.21.15.6:0/750272102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470e5480 0x560a470ef800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.951+0000 7f8b10fa6640 1 -- 172.21.15.6:0/750272102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a470e4b60 msgr2=0x560a470e4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:46.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.951+0000 7f8b10fa6640 1 --2- 172.21.15.6:0/750272102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a470e4b60 0x560a470e4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.951+0000 7f8b10fa6640 1 -- 172.21.15.6:0/750272102 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a46ffe1f0 con 0x560a470efd40 2024-09-12T03:25:46.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.951+0000 7f8b10fa6640 1 --2- 172.21.15.6:0/750272102 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560a470efd40 0x560a470f2130 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8b08009de0 tx=0x7f8b0800e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b90ee5421da217bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:46.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.953+0000 7f8b027fc640 1 --2- 172.21.15.6:0/138405378 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8af80a2cd0 0x7f8af80a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:46.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.953+0000 7f8af77fe640 1 -- 172.21.15.6:0/750272102 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b08017070 con 0x560a470efd40 2024-09-12T03:25:46.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.953+0000 7f8af77fe640 1 -- 172.21.15.6:0/750272102 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b08013950 con 0x560a470efd40 2024-09-12T03:25:46.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.953+0000 7f8af77fe640 1 -- 172.21.15.6:0/750272102 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b080129e0 con 0x560a470efd40 2024-09-12T03:25:46.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.953+0000 7f8b027fc640 1 -- 172.21.15.6:0/138405378 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f8b040ce6a0 con 0x7f8b0c100e20 2024-09-12T03:25:46.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.954+0000 7f8b11fa8640 1 --2- 172.21.15.6:0/138405378 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8af80a2cd0 0x7f8af80a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:46.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.954+0000 7f8b15de4740 1 -- 172.21.15.6:0/750272102 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560a470efd40 msgr2=0x560a470f2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:46.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.954+0000 7f8b15de4740 1 --2- 172.21.15.6:0/750272102 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560a470efd40 0x560a470f2130 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8b08009de0 tx=0x7f8b0800e5a0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.955+0000 7f8b15de4740 1 -- 172.21.15.6:0/750272102 shutdown_connections 2024-09-12T03:25:46.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.955+0000 7f8b15de4740 1 --2- 172.21.15.6:0/750272102 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560a470efd40 0x560a470f2130 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.955+0000 7f8b15de4740 1 --2- 172.21.15.6:0/750272102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470e5480 0x560a470ef800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.955+0000 7f8b15de4740 1 --2- 172.21.15.6:0/750272102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a470e4b60 0x560a470e4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.955+0000 7f8b15de4740 1 -- 172.21.15.6:0/750272102 >> 172.21.15.6:0/750272102 conn(0x560a470e10f0 msgr2=0x560a470e14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:46.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.957+0000 7f8b11fa8640 1 --2- 172.21.15.6:0/138405378 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8af80a2cd0 0x7f8af80a5190 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8afc03ba90 tx=0x7f8afc03ba20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:46.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.960+0000 7f8b15de4740 1 -- 172.21.15.6:0/750272102 shutdown_connections 2024-09-12T03:25:46.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:46 smithi118 ceph-mon[35048]: pgmap v475: 97 pgs: 97 active+clean; 13 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 320 KiB/s rd, 79 MiB/s wr, 7.55k op/s 2024-09-12T03:25:46.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.963+0000 7f8b15de4740 1 -- 172.21.15.6:0/750272102 wait complete. 2024-09-12T03:25:46.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.963+0000 7f8b15de4740 1 -- start start 2024-09-12T03:25:46.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.964+0000 7f8b15de4740 1 Processor -- start 2024-09-12T03:25:46.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.965+0000 7f8b15de4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470e4b60 0x560a470ecb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:46.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.965+0000 7f8b15de4740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a470e5480 0x560a470ed0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:46.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.965+0000 7f8b15de4740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560a470efd40 0x560a470ed5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:46.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.965+0000 7f8b15de4740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x560a470f2910 con 0x560a470e5480 2024-09-12T03:25:46.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.965+0000 7f8b15de4740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x560a47004f10 con 0x560a470efd40 2024-09-12T03:25:46.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.965+0000 7f8b15de4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560a470f2a90 con 0x560a470e4b60 2024-09-12T03:25:46.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.965+0000 7f8b15de4740 10 client.0 _ll_register_callbacks cb 0x560a46fcda40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:46.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.965+0000 7f8b10fa6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a470e5480 0x560a470ed0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:46.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.966+0000 7f8b10fa6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a470e5480 0x560a470ed0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55104/0 (socket says 172.21.15.6:55104) 2024-09-12T03:25:46.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.966+0000 7f8b10fa6640 1 -- 172.21.15.6:0/3313106954 learned_addr learned my addr 172.21.15.6:0/3313106954 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:46.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.966+0000 7f8b117a7640 1 --2- 172.21.15.6:0/3313106954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470e4b60 0x560a470ecb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:46.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.967+0000 7f8b117a7640 1 -- 172.21.15.6:0/3313106954 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560a470efd40 msgr2=0x560a470ed5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:46.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.967+0000 7f8b117a7640 1 --2- 172.21.15.6:0/3313106954 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560a470efd40 0x560a470ed5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.967+0000 7f8b117a7640 1 -- 172.21.15.6:0/3313106954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a470e5480 msgr2=0x560a470ed0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:46.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.967+0000 7f8b117a7640 1 --2- 172.21.15.6:0/3313106954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a470e5480 0x560a470ed0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.967+0000 7f8b117a7640 1 -- 172.21.15.6:0/3313106954 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a46dbb6a0 con 0x560a470e4b60 2024-09-12T03:25:46.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.968+0000 7f8b117a7640 1 --2- 172.21.15.6:0/3313106954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470e4b60 0x560a470ecb70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x560a470e1f50 tx=0x7f8b0409ad60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b45a6d8e68392f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:46.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.968+0000 7f8af4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:25:46.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.970+0000 7f8af4ff9640 1 -- 172.21.15.6:0/3313106954 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0409b5d0 con 0x560a470e4b60 2024-09-12T03:25:46.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.970+0000 7f8af4ff9640 1 -- 172.21.15.6:0/3313106954 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f8b04097540 con 0x560a470e4b60 2024-09-12T03:25:46.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.971+0000 7f8b15de4740 1 -- 172.21.15.6:0/3313106954 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b08013950 con 0x560a470e4b60 2024-09-12T03:25:46.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.975+0000 7f8af4ff9640 1 -- 172.21.15.6:0/3313106954 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8b0409aed0 con 0x560a470e4b60 2024-09-12T03:25:46.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.975+0000 7f8b15de4740 10 client.25603 fetch_fsmap learned FSMap version 27 2024-09-12T03:25:46.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.975+0000 7f8b15de4740 1 -- 172.21.15.6:0/3313106954 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560a470fe4f0 con 0x560a470e4b60 2024-09-12T03:25:46.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.976+0000 7f8af4ff9640 1 -- 172.21.15.6:0/3313106954 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b08013950 con 0x560a470e4b60 2024-09-12T03:25:46.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.976+0000 7f8af4ff9640 1 -- 172.21.15.6:0/3313106954 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8b0409a320 con 0x560a470e4b60 2024-09-12T03:25:46.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.976+0000 7f8b15de4740 10 client.25603 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:25:46.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.976+0000 7f8b15de4740 10 client.25603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:25:46.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.976+0000 7f8b15de4740 20 client.25603 populate_metadata read hostname 'smithi006' 2024-09-12T03:25:46.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.976+0000 7f8b15de4740 1 --2- 172.21.15.6:0/3313106954 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x560a470ffef0 0x560a47102380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:46.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.976+0000 7f8b15de4740 4 client.25603 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:25:46.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.976+0000 7f8b15de4740 1 -- 172.21.15.6:0/3313106954 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a470577f0 con 0x560a470ffef0 2024-09-12T03:25:46.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.977+0000 7f8b10fa6640 1 --2- 172.21.15.6:0/3313106954 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x560a470ffef0 0x560a47102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:46.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.977+0000 7f8b10fa6640 1 --2- 172.21.15.6:0/3313106954 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x560a470ffef0 0x560a47102380 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=810d53389f7247c0 server_cookie=375f3c913d86b2a8 in_seq=0 out_seq=0 2024-09-12T03:25:46.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.978+0000 7f8af4ff9640 10 client.25603 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:46.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.987+0000 7f8af4ff9640 1 -- 172.21.15.6:0/3313106954 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a470577f0 con 0x560a470ffef0 2024-09-12T03:25:46.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.987+0000 7f8af4ff9640 10 client.25603 handle_command_reply: tid=0 2024-09-12T03:25:46.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.987+0000 7f8b15de4740 1 client.25603 shutdown 2024-09-12T03:25:46.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.987+0000 7f8b15de4740 1 -- 172.21.15.6:0/3313106954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470e4b60 msgr2=0x560a470ecb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:46.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.987+0000 7f8b15de4740 1 --2- 172.21.15.6:0/3313106954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470e4b60 0x560a470ecb70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x560a470e1f50 tx=0x7f8b0409ad60 comp rx=0 tx=0).stop 2024-09-12T03:25:46.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.996+0000 7f8b15de4740 1 -- 172.21.15.6:0/3313106954 shutdown_connections 2024-09-12T03:25:46.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.996+0000 7f8b15de4740 1 --2- 172.21.15.6:0/3313106954 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x560a470ffef0 0x560a47102380 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.996+0000 7f8b15de4740 1 --2- 172.21.15.6:0/3313106954 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560a470efd40 0x560a470ed5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.996+0000 7f8b15de4740 1 --2- 172.21.15.6:0/3313106954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a470e5480 0x560a470ed0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.996+0000 7f8b15de4740 1 --2- 172.21.15.6:0/3313106954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470e4b60 0x560a470ecb70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.996+0000 7f8af4ff9640 0 client.25603 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:46.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:46.996+0000 7f8b15de4740 1 -- 172.21.15.6:0/3313106954 >> 172.21.15.6:0/3313106954 conn(0x560a470e10f0 msgr2=0x560a470e6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:47.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.001+0000 7f8b15de4740 1 -- 172.21.15.6:0/3313106954 shutdown_connections 2024-09-12T03:25:47.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.001+0000 7f8b15de4740 1 -- 172.21.15.6:0/3313106954 wait complete. 2024-09-12T03:25:47.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.001+0000 7f8b15de4740 20 client.25603 trim_cache size 0 max 16384 2024-09-12T03:25:47.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.038+0000 7f8b15de4740 1 Processor -- start 2024-09-12T03:25:47.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.038+0000 7f8b15de4740 1 -- start start 2024-09-12T03:25:47.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.038+0000 7f8b15de4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470e4b60 0x560a470e0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.038+0000 7f8b15de4740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a470e5480 0x560a47107c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.038+0000 7f8b15de4740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560a470efd40 0x560a471081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.038+0000 7f8b15de4740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8af0010cb0 con 0x560a470e5480 2024-09-12T03:25:47.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.038+0000 7f8b15de4740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f8af000e580 con 0x560a470efd40 2024-09-12T03:25:47.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.038+0000 7f8b15de4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8af000c3d0 con 0x560a470e4b60 2024-09-12T03:25:47.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.039+0000 7f8b117a7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470e4b60 0x560a470e0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.039+0000 7f8b117a7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470e4b60 0x560a470e0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:44260/0 (socket says 172.21.15.6:44260) 2024-09-12T03:25:47.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.039+0000 7f8b117a7640 1 -- 172.21.15.6:0/283718219 learned_addr learned my addr 172.21.15.6:0/283718219 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:47.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.039+0000 7f8b11fa8640 1 --2- 172.21.15.6:0/283718219 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560a470efd40 0x560a471081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.039+0000 7f8b117a7640 1 -- 172.21.15.6:0/283718219 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560a470efd40 msgr2=0x560a471081d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:47.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.039+0000 7f8b117a7640 1 --2- 172.21.15.6:0/283718219 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560a470efd40 0x560a471081d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.039+0000 7f8b117a7640 1 -- 172.21.15.6:0/283718219 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a470e5480 msgr2=0x560a47107c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:47.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.039+0000 7f8b117a7640 1 --2- 172.21.15.6:0/283718219 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a470e5480 0x560a47107c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.039+0000 7f8b117a7640 1 -- 172.21.15.6:0/283718219 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a470e0f20 con 0x560a470e4b60 2024-09-12T03:25:47.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.040+0000 7f8b117a7640 1 --2- 172.21.15.6:0/283718219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470e4b60 0x560a470e0160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8b0409abd0 tx=0x7f8b0409a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b2fa8e37f33fc45 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:47.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.041+0000 7f8af6ffd640 1 -- 172.21.15.6:0/283718219 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b040a3870 con 0x560a470e4b60 2024-09-12T03:25:47.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.041+0000 7f8af6ffd640 1 -- 172.21.15.6:0/283718219 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b0409f280 con 0x560a470e4b60 2024-09-12T03:25:47.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.041+0000 7f8af6ffd640 1 -- 172.21.15.6:0/283718219 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b04009040 con 0x560a470e4b60 2024-09-12T03:25:47.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.042+0000 7f8b15de4740 1 -- 172.21.15.6:0/283718219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470e4b60 msgr2=0x560a470e0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:47.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.043+0000 7f8b15de4740 1 --2- 172.21.15.6:0/283718219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470e4b60 0x560a470e0160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8b0409abd0 tx=0x7f8b0409a780 comp rx=0 tx=0).stop 2024-09-12T03:25:47.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.048+0000 7f8b15de4740 1 -- 172.21.15.6:0/283718219 shutdown_connections 2024-09-12T03:25:47.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.048+0000 7f8b15de4740 1 --2- 172.21.15.6:0/283718219 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560a470efd40 0x560a471081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.048+0000 7f8b15de4740 1 --2- 172.21.15.6:0/283718219 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a470e5480 0x560a47107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.048+0000 7f8b15de4740 1 --2- 172.21.15.6:0/283718219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470e4b60 0x560a470e0160 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.048+0000 7f8b15de4740 1 -- 172.21.15.6:0/283718219 >> 172.21.15.6:0/283718219 conn(0x560a470e10f0 msgr2=0x560a470dfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:47.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.050+0000 7f8b15de4740 1 -- 172.21.15.6:0/283718219 shutdown_connections 2024-09-12T03:25:47.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.050+0000 7f8b15de4740 1 -- 172.21.15.6:0/283718219 wait complete. 2024-09-12T03:25:47.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.051+0000 7f8b15de4740 1 -- start start 2024-09-12T03:25:47.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.051+0000 7f8b15de4740 1 Processor -- start 2024-09-12T03:25:47.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.053+0000 7f8b15de4740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560a470e4b60 0x560a470f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.053+0000 7f8b15de4740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a470e5480 0x560a470e95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.053+0000 7f8b15de4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470efd40 0x560a470e9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.054+0000 7f8b15de4740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8af000ca40 con 0x560a470e5480 2024-09-12T03:25:47.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.054+0000 7f8b15de4740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f8af000deb0 con 0x560a470e4b60 2024-09-12T03:25:47.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.054+0000 7f8b15de4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8af000d110 con 0x560a470efd40 2024-09-12T03:25:47.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.054+0000 7f8b15de4740 10 client.0 _ll_register_callbacks cb 0x7f8b040099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:47.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.056+0000 7f8b11fa8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470efd40 0x560a470e9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.056+0000 7f8b11fa8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470efd40 0x560a470e9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:44272/0 (socket says 172.21.15.6:44272) 2024-09-12T03:25:47.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.056+0000 7f8b11fa8640 1 -- 172.21.15.6:0/3129370750 learned_addr learned my addr 172.21.15.6:0/3129370750 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:47.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.056+0000 7f8b117a7640 1 --2- 172.21.15.6:0/3129370750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560a470e4b60 0x560a470f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.056+0000 7f8b10fa6640 1 --2- 172.21.15.6:0/3129370750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a470e5480 0x560a470e95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.056+0000 7f8b11fa8640 1 -- 172.21.15.6:0/3129370750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560a470e4b60 msgr2=0x560a470f81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:47.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.056+0000 7f8b11fa8640 1 --2- 172.21.15.6:0/3129370750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560a470e4b60 0x560a470f81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.056+0000 7f8b11fa8640 1 -- 172.21.15.6:0/3129370750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a470e5480 msgr2=0x560a470e95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:47.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.056+0000 7f8b11fa8640 1 --2- 172.21.15.6:0/3129370750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a470e5480 0x560a470e95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.056+0000 7f8b11fa8640 1 -- 172.21.15.6:0/3129370750 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a470ea040 con 0x560a470efd40 2024-09-12T03:25:47.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.057+0000 7f8b10fa6640 1 --2- 172.21.15.6:0/3129370750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a470e5480 0x560a470e95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:25:47.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.061+0000 7f8b11fa8640 1 --2- 172.21.15.6:0/3129370750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470efd40 0x560a470e9b00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8afc005c40 tx=0x7f8afc03b520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3794e3e341a80693 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:47.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.061+0000 7f8af4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:25:47.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.065+0000 7f8af4ff9640 1 -- 172.21.15.6:0/3129370750 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc046040 con 0x560a470efd40 2024-09-12T03:25:47.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.066+0000 7f8af4ff9640 1 -- 172.21.15.6:0/3129370750 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f8afc005130 con 0x560a470efd40 2024-09-12T03:25:47.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.068+0000 7f8b15de4740 1 -- 172.21.15.6:0/3129370750 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b0409f280 con 0x560a470efd40 2024-09-12T03:25:47.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.074+0000 7f8af4ff9640 1 -- 172.21.15.6:0/3129370750 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8afc0044b0 con 0x560a470efd40 2024-09-12T03:25:47.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.074+0000 7f8b15de4740 10 client.25609 fetch_fsmap learned FSMap version 27 2024-09-12T03:25:47.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.074+0000 7f8b15de4740 1 -- 172.21.15.6:0/3129370750 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560a4716a970 con 0x560a470efd40 2024-09-12T03:25:47.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.077+0000 7f8af4ff9640 1 -- 172.21.15.6:0/3129370750 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b0409f280 con 0x560a470efd40 2024-09-12T03:25:47.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.077+0000 7f8af4ff9640 1 -- 172.21.15.6:0/3129370750 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8afc062470 con 0x560a470efd40 2024-09-12T03:25:47.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.077+0000 7f8b15de4740 10 client.25609 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:25:47.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.077+0000 7f8b15de4740 10 client.25609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:25:47.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.077+0000 7f8b15de4740 20 client.25609 populate_metadata read hostname 'smithi006' 2024-09-12T03:25:47.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.078+0000 7f8b15de4740 1 --2- 172.21.15.6:0/3129370750 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x560a4716b060 0x560a4716d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.078+0000 7f8b15de4740 4 client.25609 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:25:47.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.078+0000 7f8b15de4740 1 -- 172.21.15.6:0/3129370750 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560a47051d00 con 0x560a4716b060 2024-09-12T03:25:47.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.079+0000 7f8b117a7640 1 --2- 172.21.15.6:0/3129370750 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x560a4716b060 0x560a4716d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.091+0000 7f8b117a7640 1 --2- 172.21.15.6:0/3129370750 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x560a4716b060 0x560a4716d480 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=135f1fdac6cc9bd4 server_cookie=12fd288a88ececaa in_seq=0 out_seq=0 2024-09-12T03:25:47.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.091+0000 7f8af4ff9640 10 client.25609 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:47.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.455+0000 7f8af4ff9640 1 -- 172.21.15.6:0/3129370750 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560a47051d00 con 0x560a4716b060 2024-09-12T03:25:47.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.455+0000 7f8af4ff9640 10 client.25609 handle_command_reply: tid=0 2024-09-12T03:25:47.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.455+0000 7f8b15de4740 1 client.25609 shutdown 2024-09-12T03:25:47.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.455+0000 7f8b15de4740 1 -- 172.21.15.6:0/3129370750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470efd40 msgr2=0x560a470e9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:47.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.455+0000 7f8b15de4740 1 --2- 172.21.15.6:0/3129370750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470efd40 0x560a470e9b00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8afc005c40 tx=0x7f8afc03b520 comp rx=0 tx=0).stop 2024-09-12T03:25:47.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.456+0000 7f8b15de4740 1 -- 172.21.15.6:0/3129370750 shutdown_connections 2024-09-12T03:25:47.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.456+0000 7f8b15de4740 1 --2- 172.21.15.6:0/3129370750 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x560a4716b060 0x560a4716d480 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.456+0000 7f8b15de4740 1 --2- 172.21.15.6:0/3129370750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a470efd40 0x560a470e9b00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.456+0000 7f8b15de4740 1 --2- 172.21.15.6:0/3129370750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a470e5480 0x560a470e95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.456+0000 7f8b15de4740 1 --2- 172.21.15.6:0/3129370750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560a470e4b60 0x560a470f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.456+0000 7f8af4ff9640 0 client.25609 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:47.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.456+0000 7f8b15de4740 1 -- 172.21.15.6:0/3129370750 >> 172.21.15.6:0/3129370750 conn(0x560a470e10f0 msgr2=0x560a470e6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:47.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.457+0000 7f8b15de4740 1 -- 172.21.15.6:0/3129370750 shutdown_connections 2024-09-12T03:25:47.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.457+0000 7f8b15de4740 1 -- 172.21.15.6:0/3129370750 wait complete. 2024-09-12T03:25:47.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.457+0000 7f8b15de4740 20 client.25609 trim_cache size 0 max 16384 2024-09-12T03:25:47.458 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:25:47.458 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2024-09-12T03:25:47.458 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "01562442-7858-449c-a9dd-3b0d01987c49", 2024-09-12T03:25:47.458 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2024-09-12T03:25:47.458 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:25:47.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.458+0000 7f8b13a0b640 1 -- 172.21.15.6:0/138405378 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8af80a2cd0 msgr2=0x7f8af80a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:47.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.458+0000 7f8b13a0b640 1 --2- 172.21.15.6:0/138405378 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8af80a2cd0 0x7f8af80a5190 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8afc03ba90 tx=0x7f8afc03ba20 comp rx=0 tx=0).stop 2024-09-12T03:25:47.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.458+0000 7f8b13a0b640 1 -- 172.21.15.6:0/138405378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b0c100e20 msgr2=0x7f8b0c073c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:47.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.458+0000 7f8b13a0b640 1 --2- 172.21.15.6:0/138405378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b0c100e20 0x7f8b0c073c40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8b0400ba50 tx=0x7f8b0400be50 comp rx=0 tx=0).stop 2024-09-12T03:25:47.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.458+0000 7f8b13a0b640 1 -- 172.21.15.6:0/138405378 shutdown_connections 2024-09-12T03:25:47.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.458+0000 7f8b13a0b640 1 --2- 172.21.15.6:0/138405378 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8af80a2cd0 0x7f8af80a5190 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.458+0000 7f8b13a0b640 1 --2- 172.21.15.6:0/138405378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b0c102ef0 0x7f8b0c06e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.458+0000 7f8b13a0b640 1 --2- 172.21.15.6:0/138405378 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b0c102570 0x7f8b0c06dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.458+0000 7f8b13a0b640 1 --2- 172.21.15.6:0/138405378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b0c100e20 0x7f8b0c073c40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.458+0000 7f8b13a0b640 1 -- 172.21.15.6:0/138405378 >> 172.21.15.6:0/138405378 conn(0x7f8b0c0fc7d0 msgr2=0x7f8b0c1047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:47.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.458+0000 7f8b13a0b640 1 -- 172.21.15.6:0/138405378 shutdown_connections 2024-09-12T03:25:47.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:47.459+0000 7f8b13a0b640 1 -- 172.21.15.6:0/138405378 wait complete. 2024-09-12T03:25:48.637 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:25:48.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:48 smithi006 ceph-mon[30991]: pgmap v476: 97 pgs: 97 active+clean; 13 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 320 KiB/s rd, 129 MiB/s wr, 11.87k op/s 2024-09-12T03:25:48.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:48 smithi006 ceph-mon[30991]: scrub queued for path: / 2024-09-12T03:25:48.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:48 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:48.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:48 smithi006 ceph-mon[30991]: scrub summary: active paths [/] 2024-09-12T03:25:48.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:48 smithi161 ceph-mon[35485]: pgmap v476: 97 pgs: 97 active+clean; 13 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 320 KiB/s rd, 129 MiB/s wr, 11.87k op/s 2024-09-12T03:25:48.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:48 smithi161 ceph-mon[35485]: scrub queued for path: / 2024-09-12T03:25:48.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:48 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:48.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:48 smithi161 ceph-mon[35485]: scrub summary: active paths [/] 2024-09-12T03:25:48.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:48 smithi118 ceph-mon[35048]: pgmap v476: 97 pgs: 97 active+clean; 13 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 320 KiB/s rd, 129 MiB/s wr, 11.87k op/s 2024-09-12T03:25:48.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:48 smithi118 ceph-mon[35048]: scrub queued for path: / 2024-09-12T03:25:48.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:48 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:48.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:48 smithi118 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-12T03:25:49.226 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:50.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.020+0000 7f973bfff640 1 -- 172.21.15.6:0/1724883640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f973c10e090 msgr2=0x7f973c10e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:50.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.020+0000 7f973bfff640 1 --2- 172.21.15.6:0/1724883640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f973c10e090 0x7f973c10e470 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f972c009860 tx=0x7f972c02a380 comp rx=0 tx=0).stop 2024-09-12T03:25:50.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.020+0000 7f973bfff640 1 -- 172.21.15.6:0/1724883640 shutdown_connections 2024-09-12T03:25:50.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.020+0000 7f973bfff640 1 --2- 172.21.15.6:0/1724883640 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f973c06ee70 0x7f973c07a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.020+0000 7f973bfff640 1 --2- 172.21.15.6:0/1724883640 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f973c06e4f0 0x7f973c06e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.020+0000 7f973bfff640 1 --2- 172.21.15.6:0/1724883640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f973c10e090 0x7f973c10e470 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.020+0000 7f973bfff640 1 -- 172.21.15.6:0/1724883640 >> 172.21.15.6:0/1724883640 conn(0x7f973c073180 msgr2=0x7f973c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:50.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.020+0000 7f973bfff640 1 -- 172.21.15.6:0/1724883640 shutdown_connections 2024-09-12T03:25:50.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.020+0000 7f973bfff640 1 -- 172.21.15.6:0/1724883640 wait complete. 2024-09-12T03:25:50.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.021+0000 7f973bfff640 1 Processor -- start 2024-09-12T03:25:50.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.021+0000 7f973bfff640 1 -- start start 2024-09-12T03:25:50.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.021+0000 7f973bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f973c06e4f0 0x7f973c084e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.021+0000 7f973bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f973c06ee70 0x7f973c07ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.022+0000 7f973bfff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f973c10e090 0x7f973c07f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.022+0000 7f973bfff640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f973c07d830 con 0x7f973c06e4f0 2024-09-12T03:25:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.022+0000 7f973bfff640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f973c07d6b0 con 0x7f973c10e090 2024-09-12T03:25:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.022+0000 7f973bfff640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f973c07d9b0 con 0x7f973c06ee70 2024-09-12T03:25:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.022+0000 7f973a7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f973c06ee70 0x7f973c07ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.022+0000 7f973affd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f973c06e4f0 0x7f973c084e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.022+0000 7f973b7fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f973c10e090 0x7f973c07f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.022+0000 7f973affd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f973c06e4f0 0x7f973c084e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43532/0 (socket says 172.21.15.6:43532) 2024-09-12T03:25:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.022+0000 7f973affd640 1 -- 172.21.15.6:0/2613620469 learned_addr learned my addr 172.21.15.6:0/2613620469 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.022+0000 7f973b7fe640 1 -- 172.21.15.6:0/2613620469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f973c06ee70 msgr2=0x7f973c07ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:50.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.022+0000 7f973b7fe640 1 --2- 172.21.15.6:0/2613620469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f973c06ee70 0x7f973c07ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.022+0000 7f973b7fe640 1 -- 172.21.15.6:0/2613620469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f973c06e4f0 msgr2=0x7f973c084e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:50.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.022+0000 7f973b7fe640 1 --2- 172.21.15.6:0/2613620469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f973c06e4f0 0x7f973c084e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.022+0000 7f973b7fe640 1 -- 172.21.15.6:0/2613620469 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973c1c4390 con 0x7f973c10e090 2024-09-12T03:25:50.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.023+0000 7f973b7fe640 1 --2- 172.21.15.6:0/2613620469 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f973c10e090 0x7f973c07f580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f97300095a0 tx=0x7f9730009a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:50.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.023+0000 7f9740a5c640 1 -- 172.21.15.6:0/2613620469 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730018030 con 0x7f973c10e090 2024-09-12T03:25:50.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.023+0000 7f9740a5c640 1 -- 172.21.15.6:0/2613620469 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97300146e0 con 0x7f973c10e090 2024-09-12T03:25:50.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.023+0000 7f973bfff640 1 -- 172.21.15.6:0/2613620469 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f973c1c45c0 con 0x7f973c10e090 2024-09-12T03:25:50.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.023+0000 7f973bfff640 1 -- 172.21.15.6:0/2613620469 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f973c1c4ad0 con 0x7f973c10e090 2024-09-12T03:25:50.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.023+0000 7f9740a5c640 1 -- 172.21.15.6:0/2613620469 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730003920 con 0x7f973c10e090 2024-09-12T03:25:50.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.027+0000 7f9743e37740 1 Processor -- start 2024-09-12T03:25:50.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.033+0000 7f9740a5c640 1 -- 172.21.15.6:0/2613620469 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f973001b070 con 0x7f973c10e090 2024-09-12T03:25:50.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.033+0000 7f9743e37740 1 -- start start 2024-09-12T03:25:50.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.033+0000 7f9743e37740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa84d670 0x5605aa84da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.033+0000 7f9743e37740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5605aa84df90 0x5605aa858310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.033+0000 7f9743e37740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa858850 0x5605aa85ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.033+0000 7f9743e37740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5605aa7b32e0 con 0x5605aa84d670 2024-09-12T03:25:50.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.033+0000 7f9743e37740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5605aa75ace0 con 0x5605aa84df90 2024-09-12T03:25:50.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.033+0000 7f9743e37740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5605aa7af2a0 con 0x5605aa858850 2024-09-12T03:25:50.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.035+0000 7f973a7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa858850 0x5605aa85ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.035+0000 7f973a7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa858850 0x5605aa85ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:49252/0 (socket says 172.21.15.6:49252) 2024-09-12T03:25:50.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.035+0000 7f973a7fc640 1 -- 172.21.15.6:0/3974649635 learned_addr learned my addr 172.21.15.6:0/3974649635 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:50.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.035+0000 7f973a7fc640 1 --2- 172.21.15.6:0/3974649635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa84d670 0x5605aa84da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.035+0000 7f973a7fc640 1 -- 172.21.15.6:0/3974649635 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5605aa84df90 msgr2=0x5605aa858310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:50.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.035+0000 7f973a7fc640 1 --2- 172.21.15.6:0/3974649635 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5605aa84df90 0x5605aa858310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.035+0000 7f973a7fc640 1 -- 172.21.15.6:0/3974649635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa84d670 msgr2=0x5605aa84da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:50.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.035+0000 7f973a7fc640 1 --2- 172.21.15.6:0/3974649635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa84d670 0x5605aa84da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.035+0000 7f973a7fc640 1 -- 172.21.15.6:0/3974649635 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605aa73f440 con 0x5605aa858850 2024-09-12T03:25:50.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.037+0000 7f973a7fc640 1 --2- 172.21.15.6:0/3974649635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa858850 0x5605aa85ac40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9734009970 tx=0x7f9734009a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f9327e9cdf9eec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:50.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.037+0000 7f9721ffb640 1 -- 172.21.15.6:0/3974649635 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97340110d0 con 0x5605aa858850 2024-09-12T03:25:50.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.038+0000 7f9721ffb640 1 -- 172.21.15.6:0/3974649635 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9734011270 con 0x5605aa858850 2024-09-12T03:25:50.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.038+0000 7f9721ffb640 1 -- 172.21.15.6:0/3974649635 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734005280 con 0x5605aa858850 2024-09-12T03:25:50.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.041+0000 7f9743e37740 1 -- 172.21.15.6:0/3974649635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa858850 msgr2=0x5605aa85ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:50.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.041+0000 7f9743e37740 1 --2- 172.21.15.6:0/3974649635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa858850 0x5605aa85ac40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9734009970 tx=0x7f9734009a80 comp rx=0 tx=0).stop 2024-09-12T03:25:50.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.042+0000 7f9740a5c640 1 --2- 172.21.15.6:0/2613620469 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f97240a2c80 0x7f97240a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.042+0000 7f9743e37740 1 -- 172.21.15.6:0/3974649635 shutdown_connections 2024-09-12T03:25:50.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.042+0000 7f9743e37740 1 --2- 172.21.15.6:0/3974649635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa858850 0x5605aa85ac40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.042+0000 7f9743e37740 1 --2- 172.21.15.6:0/3974649635 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5605aa84df90 0x5605aa858310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.042+0000 7f9743e37740 1 --2- 172.21.15.6:0/3974649635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa84d670 0x5605aa84da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.042+0000 7f9743e37740 1 -- 172.21.15.6:0/3974649635 >> 172.21.15.6:0/3974649635 conn(0x5605aa849c00 msgr2=0x5605aa849fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:50.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.042+0000 7f973a7fc640 1 --2- 172.21.15.6:0/2613620469 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f97240a2c80 0x7f97240a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.043+0000 7f9740a5c640 1 -- 172.21.15.6:0/2613620469 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f97300ce250 con 0x7f973c10e090 2024-09-12T03:25:50.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.043+0000 7f9743e37740 1 -- 172.21.15.6:0/3974649635 shutdown_connections 2024-09-12T03:25:50.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.043+0000 7f9743e37740 1 -- 172.21.15.6:0/3974649635 wait complete. 2024-09-12T03:25:50.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.043+0000 7f9743e37740 1 -- start start 2024-09-12T03:25:50.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.044+0000 7f9743e37740 1 Processor -- start 2024-09-12T03:25:50.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.044+0000 7f9743e37740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5605aa84d670 0x5605aa858220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.044+0000 7f9743e37740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa84df90 0x5605aa8554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.044+0000 7f9743e37740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa858850 0x5605aa855ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.044+0000 7f9743e37740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5605aa85b420 con 0x5605aa858850 2024-09-12T03:25:50.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.044+0000 7f9743e37740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5605aa76e050 con 0x5605aa84d670 2024-09-12T03:25:50.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.044+0000 7f9743e37740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5605aa85b5a0 con 0x5605aa84df90 2024-09-12T03:25:50.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.044+0000 7f9743e37740 10 client.0 _ll_register_callbacks cb 0x5605aa7ae160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:50.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.044+0000 7f973a7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa84df90 0x5605aa8554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.044+0000 7f973b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa858850 0x5605aa855ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.044+0000 7f973a7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa84df90 0x5605aa8554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:49266/0 (socket says 172.21.15.6:49266) 2024-09-12T03:25:50.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.044+0000 7f973a7fc640 1 -- 172.21.15.6:0/1669258291 learned_addr learned my addr 172.21.15.6:0/1669258291 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:50.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.045+0000 7f973b7fe640 1 -- 172.21.15.6:0/1669258291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa84df90 msgr2=0x5605aa8554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:50.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.045+0000 7f973b7fe640 1 --2- 172.21.15.6:0/1669258291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa84df90 0x5605aa8554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.045+0000 7f973b7fe640 1 -- 172.21.15.6:0/1669258291 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5605aa84d670 msgr2=0x5605aa858220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:25:50.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.045+0000 7f973b7fe640 1 --2- 172.21.15.6:0/1669258291 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5605aa84d670 0x5605aa858220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.045+0000 7f973b7fe640 1 -- 172.21.15.6:0/1669258291 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605aa5243c0 con 0x5605aa858850 2024-09-12T03:25:50.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.045+0000 7f973b7fe640 1 --2- 172.21.15.6:0/1669258291 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa858850 0x5605aa855ac0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f973000ab50 tx=0x7f973009b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1f811f8a2fd7b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:50.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.046+0000 7f971affd640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:25:50.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.047+0000 7f971affd640 1 -- 172.21.15.6:0/1669258291 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730014890 con 0x5605aa858850 2024-09-12T03:25:50.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.047+0000 7f971affd640 1 -- 172.21.15.6:0/1669258291 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f9730014c00 con 0x5605aa858850 2024-09-12T03:25:50.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.047+0000 7f9743e37740 1 -- 172.21.15.6:0/1669258291 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9734011270 con 0x5605aa858850 2024-09-12T03:25:50.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.047+0000 7f971affd640 1 -- 172.21.15.6:0/1669258291 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9730014e00 con 0x5605aa858850 2024-09-12T03:25:50.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.047+0000 7f9743e37740 10 client.15993 fetch_fsmap learned FSMap version 27 2024-09-12T03:25:50.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.047+0000 7f9743e37740 1 -- 172.21.15.6:0/1669258291 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5605aa877ef0 con 0x5605aa858850 2024-09-12T03:25:50.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.048+0000 7f973a7fc640 1 --2- 172.21.15.6:0/2613620469 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f97240a2c80 0x7f97240a5140 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x5605aa856a80 tx=0x7f973400ee30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:50.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.048+0000 7f971affd640 1 -- 172.21.15.6:0/1669258291 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9734011270 con 0x5605aa858850 2024-09-12T03:25:50.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.049+0000 7f971affd640 1 -- 172.21.15.6:0/1669258291 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9730099ed0 con 0x5605aa858850 2024-09-12T03:25:50.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.049+0000 7f9743e37740 10 client.15993 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:25:50.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.049+0000 7f9743e37740 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:25:50.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.049+0000 7f9743e37740 20 client.15993 populate_metadata read hostname 'smithi006' 2024-09-12T03:25:50.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.049+0000 7f9743e37740 1 --2- 172.21.15.6:0/1669258291 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5605aa879350 0x5605aa87b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.049+0000 7f9743e37740 4 client.15993 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:25:50.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.049+0000 7f9743e37740 1 -- 172.21.15.6:0/1669258291 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605aa7c0850 con 0x5605aa879350 2024-09-12T03:25:50.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.049+0000 7f973affd640 1 --2- 172.21.15.6:0/1669258291 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5605aa879350 0x5605aa87b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.051+0000 7f973affd640 1 --2- 172.21.15.6:0/1669258291 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5605aa879350 0x5605aa87b7d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7951aca0dcf40e3a server_cookie=598adfccf2abfab4 in_seq=0 out_seq=0 2024-09-12T03:25:50.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.051+0000 7f971affd640 10 client.15993 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:50.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.052+0000 7f971affd640 1 -- 172.21.15.6:0/1669258291 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605aa7c0850 con 0x5605aa879350 2024-09-12T03:25:50.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.052+0000 7f971affd640 10 client.15993 handle_command_reply: tid=0 2024-09-12T03:25:50.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.052+0000 7f9743e37740 1 client.15993 shutdown 2024-09-12T03:25:50.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.053+0000 7f9743e37740 1 -- 172.21.15.6:0/1669258291 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa858850 msgr2=0x5605aa855ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:50.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.053+0000 7f9743e37740 1 --2- 172.21.15.6:0/1669258291 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa858850 0x5605aa855ac0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f973000ab50 tx=0x7f973009b2c0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.054+0000 7f9743e37740 1 -- 172.21.15.6:0/1669258291 shutdown_connections 2024-09-12T03:25:50.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.054+0000 7f9743e37740 1 --2- 172.21.15.6:0/1669258291 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5605aa879350 0x5605aa87b7d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.054+0000 7f9743e37740 1 --2- 172.21.15.6:0/1669258291 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa858850 0x5605aa855ac0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.054+0000 7f9743e37740 1 --2- 172.21.15.6:0/1669258291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa84df90 0x5605aa8554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.054+0000 7f9743e37740 1 --2- 172.21.15.6:0/1669258291 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5605aa84d670 0x5605aa858220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.054+0000 7f9743e37740 1 -- 172.21.15.6:0/1669258291 >> 172.21.15.6:0/1669258291 conn(0x5605aa849c00 msgr2=0x5605aa84f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:50.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.054+0000 7f971affd640 0 client.15993 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:50.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.055+0000 7f9743e37740 1 -- 172.21.15.6:0/1669258291 shutdown_connections 2024-09-12T03:25:50.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.055+0000 7f9743e37740 1 -- 172.21.15.6:0/1669258291 wait complete. 2024-09-12T03:25:50.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.055+0000 7f9743e37740 20 client.15993 trim_cache size 0 max 16384 2024-09-12T03:25:50.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.092+0000 7f9743e37740 1 Processor -- start 2024-09-12T03:25:50.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.096+0000 7f9743e37740 1 -- start start 2024-09-12T03:25:50.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.096+0000 7f9743e37740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa84d670 0x5605aa8810a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.096+0000 7f9743e37740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa84df90 0x5605aa8815e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.097+0000 7f9743e37740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5605aa858850 0x5605aa852790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.097+0000 7f9743e37740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f973c1c71d0 con 0x5605aa84df90 2024-09-12T03:25:50.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.097+0000 7f9743e37740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f973c139340 con 0x5605aa858850 2024-09-12T03:25:50.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.097+0000 7f9743e37740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f973c06e930 con 0x5605aa84d670 2024-09-12T03:25:50.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.097+0000 7f973b7fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5605aa858850 0x5605aa852790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.097+0000 7f973b7fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5605aa858850 0x5605aa852790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:60382/0 (socket says 172.21.15.6:60382) 2024-09-12T03:25:50.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.097+0000 7f973b7fe640 1 -- 172.21.15.6:0/779513471 learned_addr learned my addr 172.21.15.6:0/779513471 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:50.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.097+0000 7f973a7fc640 1 --2- 172.21.15.6:0/779513471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa84df90 0x5605aa8815e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.098+0000 7f973affd640 1 --2- 172.21.15.6:0/779513471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa84d670 0x5605aa8810a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.098+0000 7f973b7fe640 1 -- 172.21.15.6:0/779513471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa84d670 msgr2=0x5605aa8810a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:50.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.098+0000 7f973b7fe640 1 --2- 172.21.15.6:0/779513471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa84d670 0x5605aa8810a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.098+0000 7f973b7fe640 1 -- 172.21.15.6:0/779513471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa84df90 msgr2=0x5605aa8815e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:50.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.098+0000 7f973b7fe640 1 --2- 172.21.15.6:0/779513471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa84df90 0x5605aa8815e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.098+0000 7f973b7fe640 1 -- 172.21.15.6:0/779513471 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605aa858590 con 0x5605aa858850 2024-09-12T03:25:50.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.098+0000 7f973a7fc640 1 --2- 172.21.15.6:0/779513471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa84df90 0x5605aa8815e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:50.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.099+0000 7f973b7fe640 1 --2- 172.21.15.6:0/779513471 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5605aa858850 0x5605aa852790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9730098d30 tx=0x7f9730005a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99c04a0bd77bf4b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:50.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.099+0000 7f97217fa640 1 -- 172.21.15.6:0/779513471 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973009a3e0 con 0x5605aa858850 2024-09-12T03:25:50.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.099+0000 7f97217fa640 1 -- 172.21.15.6:0/779513471 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9730099ed0 con 0x5605aa858850 2024-09-12T03:25:50.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.100+0000 7f97217fa640 1 -- 172.21.15.6:0/779513471 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97300b91b0 con 0x5605aa858850 2024-09-12T03:25:50.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.101+0000 7f9743e37740 1 -- 172.21.15.6:0/779513471 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5605aa858850 msgr2=0x5605aa852790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:50.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.101+0000 7f9743e37740 1 --2- 172.21.15.6:0/779513471 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5605aa858850 0x5605aa852790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9730098d30 tx=0x7f9730005a50 comp rx=0 tx=0).stop 2024-09-12T03:25:50.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.101+0000 7f9743e37740 1 -- 172.21.15.6:0/779513471 shutdown_connections 2024-09-12T03:25:50.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.101+0000 7f9743e37740 1 --2- 172.21.15.6:0/779513471 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5605aa858850 0x5605aa852790 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.101+0000 7f9743e37740 1 --2- 172.21.15.6:0/779513471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa84df90 0x5605aa8815e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.101+0000 7f9743e37740 1 --2- 172.21.15.6:0/779513471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa84d670 0x5605aa8810a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.101+0000 7f9743e37740 1 -- 172.21.15.6:0/779513471 >> 172.21.15.6:0/779513471 conn(0x5605aa849c00 msgr2=0x5605aa857550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:50.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.101+0000 7f9743e37740 1 -- 172.21.15.6:0/779513471 shutdown_connections 2024-09-12T03:25:50.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.101+0000 7f9743e37740 1 -- 172.21.15.6:0/779513471 wait complete. 2024-09-12T03:25:50.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.102+0000 7f9743e37740 1 -- start start 2024-09-12T03:25:50.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.103+0000 7f9743e37740 1 Processor -- start 2024-09-12T03:25:50.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.103+0000 7f9743e37740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa84d670 0x5605aa860820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.104+0000 7f9743e37740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa84df90 0x5605aa860d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.104+0000 7f973affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa84d670 0x5605aa860820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.104+0000 7f973affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa84d670 0x5605aa860820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:49286/0 (socket says 172.21.15.6:49286) 2024-09-12T03:25:50.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.104+0000 7f973affd640 1 -- 172.21.15.6:0/1357847647 learned_addr learned my addr 172.21.15.6:0/1357847647 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:25:50.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.104+0000 7f9743e37740 1 --2- 172.21.15.6:0/1357847647 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5605aa858850 0x5605aa85d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.105+0000 7f973a7fc640 1 --2- 172.21.15.6:0/1357847647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa84df90 0x5605aa860d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.105+0000 7f9743e37740 1 -- 172.21.15.6:0/1357847647 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f973c137820 con 0x5605aa84df90 2024-09-12T03:25:50.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.105+0000 7f9743e37740 1 -- 172.21.15.6:0/1357847647 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f973c138c70 con 0x5605aa858850 2024-09-12T03:25:50.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.105+0000 7f9743e37740 1 -- 172.21.15.6:0/1357847647 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f973c137ed0 con 0x5605aa84d670 2024-09-12T03:25:50.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.105+0000 7f9743e37740 10 client.0 _ll_register_callbacks cb 0x7f9730096600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:50.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.105+0000 7f973b7fe640 1 --2- 172.21.15.6:0/1357847647 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5605aa858850 0x5605aa85d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.106+0000 7f973affd640 1 -- 172.21.15.6:0/1357847647 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5605aa858850 msgr2=0x5605aa85d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:50.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.106+0000 7f973affd640 1 --2- 172.21.15.6:0/1357847647 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5605aa858850 0x5605aa85d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.106+0000 7f973affd640 1 -- 172.21.15.6:0/1357847647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa84df90 msgr2=0x5605aa860d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:50.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.106+0000 7f973affd640 1 --2- 172.21.15.6:0/1357847647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa84df90 0x5605aa860d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.106+0000 7f973affd640 1 -- 172.21.15.6:0/1357847647 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605aa861400 con 0x5605aa84d670 2024-09-12T03:25:50.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.106+0000 7f973affd640 1 --2- 172.21.15.6:0/1357847647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa84d670 0x5605aa860820 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f972c02a940 tx=0x7f972c030340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abb87f498c7a2514 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:50.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.106+0000 7f971affd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:25:50.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.107+0000 7f971affd640 1 -- 172.21.15.6:0/1357847647 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c037040 con 0x5605aa84d670 2024-09-12T03:25:50.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.107+0000 7f971affd640 1 -- 172.21.15.6:0/1357847647 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f972c004030 con 0x5605aa84d670 2024-09-12T03:25:50.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.107+0000 7f9743e37740 1 -- 172.21.15.6:0/1357847647 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9730099ed0 con 0x5605aa84d670 2024-09-12T03:25:50.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.111+0000 7f971affd640 1 -- 172.21.15.6:0/1357847647 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f972c004230 con 0x5605aa84d670 2024-09-12T03:25:50.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.111+0000 7f9743e37740 10 client.25624 fetch_fsmap learned FSMap version 27 2024-09-12T03:25:50.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.111+0000 7f9743e37740 1 -- 172.21.15.6:0/1357847647 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5605aa8e4620 con 0x5605aa84d670 2024-09-12T03:25:50.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.112+0000 7f971affd640 1 -- 172.21.15.6:0/1357847647 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f972c0043d0 con 0x5605aa84d670 2024-09-12T03:25:50.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.112+0000 7f971affd640 1 -- 172.21.15.6:0/1357847647 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f972c003b90 con 0x5605aa84d670 2024-09-12T03:25:50.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.112+0000 7f9743e37740 10 client.25624 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:25:50.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.112+0000 7f9743e37740 10 client.25624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:25:50.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.112+0000 7f9743e37740 20 client.25624 populate_metadata read hostname 'smithi006' 2024-09-12T03:25:50.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.112+0000 7f9743e37740 1 --2- 172.21.15.6:0/1357847647 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5605aa8e4df0 0x5605aa8e7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.112+0000 7f9743e37740 4 client.25624 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:25:50.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.112+0000 7f9743e37740 1 -- 172.21.15.6:0/1357847647 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f9730099ed0 con 0x5605aa8e4df0 2024-09-12T03:25:50.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.113+0000 7f973a7fc640 1 --2- 172.21.15.6:0/1357847647 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5605aa8e4df0 0x5605aa8e7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.114+0000 7f973a7fc640 1 --2- 172.21.15.6:0/1357847647 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5605aa8e4df0 0x5605aa8e7210 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a414a4597e1403de server_cookie=55b86fc6a8844167 in_seq=0 out_seq=0 2024-09-12T03:25:50.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.114+0000 7f971affd640 10 client.25624 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:50.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.114+0000 7f971affd640 1 -- 172.21.15.6:0/1357847647 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x7f9730099ed0 con 0x5605aa8e4df0 2024-09-12T03:25:50.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.114+0000 7f971affd640 10 client.25624 handle_command_reply: tid=0 2024-09-12T03:25:50.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.114+0000 7f9743e37740 1 client.25624 shutdown 2024-09-12T03:25:50.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.115+0000 7f9743e37740 1 -- 172.21.15.6:0/1357847647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa84d670 msgr2=0x5605aa860820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:50.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.115+0000 7f9743e37740 1 --2- 172.21.15.6:0/1357847647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa84d670 0x5605aa860820 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f972c02a940 tx=0x7f972c030340 comp rx=0 tx=0).stop 2024-09-12T03:25:50.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.115+0000 7f9743e37740 1 -- 172.21.15.6:0/1357847647 shutdown_connections 2024-09-12T03:25:50.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.115+0000 7f9743e37740 1 --2- 172.21.15.6:0/1357847647 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5605aa8e4df0 0x5605aa8e7210 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.115+0000 7f9743e37740 1 --2- 172.21.15.6:0/1357847647 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5605aa858850 0x5605aa85d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.115+0000 7f971affd640 0 client.25624 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:25:50.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.115+0000 7f9743e37740 1 --2- 172.21.15.6:0/1357847647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605aa84df90 0x5605aa860d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.115+0000 7f9743e37740 1 --2- 172.21.15.6:0/1357847647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605aa84d670 0x5605aa860820 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f972c02a940 tx=0x7f972c030340 comp rx=0 tx=0).stop 2024-09-12T03:25:50.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.115+0000 7f9743e37740 1 -- 172.21.15.6:0/1357847647 >> 172.21.15.6:0/1357847647 conn(0x5605aa849c00 msgr2=0x5605aa84f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:50.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.116+0000 7f9743e37740 1 -- 172.21.15.6:0/1357847647 shutdown_connections 2024-09-12T03:25:50.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.116+0000 7f9743e37740 1 -- 172.21.15.6:0/1357847647 wait complete. 2024-09-12T03:25:50.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.116+0000 7f9743e37740 20 client.25624 trim_cache size 0 max 16384 2024-09-12T03:25:50.126 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:25:50.126 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "scrub active (193 inodes in the stack)", 2024-09-12T03:25:50.126 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2024-09-12T03:25:50.126 INFO:teuthology.orchestra.run.smithi006.stdout: "01562442-7858-449c-a9dd-3b0d01987c49": { 2024-09-12T03:25:50.126 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-09-12T03:25:50.126 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "01562442-7858-449c-a9dd-3b0d01987c49", 2024-09-12T03:25:50.127 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2024-09-12T03:25:50.127 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:25:50.127 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:25:50.127 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:25:50.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.129+0000 7f973bfff640 1 -- 172.21.15.6:0/2613620469 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f97240a2c80 msgr2=0x7f97240a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:50.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.129+0000 7f973bfff640 1 --2- 172.21.15.6:0/2613620469 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f97240a2c80 0x7f97240a5140 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x5605aa856a80 tx=0x7f973400ee30 comp rx=0 tx=0).stop 2024-09-12T03:25:50.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.129+0000 7f973bfff640 1 -- 172.21.15.6:0/2613620469 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f973c10e090 msgr2=0x7f973c07f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:50.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.129+0000 7f973bfff640 1 --2- 172.21.15.6:0/2613620469 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f973c10e090 0x7f973c07f580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f97300095a0 tx=0x7f9730009a60 comp rx=0 tx=0).stop 2024-09-12T03:25:50.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.131+0000 7f973bfff640 1 -- 172.21.15.6:0/2613620469 shutdown_connections 2024-09-12T03:25:50.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.131+0000 7f973bfff640 1 --2- 172.21.15.6:0/2613620469 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f97240a2c80 0x7f97240a5140 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.131+0000 7f973bfff640 1 --2- 172.21.15.6:0/2613620469 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f973c10e090 0x7f973c07f580 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.131+0000 7f973bfff640 1 --2- 172.21.15.6:0/2613620469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f973c06ee70 0x7f973c07ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.131+0000 7f973bfff640 1 --2- 172.21.15.6:0/2613620469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f973c06e4f0 0x7f973c084e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.131+0000 7f973bfff640 1 -- 172.21.15.6:0/2613620469 >> 172.21.15.6:0/2613620469 conn(0x7f973c073180 msgr2=0x7f973c06d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:50.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.133+0000 7f973bfff640 1 -- 172.21.15.6:0/2613620469 shutdown_connections 2024-09-12T03:25:50.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:25:50.133+0000 7f973bfff640 1 -- 172.21.15.6:0/2613620469 wait complete. 2024-09-12T03:25:50.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:50 smithi006 ceph-mon[30991]: pgmap v477: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 451 KiB/s rd, 153 MiB/s wr, 14.09k op/s 2024-09-12T03:25:50.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:50 smithi161 ceph-mon[35485]: pgmap v477: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 451 KiB/s rd, 153 MiB/s wr, 14.09k op/s 2024-09-12T03:25:50.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:50 smithi118 ceph-mon[35048]: pgmap v477: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 451 KiB/s rd, 153 MiB/s wr, 14.09k op/s 2024-09-12T03:25:51.260 INFO:tasks.cephfs.filesystem:scrub status for tag:01562442-7858-449c-a9dd-3b0d01987c49 - {'path': '/', 'tag': '01562442-7858-449c-a9dd-3b0d01987c49', 'options': 'recursive,force'} 2024-09-12T03:25:52.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:52 smithi006 ceph-mon[30991]: pgmap v478: 97 pgs: 97 active+clean; 13 GiB data, 89 GiB used, 983 GiB / 1.0 TiB avail; 849 KiB/s rd, 189 MiB/s wr, 17.25k op/s 2024-09-12T03:25:52.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:52 smithi161 ceph-mon[35485]: pgmap v478: 97 pgs: 97 active+clean; 13 GiB data, 89 GiB used, 983 GiB / 1.0 TiB avail; 849 KiB/s rd, 189 MiB/s wr, 17.25k op/s 2024-09-12T03:25:52.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:52 smithi118 ceph-mon[35048]: pgmap v478: 97 pgs: 97 active+clean; 13 GiB data, 89 GiB used, 983 GiB / 1.0 TiB avail; 849 KiB/s rd, 189 MiB/s wr, 17.25k op/s 2024-09-12T03:25:54.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:54 smithi006 ceph-mon[30991]: pgmap v479: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 741 KiB/s rd, 212 MiB/s wr, 19.09k op/s 2024-09-12T03:25:54.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:54 smithi161 ceph-mon[35485]: pgmap v479: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 741 KiB/s rd, 212 MiB/s wr, 19.09k op/s 2024-09-12T03:25:54.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:54 smithi118 ceph-mon[35048]: pgmap v479: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 741 KiB/s rd, 212 MiB/s wr, 19.09k op/s 2024-09-12T03:25:56.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:56 smithi006 ceph-mon[30991]: pgmap v480: 97 pgs: 97 active+clean; 13 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 741 KiB/s rd, 208 MiB/s wr, 18.79k op/s 2024-09-12T03:25:56.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:56 smithi161 ceph-mon[35485]: pgmap v480: 97 pgs: 97 active+clean; 13 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 741 KiB/s rd, 208 MiB/s wr, 18.79k op/s 2024-09-12T03:25:56.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:56 smithi118 ceph-mon[35048]: pgmap v480: 97 pgs: 97 active+clean; 13 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 741 KiB/s rd, 208 MiB/s wr, 18.79k op/s 2024-09-12T03:25:58.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:25:58 smithi006 ceph-mon[30991]: pgmap v481: 97 pgs: 97 active+clean; 13 GiB data, 90 GiB used, 983 GiB / 1.0 TiB avail; 820 KiB/s rd, 244 MiB/s wr, 22.26k op/s 2024-09-12T03:25:58.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:25:58 smithi161 ceph-mon[35485]: pgmap v481: 97 pgs: 97 active+clean; 13 GiB data, 90 GiB used, 983 GiB / 1.0 TiB avail; 820 KiB/s rd, 244 MiB/s wr, 22.26k op/s 2024-09-12T03:25:58.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:25:58 smithi118 ceph-mon[35048]: pgmap v481: 97 pgs: 97 active+clean; 13 GiB data, 90 GiB used, 983 GiB / 1.0 TiB avail; 820 KiB/s rd, 244 MiB/s wr, 22.26k op/s 2024-09-12T03:26:00.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:00 smithi006 ceph-mon[30991]: pgmap v482: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 819 KiB/s rd, 211 MiB/s wr, 19.57k op/s 2024-09-12T03:26:00.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:00 smithi161 ceph-mon[35485]: pgmap v482: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 819 KiB/s rd, 211 MiB/s wr, 19.57k op/s 2024-09-12T03:26:00.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:00 smithi118 ceph-mon[35048]: pgmap v482: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 819 KiB/s rd, 211 MiB/s wr, 19.57k op/s 2024-09-12T03:26:02.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:02 smithi006 ceph-mon[30991]: pgmap v483: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 213 MiB/s wr, 19.83k op/s 2024-09-12T03:26:02.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:02 smithi161 ceph-mon[35485]: pgmap v483: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 213 MiB/s wr, 19.83k op/s 2024-09-12T03:26:02.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:02 smithi118 ceph-mon[35048]: pgmap v483: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 213 MiB/s wr, 19.83k op/s 2024-09-12T03:26:04.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:04 smithi006 ceph-mon[30991]: pgmap v484: 97 pgs: 97 active+clean; 13 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 630 KiB/s rd, 219 MiB/s wr, 20.75k op/s 2024-09-12T03:26:04.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:04 smithi161 ceph-mon[35485]: pgmap v484: 97 pgs: 97 active+clean; 13 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 630 KiB/s rd, 219 MiB/s wr, 20.75k op/s 2024-09-12T03:26:04.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:04 smithi118 ceph-mon[35048]: pgmap v484: 97 pgs: 97 active+clean; 13 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 630 KiB/s rd, 219 MiB/s wr, 20.75k op/s 2024-09-12T03:26:06.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:06 smithi161 ceph-mon[35485]: pgmap v485: 97 pgs: 97 active+clean; 13 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 700 KiB/s rd, 203 MiB/s wr, 19.46k op/s 2024-09-12T03:26:06.873 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:06 smithi006 ceph-mon[30991]: pgmap v485: 97 pgs: 97 active+clean; 13 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 700 KiB/s rd, 203 MiB/s wr, 19.46k op/s 2024-09-12T03:26:06.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:06 smithi118 ceph-mon[35048]: pgmap v485: 97 pgs: 97 active+clean; 13 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 700 KiB/s rd, 203 MiB/s wr, 19.46k op/s 2024-09-12T03:26:08.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:08 smithi006 ceph-mon[30991]: pgmap v486: 97 pgs: 97 active+clean; 13 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 701 KiB/s rd, 233 MiB/s wr, 22.61k op/s 2024-09-12T03:26:08.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:08 smithi161 ceph-mon[35485]: pgmap v486: 97 pgs: 97 active+clean; 13 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 701 KiB/s rd, 233 MiB/s wr, 22.61k op/s 2024-09-12T03:26:08.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:08 smithi118 ceph-mon[35048]: pgmap v486: 97 pgs: 97 active+clean; 13 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 701 KiB/s rd, 233 MiB/s wr, 22.61k op/s 2024-09-12T03:26:10.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:10 smithi006 ceph-mon[30991]: pgmap v487: 97 pgs: 97 active+clean; 13 GiB data, 95 GiB used, 978 GiB / 1.0 TiB avail; 685 KiB/s rd, 195 MiB/s wr, 19.12k op/s 2024-09-12T03:26:10.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:10 smithi161 ceph-mon[35485]: pgmap v487: 97 pgs: 97 active+clean; 13 GiB data, 95 GiB used, 978 GiB / 1.0 TiB avail; 685 KiB/s rd, 195 MiB/s wr, 19.12k op/s 2024-09-12T03:26:10.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:10 smithi118 ceph-mon[35048]: pgmap v487: 97 pgs: 97 active+clean; 13 GiB data, 95 GiB used, 978 GiB / 1.0 TiB avail; 685 KiB/s rd, 195 MiB/s wr, 19.12k op/s 2024-09-12T03:26:12.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:12 smithi006 ceph-mon[30991]: pgmap v488: 97 pgs: 97 active+clean; 13 GiB data, 95 GiB used, 978 GiB / 1.0 TiB avail; 685 KiB/s rd, 187 MiB/s wr, 18.47k op/s 2024-09-12T03:26:12.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:12 smithi161 ceph-mon[35485]: pgmap v488: 97 pgs: 97 active+clean; 13 GiB data, 95 GiB used, 978 GiB / 1.0 TiB avail; 685 KiB/s rd, 187 MiB/s wr, 18.47k op/s 2024-09-12T03:26:12.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:12 smithi118 ceph-mon[35048]: pgmap v488: 97 pgs: 97 active+clean; 13 GiB data, 95 GiB used, 978 GiB / 1.0 TiB avail; 685 KiB/s rd, 187 MiB/s wr, 18.47k op/s 2024-09-12T03:26:14.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:14 smithi118 ceph-mon[35048]: pgmap v489: 97 pgs: 97 active+clean; 13 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 524 KiB/s rd, 209 MiB/s wr, 20.96k op/s 2024-09-12T03:26:15.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:14 smithi006 ceph-mon[30991]: pgmap v489: 97 pgs: 97 active+clean; 13 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 524 KiB/s rd, 209 MiB/s wr, 20.96k op/s 2024-09-12T03:26:15.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:14 smithi161 ceph-mon[35485]: pgmap v489: 97 pgs: 97 active+clean; 13 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 524 KiB/s rd, 209 MiB/s wr, 20.96k op/s 2024-09-12T03:26:16.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:16 smithi118 ceph-mon[35048]: pgmap v490: 97 pgs: 97 active+clean; 13 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 524 KiB/s rd, 182 MiB/s wr, 18.47k op/s 2024-09-12T03:26:17.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:16 smithi006 ceph-mon[30991]: pgmap v490: 97 pgs: 97 active+clean; 13 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 524 KiB/s rd, 182 MiB/s wr, 18.47k op/s 2024-09-12T03:26:17.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:16 smithi161 ceph-mon[35485]: pgmap v490: 97 pgs: 97 active+clean; 13 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 524 KiB/s rd, 182 MiB/s wr, 18.47k op/s 2024-09-12T03:26:18.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:18 smithi118 ceph-mon[35048]: pgmap v491: 97 pgs: 97 active+clean; 13 GiB data, 98 GiB used, 975 GiB / 1.0 TiB avail; 583 KiB/s rd, 207 MiB/s wr, 20.84k op/s 2024-09-12T03:26:19.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:18 smithi006 ceph-mon[30991]: pgmap v491: 97 pgs: 97 active+clean; 13 GiB data, 98 GiB used, 975 GiB / 1.0 TiB avail; 583 KiB/s rd, 207 MiB/s wr, 20.84k op/s 2024-09-12T03:26:19.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:18 smithi161 ceph-mon[35485]: pgmap v491: 97 pgs: 97 active+clean; 13 GiB data, 98 GiB used, 975 GiB / 1.0 TiB avail; 583 KiB/s rd, 207 MiB/s wr, 20.84k op/s 2024-09-12T03:26:20.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:20 smithi118 ceph-mon[35048]: pgmap v492: 97 pgs: 97 active+clean; 13 GiB data, 98 GiB used, 975 GiB / 1.0 TiB avail; 582 KiB/s rd, 171 MiB/s wr, 17.19k op/s 2024-09-12T03:26:21.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:20 smithi006 ceph-mon[30991]: pgmap v492: 97 pgs: 97 active+clean; 13 GiB data, 98 GiB used, 975 GiB / 1.0 TiB avail; 582 KiB/s rd, 171 MiB/s wr, 17.19k op/s 2024-09-12T03:26:21.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:20 smithi161 ceph-mon[35485]: pgmap v492: 97 pgs: 97 active+clean; 13 GiB data, 98 GiB used, 975 GiB / 1.0 TiB avail; 582 KiB/s rd, 171 MiB/s wr, 17.19k op/s 2024-09-12T03:26:21.261 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:26:21.880 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:22.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.877+0000 7fe616b35640 1 -- 172.21.15.6:0/1624111839 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe610109830 msgr2=0x7fe610109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:22.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.877+0000 7fe616b35640 1 --2- 172.21.15.6:0/1624111839 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe610109830 0x7fe610109c90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe600009be0 tx=0x7fe60002a6e0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.880 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:22 smithi006 ceph-mon[30991]: pgmap v493: 97 pgs: 97 active+clean; 13 GiB data, 99 GiB used, 974 GiB / 1.0 TiB avail; 311 KiB/s rd, 180 MiB/s wr, 18.16k op/s 2024-09-12T03:26:22.880 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:22 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:26:22.880 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:22 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:22.880 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:22 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:22.880 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:22 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:26:22.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.878+0000 7fe616b35640 1 -- 172.21.15.6:0/1624111839 shutdown_connections 2024-09-12T03:26:22.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.878+0000 7fe616b35640 1 --2- 172.21.15.6:0/1624111839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe61011b140 0x7fe61011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.878+0000 7fe616b35640 1 --2- 172.21.15.6:0/1624111839 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe610109830 0x7fe610109c90 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.878+0000 7fe616b35640 1 --2- 172.21.15.6:0/1624111839 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe610109450 0x7fe610110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.878+0000 7fe616b35640 1 -- 172.21.15.6:0/1624111839 >> 172.21.15.6:0/1624111839 conn(0x7fe61006e140 msgr2=0x7fe61006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:22.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.878+0000 7fe616b35640 1 -- 172.21.15.6:0/1624111839 shutdown_connections 2024-09-12T03:26:22.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.879+0000 7fe616b35640 1 -- 172.21.15.6:0/1624111839 wait complete. 2024-09-12T03:26:22.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.879+0000 7fe616b35640 1 Processor -- start 2024-09-12T03:26:22.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.880+0000 7fe616b35640 1 -- start start 2024-09-12T03:26:22.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.880+0000 7fe616b35640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe610109450 0x7fe610118650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.880+0000 7fe616b35640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe610109830 0x7fe610118b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.880+0000 7fe616b35640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe61011b140 0x7fe6101118c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.880+0000 7fe616b35640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe610120df0 con 0x7fe61011b140 2024-09-12T03:26:22.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.880+0000 7fe616b35640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe610120c70 con 0x7fe610109450 2024-09-12T03:26:22.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.880+0000 7fe616b35640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe610120f70 con 0x7fe610109830 2024-09-12T03:26:22.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.881+0000 7fe6148d1640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe610109450 0x7fe610118650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:22.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.881+0000 7fe6148d1640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe610109450 0x7fe610118650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:50814/0 (socket says 172.21.15.6:50814) 2024-09-12T03:26:22.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.881+0000 7fe6148d1640 1 -- 172.21.15.6:0/3536676396 learned_addr learned my addr 172.21.15.6:0/3536676396 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:26:22.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.881+0000 7fe6148d1640 1 -- 172.21.15.6:0/3536676396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe610109830 msgr2=0x7fe610118b90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:26:22.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.881+0000 7fe6148d1640 1 --2- 172.21.15.6:0/3536676396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe610109830 0x7fe610118b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.881+0000 7fe6148d1640 1 -- 172.21.15.6:0/3536676396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe61011b140 msgr2=0x7fe6101118c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:26:22.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.881+0000 7fe6148d1640 1 --2- 172.21.15.6:0/3536676396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe61011b140 0x7fe6101118c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.881+0000 7fe6148d1640 1 -- 172.21.15.6:0/3536676396 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe610112180 con 0x7fe610109450 2024-09-12T03:26:22.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.882+0000 7fe6148d1640 1 --2- 172.21.15.6:0/3536676396 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe610109450 0x7fe610118650 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe60800d950 tx=0x7fe60800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:22.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.883+0000 7fe60dffb640 1 -- 172.21.15.6:0/3536676396 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608017070 con 0x7fe610109450 2024-09-12T03:26:22.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.883+0000 7fe616b35640 1 -- 172.21.15.6:0/3536676396 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe610112410 con 0x7fe610109450 2024-09-12T03:26:22.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.883+0000 7fe616b35640 1 -- 172.21.15.6:0/3536676396 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe610116460 con 0x7fe610109450 2024-09-12T03:26:22.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.888+0000 7fe618f0e740 1 Processor -- start 2024-09-12T03:26:22.889 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.889+0000 7fe618f0e740 1 -- start start 2024-09-12T03:26:22.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.889+0000 7fe618f0e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1e984a0 0x5651a1e98880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.889+0000 7fe618f0e740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651a1e98dc0 0x5651a1ea3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.889+0000 7fe618f0e740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1ea3680 0x5651a1ea5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.889+0000 7fe618f0e740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5651a1dfe2e0 con 0x5651a1ea3680 2024-09-12T03:26:22.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.889+0000 7fe618f0e740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5651a1da5ce0 con 0x5651a1e98dc0 2024-09-12T03:26:22.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.889+0000 7fe618f0e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5651a1dfa2a0 con 0x5651a1e984a0 2024-09-12T03:26:22.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.890+0000 7fe60dffb640 1 -- 172.21.15.6:0/3536676396 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe608009c30 con 0x7fe610109450 2024-09-12T03:26:22.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.890+0000 7fe60dffb640 1 -- 172.21.15.6:0/3536676396 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60800f620 con 0x7fe610109450 2024-09-12T03:26:22.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.891+0000 7fe60ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1ea3680 0x5651a1ea5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:22.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.891+0000 7fe60ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1ea3680 0x5651a1ea5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41502/0 (socket says 172.21.15.6:41502) 2024-09-12T03:26:22.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.891+0000 7fe60ffff640 1 -- 172.21.15.6:0/555075012 learned_addr learned my addr 172.21.15.6:0/555075012 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:26:22.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.891+0000 7fe6150d2640 1 --2- 172.21.15.6:0/555075012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1e984a0 0x5651a1e98880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:22.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.891+0000 7fe6150d2640 1 -- 172.21.15.6:0/555075012 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651a1e98dc0 msgr2=0x5651a1ea3140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:26:22.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.891+0000 7fe6150d2640 1 --2- 172.21.15.6:0/555075012 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651a1e98dc0 0x5651a1ea3140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.891+0000 7fe6150d2640 1 -- 172.21.15.6:0/555075012 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1ea3680 msgr2=0x5651a1ea5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:22.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.891+0000 7fe6150d2640 1 --2- 172.21.15.6:0/555075012 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1ea3680 0x5651a1ea5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.891+0000 7fe6150d2640 1 -- 172.21.15.6:0/555075012 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a1d8a440 con 0x5651a1e984a0 2024-09-12T03:26:22.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.892+0000 7fe6150d2640 1 --2- 172.21.15.6:0/555075012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1e984a0 0x5651a1e98880 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80099c0 tx=0x7fe5f8009e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c437c52ec20924b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:22.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.892+0000 7fe606ffd640 1 -- 172.21.15.6:0/555075012 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8004220 con 0x5651a1e984a0 2024-09-12T03:26:22.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.893+0000 7fe606ffd640 1 -- 172.21.15.6:0/555075012 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5f800ee50 con 0x5651a1e984a0 2024-09-12T03:26:22.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.893+0000 7fe618f0e740 1 -- 172.21.15.6:0/555075012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1e984a0 msgr2=0x5651a1e98880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:22.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.893+0000 7fe618f0e740 1 --2- 172.21.15.6:0/555075012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1e984a0 0x5651a1e98880 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80099c0 tx=0x7fe5f8009e80 comp rx=0 tx=0).stop 2024-09-12T03:26:22.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.894+0000 7fe618f0e740 1 -- 172.21.15.6:0/555075012 shutdown_connections 2024-09-12T03:26:22.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.894+0000 7fe618f0e740 1 --2- 172.21.15.6:0/555075012 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1ea3680 0x5651a1ea5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.894+0000 7fe618f0e740 1 --2- 172.21.15.6:0/555075012 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651a1e98dc0 0x5651a1ea3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.894+0000 7fe618f0e740 1 --2- 172.21.15.6:0/555075012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1e984a0 0x5651a1e98880 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.894+0000 7fe618f0e740 1 -- 172.21.15.6:0/555075012 >> 172.21.15.6:0/555075012 conn(0x5651a1e94a30 msgr2=0x5651a1e94e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:22.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.896+0000 7fe618f0e740 1 -- 172.21.15.6:0/555075012 shutdown_connections 2024-09-12T03:26:22.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.896+0000 7fe618f0e740 1 -- 172.21.15.6:0/555075012 wait complete. 2024-09-12T03:26:22.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.897+0000 7fe618f0e740 1 -- start start 2024-09-12T03:26:22.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.897+0000 7fe618f0e740 1 Processor -- start 2024-09-12T03:26:22.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.898+0000 7fe618f0e740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1e984a0 0x5651a1ea2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.898+0000 7fe618f0e740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651a1e98dc0 0x5651a1ea0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.898+0000 7fe618f0e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1ea3680 0x5651a1ea07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.898+0000 7fe618f0e740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5651a1ea62a0 con 0x5651a1e984a0 2024-09-12T03:26:22.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.898+0000 7fe618f0e740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5651a1db9050 con 0x5651a1e98dc0 2024-09-12T03:26:22.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.898+0000 7fe618f0e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5651a1ea6420 con 0x5651a1ea3680 2024-09-12T03:26:22.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.898+0000 7fe618f0e740 10 client.0 _ll_register_callbacks cb 0x5651a1df9160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:22.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.899+0000 7fe6150d2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1e984a0 0x5651a1ea2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:22.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.899+0000 7fe6150d2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1e984a0 0x5651a1ea2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41516/0 (socket says 172.21.15.6:41516) 2024-09-12T03:26:22.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.899+0000 7fe6150d2640 1 -- 172.21.15.6:0/2686859931 learned_addr learned my addr 172.21.15.6:0/2686859931 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:26:22.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.900+0000 7fe6148d1640 1 --2- 172.21.15.6:0/2686859931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651a1e98dc0 0x5651a1ea0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:22.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.900+0000 7fe6150d2640 1 -- 172.21.15.6:0/2686859931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1ea3680 msgr2=0x5651a1ea07f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:26:22.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.900+0000 7fe6150d2640 1 --2- 172.21.15.6:0/2686859931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1ea3680 0x5651a1ea07f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.900+0000 7fe6150d2640 1 -- 172.21.15.6:0/2686859931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651a1e98dc0 msgr2=0x5651a1ea0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:22.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.900+0000 7fe6150d2640 1 --2- 172.21.15.6:0/2686859931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651a1e98dc0 0x5651a1ea0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.900+0000 7fe6150d2640 1 -- 172.21.15.6:0/2686859931 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a1b6f3c0 con 0x5651a1e984a0 2024-09-12T03:26:22.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.901+0000 7fe6150d2640 1 --2- 172.21.15.6:0/2686859931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1e984a0 0x5651a1ea2fe0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8000c00 tx=0x7fe5f800eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75aba05ad6acac40 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:22.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.901+0000 7fe5effff640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:26:22.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.901+0000 7fe5effff640 1 -- 172.21.15.6:0/2686859931 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8004ad0 con 0x5651a1e984a0 2024-09-12T03:26:22.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.902+0000 7fe618f0e740 1 -- 172.21.15.6:0/2686859931 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5f800ee50 con 0x5651a1e984a0 2024-09-12T03:26:22.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.902+0000 7fe5effff640 1 -- 172.21.15.6:0/2686859931 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe5f80048d0 con 0x5651a1e984a0 2024-09-12T03:26:22.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.902+0000 7fe5effff640 1 -- 172.21.15.6:0/2686859931 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe5f8004ad0 con 0x5651a1e984a0 2024-09-12T03:26:22.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.903+0000 7fe618f0e740 10 client.16008 fetch_fsmap learned FSMap version 27 2024-09-12T03:26:22.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.903+0000 7fe618f0e740 1 -- 172.21.15.6:0/2686859931 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5651a1eb1db0 con 0x5651a1e984a0 2024-09-12T03:26:22.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.903+0000 7fe5effff640 1 -- 172.21.15.6:0/2686859931 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5f800ee50 con 0x5651a1e984a0 2024-09-12T03:26:22.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.905+0000 7fe5effff640 1 -- 172.21.15.6:0/2686859931 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe5f802a030 con 0x5651a1e984a0 2024-09-12T03:26:22.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.905+0000 7fe618f0e740 10 client.16008 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:26:22.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.905+0000 7fe618f0e740 10 client.16008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:26:22.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.905+0000 7fe618f0e740 20 client.16008 populate_metadata read hostname 'smithi006' 2024-09-12T03:26:22.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.905+0000 7fe618f0e740 1 --2- 172.21.15.6:0/2686859931 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5651a1eb3770 0x5651a1eb5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.905+0000 7fe618f0e740 4 client.16008 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:26:22.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.905+0000 7fe618f0e740 1 -- 172.21.15.6:0/2686859931 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651a1e0b850 con 0x5651a1eb3770 2024-09-12T03:26:22.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.914+0000 7fe60dffb640 1 -- 172.21.15.6:0/3536676396 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe60800f880 con 0x7fe610109450 2024-09-12T03:26:22.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.915+0000 7fe60dffb640 1 --2- 172.21.15.6:0/3536676396 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5fc0a2b80 0x7fe5fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.915+0000 7fe60dffb640 1 -- 172.21.15.6:0/3536676396 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe6080cebc0 con 0x7fe610109450 2024-09-12T03:26:22.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.917+0000 7fe6148d1640 1 --2- 172.21.15.6:0/2686859931 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5651a1eb3770 0x5651a1eb5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:22.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.918+0000 7fe60ffff640 1 --2- 172.21.15.6:0/3536676396 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5fc0a2b80 0x7fe5fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:22.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.923+0000 7fe6148d1640 1 --2- 172.21.15.6:0/2686859931 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5651a1eb3770 0x5651a1eb5c30 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=520694e01417037c server_cookie=2d8a4c3aeefcec11 in_seq=0 out_seq=0 2024-09-12T03:26:22.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.923+0000 7fe5effff640 10 client.16008 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:26:22.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.926+0000 7fe5effff640 1 -- 172.21.15.6:0/2686859931 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5651a1e0b850 con 0x5651a1eb3770 2024-09-12T03:26:22.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.927+0000 7fe5effff640 10 client.16008 handle_command_reply: tid=0 2024-09-12T03:26:22.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.927+0000 7fe618f0e740 1 client.16008 shutdown 2024-09-12T03:26:22.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.927+0000 7fe618f0e740 1 -- 172.21.15.6:0/2686859931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1e984a0 msgr2=0x5651a1ea2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:22.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.927+0000 7fe618f0e740 1 --2- 172.21.15.6:0/2686859931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1e984a0 0x5651a1ea2fe0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8000c00 tx=0x7fe5f800eba0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.927+0000 7fe618f0e740 1 -- 172.21.15.6:0/2686859931 shutdown_connections 2024-09-12T03:26:22.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.927+0000 7fe618f0e740 1 --2- 172.21.15.6:0/2686859931 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5651a1eb3770 0x5651a1eb5c30 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.927+0000 7fe5effff640 0 client.16008 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:26:22.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.927+0000 7fe618f0e740 1 --2- 172.21.15.6:0/2686859931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1ea3680 0x5651a1ea07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.927+0000 7fe618f0e740 1 --2- 172.21.15.6:0/2686859931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651a1e98dc0 0x5651a1ea0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.927+0000 7fe618f0e740 1 --2- 172.21.15.6:0/2686859931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1e984a0 0x5651a1ea2fe0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.927+0000 7fe618f0e740 1 -- 172.21.15.6:0/2686859931 >> 172.21.15.6:0/2686859931 conn(0x5651a1e94a30 msgr2=0x5651a1e9a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:22.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.927+0000 7fe618f0e740 1 -- 172.21.15.6:0/2686859931 shutdown_connections 2024-09-12T03:26:22.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.927+0000 7fe618f0e740 1 -- 172.21.15.6:0/2686859931 wait complete. 2024-09-12T03:26:22.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.928+0000 7fe618f0e740 20 client.16008 trim_cache size 0 max 16384 2024-09-12T03:26:22.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.939+0000 7fe60ffff640 1 --2- 172.21.15.6:0/3536676396 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5fc0a2b80 0x7fe5fc0a5040 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe600006010 tx=0x7fe600009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:22.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:22 smithi118 ceph-mon[35048]: pgmap v493: 97 pgs: 97 active+clean; 13 GiB data, 99 GiB used, 974 GiB / 1.0 TiB avail; 311 KiB/s rd, 180 MiB/s wr, 18.16k op/s 2024-09-12T03:26:22.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:22 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:26:22.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:22 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:22.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:22 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:22.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:22 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:26:22.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.969+0000 7fe618f0e740 1 Processor -- start 2024-09-12T03:26:22.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.969+0000 7fe618f0e740 1 -- start start 2024-09-12T03:26:22.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.970+0000 7fe618f0e740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651a1e984a0 0x5651a1e9eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.970+0000 7fe618f0e740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1e98dc0 0x5651a1ebb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.970+0000 7fe6148d1640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651a1e984a0 0x5651a1e9eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:22.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.970+0000 7fe6148d1640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651a1e984a0 0x5651a1e9eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:50848/0 (socket says 172.21.15.6:50848) 2024-09-12T03:26:22.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.970+0000 7fe618f0e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1ea3680 0x5651a1ebba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.971+0000 7fe60ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1e98dc0 0x5651a1ebb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:22.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.971+0000 7fe60ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1e98dc0 0x5651a1ebb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41522/0 (socket says 172.21.15.6:41522) 2024-09-12T03:26:22.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.971+0000 7fe6148d1640 1 -- 172.21.15.6:0/3622347797 learned_addr learned my addr 172.21.15.6:0/3622347797 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:26:22.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.971+0000 7fe6150d2640 1 --2- 172.21.15.6:0/3622347797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1ea3680 0x5651a1ebba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:22.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.971+0000 7fe618f0e740 1 -- 172.21.15.6:0/3622347797 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f0010f70 con 0x5651a1e98dc0 2024-09-12T03:26:22.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.971+0000 7fe618f0e740 1 -- 172.21.15.6:0/3622347797 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f000e840 con 0x5651a1e984a0 2024-09-12T03:26:22.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.972+0000 7fe618f0e740 1 -- 172.21.15.6:0/3622347797 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f000c630 con 0x5651a1ea3680 2024-09-12T03:26:22.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.972+0000 7fe6148d1640 1 -- 172.21.15.6:0/3622347797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1ea3680 msgr2=0x5651a1ebba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:22.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.972+0000 7fe6148d1640 1 --2- 172.21.15.6:0/3622347797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1ea3680 0x5651a1ebba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.972+0000 7fe6148d1640 1 -- 172.21.15.6:0/3622347797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1e98dc0 msgr2=0x5651a1ebb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:22.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.972+0000 7fe6148d1640 1 --2- 172.21.15.6:0/3622347797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1e98dc0 0x5651a1ebb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.972+0000 7fe6148d1640 1 -- 172.21.15.6:0/3622347797 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a1e9f280 con 0x5651a1e984a0 2024-09-12T03:26:22.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.973+0000 7fe6148d1640 1 --2- 172.21.15.6:0/3622347797 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651a1e984a0 0x5651a1e9eba0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe60809b380 tx=0x7fe6080d1050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=772e99a04c174c64 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:22.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.974+0000 7fe6067fc640 1 -- 172.21.15.6:0/3622347797 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608020040 con 0x5651a1e984a0 2024-09-12T03:26:22.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.974+0000 7fe6067fc640 1 -- 172.21.15.6:0/3622347797 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6080a4060 con 0x5651a1e984a0 2024-09-12T03:26:22.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.975+0000 7fe618f0e740 1 -- 172.21.15.6:0/3622347797 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651a1e984a0 msgr2=0x5651a1e9eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:22.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.975+0000 7fe618f0e740 1 --2- 172.21.15.6:0/3622347797 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651a1e984a0 0x5651a1e9eba0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe60809b380 tx=0x7fe6080d1050 comp rx=0 tx=0).stop 2024-09-12T03:26:22.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.975+0000 7fe618f0e740 1 -- 172.21.15.6:0/3622347797 shutdown_connections 2024-09-12T03:26:22.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.975+0000 7fe618f0e740 1 --2- 172.21.15.6:0/3622347797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1ea3680 0x5651a1ebba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.975+0000 7fe618f0e740 1 --2- 172.21.15.6:0/3622347797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1e98dc0 0x5651a1ebb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.975+0000 7fe618f0e740 1 --2- 172.21.15.6:0/3622347797 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651a1e984a0 0x5651a1e9eba0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.975+0000 7fe618f0e740 1 -- 172.21.15.6:0/3622347797 >> 172.21.15.6:0/3622347797 conn(0x5651a1e94a30 msgr2=0x5651a1ea2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:22.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.975+0000 7fe618f0e740 1 -- 172.21.15.6:0/3622347797 shutdown_connections 2024-09-12T03:26:22.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.975+0000 7fe618f0e740 1 -- 172.21.15.6:0/3622347797 wait complete. 2024-09-12T03:26:22.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.976+0000 7fe618f0e740 1 -- start start 2024-09-12T03:26:22.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.976+0000 7fe618f0e740 1 Processor -- start 2024-09-12T03:26:22.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.977+0000 7fe618f0e740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1e98dc0 0x5651a1eabc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.977+0000 7fe618f0e740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651a1ea3680 0x5651a1ea8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.977+0000 7fe618f0e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1ea9450 0x5651a1ea98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.977+0000 7fe618f0e740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f000cd00 con 0x5651a1e98dc0 2024-09-12T03:26:22.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.977+0000 7fe618f0e740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f000e170 con 0x5651a1ea3680 2024-09-12T03:26:22.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.977+0000 7fe618f0e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f000d3d0 con 0x5651a1ea9450 2024-09-12T03:26:22.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.977+0000 7fe618f0e740 10 client.0 _ll_register_callbacks cb 0x7fe600034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:22.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.977+0000 7fe6150d2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1ea9450 0x5651a1ea98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:22.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.977+0000 7fe6150d2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1ea9450 0x5651a1ea98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:56156/0 (socket says 172.21.15.6:56156) 2024-09-12T03:26:22.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.977+0000 7fe6150d2640 1 -- 172.21.15.6:0/4220699936 learned_addr learned my addr 172.21.15.6:0/4220699936 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:26:22.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.977+0000 7fe60ffff640 1 --2- 172.21.15.6:0/4220699936 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651a1ea3680 0x5651a1ea8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:22.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.978+0000 7fe6150d2640 1 -- 172.21.15.6:0/4220699936 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651a1ea3680 msgr2=0x5651a1ea8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:22.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.978+0000 7fe6150d2640 1 --2- 172.21.15.6:0/4220699936 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651a1ea3680 0x5651a1ea8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.978+0000 7fe6150d2640 1 -- 172.21.15.6:0/4220699936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1e98dc0 msgr2=0x5651a1eabc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:22.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.978+0000 7fe6150d2640 1 --2- 172.21.15.6:0/4220699936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1e98dc0 0x5651a1eabc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.978+0000 7fe6150d2640 1 -- 172.21.15.6:0/4220699936 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a1ea5de0 con 0x5651a1ea9450 2024-09-12T03:26:22.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.979+0000 7fe6150d2640 1 --2- 172.21.15.6:0/4220699936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1ea9450 0x5651a1ea98d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8029f40 tx=0x7fe5f8018c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45e176ad12e82349 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:22.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.979+0000 7fe5effff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:26:22.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.979+0000 7fe5effff640 1 -- 172.21.15.6:0/4220699936 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8004ad0 con 0x5651a1ea9450 2024-09-12T03:26:22.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.980+0000 7fe618f0e740 1 -- 172.21.15.6:0/4220699936 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5651a1e05d60 con 0x5651a1ea9450 2024-09-12T03:26:22.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.980+0000 7fe5effff640 1 -- 172.21.15.6:0/4220699936 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe5f802ae40 con 0x5651a1ea9450 2024-09-12T03:26:22.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.980+0000 7fe5effff640 1 -- 172.21.15.6:0/4220699936 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe5f800fbc0 con 0x5651a1ea9450 2024-09-12T03:26:22.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.980+0000 7fe618f0e740 10 client.25636 fetch_fsmap learned FSMap version 27 2024-09-12T03:26:22.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.980+0000 7fe618f0e740 1 -- 172.21.15.6:0/4220699936 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5651a1f1e590 con 0x5651a1ea9450 2024-09-12T03:26:22.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.981+0000 7fe5effff640 1 -- 172.21.15.6:0/4220699936 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5651a1e05d60 con 0x5651a1ea9450 2024-09-12T03:26:22.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.981+0000 7fe5effff640 1 -- 172.21.15.6:0/4220699936 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe5f802a030 con 0x5651a1ea9450 2024-09-12T03:26:22.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.981+0000 7fe618f0e740 10 client.25636 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:26:22.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.981+0000 7fe618f0e740 10 client.25636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:26:22.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.981+0000 7fe618f0e740 20 client.25636 populate_metadata read hostname 'smithi006' 2024-09-12T03:26:22.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.981+0000 7fe618f0e740 1 --2- 172.21.15.6:0/4220699936 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5651a1f1eac0 0x5651a1f20eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.981+0000 7fe618f0e740 4 client.25636 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:26:22.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.981+0000 7fe618f0e740 1 -- 172.21.15.6:0/4220699936 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5651a1d82210 con 0x5651a1f1eac0 2024-09-12T03:26:22.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.982+0000 7fe6148d1640 1 --2- 172.21.15.6:0/4220699936 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5651a1f1eac0 0x5651a1f20eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:22.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.983+0000 7fe5effff640 10 client.25636 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:26:22.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.984+0000 7fe6148d1640 1 --2- 172.21.15.6:0/4220699936 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5651a1f1eac0 0x5651a1f20eb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e34cb7eef6503360 server_cookie=1c78c33286cba2f4 in_seq=0 out_seq=0 2024-09-12T03:26:22.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.984+0000 7fe5effff640 1 -- 172.21.15.6:0/4220699936 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5651a1d82210 con 0x5651a1f1eac0 2024-09-12T03:26:22.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.984+0000 7fe5effff640 10 client.25636 handle_command_reply: tid=0 2024-09-12T03:26:22.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.984+0000 7fe618f0e740 1 client.25636 shutdown 2024-09-12T03:26:22.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.984+0000 7fe618f0e740 1 -- 172.21.15.6:0/4220699936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1ea9450 msgr2=0x5651a1ea98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:22.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.984+0000 7fe618f0e740 1 --2- 172.21.15.6:0/4220699936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1ea9450 0x5651a1ea98d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8029f40 tx=0x7fe5f8018c30 comp rx=0 tx=0).stop 2024-09-12T03:26:22.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.985+0000 7fe618f0e740 1 -- 172.21.15.6:0/4220699936 shutdown_connections 2024-09-12T03:26:22.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.985+0000 7fe618f0e740 1 --2- 172.21.15.6:0/4220699936 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5651a1f1eac0 0x5651a1f20eb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.985+0000 7fe618f0e740 1 --2- 172.21.15.6:0/4220699936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a1ea9450 0x5651a1ea98d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.985+0000 7fe618f0e740 1 --2- 172.21.15.6:0/4220699936 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651a1ea3680 0x5651a1ea8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.985+0000 7fe5effff640 0 client.25636 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:26:22.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.985+0000 7fe618f0e740 1 --2- 172.21.15.6:0/4220699936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5651a1e98dc0 0x5651a1eabc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.985+0000 7fe618f0e740 1 -- 172.21.15.6:0/4220699936 >> 172.21.15.6:0/4220699936 conn(0x5651a1e94a30 msgr2=0x5651a1e93370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:22.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.985+0000 7fe618f0e740 1 -- 172.21.15.6:0/4220699936 shutdown_connections 2024-09-12T03:26:22.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.985+0000 7fe618f0e740 1 -- 172.21.15.6:0/4220699936 wait complete. 2024-09-12T03:26:22.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.985+0000 7fe618f0e740 20 client.25636 trim_cache size 0 max 16384 2024-09-12T03:26:22.991 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:26:22.991 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "scrub active (125 inodes in the stack)", 2024-09-12T03:26:22.991 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2024-09-12T03:26:22.992 INFO:teuthology.orchestra.run.smithi006.stdout: "01562442-7858-449c-a9dd-3b0d01987c49": { 2024-09-12T03:26:22.992 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-09-12T03:26:22.992 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "01562442-7858-449c-a9dd-3b0d01987c49", 2024-09-12T03:26:22.992 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2024-09-12T03:26:22.992 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:26:22.992 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:26:22.992 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:26:22.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.992+0000 7fe616b35640 1 -- 172.21.15.6:0/3536676396 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5fc0a2b80 msgr2=0x7fe5fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:22.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.992+0000 7fe616b35640 1 --2- 172.21.15.6:0/3536676396 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5fc0a2b80 0x7fe5fc0a5040 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe600006010 tx=0x7fe600009570 comp rx=0 tx=0).stop 2024-09-12T03:26:22.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.992+0000 7fe616b35640 1 -- 172.21.15.6:0/3536676396 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe610109450 msgr2=0x7fe610118650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:22.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.992+0000 7fe616b35640 1 --2- 172.21.15.6:0/3536676396 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe610109450 0x7fe610118650 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe60800d950 tx=0x7fe60800de10 comp rx=0 tx=0).stop 2024-09-12T03:26:22.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.992+0000 7fe616b35640 1 -- 172.21.15.6:0/3536676396 shutdown_connections 2024-09-12T03:26:22.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.992+0000 7fe616b35640 1 --2- 172.21.15.6:0/3536676396 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe5fc0a2b80 0x7fe5fc0a5040 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.992+0000 7fe616b35640 1 --2- 172.21.15.6:0/3536676396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe61011b140 0x7fe6101118c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.992+0000 7fe616b35640 1 --2- 172.21.15.6:0/3536676396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe610109830 0x7fe610118b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.992+0000 7fe616b35640 1 --2- 172.21.15.6:0/3536676396 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe610109450 0x7fe610118650 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.992+0000 7fe616b35640 1 -- 172.21.15.6:0/3536676396 >> 172.21.15.6:0/3536676396 conn(0x7fe61006e140 msgr2=0x7fe61011be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:22.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.992+0000 7fe616b35640 1 -- 172.21.15.6:0/3536676396 shutdown_connections 2024-09-12T03:26:22.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:22.993+0000 7fe616b35640 1 -- 172.21.15.6:0/3536676396 wait complete. 2024-09-12T03:26:23.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:22 smithi161 ceph-mon[35485]: pgmap v493: 97 pgs: 97 active+clean; 13 GiB data, 99 GiB used, 974 GiB / 1.0 TiB avail; 311 KiB/s rd, 180 MiB/s wr, 18.16k op/s 2024-09-12T03:26:23.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:22 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:26:23.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:22 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:23.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:22 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:23.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:22 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:26:24.243 INFO:tasks.cephfs.filesystem:scrub status for tag:01562442-7858-449c-a9dd-3b0d01987c49 - {'path': '/', 'tag': '01562442-7858-449c-a9dd-3b0d01987c49', 'options': 'recursive,force'} 2024-09-12T03:26:24.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:24 smithi118 ceph-mon[35048]: pgmap v494: 97 pgs: 97 active+clean; 13 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 504 KiB/s rd, 214 MiB/s wr, 21.39k op/s 2024-09-12T03:26:25.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:24 smithi006 ceph-mon[30991]: pgmap v494: 97 pgs: 97 active+clean; 13 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 504 KiB/s rd, 214 MiB/s wr, 21.39k op/s 2024-09-12T03:26:25.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:24 smithi161 ceph-mon[35485]: pgmap v494: 97 pgs: 97 active+clean; 13 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 504 KiB/s rd, 214 MiB/s wr, 21.39k op/s 2024-09-12T03:26:26.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:26 smithi118 ceph-mon[35048]: pgmap v495: 97 pgs: 97 active+clean; 13 GiB data, 98 GiB used, 975 GiB / 1.0 TiB avail; 322 KiB/s rd, 172 MiB/s wr, 17.13k op/s 2024-09-12T03:26:27.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:26 smithi006 ceph-mon[30991]: pgmap v495: 97 pgs: 97 active+clean; 13 GiB data, 98 GiB used, 975 GiB / 1.0 TiB avail; 322 KiB/s rd, 172 MiB/s wr, 17.13k op/s 2024-09-12T03:26:27.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:26 smithi161 ceph-mon[35485]: pgmap v495: 97 pgs: 97 active+clean; 13 GiB data, 98 GiB used, 975 GiB / 1.0 TiB avail; 322 KiB/s rd, 172 MiB/s wr, 17.13k op/s 2024-09-12T03:26:28.965 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:28 smithi118 ceph-mon[35048]: pgmap v496: 97 pgs: 97 active+clean; 13 GiB data, 98 GiB used, 974 GiB / 1.0 TiB avail; 505 KiB/s rd, 197 MiB/s wr, 19.66k op/s 2024-09-12T03:26:29.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:28 smithi006 ceph-mon[30991]: pgmap v496: 97 pgs: 97 active+clean; 13 GiB data, 98 GiB used, 974 GiB / 1.0 TiB avail; 505 KiB/s rd, 197 MiB/s wr, 19.66k op/s 2024-09-12T03:26:29.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:28 smithi161 ceph-mon[35485]: pgmap v496: 97 pgs: 97 active+clean; 13 GiB data, 98 GiB used, 974 GiB / 1.0 TiB avail; 505 KiB/s rd, 197 MiB/s wr, 19.66k op/s 2024-09-12T03:26:30.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:30 smithi118 ceph-mon[35048]: pgmap v497: 97 pgs: 97 active+clean; 13 GiB data, 96 GiB used, 977 GiB / 1.0 TiB avail; 376 KiB/s rd, 172 MiB/s wr, 17.42k op/s 2024-09-12T03:26:31.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:30 smithi006 ceph-mon[30991]: pgmap v497: 97 pgs: 97 active+clean; 13 GiB data, 96 GiB used, 977 GiB / 1.0 TiB avail; 376 KiB/s rd, 172 MiB/s wr, 17.42k op/s 2024-09-12T03:26:31.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:30 smithi161 ceph-mon[35485]: pgmap v497: 97 pgs: 97 active+clean; 13 GiB data, 96 GiB used, 977 GiB / 1.0 TiB avail; 376 KiB/s rd, 172 MiB/s wr, 17.42k op/s 2024-09-12T03:26:33.117 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:32 smithi161 ceph-mon[35485]: pgmap v498: 97 pgs: 97 active+clean; 13 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 420 KiB/s rd, 183 MiB/s wr, 18.57k op/s 2024-09-12T03:26:33.118 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:32 smithi006 ceph-mon[30991]: pgmap v498: 97 pgs: 97 active+clean; 13 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 420 KiB/s rd, 183 MiB/s wr, 18.57k op/s 2024-09-12T03:26:33.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:32 smithi118 ceph-mon[35048]: pgmap v498: 97 pgs: 97 active+clean; 13 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 420 KiB/s rd, 183 MiB/s wr, 18.57k op/s 2024-09-12T03:26:35.014 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:34 smithi006 ceph-mon[30991]: pgmap v499: 97 pgs: 97 active+clean; 13 GiB data, 98 GiB used, 975 GiB / 1.0 TiB avail; 501 KiB/s rd, 211 MiB/s wr, 21.47k op/s 2024-09-12T03:26:35.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:34 smithi161 ceph-mon[35485]: pgmap v499: 97 pgs: 97 active+clean; 13 GiB data, 98 GiB used, 975 GiB / 1.0 TiB avail; 501 KiB/s rd, 211 MiB/s wr, 21.47k op/s 2024-09-12T03:26:35.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:34 smithi118 ceph-mon[35048]: pgmap v499: 97 pgs: 97 active+clean; 13 GiB data, 98 GiB used, 975 GiB / 1.0 TiB avail; 501 KiB/s rd, 211 MiB/s wr, 21.47k op/s 2024-09-12T03:26:37.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:36 smithi006 ceph-mon[30991]: pgmap v500: 97 pgs: 97 active+clean; 13 GiB data, 99 GiB used, 974 GiB / 1.0 TiB avail; 606 KiB/s rd, 165 MiB/s wr, 17.18k op/s 2024-09-12T03:26:37.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:36 smithi161 ceph-mon[35485]: pgmap v500: 97 pgs: 97 active+clean; 13 GiB data, 99 GiB used, 974 GiB / 1.0 TiB avail; 606 KiB/s rd, 165 MiB/s wr, 17.18k op/s 2024-09-12T03:26:37.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:36 smithi118 ceph-mon[35048]: pgmap v500: 97 pgs: 97 active+clean; 13 GiB data, 99 GiB used, 974 GiB / 1.0 TiB avail; 606 KiB/s rd, 165 MiB/s wr, 17.18k op/s 2024-09-12T03:26:39.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:38 smithi006 ceph-mon[30991]: pgmap v501: 97 pgs: 97 active+clean; 13 GiB data, 99 GiB used, 973 GiB / 1.0 TiB avail; 606 KiB/s rd, 185 MiB/s wr, 19.11k op/s 2024-09-12T03:26:39.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:38 smithi161 ceph-mon[35485]: pgmap v501: 97 pgs: 97 active+clean; 13 GiB data, 99 GiB used, 973 GiB / 1.0 TiB avail; 606 KiB/s rd, 185 MiB/s wr, 19.11k op/s 2024-09-12T03:26:39.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:38 smithi118 ceph-mon[35048]: pgmap v501: 97 pgs: 97 active+clean; 13 GiB data, 99 GiB used, 973 GiB / 1.0 TiB avail; 606 KiB/s rd, 185 MiB/s wr, 19.11k op/s 2024-09-12T03:26:41.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:40 smithi006 ceph-mon[30991]: pgmap v502: 97 pgs: 97 active+clean; 13 GiB data, 100 GiB used, 973 GiB / 1.0 TiB avail; 424 KiB/s rd, 184 MiB/s wr, 19.10k op/s 2024-09-12T03:26:41.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:40 smithi161 ceph-mon[35485]: pgmap v502: 97 pgs: 97 active+clean; 13 GiB data, 100 GiB used, 973 GiB / 1.0 TiB avail; 424 KiB/s rd, 184 MiB/s wr, 19.10k op/s 2024-09-12T03:26:41.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:40 smithi118 ceph-mon[35048]: pgmap v502: 97 pgs: 97 active+clean; 13 GiB data, 100 GiB used, 973 GiB / 1.0 TiB avail; 424 KiB/s rd, 184 MiB/s wr, 19.10k op/s 2024-09-12T03:26:43.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:42 smithi006 ceph-mon[30991]: pgmap v503: 97 pgs: 97 active+clean; 13 GiB data, 101 GiB used, 972 GiB / 1.0 TiB avail; 441 KiB/s rd, 183 MiB/s wr, 19.01k op/s 2024-09-12T03:26:43.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:42 smithi161 ceph-mon[35485]: pgmap v503: 97 pgs: 97 active+clean; 13 GiB data, 101 GiB used, 972 GiB / 1.0 TiB avail; 441 KiB/s rd, 183 MiB/s wr, 19.01k op/s 2024-09-12T03:26:43.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:42 smithi118 ceph-mon[35048]: pgmap v503: 97 pgs: 97 active+clean; 13 GiB data, 101 GiB used, 972 GiB / 1.0 TiB avail; 441 KiB/s rd, 183 MiB/s wr, 19.01k op/s 2024-09-12T03:26:45.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:44 smithi161 ceph-mon[35485]: pgmap v504: 97 pgs: 97 active+clean; 13 GiB data, 102 GiB used, 971 GiB / 1.0 TiB avail; 659 KiB/s rd, 215 MiB/s wr, 22.37k op/s 2024-09-12T03:26:45.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:44 smithi006 ceph-mon[30991]: pgmap v504: 97 pgs: 97 active+clean; 13 GiB data, 102 GiB used, 971 GiB / 1.0 TiB avail; 659 KiB/s rd, 215 MiB/s wr, 22.37k op/s 2024-09-12T03:26:45.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:44 smithi118 ceph-mon[35048]: pgmap v504: 97 pgs: 97 active+clean; 13 GiB data, 102 GiB used, 971 GiB / 1.0 TiB avail; 659 KiB/s rd, 215 MiB/s wr, 22.37k op/s 2024-09-12T03:26:47.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:46 smithi006 ceph-mon[30991]: pgmap v505: 97 pgs: 97 active+clean; 13 GiB data, 102 GiB used, 971 GiB / 1.0 TiB avail; 663 KiB/s rd, 167 MiB/s wr, 17.25k op/s 2024-09-12T03:26:47.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:46 smithi161 ceph-mon[35485]: pgmap v505: 97 pgs: 97 active+clean; 13 GiB data, 102 GiB used, 971 GiB / 1.0 TiB avail; 663 KiB/s rd, 167 MiB/s wr, 17.25k op/s 2024-09-12T03:26:47.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:46 smithi118 ceph-mon[35048]: pgmap v505: 97 pgs: 97 active+clean; 13 GiB data, 102 GiB used, 971 GiB / 1.0 TiB avail; 663 KiB/s rd, 167 MiB/s wr, 17.25k op/s 2024-09-12T03:26:49.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:48 smithi006 ceph-mon[30991]: pgmap v506: 97 pgs: 97 active+clean; 13 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 366 KiB/s rd, 186 MiB/s wr, 19.30k op/s 2024-09-12T03:26:49.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:48 smithi161 ceph-mon[35485]: pgmap v506: 97 pgs: 97 active+clean; 13 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 366 KiB/s rd, 186 MiB/s wr, 19.30k op/s 2024-09-12T03:26:49.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:48 smithi118 ceph-mon[35048]: pgmap v506: 97 pgs: 97 active+clean; 13 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 366 KiB/s rd, 186 MiB/s wr, 19.30k op/s 2024-09-12T03:26:51.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:50 smithi006 ceph-mon[30991]: pgmap v507: 97 pgs: 97 active+clean; 13 GiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 366 KiB/s rd, 200 MiB/s wr, 20.89k op/s 2024-09-12T03:26:51.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:50 smithi161 ceph-mon[35485]: pgmap v507: 97 pgs: 97 active+clean; 13 GiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 366 KiB/s rd, 200 MiB/s wr, 20.89k op/s 2024-09-12T03:26:51.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:50 smithi118 ceph-mon[35048]: pgmap v507: 97 pgs: 97 active+clean; 13 GiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 366 KiB/s rd, 200 MiB/s wr, 20.89k op/s 2024-09-12T03:26:53.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:52 smithi006 ceph-mon[30991]: pgmap v508: 97 pgs: 97 active+clean; 13 GiB data, 105 GiB used, 968 GiB / 1.0 TiB avail; 606 KiB/s rd, 173 MiB/s wr, 17.97k op/s 2024-09-12T03:26:53.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:52 smithi161 ceph-mon[35485]: pgmap v508: 97 pgs: 97 active+clean; 13 GiB data, 105 GiB used, 968 GiB / 1.0 TiB avail; 606 KiB/s rd, 173 MiB/s wr, 17.97k op/s 2024-09-12T03:26:53.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:52 smithi118 ceph-mon[35048]: pgmap v508: 97 pgs: 97 active+clean; 13 GiB data, 105 GiB used, 968 GiB / 1.0 TiB avail; 606 KiB/s rd, 173 MiB/s wr, 17.97k op/s 2024-09-12T03:26:54.244 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:26:54.730 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-12T03:26:54.731 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:26:54.816 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:55.106 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:54 smithi006 ceph-mon[30991]: pgmap v509: 97 pgs: 97 active+clean; 13 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 724 KiB/s rd, 212 MiB/s wr, 21.81k op/s 2024-09-12T03:26:55.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:54 smithi161 ceph-mon[35485]: pgmap v509: 97 pgs: 97 active+clean; 13 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 724 KiB/s rd, 212 MiB/s wr, 21.81k op/s 2024-09-12T03:26:55.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:54 smithi118 ceph-mon[35048]: pgmap v509: 97 pgs: 97 active+clean; 13 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 724 KiB/s rd, 212 MiB/s wr, 21.81k op/s 2024-09-12T03:26:55.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.902+0000 7f938a392640 1 -- 172.21.15.6:0/4235756945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9384101270 msgr2=0x7f938410d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:55.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.902+0000 7f938a392640 1 --2- 172.21.15.6:0/4235756945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9384101270 0x7f938410d7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f938001a940 tx=0x7f938003b450 comp rx=0 tx=0).stop 2024-09-12T03:26:55.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.902+0000 7f938a392640 1 -- 172.21.15.6:0/4235756945 shutdown_connections 2024-09-12T03:26:55.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.902+0000 7f938a392640 1 --2- 172.21.15.6:0/4235756945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9384101270 0x7f938410d7a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.902+0000 7f938a392640 1 --2- 172.21.15.6:0/4235756945 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93841008d0 0x7f9384100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.902+0000 7f938a392640 1 --2- 172.21.15.6:0/4235756945 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93841068d0 0x7f9384106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.902+0000 7f938a392640 1 -- 172.21.15.6:0/4235756945 >> 172.21.15.6:0/4235756945 conn(0x7f93840fc610 msgr2=0x7f93840fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:55.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.902+0000 7f938a392640 1 -- 172.21.15.6:0/4235756945 shutdown_connections 2024-09-12T03:26:55.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.902+0000 7f938a392640 1 -- 172.21.15.6:0/4235756945 wait complete. 2024-09-12T03:26:55.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.903+0000 7f938a392640 1 Processor -- start 2024-09-12T03:26:55.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.903+0000 7f938a392640 1 -- start start 2024-09-12T03:26:55.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.903+0000 7f938a392640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93841008d0 0x7f9384073e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:55.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.903+0000 7f938a392640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9384101270 0x7f938406dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:55.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.903+0000 7f938a392640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93841068d0 0x7f938406e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:55.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.903+0000 7f938a392640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f938406c640 con 0x7f9384101270 2024-09-12T03:26:55.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.904+0000 7f938a392640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f938406c4c0 con 0x7f93841008d0 2024-09-12T03:26:55.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.904+0000 7f938a392640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f938406c7c0 con 0x7f93841068d0 2024-09-12T03:26:55.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.904+0000 7f9389390640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93841008d0 0x7f9384073e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:55.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.904+0000 7f9389390640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93841008d0 0x7f9384073e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:59024/0 (socket says 172.21.15.6:59024) 2024-09-12T03:26:55.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.904+0000 7f9389390640 1 -- 172.21.15.6:0/1787768983 learned_addr learned my addr 172.21.15.6:0/1787768983 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:26:55.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.904+0000 7f9388b8f640 1 --2- 172.21.15.6:0/1787768983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9384101270 0x7f938406dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:55.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.904+0000 7f9389b91640 1 --2- 172.21.15.6:0/1787768983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93841068d0 0x7f938406e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:55.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.904+0000 7f9389390640 1 -- 172.21.15.6:0/1787768983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93841068d0 msgr2=0x7f938406e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:55.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.904+0000 7f9389390640 1 --2- 172.21.15.6:0/1787768983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93841068d0 0x7f938406e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.904+0000 7f9389390640 1 -- 172.21.15.6:0/1787768983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9384101270 msgr2=0x7f938406dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:55.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.904+0000 7f9389390640 1 --2- 172.21.15.6:0/1787768983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9384101270 0x7f938406dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.904+0000 7f9389390640 1 -- 172.21.15.6:0/1787768983 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938406eb00 con 0x7f93841008d0 2024-09-12T03:26:55.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.904+0000 7f9389b91640 1 --2- 172.21.15.6:0/1787768983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93841068d0 0x7f938406e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:55.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.908+0000 7f9389390640 1 --2- 172.21.15.6:0/1787768983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93841008d0 0x7f9384073e40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f937c00ba50 tx=0x7f937c00be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:55.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.911+0000 7f937a7fc640 1 -- 172.21.15.6:0/1787768983 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c019070 con 0x7f93841008d0 2024-09-12T03:26:55.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.911+0000 7f937a7fc640 1 -- 172.21.15.6:0/1787768983 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f937c014440 con 0x7f93841008d0 2024-09-12T03:26:55.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.911+0000 7f937a7fc640 1 -- 172.21.15.6:0/1787768983 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c0075d0 con 0x7f93841008d0 2024-09-12T03:26:55.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.913+0000 7f938a392640 1 -- 172.21.15.6:0/1787768983 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93841151e0 con 0x7f93841008d0 2024-09-12T03:26:55.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.913+0000 7f938a392640 1 -- 172.21.15.6:0/1787768983 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9384115640 con 0x7f93841008d0 2024-09-12T03:26:55.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.917+0000 7f938d9cd740 1 Processor -- start 2024-09-12T03:26:55.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.917+0000 7f938d9cd740 1 -- start start 2024-09-12T03:26:55.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.917+0000 7f938d9cd740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d3061f8fa0 0x55d3061f7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:55.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.917+0000 7f938d9cd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3061f7cb0 0x55d3061f80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:55.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.917+0000 7f938d9cd740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d306202610 0x55d306204a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:55.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.917+0000 7f938d9cd740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55d30615d2e0 con 0x55d3061f8fa0 2024-09-12T03:26:55.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.917+0000 7f938d9cd740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55d306104ce0 con 0x55d306202610 2024-09-12T03:26:55.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.917+0000 7f938d9cd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d3061592a0 con 0x55d3061f7cb0 2024-09-12T03:26:55.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.919+0000 7f937a7fc640 1 -- 172.21.15.6:0/1787768983 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f937c0145f0 con 0x7f93841008d0 2024-09-12T03:26:55.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.920+0000 7f937a7fc640 1 --2- 172.21.15.6:0/1787768983 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f93700a2d90 0x7f93700a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:55.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.921+0000 7f937a7fc640 1 -- 172.21.15.6:0/1787768983 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f937c0ce4b0 con 0x7f93841008d0 2024-09-12T03:26:55.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.921+0000 7f9389b91640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d3061f8fa0 0x55d3061f7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:55.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.921+0000 7f9389b91640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d3061f8fa0 0x55d3061f7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50736/0 (socket says 172.21.15.6:50736) 2024-09-12T03:26:55.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.921+0000 7f9389b91640 1 -- 172.21.15.6:0/2509281948 learned_addr learned my addr 172.21.15.6:0/2509281948 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:26:55.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.921+0000 7f9389390640 1 --2- 172.21.15.6:0/2509281948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3061f7cb0 0x55d3061f80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:55.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.921+0000 7f9388b8f640 1 --2- 172.21.15.6:0/2509281948 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d306202610 0x55d306204a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:55.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.921+0000 7f9389b91640 1 -- 172.21.15.6:0/2509281948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3061f7cb0 msgr2=0x55d3061f80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:55.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.921+0000 7f9389b91640 1 --2- 172.21.15.6:0/2509281948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3061f7cb0 0x55d3061f80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.921+0000 7f9389b91640 1 -- 172.21.15.6:0/2509281948 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d306202610 msgr2=0x55d306204a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:55.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.921+0000 7f9389b91640 1 --2- 172.21.15.6:0/2509281948 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d306202610 0x55d306204a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.921+0000 7f9389b91640 1 -- 172.21.15.6:0/2509281948 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3060e9440 con 0x55d3061f8fa0 2024-09-12T03:26:55.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.921+0000 7f9389b91640 1 --2- 172.21.15.6:0/2509281948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d3061f8fa0 0x55d3061f7770 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f938001a940 tx=0x7f93800052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d976f4a6bb99aa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:55.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.921+0000 7f9389b91640 1 --2- 172.21.15.6:0/1787768983 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f93700a2d90 0x7f93700a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:55.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.923+0000 7f938a392640 1 -- 172.21.15.6:0/2509281948 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380046030 con 0x55d3061f8fa0 2024-09-12T03:26:55.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.923+0000 7f938a392640 1 -- 172.21.15.6:0/2509281948 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9380004d60 con 0x55d3061f8fa0 2024-09-12T03:26:55.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.923+0000 7f938a392640 1 -- 172.21.15.6:0/2509281948 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380042660 con 0x55d3061f8fa0 2024-09-12T03:26:55.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.925+0000 7f938d9cd740 1 -- 172.21.15.6:0/2509281948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d3061f8fa0 msgr2=0x55d3061f7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:55.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.925+0000 7f938d9cd740 1 --2- 172.21.15.6:0/2509281948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d3061f8fa0 0x55d3061f7770 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f938001a940 tx=0x7f93800052e0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.926+0000 7f938d9cd740 1 -- 172.21.15.6:0/2509281948 shutdown_connections 2024-09-12T03:26:55.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.926+0000 7f938d9cd740 1 --2- 172.21.15.6:0/2509281948 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d306202610 0x55d306204a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.926+0000 7f938d9cd740 1 --2- 172.21.15.6:0/2509281948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3061f7cb0 0x55d3061f80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.926+0000 7f938d9cd740 1 --2- 172.21.15.6:0/2509281948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d3061f8fa0 0x55d3061f7770 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.926+0000 7f938d9cd740 1 -- 172.21.15.6:0/2509281948 >> 172.21.15.6:0/2509281948 conn(0x55d3061f3f00 msgr2=0x55d3061f42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:55.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.926+0000 7f938d9cd740 1 -- 172.21.15.6:0/2509281948 shutdown_connections 2024-09-12T03:26:55.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.926+0000 7f9389b91640 1 --2- 172.21.15.6:0/1787768983 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f93700a2d90 0x7f93700a5250 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9380058000 tx=0x7f9380059000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:55.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.926+0000 7f938d9cd740 1 -- 172.21.15.6:0/2509281948 wait complete. 2024-09-12T03:26:55.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.927+0000 7f938d9cd740 1 -- start start 2024-09-12T03:26:55.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.927+0000 7f938d9cd740 1 Processor -- start 2024-09-12T03:26:55.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.928+0000 7f938d9cd740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d3061f7cb0 0x55d306201f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:55.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.928+0000 7f938d9cd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3061f8fa0 0x55d3061ff210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:55.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.928+0000 7f938d9cd740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d306202610 0x55d3061ff770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:55.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.929+0000 7f938d9cd740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55d306205480 con 0x55d306202610 2024-09-12T03:26:55.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.929+0000 7f938d9cd740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55d306118050 con 0x55d3061f7cb0 2024-09-12T03:26:55.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.929+0000 7f938d9cd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d306205600 con 0x55d3061f8fa0 2024-09-12T03:26:55.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.929+0000 7f938d9cd740 10 client.0 _ll_register_callbacks cb 0x55d306158160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:55.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.930+0000 7f9389b91640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d306202610 0x55d3061ff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:55.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.930+0000 7f9389b91640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d306202610 0x55d3061ff770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50742/0 (socket says 172.21.15.6:50742) 2024-09-12T03:26:55.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.930+0000 7f9389b91640 1 -- 172.21.15.6:0/3424840851 learned_addr learned my addr 172.21.15.6:0/3424840851 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:26:55.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.930+0000 7f9388b8f640 1 --2- 172.21.15.6:0/3424840851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3061f8fa0 0x55d3061ff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:55.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.930+0000 7f9389390640 1 --2- 172.21.15.6:0/3424840851 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d3061f7cb0 0x55d306201f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:55.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.934+0000 7f9388b8f640 1 -- 172.21.15.6:0/3424840851 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d3061f7cb0 msgr2=0x55d306201f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:55.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.934+0000 7f9388b8f640 1 --2- 172.21.15.6:0/3424840851 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d3061f7cb0 0x55d306201f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.934+0000 7f9388b8f640 1 -- 172.21.15.6:0/3424840851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d306202610 msgr2=0x55d3061ff770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:55.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.934+0000 7f9388b8f640 1 --2- 172.21.15.6:0/3424840851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d306202610 0x55d3061ff770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.934+0000 7f9388b8f640 1 -- 172.21.15.6:0/3424840851 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d305ece3c0 con 0x55d3061f8fa0 2024-09-12T03:26:55.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.935+0000 7f9389390640 1 --2- 172.21.15.6:0/3424840851 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d3061f7cb0 0x55d306201f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:55.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.937+0000 7f9388b8f640 1 --2- 172.21.15.6:0/3424840851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3061f8fa0 0x55d3061ff210 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f938406f680 tx=0x7f9374012bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63aad429b9f43ef9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:55.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.937+0000 7f936cff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:26:55.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.939+0000 7f936cff9640 1 -- 172.21.15.6:0/3424840851 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937401f060 con 0x55d3061f8fa0 2024-09-12T03:26:55.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.939+0000 7f936cff9640 1 -- 172.21.15.6:0/3424840851 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f9374010d90 con 0x55d3061f8fa0 2024-09-12T03:26:55.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.939+0000 7f938d9cd740 1 -- 172.21.15.6:0/3424840851 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d30616a850 con 0x55d3061f8fa0 2024-09-12T03:26:55.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.939+0000 7f936cff9640 1 -- 172.21.15.6:0/3424840851 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f937401a430 con 0x55d3061f8fa0 2024-09-12T03:26:55.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.940+0000 7f938d9cd740 10 client.25645 fetch_fsmap learned FSMap version 27 2024-09-12T03:26:55.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.940+0000 7f938d9cd740 1 -- 172.21.15.6:0/3424840851 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d306210ef0 con 0x55d3061f8fa0 2024-09-12T03:26:55.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.941+0000 7f936cff9640 1 -- 172.21.15.6:0/3424840851 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d30616a850 con 0x55d3061f8fa0 2024-09-12T03:26:55.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.941+0000 7f936cff9640 1 -- 172.21.15.6:0/3424840851 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f93740235d0 con 0x55d3061f8fa0 2024-09-12T03:26:55.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.941+0000 7f938d9cd740 10 client.25645 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:26:55.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.941+0000 7f938d9cd740 10 client.25645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:26:55.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.941+0000 7f938d9cd740 20 client.25645 populate_metadata read hostname 'smithi006' 2024-09-12T03:26:55.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.941+0000 7f938d9cd740 1 --2- 172.21.15.6:0/3424840851 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55d306212300 0x55d306214740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:55.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.941+0000 7f938d9cd740 4 client.25645 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:26:55.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.941+0000 7f938d9cd740 1 -- 172.21.15.6:0/3424840851 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d306164d60 con 0x55d306212300 2024-09-12T03:26:55.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.941+0000 7f9389390640 1 --2- 172.21.15.6:0/3424840851 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55d306212300 0x55d306214740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:55.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.942+0000 7f9389390640 1 --2- 172.21.15.6:0/3424840851 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55d306212300 0x55d306214740 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0d578aada77ea0 server_cookie=47c27bad43831a9 in_seq=0 out_seq=0 2024-09-12T03:26:55.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.942+0000 7f936cff9640 10 client.25645 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:26:55.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.945+0000 7f936cff9640 1 -- 172.21.15.6:0/3424840851 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d306164d60 con 0x55d306212300 2024-09-12T03:26:55.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.945+0000 7f936cff9640 10 client.25645 handle_command_reply: tid=0 2024-09-12T03:26:55.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.945+0000 7f938d9cd740 1 client.25645 shutdown 2024-09-12T03:26:55.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.945+0000 7f938d9cd740 1 -- 172.21.15.6:0/3424840851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3061f8fa0 msgr2=0x55d3061ff210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:55.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.945+0000 7f938d9cd740 1 --2- 172.21.15.6:0/3424840851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3061f8fa0 0x55d3061ff210 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f938406f680 tx=0x7f9374012bf0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.948+0000 7f938d9cd740 1 -- 172.21.15.6:0/3424840851 shutdown_connections 2024-09-12T03:26:55.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.948+0000 7f938d9cd740 1 --2- 172.21.15.6:0/3424840851 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55d306212300 0x55d306214740 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.948+0000 7f938d9cd740 1 --2- 172.21.15.6:0/3424840851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d306202610 0x55d3061ff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.948+0000 7f938d9cd740 1 --2- 172.21.15.6:0/3424840851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3061f8fa0 0x55d3061ff210 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.948+0000 7f938d9cd740 1 --2- 172.21.15.6:0/3424840851 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d3061f7cb0 0x55d306201f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.949+0000 7f938d9cd740 1 -- 172.21.15.6:0/3424840851 >> 172.21.15.6:0/3424840851 conn(0x55d3061f3f00 msgr2=0x55d3061f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:55.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.949+0000 7f936cff9640 0 client.25645 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:26:55.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.950+0000 7f938d9cd740 1 -- 172.21.15.6:0/3424840851 shutdown_connections 2024-09-12T03:26:55.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.951+0000 7f938d9cd740 1 -- 172.21.15.6:0/3424840851 wait complete. 2024-09-12T03:26:55.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.952+0000 7f938d9cd740 20 client.25645 trim_cache size 0 max 16384 2024-09-12T03:26:55.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.994+0000 7f938d9cd740 1 Processor -- start 2024-09-12T03:26:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.994+0000 7f938d9cd740 1 -- start start 2024-09-12T03:26:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.994+0000 7f938d9cd740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d3061f7cb0 0x55d3061f3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:55.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.994+0000 7f938d9cd740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d3061f8fa0 0x55d3061fe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:55.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.994+0000 7f938d9cd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d306202610 0x55d3061fa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:55.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.994+0000 7f938d9cd740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f93841a5240 con 0x55d3061f8fa0 2024-09-12T03:26:55.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.994+0000 7f938d9cd740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f938411de00 con 0x55d3061f7cb0 2024-09-12T03:26:55.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.994+0000 7f938d9cd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f938411bc50 con 0x55d306202610 2024-09-12T03:26:55.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.995+0000 7f9389390640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d3061f7cb0 0x55d3061f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:55.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.995+0000 7f9389b91640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d306202610 0x55d3061fa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:55.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.995+0000 7f9389390640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d3061f7cb0 0x55d3061f3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:59064/0 (socket says 172.21.15.6:59064) 2024-09-12T03:26:55.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.995+0000 7f9389b91640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d306202610 0x55d3061fa420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:58620/0 (socket says 172.21.15.6:58620) 2024-09-12T03:26:55.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.995+0000 7f9389390640 1 -- 172.21.15.6:0/3048891682 learned_addr learned my addr 172.21.15.6:0/3048891682 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:26:55.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.995+0000 7f9388b8f640 1 --2- 172.21.15.6:0/3048891682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d3061f8fa0 0x55d3061fe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:56.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.995+0000 7f9389b91640 1 -- 172.21.15.6:0/3048891682 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d3061f7cb0 msgr2=0x55d3061f3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:56.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.995+0000 7f9389b91640 1 --2- 172.21.15.6:0/3048891682 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d3061f7cb0 0x55d3061f3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.995+0000 7f9389b91640 1 -- 172.21.15.6:0/3048891682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d3061f8fa0 msgr2=0x55d3061fe1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:56.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.995+0000 7f9389b91640 1 --2- 172.21.15.6:0/3048891682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d3061f8fa0 0x55d3061fe1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.995+0000 7f9389b91640 1 -- 172.21.15.6:0/3048891682 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3062022d0 con 0x55d306202610 2024-09-12T03:26:56.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.996+0000 7f9389b91640 1 --2- 172.21.15.6:0/3048891682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d306202610 0x55d3061fa420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f938001a940 tx=0x7f938003b450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5d0ab3db5f5b625 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:56.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.996+0000 7f936effd640 1 -- 172.21.15.6:0/3048891682 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380046040 con 0x55d306202610 2024-09-12T03:26:56.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.996+0000 7f936effd640 1 -- 172.21.15.6:0/3048891682 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93800046c0 con 0x55d306202610 2024-09-12T03:26:56.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.998+0000 7f938d9cd740 1 -- 172.21.15.6:0/3048891682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d306202610 msgr2=0x55d3061fa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:56.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.998+0000 7f938d9cd740 1 --2- 172.21.15.6:0/3048891682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d306202610 0x55d3061fa420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f938001a940 tx=0x7f938003b450 comp rx=0 tx=0).stop 2024-09-12T03:26:56.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.998+0000 7f938d9cd740 1 -- 172.21.15.6:0/3048891682 shutdown_connections 2024-09-12T03:26:56.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.998+0000 7f938d9cd740 1 --2- 172.21.15.6:0/3048891682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d306202610 0x55d3061fa420 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.998+0000 7f938d9cd740 1 --2- 172.21.15.6:0/3048891682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d3061f8fa0 0x55d3061fe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.998+0000 7f938d9cd740 1 --2- 172.21.15.6:0/3048891682 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d3061f7cb0 0x55d3061f3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.998+0000 7f938d9cd740 1 -- 172.21.15.6:0/3048891682 >> 172.21.15.6:0/3048891682 conn(0x55d3061f3f00 msgr2=0x55d3061fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:56.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.998+0000 7f938d9cd740 1 -- 172.21.15.6:0/3048891682 shutdown_connections 2024-09-12T03:26:56.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.999+0000 7f938d9cd740 1 -- 172.21.15.6:0/3048891682 wait complete. 2024-09-12T03:26:56.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.999+0000 7f938d9cd740 1 -- start start 2024-09-12T03:26:56.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:55.999+0000 7f938d9cd740 1 Processor -- start 2024-09-12T03:26:56.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.000+0000 7f938d9cd740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d3061f7cb0 0x55d30620b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:56.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.000+0000 7f938d9cd740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d3061f8fa0 0x55d3062064c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:56.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.000+0000 7f938d9cd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d306202610 0x55d306206c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:56.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.000+0000 7f938d9cd740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f938411c2c0 con 0x55d3061f8fa0 2024-09-12T03:26:56.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.000+0000 7f938d9cd740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f938411d730 con 0x55d3061f7cb0 2024-09-12T03:26:56.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.000+0000 7f938d9cd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f938411c990 con 0x55d306202610 2024-09-12T03:26:56.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.000+0000 7f938d9cd740 10 client.0 _ll_register_callbacks cb 0x7f937400a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:56.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.000+0000 7f9388b8f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d3061f8fa0 0x55d3062064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:56.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.000+0000 7f9389390640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d3061f7cb0 0x55d30620b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:56.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.000+0000 7f9389390640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d3061f7cb0 0x55d30620b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:59070/0 (socket says 172.21.15.6:59070) 2024-09-12T03:26:56.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.001+0000 7f9389390640 1 -- 172.21.15.6:0/2966291158 learned_addr learned my addr 172.21.15.6:0/2966291158 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:26:56.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.001+0000 7f9388b8f640 1 -- 172.21.15.6:0/2966291158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d306202610 msgr2=0x55d306206c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:56.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.001+0000 7f9388b8f640 1 --2- 172.21.15.6:0/2966291158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d306202610 0x55d306206c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.001+0000 7f9388b8f640 1 -- 172.21.15.6:0/2966291158 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d3061f7cb0 msgr2=0x55d30620b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:56.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.001+0000 7f9388b8f640 1 --2- 172.21.15.6:0/2966291158 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d3061f7cb0 0x55d30620b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.001+0000 7f9388b8f640 1 -- 172.21.15.6:0/2966291158 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d30627bfe0 con 0x55d3061f8fa0 2024-09-12T03:26:56.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.001+0000 7f9388b8f640 1 --2- 172.21.15.6:0/2966291158 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d3061f8fa0 0x55d3062064c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x55d3062007c0 tx=0x7f93740122f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf92d9bd85ec3e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:56.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.001+0000 7f936cff9640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:26:56.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.002+0000 7f936cff9640 1 -- 172.21.15.6:0/2966291158 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374031020 con 0x55d3061f8fa0 2024-09-12T03:26:56.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.002+0000 7f938d9cd740 1 -- 172.21.15.6:0/2966291158 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d3060e1210 con 0x55d3061f8fa0 2024-09-12T03:26:56.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.002+0000 7f936cff9640 1 -- 172.21.15.6:0/2966291158 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f93740233a0 con 0x55d3061f8fa0 2024-09-12T03:26:56.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.003+0000 7f936cff9640 1 -- 172.21.15.6:0/2966291158 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f937401f020 con 0x55d3061f8fa0 2024-09-12T03:26:56.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.004+0000 7f938d9cd740 10 client.16026 fetch_fsmap learned FSMap version 27 2024-09-12T03:26:56.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.004+0000 7f938d9cd740 1 -- 172.21.15.6:0/2966291158 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d30627d240 con 0x55d3061f8fa0 2024-09-12T03:26:56.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.005+0000 7f936cff9640 1 -- 172.21.15.6:0/2966291158 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d3060e1210 con 0x55d3061f8fa0 2024-09-12T03:26:56.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.006+0000 7f936cff9640 1 -- 172.21.15.6:0/2966291158 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9374004660 con 0x55d3061f8fa0 2024-09-12T03:26:56.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.006+0000 7f938d9cd740 10 client.16026 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:26:56.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.006+0000 7f938d9cd740 10 client.16026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:26:56.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.006+0000 7f938d9cd740 20 client.16026 populate_metadata read hostname 'smithi006' 2024-09-12T03:26:56.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.006+0000 7f938d9cd740 1 --2- 172.21.15.6:0/2966291158 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55d30627d950 0x55d30627fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:56.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.006+0000 7f938d9cd740 4 client.16026 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:26:56.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.006+0000 7f938d9cd740 1 -- 172.21.15.6:0/2966291158 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d30612e9a0 con 0x55d30627d950 2024-09-12T03:26:56.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.022+0000 7f9389390640 1 --2- 172.21.15.6:0/2966291158 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55d30627d950 0x55d30627fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:56.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.033+0000 7f9389390640 1 --2- 172.21.15.6:0/2966291158 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55d30627d950 0x55d30627fd70 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3867c1396608d10 server_cookie=1ffdede030249e0 in_seq=0 out_seq=0 2024-09-12T03:26:56.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.033+0000 7f936cff9640 10 client.16026 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:26:56.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.033+0000 7f936cff9640 1 -- 172.21.15.6:0/2966291158 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55d30612e9a0 con 0x55d30627d950 2024-09-12T03:26:56.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.033+0000 7f936cff9640 10 client.16026 handle_command_reply: tid=0 2024-09-12T03:26:56.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.033+0000 7f938d9cd740 1 client.16026 shutdown 2024-09-12T03:26:56.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.034+0000 7f938d9cd740 1 -- 172.21.15.6:0/2966291158 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d3061f8fa0 msgr2=0x55d3062064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:56.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.034+0000 7f938d9cd740 1 --2- 172.21.15.6:0/2966291158 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d3061f8fa0 0x55d3062064c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x55d3062007c0 tx=0x7f93740122f0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.034+0000 7f938d9cd740 1 -- 172.21.15.6:0/2966291158 shutdown_connections 2024-09-12T03:26:56.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.034+0000 7f938d9cd740 1 --2- 172.21.15.6:0/2966291158 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55d30627d950 0x55d30627fd70 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.034+0000 7f938d9cd740 1 --2- 172.21.15.6:0/2966291158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d306202610 0x55d306206c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.034+0000 7f938d9cd740 1 --2- 172.21.15.6:0/2966291158 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d3061f8fa0 0x55d3062064c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.034+0000 7f938d9cd740 1 --2- 172.21.15.6:0/2966291158 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d3061f7cb0 0x55d30620b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.034+0000 7f938d9cd740 1 -- 172.21.15.6:0/2966291158 >> 172.21.15.6:0/2966291158 conn(0x55d3061f3f00 msgr2=0x55d3061fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:56.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.034+0000 7f936cff9640 0 client.16026 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:26:56.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.035+0000 7f938d9cd740 1 -- 172.21.15.6:0/2966291158 shutdown_connections 2024-09-12T03:26:56.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.035+0000 7f938d9cd740 1 -- 172.21.15.6:0/2966291158 wait complete. 2024-09-12T03:26:56.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.035+0000 7f938d9cd740 20 client.16026 trim_cache size 0 max 16384 2024-09-12T03:26:56.037 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:26:56.037 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "scrub active (113 inodes in the stack)", 2024-09-12T03:26:56.037 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2024-09-12T03:26:56.037 INFO:teuthology.orchestra.run.smithi006.stdout: "01562442-7858-449c-a9dd-3b0d01987c49": { 2024-09-12T03:26:56.037 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-09-12T03:26:56.037 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "01562442-7858-449c-a9dd-3b0d01987c49", 2024-09-12T03:26:56.037 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2024-09-12T03:26:56.037 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:26:56.037 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:26:56.038 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:26:56.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.037+0000 7f936ffff640 1 -- 172.21.15.6:0/1787768983 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f93700a2d90 msgr2=0x7f93700a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:56.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.037+0000 7f936ffff640 1 --2- 172.21.15.6:0/1787768983 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f93700a2d90 0x7f93700a5250 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9380058000 tx=0x7f9380059000 comp rx=0 tx=0).stop 2024-09-12T03:26:56.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.037+0000 7f936ffff640 1 -- 172.21.15.6:0/1787768983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93841008d0 msgr2=0x7f9384073e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:56.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.037+0000 7f936ffff640 1 --2- 172.21.15.6:0/1787768983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93841008d0 0x7f9384073e40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f937c00ba50 tx=0x7f937c00be50 comp rx=0 tx=0).stop 2024-09-12T03:26:56.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.041+0000 7f936ffff640 1 -- 172.21.15.6:0/1787768983 shutdown_connections 2024-09-12T03:26:56.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.041+0000 7f936ffff640 1 --2- 172.21.15.6:0/1787768983 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f93700a2d90 0x7f93700a5250 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.041+0000 7f936ffff640 1 --2- 172.21.15.6:0/1787768983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93841068d0 0x7f938406e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.041+0000 7f936ffff640 1 --2- 172.21.15.6:0/1787768983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9384101270 0x7f938406dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.041+0000 7f936ffff640 1 --2- 172.21.15.6:0/1787768983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93841008d0 0x7f9384073e40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.041+0000 7f936ffff640 1 -- 172.21.15.6:0/1787768983 >> 172.21.15.6:0/1787768983 conn(0x7f93840fc610 msgr2=0x7f93840fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:56.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.041+0000 7f936ffff640 1 -- 172.21.15.6:0/1787768983 shutdown_connections 2024-09-12T03:26:56.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:56.042+0000 7f936ffff640 1 -- 172.21.15.6:0/1787768983 wait complete. 2024-09-12T03:26:56.485 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:57.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:56 smithi118 ceph-mon[35048]: pgmap v510: 97 pgs: 97 active+clean; 13 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 462 KiB/s rd, 159 MiB/s wr, 16.39k op/s 2024-09-12T03:26:57.249 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:56 smithi006 ceph-mon[30991]: pgmap v510: 97 pgs: 97 active+clean; 13 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 462 KiB/s rd, 159 MiB/s wr, 16.39k op/s 2024-09-12T03:26:57.322 INFO:tasks.cephfs.filesystem:scrub status for tag:01562442-7858-449c-a9dd-3b0d01987c49 - {'path': '/', 'tag': '01562442-7858-449c-a9dd-3b0d01987c49', 'options': 'recursive,force'} 2024-09-12T03:26:57.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:56 smithi161 ceph-mon[35485]: pgmap v510: 97 pgs: 97 active+clean; 13 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 462 KiB/s rd, 159 MiB/s wr, 16.39k op/s 2024-09-12T03:26:58.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.050+0000 7fe76ac51640 1 -- 172.21.15.6:0/2322498457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe75c09a4c0 msgr2=0x7fe75c09a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:58.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.050+0000 7fe768c4d640 1 -- 172.21.15.6:0/2322498457 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760037070 con 0x7fe75c09a4c0 2024-09-12T03:26:58.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.050+0000 7fe76ac51640 1 --2- 172.21.15.6:0/2322498457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe75c09a4c0 0x7fe75c09a8a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe760009860 tx=0x7fe76002a380 comp rx=0 tx=0).stop 2024-09-12T03:26:58.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.050+0000 7fe76ac51640 1 -- 172.21.15.6:0/2322498457 shutdown_connections 2024-09-12T03:26:58.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.050+0000 7fe76ac51640 1 --2- 172.21.15.6:0/2322498457 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe75c005710 0x7fe75c097fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:58.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.051+0000 7fe76ac51640 1 --2- 172.21.15.6:0/2322498457 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe75c004c80 0x7fe75c0050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:58.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.051+0000 7fe76ac51640 1 --2- 172.21.15.6:0/2322498457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe75c09a4c0 0x7fe75c09a8a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:58.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.051+0000 7fe76ac51640 1 -- 172.21.15.6:0/2322498457 >> 172.21.15.6:0/2322498457 conn(0x7fe75c00a1f0 msgr2=0x7fe75c00a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:58.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.051+0000 7fe76ac51640 1 -- 172.21.15.6:0/2322498457 shutdown_connections 2024-09-12T03:26:58.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.051+0000 7fe76ac51640 1 -- 172.21.15.6:0/2322498457 wait complete. 2024-09-12T03:26:58.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.052+0000 7fe76ac51640 1 Processor -- start 2024-09-12T03:26:58.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.053+0000 7fe76ac51640 1 -- start start 2024-09-12T03:26:58.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.053+0000 7fe76ac51640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe75c004c80 0x7fe75c12f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:58.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.054+0000 7fe76ac51640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe75c005710 0x7fe75c12fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:58.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.054+0000 7fe76ac51640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe75c09a4c0 0x7fe75c129a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:58.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.054+0000 7fe76ac51640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c09d7b0 con 0x7fe75c09a4c0 2024-09-12T03:26:58.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.054+0000 7fe76ac51640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c09d630 con 0x7fe75c005710 2024-09-12T03:26:58.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.054+0000 7fe76ac51640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c09d930 con 0x7fe75c004c80 2024-09-12T03:26:58.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.055+0000 7fe76a450640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe75c09a4c0 0x7fe75c129a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:58.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.055+0000 7fe76a450640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe75c09a4c0 0x7fe75c129a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50764/0 (socket says 172.21.15.6:50764) 2024-09-12T03:26:58.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.055+0000 7fe76a450640 1 -- 172.21.15.6:0/215918805 learned_addr learned my addr 172.21.15.6:0/215918805 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:26:58.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.055+0000 7fe76a450640 1 -- 172.21.15.6:0/215918805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe75c004c80 msgr2=0x7fe75c12f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:58.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.055+0000 7fe76a450640 1 --2- 172.21.15.6:0/215918805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe75c004c80 0x7fe75c12f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:58.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.055+0000 7fe76a450640 1 -- 172.21.15.6:0/215918805 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe75c005710 msgr2=0x7fe75c12fd80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:26:58.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.055+0000 7fe76a450640 1 --2- 172.21.15.6:0/215918805 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe75c005710 0x7fe75c12fd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:58.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.055+0000 7fe76a450640 1 -- 172.21.15.6:0/215918805 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75c12a130 con 0x7fe75c09a4c0 2024-09-12T03:26:58.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.056+0000 7fe76a450640 1 --2- 172.21.15.6:0/215918805 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe75c09a4c0 0x7fe75c129a00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe764067ce0 tx=0x7fe7640747f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:58.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.056+0000 7fe75affd640 1 -- 172.21.15.6:0/215918805 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764090470 con 0x7fe75c09a4c0 2024-09-12T03:26:58.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.057+0000 7fe75affd640 1 -- 172.21.15.6:0/215918805 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe764095020 con 0x7fe75c09a4c0 2024-09-12T03:26:58.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.057+0000 7fe75affd640 1 -- 172.21.15.6:0/215918805 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76408f6a0 con 0x7fe75c09a4c0 2024-09-12T03:26:58.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.057+0000 7fe76ac51640 1 -- 172.21.15.6:0/215918805 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe75c12a420 con 0x7fe75c09a4c0 2024-09-12T03:26:58.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.057+0000 7fe76ac51640 1 -- 172.21.15.6:0/215918805 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe75c136710 con 0x7fe75c09a4c0 2024-09-12T03:26:58.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.061+0000 7fe75affd640 1 -- 172.21.15.6:0/215918805 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe76408f840 con 0x7fe75c09a4c0 2024-09-12T03:26:58.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.062+0000 7fe75affd640 1 --2- 172.21.15.6:0/215918805 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe7500a2da0 0x7fe7500a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:58.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.062+0000 7fe76ac51640 1 -- 172.21.15.6:0/215918805 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7480052c0 con 0x7fe75c09a4c0 2024-09-12T03:26:58.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.062+0000 7fe75affd640 1 -- 172.21.15.6:0/215918805 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe76414e0f0 con 0x7fe75c09a4c0 2024-09-12T03:26:58.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.063+0000 7fe769c4f640 1 --2- 172.21.15.6:0/215918805 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe7500a2da0 0x7fe7500a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:58.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.067+0000 7fe769c4f640 1 --2- 172.21.15.6:0/215918805 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe7500a2da0 0x7fe7500a5260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe7600304e0 tx=0x7fe7600093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:58.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.076+0000 7fe75affd640 1 -- 172.21.15.6:0/215918805 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe76409f020 con 0x7fe75c09a4c0 2024-09-12T03:26:58.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.526+0000 7fe76ac51640 1 -- 172.21.15.6:0/215918805 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7480055b0 con 0x7fe75c09a4c0 2024-09-12T03:26:58.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.531+0000 7fe75affd640 1 -- 172.21.15.6:0/215918805 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11908 (secure 0 0 0) 0x7fe76411b020 con 0x7fe75c09a4c0 2024-09-12T03:26:58.533 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:26:58.533 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"btime":"2024-09-12T03:24:26:920603+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:24:26.080047+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24463]},"id":1}]} 2024-09-12T03:26:58.534 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 27 2024-09-12T03:26:58.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.537+0000 7fe76ac51640 1 -- 172.21.15.6:0/215918805 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe7500a2da0 msgr2=0x7fe7500a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:58.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.537+0000 7fe76ac51640 1 --2- 172.21.15.6:0/215918805 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe7500a2da0 0x7fe7500a5260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe7600304e0 tx=0x7fe7600093f0 comp rx=0 tx=0).stop 2024-09-12T03:26:58.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.537+0000 7fe76ac51640 1 -- 172.21.15.6:0/215918805 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe75c09a4c0 msgr2=0x7fe75c129a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:58.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.537+0000 7fe76ac51640 1 --2- 172.21.15.6:0/215918805 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe75c09a4c0 0x7fe75c129a00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe764067ce0 tx=0x7fe7640747f0 comp rx=0 tx=0).stop 2024-09-12T03:26:58.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.537+0000 7fe76ac51640 1 -- 172.21.15.6:0/215918805 shutdown_connections 2024-09-12T03:26:58.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.537+0000 7fe76ac51640 1 --2- 172.21.15.6:0/215918805 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe7500a2da0 0x7fe7500a5260 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:58.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.537+0000 7fe76ac51640 1 --2- 172.21.15.6:0/215918805 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe75c09a4c0 0x7fe75c129a00 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:58.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.537+0000 7fe76ac51640 1 --2- 172.21.15.6:0/215918805 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe75c005710 0x7fe75c12fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:58.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.537+0000 7fe76ac51640 1 --2- 172.21.15.6:0/215918805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe75c004c80 0x7fe75c12f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:58.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.537+0000 7fe76ac51640 1 -- 172.21.15.6:0/215918805 >> 172.21.15.6:0/215918805 conn(0x7fe75c00a1f0 msgr2=0x7fe75c096fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:58.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.537+0000 7fe76ac51640 1 -- 172.21.15.6:0/215918805 shutdown_connections 2024-09-12T03:26:58.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:26:58.537+0000 7fe76ac51640 1 -- 172.21.15.6:0/215918805 wait complete. 2024-09-12T03:26:59.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:58 smithi006 ceph-mon[30991]: pgmap v511: 97 pgs: 97 active+clean; 13 GiB data, 107 GiB used, 966 GiB / 1.0 TiB avail; 377 KiB/s rd, 179 MiB/s wr, 18.36k op/s 2024-09-12T03:26:59.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:26:58 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/215918805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:59.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:58 smithi118 ceph-mon[35048]: pgmap v511: 97 pgs: 97 active+clean; 13 GiB data, 107 GiB used, 966 GiB / 1.0 TiB avail; 377 KiB/s rd, 179 MiB/s wr, 18.36k op/s 2024-09-12T03:26:59.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:26:58 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/215918805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:59.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:58 smithi161 ceph-mon[35485]: pgmap v511: 97 pgs: 97 active+clean; 13 GiB data, 107 GiB used, 966 GiB / 1.0 TiB avail; 377 KiB/s rd, 179 MiB/s wr, 18.36k op/s 2024-09-12T03:26:59.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:26:58 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/215918805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:59.653 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-12T03:26:59.654 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-12T03:27:00.232 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:01.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:00 smithi006 ceph-mon[30991]: pgmap v512: 97 pgs: 97 active+clean; 13 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 585 KiB/s rd, 192 MiB/s wr, 19.79k op/s 2024-09-12T03:27:01.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:00 smithi118 ceph-mon[35048]: pgmap v512: 97 pgs: 97 active+clean; 13 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 585 KiB/s rd, 192 MiB/s wr, 19.79k op/s 2024-09-12T03:27:01.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.308+0000 7fba1d874640 1 -- 172.21.15.6:0/3166353600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba1811b140 msgr2=0x7fba1811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.308+0000 7fba1d874640 1 --2- 172.21.15.6:0/3166353600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba1811b140 0x7fba1811d530 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fba1001acb0 tx=0x7fba1003b900 comp rx=0 tx=0).stop 2024-09-12T03:27:01.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.312+0000 7fba1d874640 1 -- 172.21.15.6:0/3166353600 shutdown_connections 2024-09-12T03:27:01.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.312+0000 7fba1d874640 1 --2- 172.21.15.6:0/3166353600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba1811b140 0x7fba1811d530 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.312+0000 7fba1d874640 1 --2- 172.21.15.6:0/3166353600 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba1806ea30 0x7fba1806ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.312+0000 7fba1d874640 1 --2- 172.21.15.6:0/3166353600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba18110190 0x7fba1806e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.312+0000 7fba1d874640 1 -- 172.21.15.6:0/3166353600 >> 172.21.15.6:0/3166353600 conn(0x7fba18073180 msgr2=0x7fba18073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:01.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.318+0000 7fba1d874640 1 -- 172.21.15.6:0/3166353600 shutdown_connections 2024-09-12T03:27:01.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.321+0000 7fba1d874640 1 -- 172.21.15.6:0/3166353600 wait complete. 2024-09-12T03:27:01.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.322+0000 7fba1d874640 1 Processor -- start 2024-09-12T03:27:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.322+0000 7fba1d874640 1 -- start start 2024-09-12T03:27:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.322+0000 7fba1d874640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba1806ea30 0x7fba1811a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.322+0000 7fba1d874640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba18110190 0x7fba18116960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.322+0000 7fba1d874640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba1811b140 0x7fba18116ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.322+0000 7fba1d874640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fba18113360 con 0x7fba1811b140 2024-09-12T03:27:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.322+0000 7fba1d874640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fba181131e0 con 0x7fba1806ea30 2024-09-12T03:27:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.322+0000 7fba1d874640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fba181134e0 con 0x7fba18110190 2024-09-12T03:27:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.322+0000 7fba17fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba18110190 0x7fba18116960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.322+0000 7fba17fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba18110190 0x7fba18116960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:43530/0 (socket says 172.21.15.6:43530) 2024-09-12T03:27:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.322+0000 7fba17fff640 1 -- 172.21.15.6:0/3961221281 learned_addr learned my addr 172.21.15.6:0/3961221281 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.322+0000 7fba1d073640 1 --2- 172.21.15.6:0/3961221281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba1811b140 0x7fba18116ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.323+0000 7fba1c872640 1 --2- 172.21.15.6:0/3961221281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba1806ea30 0x7fba1811a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.323+0000 7fba17fff640 1 -- 172.21.15.6:0/3961221281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba1806ea30 msgr2=0x7fba1811a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.323+0000 7fba17fff640 1 --2- 172.21.15.6:0/3961221281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba1806ea30 0x7fba1811a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.323+0000 7fba17fff640 1 -- 172.21.15.6:0/3961221281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba1811b140 msgr2=0x7fba18116ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.323+0000 7fba17fff640 1 --2- 172.21.15.6:0/3961221281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba1811b140 0x7fba18116ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.323+0000 7fba17fff640 1 -- 172.21.15.6:0/3961221281 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba181175d0 con 0x7fba18110190 2024-09-12T03:27:01.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.323+0000 7fba17fff640 1 --2- 172.21.15.6:0/3961221281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba18110190 0x7fba18116960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00b450 tx=0x7fba0c00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:01.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.324+0000 7fba15ffb640 1 -- 172.21.15.6:0/3961221281 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c015020 con 0x7fba18110190 2024-09-12T03:27:01.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.324+0000 7fba1d874640 1 -- 172.21.15.6:0/3961221281 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba181aaf60 con 0x7fba18110190 2024-09-12T03:27:01.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.324+0000 7fba1d874640 1 -- 172.21.15.6:0/3961221281 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba181ab360 con 0x7fba18110190 2024-09-12T03:27:01.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.324+0000 7fba15ffb640 1 -- 172.21.15.6:0/3961221281 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba0c011d70 con 0x7fba18110190 2024-09-12T03:27:01.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.324+0000 7fba15ffb640 1 -- 172.21.15.6:0/3961221281 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c00f340 con 0x7fba18110190 2024-09-12T03:27:01.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.330+0000 7fba20eaf740 1 Processor -- start 2024-09-12T03:27:01.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.333+0000 7fba20eaf740 1 -- start start 2024-09-12T03:27:01.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.333+0000 7fba15ffb640 1 -- 172.21.15.6:0/3961221281 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fba0c018020 con 0x7fba18110190 2024-09-12T03:27:01.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.333+0000 7fba20eaf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4f8fe900 0x55ed4f8fece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.333+0000 7fba20eaf740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f8ff220 0x55ed4f9095a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.333+0000 7fba20eaf740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f909ae0 0x55ed4f90bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.333+0000 7fba20eaf740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55ed4f8642a0 con 0x55ed4f8ff220 2024-09-12T03:27:01.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.333+0000 7fba20eaf740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55ed4f80bb10 con 0x55ed4f909ae0 2024-09-12T03:27:01.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.333+0000 7fba20eaf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ed4f860260 con 0x55ed4f8fe900 2024-09-12T03:27:01.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.333+0000 7fba1c872640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f8ff220 0x55ed4f9095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.333+0000 7fba1c872640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f8ff220 0x55ed4f9095a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34386/0 (socket says 172.21.15.6:34386) 2024-09-12T03:27:01.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.333+0000 7fba1c872640 1 -- 172.21.15.6:0/3077907871 learned_addr learned my addr 172.21.15.6:0/3077907871 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:01.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.335+0000 7fba15ffb640 1 --2- 172.21.15.6:0/3961221281 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fba000a2cc0 0x7fba000a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.335+0000 7fba15ffb640 1 -- 172.21.15.6:0/3961221281 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fba0c0cea50 con 0x7fba18110190 2024-09-12T03:27:01.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.336+0000 7fba1c872640 1 -- 172.21.15.6:0/3077907871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4f8fe900 msgr2=0x55ed4f8fece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.336+0000 7fba1c872640 1 --2- 172.21.15.6:0/3077907871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4f8fe900 0x55ed4f8fece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.336+0000 7fba1c872640 1 -- 172.21.15.6:0/3077907871 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f909ae0 msgr2=0x55ed4f90bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.336+0000 7fba1c872640 1 --2- 172.21.15.6:0/3077907871 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f909ae0 0x55ed4f90bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.336+0000 7fba1c872640 1 -- 172.21.15.6:0/3077907871 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed4f5d5b90 con 0x55ed4f8ff220 2024-09-12T03:27:01.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.337+0000 7fba1c872640 1 --2- 172.21.15.6:0/3077907871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f8ff220 0x55ed4f9095a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fba0800bcb0 tx=0x7fba0800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cde3d29318a5a2c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:01.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.338+0000 7fba1d073640 1 --2- 172.21.15.6:0/3961221281 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fba000a2cc0 0x7fba000a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.339+0000 7fb9feffd640 1 -- 172.21.15.6:0/3077907871 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba08015070 con 0x55ed4f8ff220 2024-09-12T03:27:01.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.339+0000 7fb9feffd640 1 -- 172.21.15.6:0/3077907871 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba0801a940 con 0x55ed4f8ff220 2024-09-12T03:27:01.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.340+0000 7fba20eaf740 1 -- 172.21.15.6:0/3077907871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f8ff220 msgr2=0x55ed4f9095a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.340+0000 7fba20eaf740 1 --2- 172.21.15.6:0/3077907871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f8ff220 0x55ed4f9095a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fba0800bcb0 tx=0x7fba0800f730 comp rx=0 tx=0).stop 2024-09-12T03:27:01.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.341+0000 7fba1d073640 1 --2- 172.21.15.6:0/3961221281 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fba000a2cc0 0x7fba000a5180 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fba10005e30 tx=0x7fba10005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:01.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.341+0000 7fba20eaf740 1 -- 172.21.15.6:0/3077907871 shutdown_connections 2024-09-12T03:27:01.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.341+0000 7fba20eaf740 1 --2- 172.21.15.6:0/3077907871 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f909ae0 0x55ed4f90bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.341+0000 7fba20eaf740 1 --2- 172.21.15.6:0/3077907871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f8ff220 0x55ed4f9095a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.341+0000 7fba20eaf740 1 --2- 172.21.15.6:0/3077907871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4f8fe900 0x55ed4f8fece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.341+0000 7fba20eaf740 1 -- 172.21.15.6:0/3077907871 >> 172.21.15.6:0/3077907871 conn(0x55ed4f8fae90 msgr2=0x55ed4f8fb270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:01.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.343+0000 7fba20eaf740 1 -- 172.21.15.6:0/3077907871 shutdown_connections 2024-09-12T03:27:01.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.343+0000 7fba20eaf740 1 -- 172.21.15.6:0/3077907871 wait complete. 2024-09-12T03:27:01.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.343+0000 7fba20eaf740 1 -- start start 2024-09-12T03:27:01.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.344+0000 7fba20eaf740 1 Processor -- start 2024-09-12T03:27:01.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.345+0000 7fba20eaf740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f8fe900 0x55ed4f909540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.345+0000 7fba20eaf740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f909ae0 0x55ed4f9067f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.345+0000 7fba20eaf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4f906ee0 0x55ed4f915c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.345+0000 7fba20eaf740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55ed4f90c700 con 0x55ed4f909ae0 2024-09-12T03:27:01.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.345+0000 7fba20eaf740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55ed4f7ec6e0 con 0x55ed4f8fe900 2024-09-12T03:27:01.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.345+0000 7fba20eaf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ed4f90c880 con 0x55ed4f906ee0 2024-09-12T03:27:01.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.345+0000 7fba20eaf740 10 client.0 _ll_register_callbacks cb 0x55ed4f810a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:01.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.348+0000 7fba17fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f909ae0 0x55ed4f9067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.348+0000 7fba17fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f909ae0 0x55ed4f9067f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34402/0 (socket says 172.21.15.6:34402) 2024-09-12T03:27:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.348+0000 7fba17fff640 1 -- 172.21.15.6:0/615829163 learned_addr learned my addr 172.21.15.6:0/615829163 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.348+0000 7fba1c872640 1 --2- 172.21.15.6:0/615829163 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f8fe900 0x55ed4f909540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.348+0000 7fba1d073640 1 --2- 172.21.15.6:0/615829163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4f906ee0 0x55ed4f915c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.349+0000 7fba1c872640 1 -- 172.21.15.6:0/615829163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4f906ee0 msgr2=0x55ed4f915c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.349+0000 7fba1c872640 1 --2- 172.21.15.6:0/615829163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4f906ee0 0x55ed4f915c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.349+0000 7fba1c872640 1 -- 172.21.15.6:0/615829163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f909ae0 msgr2=0x55ed4f9067f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.349+0000 7fba1c872640 1 --2- 172.21.15.6:0/615829163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f909ae0 0x55ed4f9067f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.349+0000 7fba1c872640 1 -- 172.21.15.6:0/615829163 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed4f916190 con 0x55ed4f8fe900 2024-09-12T03:27:01.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.349+0000 7fba1c872640 1 --2- 172.21.15.6:0/615829163 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f8fe900 0x55ed4f909540 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fba0800bcb0 tx=0x7fba0800b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e849147a8ac44f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:01.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.350+0000 7fb9f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:27:01.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.350+0000 7fb9f7fff640 1 -- 172.21.15.6:0/615829163 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba08015050 con 0x55ed4f8fe900 2024-09-12T03:27:01.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.350+0000 7fba20eaf740 1 -- 172.21.15.6:0/615829163 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ed4f8719d0 con 0x55ed4f8fe900 2024-09-12T03:27:01.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.350+0000 7fb9f7fff640 1 -- 172.21.15.6:0/615829163 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fba08005240 con 0x55ed4f8fe900 2024-09-12T03:27:01.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.351+0000 7fb9f7fff640 1 -- 172.21.15.6:0/615829163 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fba08005440 con 0x55ed4f8fe900 2024-09-12T03:27:01.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.351+0000 7fba20eaf740 10 client.25580 fetch_fsmap learned FSMap version 27 2024-09-12T03:27:01.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.351+0000 7fba20eaf740 1 -- 172.21.15.6:0/615829163 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed4f918900 con 0x55ed4f8fe900 2024-09-12T03:27:01.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.353+0000 7fb9f7fff640 1 -- 172.21.15.6:0/615829163 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ed4f8719d0 con 0x55ed4f8fe900 2024-09-12T03:27:01.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.354+0000 7fb9f7fff640 1 -- 172.21.15.6:0/615829163 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fba0801a760 con 0x55ed4f8fe900 2024-09-12T03:27:01.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.354+0000 7fba20eaf740 10 client.25580 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:27:01.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.354+0000 7fba20eaf740 10 client.25580 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:27:01.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.354+0000 7fba20eaf740 20 client.25580 populate_metadata read hostname 'smithi006' 2024-09-12T03:27:01.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.354+0000 7fba20eaf740 1 --2- 172.21.15.6:0/615829163 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ed4f91a040 0x55ed4f91c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.354+0000 7fba20eaf740 4 client.25580 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:01.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.354+0000 7fba20eaf740 1 -- 172.21.15.6:0/615829163 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed4f86bd20 con 0x55ed4f91a040 2024-09-12T03:27:01.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.355+0000 7fba17fff640 1 --2- 172.21.15.6:0/615829163 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ed4f91a040 0x55ed4f91c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.355+0000 7fba17fff640 1 --2- 172.21.15.6:0/615829163 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ed4f91a040 0x55ed4f91c4c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97df6df33bdc4d9 server_cookie=5e31ae422689cdf1 in_seq=0 out_seq=0 2024-09-12T03:27:01.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.355+0000 7fb9f7fff640 10 client.25580 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.356+0000 7fb9f7fff640 1 -- 172.21.15.6:0/615829163 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed4f86bd20 con 0x55ed4f91a040 2024-09-12T03:27:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.356+0000 7fb9f7fff640 10 client.25580 handle_command_reply: tid=0 2024-09-12T03:27:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.356+0000 7fba20eaf740 1 client.25580 shutdown 2024-09-12T03:27:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.357+0000 7fba20eaf740 1 -- 172.21.15.6:0/615829163 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f8fe900 msgr2=0x55ed4f909540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.357+0000 7fba20eaf740 1 --2- 172.21.15.6:0/615829163 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f8fe900 0x55ed4f909540 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fba0800bcb0 tx=0x7fba0800b9f0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.357+0000 7fba20eaf740 1 -- 172.21.15.6:0/615829163 shutdown_connections 2024-09-12T03:27:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.357+0000 7fba20eaf740 1 --2- 172.21.15.6:0/615829163 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ed4f91a040 0x55ed4f91c4c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.357+0000 7fba20eaf740 1 --2- 172.21.15.6:0/615829163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4f906ee0 0x55ed4f915c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.357+0000 7fba20eaf740 1 --2- 172.21.15.6:0/615829163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f909ae0 0x55ed4f9067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.357+0000 7fba20eaf740 1 --2- 172.21.15.6:0/615829163 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f8fe900 0x55ed4f909540 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.357+0000 7fba20eaf740 1 -- 172.21.15.6:0/615829163 >> 172.21.15.6:0/615829163 conn(0x55ed4f8fae90 msgr2=0x55ed4f8f97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:01.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.357+0000 7fb9f7fff640 0 client.25580 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:01.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.357+0000 7fba20eaf740 1 -- 172.21.15.6:0/615829163 shutdown_connections 2024-09-12T03:27:01.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.358+0000 7fba20eaf740 1 -- 172.21.15.6:0/615829163 wait complete. 2024-09-12T03:27:01.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.358+0000 7fba20eaf740 20 client.25580 trim_cache size 0 max 16384 2024-09-12T03:27:01.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:00 smithi161 ceph-mon[35485]: pgmap v512: 97 pgs: 97 active+clean; 13 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 585 KiB/s rd, 192 MiB/s wr, 19.79k op/s 2024-09-12T03:27:01.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.392+0000 7fba20eaf740 1 Processor -- start 2024-09-12T03:27:01.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.393+0000 7fba20eaf740 1 -- start start 2024-09-12T03:27:01.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.393+0000 7fba20eaf740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f8fe900 0x55ed4f9067c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.394+0000 7fba20eaf740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f906ee0 0x55ed4f909060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.394+0000 7fba1c872640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f8fe900 0x55ed4f9067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.394+0000 7fba1c872640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f8fe900 0x55ed4f9067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:39254/0 (socket says 172.21.15.6:39254) 2024-09-12T03:27:01.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.394+0000 7fba1c872640 1 -- 172.21.15.6:0/794799412 learned_addr learned my addr 172.21.15.6:0/794799412 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:01.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.395+0000 7fba17fff640 1 --2- 172.21.15.6:0/794799412 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f906ee0 0x55ed4f909060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.395+0000 7fba20eaf740 1 --2- 172.21.15.6:0/794799412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4f909ae0 0x55ed4f9095a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.395+0000 7fba20eaf740 1 -- 172.21.15.6:0/794799412 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fba04010b20 con 0x55ed4f906ee0 2024-09-12T03:27:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.396+0000 7fba1d073640 1 --2- 172.21.15.6:0/794799412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4f909ae0 0x55ed4f9095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.396+0000 7fba20eaf740 1 -- 172.21.15.6:0/794799412 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fba0400e3f0 con 0x55ed4f8fe900 2024-09-12T03:27:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.396+0000 7fba20eaf740 1 -- 172.21.15.6:0/794799412 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fba0400c2f0 con 0x55ed4f909ae0 2024-09-12T03:27:01.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.397+0000 7fba1c872640 1 -- 172.21.15.6:0/794799412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4f909ae0 msgr2=0x55ed4f9095a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.397+0000 7fba1c872640 1 --2- 172.21.15.6:0/794799412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4f909ae0 0x55ed4f9095a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.397+0000 7fba1c872640 1 -- 172.21.15.6:0/794799412 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f906ee0 msgr2=0x55ed4f909060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.397+0000 7fba1c872640 1 --2- 172.21.15.6:0/794799412 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f906ee0 0x55ed4f909060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.397+0000 7fba1c872640 1 -- 172.21.15.6:0/794799412 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed4f9057a0 con 0x55ed4f8fe900 2024-09-12T03:27:01.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.398+0000 7fba1c872640 1 --2- 172.21.15.6:0/794799412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f8fe900 0x55ed4f9067c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fba0801ab00 tx=0x7fba08005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1ca078cc6471fe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:01.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.398+0000 7fb9fe7fc640 1 -- 172.21.15.6:0/794799412 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba08015040 con 0x55ed4f8fe900 2024-09-12T03:27:01.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.399+0000 7fb9fe7fc640 1 -- 172.21.15.6:0/794799412 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba080055e0 con 0x55ed4f8fe900 2024-09-12T03:27:01.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.399+0000 7fb9fe7fc640 1 -- 172.21.15.6:0/794799412 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0802f800 con 0x55ed4f8fe900 2024-09-12T03:27:01.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.400+0000 7fba20eaf740 1 -- 172.21.15.6:0/794799412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f8fe900 msgr2=0x55ed4f9067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.400+0000 7fba20eaf740 1 --2- 172.21.15.6:0/794799412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f8fe900 0x55ed4f9067c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fba0801ab00 tx=0x7fba08005c50 comp rx=0 tx=0).stop 2024-09-12T03:27:01.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.405+0000 7fba20eaf740 1 -- 172.21.15.6:0/794799412 shutdown_connections 2024-09-12T03:27:01.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.405+0000 7fba20eaf740 1 --2- 172.21.15.6:0/794799412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4f909ae0 0x55ed4f9095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.405+0000 7fba20eaf740 1 --2- 172.21.15.6:0/794799412 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f906ee0 0x55ed4f909060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.405+0000 7fba20eaf740 1 --2- 172.21.15.6:0/794799412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f8fe900 0x55ed4f9067c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.405+0000 7fba20eaf740 1 -- 172.21.15.6:0/794799412 >> 172.21.15.6:0/794799412 conn(0x55ed4f8fae90 msgr2=0x55ed4f9060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:01.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.405+0000 7fba20eaf740 1 -- 172.21.15.6:0/794799412 shutdown_connections 2024-09-12T03:27:01.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.405+0000 7fba20eaf740 1 -- 172.21.15.6:0/794799412 wait complete. 2024-09-12T03:27:01.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.406+0000 7fba20eaf740 1 -- start start 2024-09-12T03:27:01.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.406+0000 7fba20eaf740 1 Processor -- start 2024-09-12T03:27:01.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.407+0000 7fba20eaf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4f8fe900 0x55ed4f9120a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.407+0000 7fba20eaf740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f906ee0 0x55ed4f921480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.407+0000 7fba20eaf740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f909ae0 0x55ed4f9219c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.408+0000 7fba20eaf740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fba0400c8b0 con 0x55ed4f906ee0 2024-09-12T03:27:01.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.408+0000 7fba20eaf740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fba0400dd20 con 0x55ed4f909ae0 2024-09-12T03:27:01.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.408+0000 7fba20eaf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fba0400cf80 con 0x55ed4f8fe900 2024-09-12T03:27:01.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.408+0000 7fba20eaf740 10 client.0 _ll_register_callbacks cb 0x7fba0800bda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:01.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.409+0000 7fba1d073640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f909ae0 0x55ed4f9219c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.409+0000 7fba1d073640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f909ae0 0x55ed4f9219c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:39256/0 (socket says 172.21.15.6:39256) 2024-09-12T03:27:01.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.409+0000 7fba1d073640 1 -- 172.21.15.6:0/2638462273 learned_addr learned my addr 172.21.15.6:0/2638462273 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:01.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.409+0000 7fba1c872640 1 --2- 172.21.15.6:0/2638462273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4f8fe900 0x55ed4f9120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.409+0000 7fba17fff640 1 --2- 172.21.15.6:0/2638462273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f906ee0 0x55ed4f921480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.410+0000 7fba1d073640 1 -- 172.21.15.6:0/2638462273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4f8fe900 msgr2=0x55ed4f9120a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.410+0000 7fba1d073640 1 --2- 172.21.15.6:0/2638462273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4f8fe900 0x55ed4f9120a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.410+0000 7fba1d073640 1 -- 172.21.15.6:0/2638462273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f906ee0 msgr2=0x55ed4f921480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.410+0000 7fba1d073640 1 --2- 172.21.15.6:0/2638462273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f906ee0 0x55ed4f921480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.410+0000 7fba1d073640 1 -- 172.21.15.6:0/2638462273 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed4f921f30 con 0x55ed4f909ae0 2024-09-12T03:27:01.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.414+0000 7fba1d073640 1 --2- 172.21.15.6:0/2638462273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f909ae0 0x55ed4f9219c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fba1001acb0 tx=0x7fba1003be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=315845c8ef3abd04 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:01.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.415+0000 7fb9f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:27:01.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.415+0000 7fb9f7fff640 1 -- 172.21.15.6:0/2638462273 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10045070 con 0x55ed4f909ae0 2024-09-12T03:27:01.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.415+0000 7fba20eaf740 1 -- 172.21.15.6:0/2638462273 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba080055e0 con 0x55ed4f909ae0 2024-09-12T03:27:01.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.420+0000 7fb9f7fff640 1 -- 172.21.15.6:0/2638462273 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fba10019040 con 0x55ed4f909ae0 2024-09-12T03:27:01.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.420+0000 7fb9f7fff640 1 -- 172.21.15.6:0/2638462273 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fba10004720 con 0x55ed4f909ae0 2024-09-12T03:27:01.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.420+0000 7fba20eaf740 10 client.25586 fetch_fsmap learned FSMap version 27 2024-09-12T03:27:01.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.420+0000 7fba20eaf740 1 -- 172.21.15.6:0/2638462273 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed4f910930 con 0x55ed4f909ae0 2024-09-12T03:27:01.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.421+0000 7fb9f7fff640 1 -- 172.21.15.6:0/2638462273 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba10068070 con 0x55ed4f909ae0 2024-09-12T03:27:01.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.422+0000 7fb9f7fff640 1 -- 172.21.15.6:0/2638462273 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fba10063730 con 0x55ed4f909ae0 2024-09-12T03:27:01.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.422+0000 7fba20eaf740 10 client.25586 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:27:01.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.422+0000 7fba20eaf740 10 client.25586 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:27:01.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.422+0000 7fba20eaf740 20 client.25586 populate_metadata read hostname 'smithi006' 2024-09-12T03:27:01.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.423+0000 7fba20eaf740 1 --2- 172.21.15.6:0/2638462273 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ed4f985360 0x55ed4f987780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.423+0000 7fba20eaf740 4 client.25586 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-12T03:27:01.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.423+0000 7fba20eaf740 1 -- 172.21.15.6:0/2638462273 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x7fba080055e0 con 0x55ed4f985360 2024-09-12T03:27:01.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.423+0000 7fba1c872640 1 --2- 172.21.15.6:0/2638462273 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ed4f985360 0x55ed4f987780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.423+0000 7fba1c872640 1 --2- 172.21.15.6:0/2638462273 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ed4f985360 0x55ed4f987780 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58a8cfd80328fdfa server_cookie=c356aaf7d08d6ae6 in_seq=0 out_seq=0 2024-09-12T03:27:01.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:01.423+0000 7fb9f7fff640 10 client.25586 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:03.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:02 smithi118 ceph-mon[35048]: pgmap v513: 97 pgs: 97 active+clean; 13 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 584 KiB/s rd, 158 MiB/s wr, 16.26k op/s 2024-09-12T03:27:03.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:02 smithi006 ceph-mon[30991]: pgmap v513: 97 pgs: 97 active+clean; 13 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 584 KiB/s rd, 158 MiB/s wr, 16.26k op/s 2024-09-12T03:27:03.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:02 smithi161 ceph-mon[35485]: pgmap v513: 97 pgs: 97 active+clean; 13 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 584 KiB/s rd, 158 MiB/s wr, 16.26k op/s 2024-09-12T03:27:05.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:04 smithi118 ceph-mon[35048]: pgmap v514: 97 pgs: 97 active+clean; 13 GiB data, 108 GiB used, 964 GiB / 1.0 TiB avail; 345 KiB/s rd, 166 MiB/s wr, 17.18k op/s 2024-09-12T03:27:05.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:04 smithi006 ceph-mon[30991]: pgmap v514: 97 pgs: 97 active+clean; 13 GiB data, 108 GiB used, 964 GiB / 1.0 TiB avail; 345 KiB/s rd, 166 MiB/s wr, 17.18k op/s 2024-09-12T03:27:05.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:04 smithi161 ceph-mon[35485]: pgmap v514: 97 pgs: 97 active+clean; 13 GiB data, 108 GiB used, 964 GiB / 1.0 TiB avail; 345 KiB/s rd, 166 MiB/s wr, 17.18k op/s 2024-09-12T03:27:05.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.554+0000 7fb9f7fff640 1 -- 172.21.15.6:0/2638462273 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fba080055e0 con 0x55ed4f985360 2024-09-12T03:27:05.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.554+0000 7fb9f7fff640 10 client.25586 handle_command_reply: tid=0 2024-09-12T03:27:05.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.554+0000 7fba20eaf740 1 client.25586 shutdown 2024-09-12T03:27:05.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.555+0000 7fba20eaf740 1 -- 172.21.15.6:0/2638462273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f909ae0 msgr2=0x55ed4f9219c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:05.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.555+0000 7fba20eaf740 1 --2- 172.21.15.6:0/2638462273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f909ae0 0x55ed4f9219c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fba1001acb0 tx=0x7fba1003be00 comp rx=0 tx=0).stop 2024-09-12T03:27:05.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.555+0000 7fba20eaf740 1 -- 172.21.15.6:0/2638462273 shutdown_connections 2024-09-12T03:27:05.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.555+0000 7fba20eaf740 1 --2- 172.21.15.6:0/2638462273 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55ed4f985360 0x55ed4f987780 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:05.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.555+0000 7fba20eaf740 1 --2- 172.21.15.6:0/2638462273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ed4f909ae0 0x55ed4f9219c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:05.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.555+0000 7fba20eaf740 1 --2- 172.21.15.6:0/2638462273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed4f906ee0 0x55ed4f921480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:05.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.555+0000 7fb9f7fff640 0 client.25586 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:05.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.555+0000 7fba20eaf740 1 --2- 172.21.15.6:0/2638462273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4f8fe900 0x55ed4f9120a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:05.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.555+0000 7fba20eaf740 1 -- 172.21.15.6:0/2638462273 >> 172.21.15.6:0/2638462273 conn(0x55ed4f8fae90 msgr2=0x55ed4f906020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:05.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.555+0000 7fba20eaf740 1 -- 172.21.15.6:0/2638462273 shutdown_connections 2024-09-12T03:27:05.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.555+0000 7fba20eaf740 1 -- 172.21.15.6:0/2638462273 wait complete. 2024-09-12T03:27:05.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.555+0000 7fba20eaf740 20 client.25586 trim_cache size 0 max 16384 2024-09-12T03:27:05.557 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:27:05.557 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 27, 2024-09-12T03:27:05.557 INFO:teuthology.orchestra.run.smithi006.stdout: "leader": 24494, 2024-09-12T03:27:05.557 INFO:teuthology.orchestra.run.smithi006.stdout: "set_version": 22, 2024-09-12T03:27:05.557 INFO:teuthology.orchestra.run.smithi006.stdout: "sets": { 2024-09-12T03:27:05.557 INFO:teuthology.orchestra.run.smithi006.stdout: "f777655b": { 2024-09-12T03:27:05.557 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 22, 2024-09-12T03:27:05.557 INFO:teuthology.orchestra.run.smithi006.stdout: "age_ref": 0.0, 2024-09-12T03:27:05.558 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:27:05.558 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "QUIESCED", 2024-09-12T03:27:05.558 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:27:05.559 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:27:05.559 INFO:teuthology.orchestra.run.smithi006.stdout: "timeout": 90.0, 2024-09-12T03:27:05.559 INFO:teuthology.orchestra.run.smithi006.stdout: "expiration": 140.1, 2024-09-12T03:27:05.559 INFO:teuthology.orchestra.run.smithi006.stdout: "members": { 2024-09-12T03:27:05.559 INFO:teuthology.orchestra.run.smithi006.stdout: "file:/": { 2024-09-12T03:27:05.559 INFO:teuthology.orchestra.run.smithi006.stdout: "excluded": false, 2024-09-12T03:27:05.560 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:27:05.560 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "QUIESCED", 2024-09-12T03:27:05.560 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:27:05.560 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:27:05.560 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:27:05.560 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:27:05.560 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:27:05.561 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:27:05.561 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:27:05.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.556+0000 7fba1d874640 1 -- 172.21.15.6:0/3961221281 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fba000a2cc0 msgr2=0x7fba000a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:05.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.556+0000 7fba1d874640 1 --2- 172.21.15.6:0/3961221281 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fba000a2cc0 0x7fba000a5180 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fba10005e30 tx=0x7fba10005ef0 comp rx=0 tx=0).stop 2024-09-12T03:27:05.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.556+0000 7fba1d874640 1 -- 172.21.15.6:0/3961221281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba18110190 msgr2=0x7fba18116960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:05.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.556+0000 7fba1d874640 1 --2- 172.21.15.6:0/3961221281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba18110190 0x7fba18116960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00b450 tx=0x7fba0c00b910 comp rx=0 tx=0).stop 2024-09-12T03:27:05.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.556+0000 7fba1d874640 1 -- 172.21.15.6:0/3961221281 shutdown_connections 2024-09-12T03:27:05.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.556+0000 7fba1d874640 1 --2- 172.21.15.6:0/3961221281 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fba000a2cc0 0x7fba000a5180 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:05.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.556+0000 7fba1d874640 1 --2- 172.21.15.6:0/3961221281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba1811b140 0x7fba18116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:05.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.556+0000 7fba1d874640 1 --2- 172.21.15.6:0/3961221281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba18110190 0x7fba18116960 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:05.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.556+0000 7fba1d874640 1 --2- 172.21.15.6:0/3961221281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba1806ea30 0x7fba1811a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:05.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.556+0000 7fba1d874640 1 -- 172.21.15.6:0/3961221281 >> 172.21.15.6:0/3961221281 conn(0x7fba18073180 msgr2=0x7fba1811b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:05.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.556+0000 7fba1d874640 1 -- 172.21.15.6:0/3961221281 shutdown_connections 2024-09-12T03:27:05.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:05.556+0000 7fba1d874640 1 -- 172.21.15.6:0/3961221281 wait complete. 2024-09-12T03:27:06.394 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f777655b', quiesce took 11.7 seconds. Will release after: 8.400000000000002 2024-09-12T03:27:07.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:06 smithi118 ceph-mon[35048]: pgmap v515: 97 pgs: 97 active+clean; 13 GiB data, 109 GiB used, 963 GiB / 1.0 TiB avail; 210 KiB/s rd, 151 MiB/s wr, 16.16k op/s 2024-09-12T03:27:07.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:06 smithi118 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-12T03:27:07.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:06 smithi118 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-12T03:27:07.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:06 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:07.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:06 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:07.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:06 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:07.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:06 smithi006 ceph-mon[30991]: pgmap v515: 97 pgs: 97 active+clean; 13 GiB data, 109 GiB used, 963 GiB / 1.0 TiB avail; 210 KiB/s rd, 151 MiB/s wr, 16.16k op/s 2024-09-12T03:27:07.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:06 smithi006 ceph-mon[30991]: scrub summary: active paths [/] 2024-09-12T03:27:07.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:06 smithi006 ceph-mon[30991]: scrub summary: active paths [/] 2024-09-12T03:27:07.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:06 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:07.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:06 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:07.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:06 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:07.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:06 smithi161 ceph-mon[35485]: pgmap v515: 97 pgs: 97 active+clean; 13 GiB data, 109 GiB used, 963 GiB / 1.0 TiB avail; 210 KiB/s rd, 151 MiB/s wr, 16.16k op/s 2024-09-12T03:27:07.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:06 smithi161 ceph-mon[35485]: scrub summary: active paths [/] 2024-09-12T03:27:07.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:06 smithi161 ceph-mon[35485]: scrub summary: active paths [/] 2024-09-12T03:27:07.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:06 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:07.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:06 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:07.364 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:06 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:09.213 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:08 smithi118 ceph-mon[35048]: pgmap v516: 97 pgs: 97 active+clean; 13 GiB data, 110 GiB used, 963 GiB / 1.0 TiB avail; 366 KiB/s rd, 166 MiB/s wr, 18.04k op/s 2024-09-12T03:27:09.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:08 smithi006 ceph-mon[30991]: pgmap v516: 97 pgs: 97 active+clean; 13 GiB data, 110 GiB used, 963 GiB / 1.0 TiB avail; 366 KiB/s rd, 166 MiB/s wr, 18.04k op/s 2024-09-12T03:27:09.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:08 smithi161 ceph-mon[35485]: pgmap v516: 97 pgs: 97 active+clean; 13 GiB data, 110 GiB used, 963 GiB / 1.0 TiB avail; 366 KiB/s rd, 166 MiB/s wr, 18.04k op/s 2024-09-12T03:27:11.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:10 smithi118 ceph-mon[35048]: pgmap v517: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 366 KiB/s rd, 160 MiB/s wr, 17.82k op/s 2024-09-12T03:27:11.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:10 smithi006 ceph-mon[30991]: pgmap v517: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 366 KiB/s rd, 160 MiB/s wr, 17.82k op/s 2024-09-12T03:27:11.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:10 smithi161 ceph-mon[35485]: pgmap v517: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 366 KiB/s rd, 160 MiB/s wr, 17.82k op/s 2024-09-12T03:27:13.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:12 smithi118 ceph-mon[35048]: pgmap v518: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 160 KiB/s rd, 121 MiB/s wr, 13.61k op/s 2024-09-12T03:27:13.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:12 smithi006 ceph-mon[30991]: pgmap v518: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 160 KiB/s rd, 121 MiB/s wr, 13.61k op/s 2024-09-12T03:27:13.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:12 smithi161 ceph-mon[35485]: pgmap v518: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 160 KiB/s rd, 121 MiB/s wr, 13.61k op/s 2024-09-12T03:27:14.796 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:27:15.056 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:15.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:14 smithi006 ceph-mon[30991]: pgmap v519: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 161 KiB/s rd, 116 MiB/s wr, 13.02k op/s 2024-09-12T03:27:15.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:14 smithi118 ceph-mon[35048]: pgmap v519: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 161 KiB/s rd, 116 MiB/s wr, 13.02k op/s 2024-09-12T03:27:15.365 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:14 smithi161 ceph-mon[35485]: pgmap v519: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 161 KiB/s rd, 116 MiB/s wr, 13.02k op/s 2024-09-12T03:27:15.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.597+0000 7f3f888e5640 1 -- 172.21.15.6:0/2049221516 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f84106f10 msgr2=0x7f3f8410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:15.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.597+0000 7f3f888e5640 1 --2- 172.21.15.6:0/2049221516 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f84106f10 0x7f3f8410ea90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009900 tx=0x7f3f7c02a410 comp rx=0 tx=0).stop 2024-09-12T03:27:15.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.597+0000 7f3f888e5640 1 -- 172.21.15.6:0/2049221516 shutdown_connections 2024-09-12T03:27:15.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.598+0000 7f3f888e5640 1 --2- 172.21.15.6:0/2049221516 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f84106f10 0x7f3f8410ea90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:15.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.598+0000 7f3f888e5640 1 --2- 172.21.15.6:0/2049221516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f84102790 0x7f3f841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:15.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.598+0000 7f3f888e5640 1 --2- 172.21.15.6:0/2049221516 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f84101de0 0x7f3f841021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:15.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.598+0000 7f3f888e5640 1 -- 172.21.15.6:0/2049221516 >> 172.21.15.6:0/2049221516 conn(0x7f3f840fd8c0 msgr2=0x7f3f840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:15.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.598+0000 7f3f888e5640 1 -- 172.21.15.6:0/2049221516 shutdown_connections 2024-09-12T03:27:15.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.598+0000 7f3f888e5640 1 -- 172.21.15.6:0/2049221516 wait complete. 2024-09-12T03:27:15.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.598+0000 7f3f888e5640 1 Processor -- start 2024-09-12T03:27:15.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.599+0000 7f3f888e5640 1 -- start start 2024-09-12T03:27:15.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.599+0000 7f3f888e5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f84101de0 0x7f3f84198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:15.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.599+0000 7f3f888e5640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f84102790 0x7f3f84198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:15.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.599+0000 7f3f888e5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f84106f10 0x7f3f8419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:15.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.599+0000 7f3f888e5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3f84111f70 con 0x7f3f84106f10 2024-09-12T03:27:15.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.599+0000 7f3f888e5640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f3f84111df0 con 0x7f3f84102790 2024-09-12T03:27:15.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.599+0000 7f3f888e5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841120f0 con 0x7f3f84101de0 2024-09-12T03:27:15.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.599+0000 7f3f82d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f84106f10 0x7f3f8419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:15.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.599+0000 7f3f82d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f84106f10 0x7f3f8419cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56490/0 (socket says 172.21.15.6:56490) 2024-09-12T03:27:15.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.599+0000 7f3f82d9d640 1 -- 172.21.15.6:0/2770514228 learned_addr learned my addr 172.21.15.6:0/2770514228 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:15.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.599+0000 7f3f81d9b640 1 --2- 172.21.15.6:0/2770514228 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f84102790 0x7f3f84198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:15.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.599+0000 7f3f8259c640 1 --2- 172.21.15.6:0/2770514228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f84101de0 0x7f3f84198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:15.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.599+0000 7f3f82d9d640 1 -- 172.21.15.6:0/2770514228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f84101de0 msgr2=0x7f3f84198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:15.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.599+0000 7f3f82d9d640 1 --2- 172.21.15.6:0/2770514228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f84101de0 0x7f3f84198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:15.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.599+0000 7f3f82d9d640 1 -- 172.21.15.6:0/2770514228 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f84102790 msgr2=0x7f3f84198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:15.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.599+0000 7f3f82d9d640 1 --2- 172.21.15.6:0/2770514228 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f84102790 0x7f3f84198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:15.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.599+0000 7f3f82d9d640 1 -- 172.21.15.6:0/2770514228 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8419d3e0 con 0x7f3f84106f10 2024-09-12T03:27:15.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.600+0000 7f3f82d9d640 1 --2- 172.21.15.6:0/2770514228 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f84106f10 0x7f3f8419cd00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c005ab0 tx=0x7f3f7c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:15.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.600+0000 7f3f7b7fe640 1 -- 172.21.15.6:0/2770514228 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c03f030 con 0x7f3f84106f10 2024-09-12T03:27:15.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.600+0000 7f3f7b7fe640 1 -- 172.21.15.6:0/2770514228 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f7c003ce0 con 0x7f3f84106f10 2024-09-12T03:27:15.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.600+0000 7f3f888e5640 1 -- 172.21.15.6:0/2770514228 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8419d670 con 0x7f3f84106f10 2024-09-12T03:27:15.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.600+0000 7f3f7b7fe640 1 -- 172.21.15.6:0/2770514228 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c035070 con 0x7f3f84106f10 2024-09-12T03:27:15.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.600+0000 7f3f888e5640 1 -- 172.21.15.6:0/2770514228 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f841a4fb0 con 0x7f3f84106f10 2024-09-12T03:27:15.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.601+0000 7f3f888e5640 1 -- 172.21.15.6:0/2770514228 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f841021c0 con 0x7f3f84106f10 2024-09-12T03:27:15.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.602+0000 7f3f7b7fe640 1 -- 172.21.15.6:0/2770514228 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3f7c042030 con 0x7f3f84106f10 2024-09-12T03:27:15.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.603+0000 7f3f7b7fe640 1 --2- 172.21.15.6:0/2770514228 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3f700a2c00 0x7f3f700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:15.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.603+0000 7f3f7b7fe640 1 -- 172.21.15.6:0/2770514228 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f3f7c0ef6e0 con 0x7f3f84106f10 2024-09-12T03:27:15.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.603+0000 7f3f8259c640 1 --2- 172.21.15.6:0/2770514228 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3f700a2c00 0x7f3f700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:15.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.604+0000 7f3f8259c640 1 --2- 172.21.15.6:0/2770514228 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3f700a2c00 0x7f3f700a50c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3f74005c40 tx=0x7f3f7400a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:15.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.607+0000 7f3f7b7fe640 1 -- 172.21.15.6:0/2770514228 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3f7c0bc020 con 0x7f3f84106f10 2024-09-12T03:27:15.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.834+0000 7f3f888e5640 1 -- 172.21.15.6:0/2770514228 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f8410a0f0 con 0x7f3f84106f10 2024-09-12T03:27:15.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.834+0000 7f3f7b7fe640 1 -- 172.21.15.6:0/2770514228 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11908 (secure 0 0 0) 0x7f3f7c0c0320 con 0x7f3f84106f10 2024-09-12T03:27:15.835 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:27:15.835 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"btime":"2024-09-12T03:24:26:920603+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:24:26.080047+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24463]},"id":1}]} 2024-09-12T03:27:15.835 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 27 2024-09-12T03:27:15.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.836+0000 7f3f888e5640 1 -- 172.21.15.6:0/2770514228 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3f700a2c00 msgr2=0x7f3f700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:15.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.836+0000 7f3f888e5640 1 --2- 172.21.15.6:0/2770514228 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3f700a2c00 0x7f3f700a50c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3f74005c40 tx=0x7f3f7400a590 comp rx=0 tx=0).stop 2024-09-12T03:27:15.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.836+0000 7f3f888e5640 1 -- 172.21.15.6:0/2770514228 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f84106f10 msgr2=0x7f3f8419cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:15.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.836+0000 7f3f888e5640 1 --2- 172.21.15.6:0/2770514228 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f84106f10 0x7f3f8419cd00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c005ab0 tx=0x7f3f7c0042f0 comp rx=0 tx=0).stop 2024-09-12T03:27:15.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.837+0000 7f3f888e5640 1 -- 172.21.15.6:0/2770514228 shutdown_connections 2024-09-12T03:27:15.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.837+0000 7f3f888e5640 1 --2- 172.21.15.6:0/2770514228 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f3f700a2c00 0x7f3f700a50c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:15.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.837+0000 7f3f888e5640 1 --2- 172.21.15.6:0/2770514228 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f84106f10 0x7f3f8419cd00 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:15.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.837+0000 7f3f888e5640 1 --2- 172.21.15.6:0/2770514228 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f84102790 0x7f3f84198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:15.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.837+0000 7f3f888e5640 1 --2- 172.21.15.6:0/2770514228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f84101de0 0x7f3f84198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:15.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.837+0000 7f3f888e5640 1 -- 172.21.15.6:0/2770514228 >> 172.21.15.6:0/2770514228 conn(0x7f3f840fd8c0 msgr2=0x7f3f840fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:15.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.837+0000 7f3f888e5640 1 -- 172.21.15.6:0/2770514228 shutdown_connections 2024-09-12T03:27:15.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:15.837+0000 7f3f888e5640 1 -- 172.21.15.6:0/2770514228 wait complete. 2024-09-12T03:27:16.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:15 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2770514228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:16.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:15 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2770514228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:16.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:15 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2770514228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:16.486 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id f777655b --release --await' 2024-09-12T03:27:16.486 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id f777655b --release --await 2024-09-12T03:27:16.743 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:17.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:16 smithi006 ceph-mon[30991]: pgmap v520: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 160 KiB/s rd, 99 MiB/s wr, 11.25k op/s 2024-09-12T03:27:17.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:16 smithi006 ceph-mon[30991]: scrub summary: idle 2024-09-12T03:27:17.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:16 smithi006 ceph-mon[30991]: scrub summary: idle 2024-09-12T03:27:17.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:16 smithi006 ceph-mon[30991]: scrub summary: idle 2024-09-12T03:27:17.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:16 smithi161 ceph-mon[35485]: pgmap v520: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 160 KiB/s rd, 99 MiB/s wr, 11.25k op/s 2024-09-12T03:27:17.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:16 smithi161 ceph-mon[35485]: scrub summary: idle 2024-09-12T03:27:17.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:16 smithi161 ceph-mon[35485]: scrub summary: idle 2024-09-12T03:27:17.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:16 smithi161 ceph-mon[35485]: scrub summary: idle 2024-09-12T03:27:17.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:16 smithi118 ceph-mon[35048]: pgmap v520: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 160 KiB/s rd, 99 MiB/s wr, 11.25k op/s 2024-09-12T03:27:17.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:16 smithi118 ceph-mon[35048]: scrub summary: idle 2024-09-12T03:27:17.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:16 smithi118 ceph-mon[35048]: scrub summary: idle 2024-09-12T03:27:17.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:16 smithi118 ceph-mon[35048]: scrub summary: idle 2024-09-12T03:27:17.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.819+0000 7fe02f9fe640 1 -- 172.21.15.6:0/2740711680 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe02811b140 msgr2=0x7fe02811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:17.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.819+0000 7fe02f9fe640 1 --2- 172.21.15.6:0/2740711680 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe02811b140 0x7fe02811d530 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe020009be0 tx=0x7fe02002a710 comp rx=0 tx=0).stop 2024-09-12T03:27:17.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.819+0000 7fe02f9fe640 1 -- 172.21.15.6:0/2740711680 shutdown_connections 2024-09-12T03:27:17.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.819+0000 7fe02f9fe640 1 --2- 172.21.15.6:0/2740711680 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe02811b140 0x7fe02811d530 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.819+0000 7fe02f9fe640 1 --2- 172.21.15.6:0/2740711680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe028109830 0x7fe028109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.819+0000 7fe02f9fe640 1 --2- 172.21.15.6:0/2740711680 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe028109450 0x7fe028110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.819+0000 7fe02f9fe640 1 -- 172.21.15.6:0/2740711680 >> 172.21.15.6:0/2740711680 conn(0x7fe02806e140 msgr2=0x7fe02806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:17.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.819+0000 7fe02f9fe640 1 -- 172.21.15.6:0/2740711680 shutdown_connections 2024-09-12T03:27:17.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.819+0000 7fe02f9fe640 1 -- 172.21.15.6:0/2740711680 wait complete. 2024-09-12T03:27:17.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.820+0000 7fe02f9fe640 1 Processor -- start 2024-09-12T03:27:17.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.820+0000 7fe02f9fe640 1 -- start start 2024-09-12T03:27:17.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.821+0000 7fe02f9fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe028109450 0x7fe0281a7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.821+0000 7fe02f9fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe028109830 0x7fe0281a7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.821+0000 7fe02f9fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe02811b140 0x7fe0281abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.821+0000 7fe02f9fe640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe028120da0 con 0x7fe02811b140 2024-09-12T03:27:17.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.821+0000 7fe02f9fe640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe028120c20 con 0x7fe028109830 2024-09-12T03:27:17.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.821+0000 7fe02f9fe640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe028120f20 con 0x7fe028109450 2024-09-12T03:27:17.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.822+0000 7fe02df9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe02811b140 0x7fe0281abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.822+0000 7fe02df9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe02811b140 0x7fe0281abaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56508/0 (socket says 172.21.15.6:56508) 2024-09-12T03:27:17.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.822+0000 7fe02df9b640 1 -- 172.21.15.6:0/3647355505 learned_addr learned my addr 172.21.15.6:0/3647355505 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:17.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.823+0000 7fe02cf99640 1 --2- 172.21.15.6:0/3647355505 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe028109830 0x7fe0281a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.823+0000 7fe02d79a640 1 --2- 172.21.15.6:0/3647355505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe028109450 0x7fe0281a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.823+0000 7fe02cf99640 1 -- 172.21.15.6:0/3647355505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe028109450 msgr2=0x7fe0281a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:17.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.824+0000 7fe02cf99640 1 --2- 172.21.15.6:0/3647355505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe028109450 0x7fe0281a7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.824+0000 7fe02cf99640 1 -- 172.21.15.6:0/3647355505 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe02811b140 msgr2=0x7fe0281abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:17.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.824+0000 7fe02cf99640 1 --2- 172.21.15.6:0/3647355505 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe02811b140 0x7fe0281abaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.824+0000 7fe02cf99640 1 -- 172.21.15.6:0/3647355505 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0281ac1d0 con 0x7fe028109830 2024-09-12T03:27:17.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.824+0000 7fe02df9b640 1 --2- 172.21.15.6:0/3647355505 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe02811b140 0x7fe0281abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:17.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.824+0000 7fe02cf99640 1 --2- 172.21.15.6:0/3647355505 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe028109830 0x7fe0281a7760 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe02400d6d0 tx=0x7fe02400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:17.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.825+0000 7fe01e7fc640 1 -- 172.21.15.6:0/3647355505 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe024017070 con 0x7fe028109830 2024-09-12T03:27:17.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.825+0000 7fe02f9fe640 1 -- 172.21.15.6:0/3647355505 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0281ac4c0 con 0x7fe028109830 2024-09-12T03:27:17.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.825+0000 7fe02f9fe640 1 -- 172.21.15.6:0/3647355505 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0281b3da0 con 0x7fe028109830 2024-09-12T03:27:17.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.826+0000 7fe01e7fc640 1 -- 172.21.15.6:0/3647355505 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe024010460 con 0x7fe028109830 2024-09-12T03:27:17.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.827+0000 7fe01e7fc640 1 -- 172.21.15.6:0/3647355505 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02400f620 con 0x7fe028109830 2024-09-12T03:27:17.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.829+0000 7fe031dd7740 1 Processor -- start 2024-09-12T03:27:17.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.829+0000 7fe031dd7740 1 -- start start 2024-09-12T03:27:17.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.829+0000 7fe031dd7740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ad4b48060 0x560ad4b48440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.829+0000 7fe031dd7740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ad4b48980 0x560ad4b52d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.829+0000 7fe031dd7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b53240 0x560ad4b55630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.830+0000 7fe031dd7740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x560ad4aad620 con 0x560ad4b48980 2024-09-12T03:27:17.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.830+0000 7fe031dd7740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x560ad4a54e50 con 0x560ad4b48060 2024-09-12T03:27:17.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.830+0000 7fe031dd7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560ad4aa95e0 con 0x560ad4b53240 2024-09-12T03:27:17.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.830+0000 7fe02df9b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ad4b48060 0x560ad4b48440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.830+0000 7fe02df9b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ad4b48060 0x560ad4b48440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:44136/0 (socket says 172.21.15.6:44136) 2024-09-12T03:27:17.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.830+0000 7fe02df9b640 1 -- 172.21.15.6:0/3814120906 learned_addr learned my addr 172.21.15.6:0/3814120906 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:17.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.830+0000 7fe02df9b640 1 -- 172.21.15.6:0/3814120906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b53240 msgr2=0x560ad4b55630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:27:17.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.830+0000 7fe02df9b640 1 --2- 172.21.15.6:0/3814120906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b53240 0x560ad4b55630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.830+0000 7fe02df9b640 1 -- 172.21.15.6:0/3814120906 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ad4b48980 msgr2=0x560ad4b52d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:27:17.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.832+0000 7fe01e7fc640 1 -- 172.21.15.6:0/3647355505 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe02401a070 con 0x7fe028109830 2024-09-12T03:27:17.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.833+0000 7fe01e7fc640 1 --2- 172.21.15.6:0/3647355505 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe0140a2c40 0x7fe0140a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.833+0000 7fe01e7fc640 1 -- 172.21.15.6:0/3647355505 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe024098f90 con 0x7fe028109830 2024-09-12T03:27:17.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.834+0000 7fe02df9b640 1 --2- 172.21.15.6:0/3814120906 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ad4b48980 0x560ad4b52d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.834+0000 7fe02df9b640 1 -- 172.21.15.6:0/3814120906 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ad4a45d40 con 0x560ad4b48060 2024-09-12T03:27:17.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.834+0000 7fe02df9b640 1 --2- 172.21.15.6:0/3814120906 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ad4b48060 0x560ad4b48440 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe0200307e0 tx=0x7fe020030810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=762fec72afdaf3b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:17.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.835+0000 7fe0137fe640 1 -- 172.21.15.6:0/3814120906 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020041030 con 0x560ad4b48060 2024-09-12T03:27:17.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.835+0000 7fe0137fe640 1 -- 172.21.15.6:0/3814120906 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe020030e60 con 0x560ad4b48060 2024-09-12T03:27:17.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.835+0000 7fe0137fe640 1 -- 172.21.15.6:0/3814120906 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020036040 con 0x560ad4b48060 2024-09-12T03:27:17.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.835+0000 7fe02cf99640 1 --2- 172.21.15.6:0/3647355505 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe0140a2c40 0x7fe0140a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.836+0000 7fe031dd7740 1 -- 172.21.15.6:0/3814120906 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ad4b48060 msgr2=0x560ad4b48440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:17.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.836+0000 7fe031dd7740 1 --2- 172.21.15.6:0/3814120906 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ad4b48060 0x560ad4b48440 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe0200307e0 tx=0x7fe020030810 comp rx=0 tx=0).stop 2024-09-12T03:27:17.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.836+0000 7fe031dd7740 1 -- 172.21.15.6:0/3814120906 shutdown_connections 2024-09-12T03:27:17.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.836+0000 7fe031dd7740 1 --2- 172.21.15.6:0/3814120906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b53240 0x560ad4b55630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.836+0000 7fe031dd7740 1 --2- 172.21.15.6:0/3814120906 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ad4b48980 0x560ad4b52d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.836+0000 7fe031dd7740 1 --2- 172.21.15.6:0/3814120906 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ad4b48060 0x560ad4b48440 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.837+0000 7fe031dd7740 1 -- 172.21.15.6:0/3814120906 >> 172.21.15.6:0/3814120906 conn(0x560ad4b445f0 msgr2=0x560ad4b449d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:17.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.837+0000 7fe031dd7740 1 -- 172.21.15.6:0/3814120906 shutdown_connections 2024-09-12T03:27:17.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.837+0000 7fe031dd7740 1 -- 172.21.15.6:0/3814120906 wait complete. 2024-09-12T03:27:17.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.837+0000 7fe031dd7740 1 -- start start 2024-09-12T03:27:17.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.837+0000 7fe031dd7740 1 Processor -- start 2024-09-12T03:27:17.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.838+0000 7fe031dd7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b48060 0x560ad4b50130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.838+0000 7fe031dd7740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ad4b48980 0x560ad4b50670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.838+0000 7fe031dd7740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ad4b53240 0x560ad4b50bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.838+0000 7fe031dd7740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x560ad4b55e10 con 0x560ad4b53240 2024-09-12T03:27:17.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.838+0000 7fe031dd7740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x560ad4a35f10 con 0x560ad4b48980 2024-09-12T03:27:17.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.838+0000 7fe031dd7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560ad4b55f90 con 0x560ad4b48060 2024-09-12T03:27:17.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.838+0000 7fe031dd7740 10 client.0 _ll_register_callbacks cb 0x560ad4a59db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:17.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.839+0000 7fe02cf99640 1 --2- 172.21.15.6:0/3647355505 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe0140a2c40 0x7fe0140a5100 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe024098340 tx=0x7fe0240a03c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:17.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.841+0000 7fe02df9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b48060 0x560ad4b50130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.841+0000 7fe02df9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b48060 0x560ad4b50130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:33810/0 (socket says 172.21.15.6:33810) 2024-09-12T03:27:17.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.841+0000 7fe02df9b640 1 -- 172.21.15.6:0/2112011659 learned_addr learned my addr 172.21.15.6:0/2112011659 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:17.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.841+0000 7fe02df9b640 1 --2- 172.21.15.6:0/2112011659 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ad4b53240 0x560ad4b50bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.841+0000 7fe02df9b640 1 -- 172.21.15.6:0/2112011659 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ad4b48980 msgr2=0x560ad4b50670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:27:17.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.841+0000 7fe02df9b640 1 --2- 172.21.15.6:0/2112011659 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ad4b48980 0x560ad4b50670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.841+0000 7fe02df9b640 1 -- 172.21.15.6:0/2112011659 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ad4b53240 msgr2=0x560ad4b50bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:17.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.841+0000 7fe02df9b640 1 --2- 172.21.15.6:0/2112011659 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ad4b53240 0x560ad4b50bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.841+0000 7fe02df9b640 1 -- 172.21.15.6:0/2112011659 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ad47bc580 con 0x560ad4b48060 2024-09-12T03:27:17.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.843+0000 7fe02df9b640 1 --2- 172.21.15.6:0/2112011659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b48060 0x560ad4b50130 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe020031e40 tx=0x7fe02003d460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c133ebe56abbb5dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:17.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.843+0000 7fe010ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:27:17.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.843+0000 7fe010ff9640 1 -- 172.21.15.6:0/2112011659 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020036030 con 0x560ad4b48060 2024-09-12T03:27:17.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.844+0000 7fe010ff9640 1 -- 172.21.15.6:0/2112011659 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe020030b80 con 0x560ad4b48060 2024-09-12T03:27:17.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.844+0000 7fe031dd7740 1 -- 172.21.15.6:0/2112011659 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe020030e60 con 0x560ad4b48060 2024-09-12T03:27:17.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.844+0000 7fe010ff9640 1 -- 172.21.15.6:0/2112011659 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe020041040 con 0x560ad4b48060 2024-09-12T03:27:17.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.844+0000 7fe031dd7740 10 client.25681 fetch_fsmap learned FSMap version 27 2024-09-12T03:27:17.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.844+0000 7fe031dd7740 1 -- 172.21.15.6:0/2112011659 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560ad4886960 con 0x560ad4b48060 2024-09-12T03:27:17.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.846+0000 7fe010ff9640 1 -- 172.21.15.6:0/2112011659 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe02004a030 con 0x560ad4b48060 2024-09-12T03:27:17.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.847+0000 7fe010ff9640 1 -- 172.21.15.6:0/2112011659 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe020046600 con 0x560ad4b48060 2024-09-12T03:27:17.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.847+0000 7fe031dd7740 10 client.25681 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:27:17.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.847+0000 7fe031dd7740 10 client.25681 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:27:17.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.847+0000 7fe031dd7740 20 client.25681 populate_metadata read hostname 'smithi006' 2024-09-12T03:27:17.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.847+0000 7fe031dd7740 1 --2- 172.21.15.6:0/2112011659 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x560ad4b62c80 0x560ad4b65070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.848+0000 7fe031dd7740 4 client.25681 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:17.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.848+0000 7fe031dd7740 1 -- 172.21.15.6:0/2112011659 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe020030e60 con 0x560ad4b62c80 2024-09-12T03:27:17.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.848+0000 7fe02d79a640 1 --2- 172.21.15.6:0/2112011659 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x560ad4b62c80 0x560ad4b65070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.857+0000 7fe010ff9640 10 client.25681 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:17.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.857+0000 7fe02d79a640 1 --2- 172.21.15.6:0/2112011659 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x560ad4b62c80 0x560ad4b65070 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=297d519e1efe1f87 server_cookie=4d7163f468ee5836 in_seq=0 out_seq=0 2024-09-12T03:27:17.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.859+0000 7fe010ff9640 1 -- 172.21.15.6:0/2112011659 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe020030e60 con 0x560ad4b62c80 2024-09-12T03:27:17.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.859+0000 7fe010ff9640 10 client.25681 handle_command_reply: tid=0 2024-09-12T03:27:17.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.859+0000 7fe031dd7740 1 client.25681 shutdown 2024-09-12T03:27:17.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.859+0000 7fe031dd7740 1 -- 172.21.15.6:0/2112011659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b48060 msgr2=0x560ad4b50130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:17.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.859+0000 7fe031dd7740 1 --2- 172.21.15.6:0/2112011659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b48060 0x560ad4b50130 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe020031e40 tx=0x7fe02003d460 comp rx=0 tx=0).stop 2024-09-12T03:27:17.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.859+0000 7fe031dd7740 1 -- 172.21.15.6:0/2112011659 shutdown_connections 2024-09-12T03:27:17.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.859+0000 7fe031dd7740 1 --2- 172.21.15.6:0/2112011659 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x560ad4b62c80 0x560ad4b65070 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.859+0000 7fe010ff9640 0 client.25681 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:17.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.860+0000 7fe031dd7740 1 --2- 172.21.15.6:0/2112011659 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ad4b53240 0x560ad4b50bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.860+0000 7fe031dd7740 1 --2- 172.21.15.6:0/2112011659 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ad4b48980 0x560ad4b50670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.860+0000 7fe031dd7740 1 --2- 172.21.15.6:0/2112011659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b48060 0x560ad4b50130 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.860+0000 7fe031dd7740 1 -- 172.21.15.6:0/2112011659 >> 172.21.15.6:0/2112011659 conn(0x560ad4b445f0 msgr2=0x560ad4b4a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:17.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.862+0000 7fe031dd7740 1 -- 172.21.15.6:0/2112011659 shutdown_connections 2024-09-12T03:27:17.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.862+0000 7fe031dd7740 1 -- 172.21.15.6:0/2112011659 wait complete. 2024-09-12T03:27:17.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.862+0000 7fe031dd7740 20 client.25681 trim_cache size 0 max 16384 2024-09-12T03:27:17.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.898+0000 7fe031dd7740 1 Processor -- start 2024-09-12T03:27:17.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.899+0000 7fe031dd7740 1 -- start start 2024-09-12T03:27:17.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.900+0000 7fe031dd7740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ad4b48060 0x560ad4b6a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.901+0000 7fe031dd7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b48980 0x560ad4b6ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.901+0000 7fe031dd7740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ad4b53240 0x560ad4b4d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.901+0000 7fe031dd7740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c010b00 con 0x560ad4b48060 2024-09-12T03:27:17.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.901+0000 7fe031dd7740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c00e3d0 con 0x560ad4b53240 2024-09-12T03:27:17.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.902+0000 7fe031dd7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c00c2d0 con 0x560ad4b48980 2024-09-12T03:27:17.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.902+0000 7fe02df9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ad4b48060 0x560ad4b6a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.902+0000 7fe02df9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ad4b48060 0x560ad4b6a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56520/0 (socket says 172.21.15.6:56520) 2024-09-12T03:27:17.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.902+0000 7fe02df9b640 1 -- 172.21.15.6:0/3479030903 learned_addr learned my addr 172.21.15.6:0/3479030903 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:17.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.902+0000 7fe02d79a640 1 --2- 172.21.15.6:0/3479030903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b48980 0x560ad4b6ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.903+0000 7fe02d79a640 1 -- 172.21.15.6:0/3479030903 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ad4b53240 msgr2=0x560ad4b4d3b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:27:17.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.903+0000 7fe02d79a640 1 --2- 172.21.15.6:0/3479030903 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ad4b53240 0x560ad4b4d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.903+0000 7fe02d79a640 1 -- 172.21.15.6:0/3479030903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ad4b48060 msgr2=0x560ad4b6a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:17.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.903+0000 7fe02d79a640 1 --2- 172.21.15.6:0/3479030903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ad4b48060 0x560ad4b6a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.903+0000 7fe02d79a640 1 -- 172.21.15.6:0/3479030903 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ad4b44090 con 0x560ad4b48980 2024-09-12T03:27:17.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.904+0000 7fe02d79a640 1 --2- 172.21.15.6:0/3479030903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b48980 0x560ad4b6ac70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x560ad4b656d0 tx=0x7fe0180046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7838ccd9514046ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:17.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.904+0000 7fe012ffd640 1 -- 172.21.15.6:0/3479030903 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01801b030 con 0x560ad4b48980 2024-09-12T03:27:17.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.904+0000 7fe012ffd640 1 -- 172.21.15.6:0/3479030903 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0180054d0 con 0x560ad4b48980 2024-09-12T03:27:17.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.905+0000 7fe031dd7740 1 -- 172.21.15.6:0/3479030903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b48980 msgr2=0x560ad4b6ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:17.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.905+0000 7fe031dd7740 1 --2- 172.21.15.6:0/3479030903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b48980 0x560ad4b6ac70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x560ad4b656d0 tx=0x7fe0180046c0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.906+0000 7fe02df9b640 1 --2- 172.21.15.6:0/3479030903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ad4b48060 0x560ad4b6a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:17.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.906+0000 7fe031dd7740 1 -- 172.21.15.6:0/3479030903 shutdown_connections 2024-09-12T03:27:17.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.906+0000 7fe031dd7740 1 --2- 172.21.15.6:0/3479030903 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ad4b53240 0x560ad4b4d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.906+0000 7fe031dd7740 1 --2- 172.21.15.6:0/3479030903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b48980 0x560ad4b6ac70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.906+0000 7fe031dd7740 1 --2- 172.21.15.6:0/3479030903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ad4b48060 0x560ad4b6a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.906+0000 7fe031dd7740 1 -- 172.21.15.6:0/3479030903 >> 172.21.15.6:0/3479030903 conn(0x560ad4b445f0 msgr2=0x560ad4b4ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:17.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.907+0000 7fe031dd7740 1 -- 172.21.15.6:0/3479030903 shutdown_connections 2024-09-12T03:27:17.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.907+0000 7fe031dd7740 1 -- 172.21.15.6:0/3479030903 wait complete. 2024-09-12T03:27:17.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.907+0000 7fe031dd7740 1 -- start start 2024-09-12T03:27:17.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.907+0000 7fe031dd7740 1 Processor -- start 2024-09-12T03:27:17.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.908+0000 7fe031dd7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b48060 0x560ad4b5b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.908+0000 7fe031dd7740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ad4b48980 0x560ad4b5b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.908+0000 7fe031dd7740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ad4b53240 0x560ad4b58210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.908+0000 7fe031dd7740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c00c890 con 0x560ad4b48980 2024-09-12T03:27:17.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.908+0000 7fe031dd7740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c00dd00 con 0x560ad4b53240 2024-09-12T03:27:17.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.908+0000 7fe031dd7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c00cf60 con 0x560ad4b48060 2024-09-12T03:27:17.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.908+0000 7fe031dd7740 10 client.0 _ll_register_callbacks cb 0x7fe020031990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:17.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.908+0000 7fe02df9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b48060 0x560ad4b5b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.908+0000 7fe02df9b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ad4b53240 0x560ad4b58210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.909+0000 7fe02df9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b48060 0x560ad4b5b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:33822/0 (socket says 172.21.15.6:33822) 2024-09-12T03:27:17.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.909+0000 7fe02df9b640 1 -- 172.21.15.6:0/2407278542 learned_addr learned my addr 172.21.15.6:0/2407278542 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:17.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.910+0000 7fe02df9b640 1 -- 172.21.15.6:0/2407278542 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ad4b53240 msgr2=0x560ad4b58210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:17.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.910+0000 7fe02df9b640 1 --2- 172.21.15.6:0/2407278542 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ad4b53240 0x560ad4b58210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.910+0000 7fe02df9b640 1 -- 172.21.15.6:0/2407278542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ad4b48980 msgr2=0x560ad4b5b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:17.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.910+0000 7fe02df9b640 1 --2- 172.21.15.6:0/2407278542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ad4b48980 0x560ad4b5b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.910+0000 7fe02df9b640 1 -- 172.21.15.6:0/2407278542 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ad4b58a40 con 0x560ad4b48060 2024-09-12T03:27:17.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.911+0000 7fe02df9b640 1 --2- 172.21.15.6:0/2407278542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b48060 0x560ad4b5b420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe020031e40 tx=0x7fe020031ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e742a6a0f48c4ce4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:17.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.911+0000 7fe010ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:27:17.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.912+0000 7fe010ff9640 1 -- 172.21.15.6:0/2407278542 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020063060 con 0x560ad4b48060 2024-09-12T03:27:17.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.912+0000 7fe031dd7740 1 -- 172.21.15.6:0/2407278542 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560ad4abab90 con 0x560ad4b48060 2024-09-12T03:27:17.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.912+0000 7fe010ff9640 1 -- 172.21.15.6:0/2407278542 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe02003d4d0 con 0x560ad4b48060 2024-09-12T03:27:17.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.912+0000 7fe010ff9640 1 -- 172.21.15.6:0/2407278542 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe020036070 con 0x560ad4b48060 2024-09-12T03:27:17.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.913+0000 7fe031dd7740 10 client.25687 fetch_fsmap learned FSMap version 27 2024-09-12T03:27:17.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.913+0000 7fe031dd7740 1 -- 172.21.15.6:0/2407278542 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560ad4bce1f0 con 0x560ad4b48060 2024-09-12T03:27:17.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.913+0000 7fe010ff9640 1 -- 172.21.15.6:0/2407278542 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560ad4abab90 con 0x560ad4b48060 2024-09-12T03:27:17.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.913+0000 7fe010ff9640 1 -- 172.21.15.6:0/2407278542 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe020045780 con 0x560ad4b48060 2024-09-12T03:27:17.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.913+0000 7fe031dd7740 10 client.25687 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:27:17.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.913+0000 7fe031dd7740 10 client.25687 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:27:17.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.913+0000 7fe031dd7740 20 client.25687 populate_metadata read hostname 'smithi006' 2024-09-12T03:27:17.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.913+0000 7fe031dd7740 1 --2- 172.21.15.6:0/2407278542 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x560ad4bce8f0 0x560ad4bd0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.913+0000 7fe031dd7740 4 client.25687 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f777655b", "release": true, "await": true}] 2024-09-12T03:27:17.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.913+0000 7fe031dd7740 1 -- 172.21.15.6:0/2407278542 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f777655b", "release": true, "await": true}) -- 0x560ad4ab50a0 con 0x560ad4bce8f0 2024-09-12T03:27:17.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.915+0000 7fe02d79a640 1 --2- 172.21.15.6:0/2407278542 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x560ad4bce8f0 0x560ad4bd0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.919+0000 7fe010ff9640 10 client.25687 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:17.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.919+0000 7fe02d79a640 1 --2- 172.21.15.6:0/2407278542 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x560ad4bce8f0 0x560ad4bd0d10 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9de493a99ba51aa5 server_cookie=b40105970e2aa87a in_seq=0 out_seq=0 2024-09-12T03:27:18.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.963+0000 7fe010ff9640 1 -- 172.21.15.6:0/2407278542 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560ad4ab50a0 con 0x560ad4bce8f0 2024-09-12T03:27:18.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.963+0000 7fe010ff9640 10 client.25687 handle_command_reply: tid=0 2024-09-12T03:27:18.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.963+0000 7fe031dd7740 1 client.25687 shutdown 2024-09-12T03:27:18.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.963+0000 7fe031dd7740 1 -- 172.21.15.6:0/2407278542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b48060 msgr2=0x560ad4b5b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:18.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.963+0000 7fe031dd7740 1 --2- 172.21.15.6:0/2407278542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b48060 0x560ad4b5b420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe020031e40 tx=0x7fe020031ea0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.972+0000 7fe031dd7740 1 -- 172.21.15.6:0/2407278542 shutdown_connections 2024-09-12T03:27:18.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.972+0000 7fe031dd7740 1 --2- 172.21.15.6:0/2407278542 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x560ad4bce8f0 0x560ad4bd0d10 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.972+0000 7fe031dd7740 1 --2- 172.21.15.6:0/2407278542 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ad4b53240 0x560ad4b58210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.972+0000 7fe031dd7740 1 --2- 172.21.15.6:0/2407278542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ad4b48980 0x560ad4b5b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.972+0000 7fe031dd7740 1 --2- 172.21.15.6:0/2407278542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ad4b48060 0x560ad4b5b420 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.972+0000 7fe031dd7740 1 -- 172.21.15.6:0/2407278542 >> 172.21.15.6:0/2407278542 conn(0x560ad4b445f0 msgr2=0x560ad4b49ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:18.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.972+0000 7fe010ff9640 0 client.25687 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:18.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.975+0000 7fe031dd7740 1 -- 172.21.15.6:0/2407278542 shutdown_connections 2024-09-12T03:27:18.022 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:27:18.022 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 27, 2024-09-12T03:27:18.022 INFO:teuthology.orchestra.run.smithi006.stdout: "leader": 24494, 2024-09-12T03:27:18.022 INFO:teuthology.orchestra.run.smithi006.stdout: "set_version": 24, 2024-09-12T03:27:18.022 INFO:teuthology.orchestra.run.smithi006.stdout: "sets": { 2024-09-12T03:27:18.022 INFO:teuthology.orchestra.run.smithi006.stdout: "f777655b": { 2024-09-12T03:27:18.022 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 24, 2024-09-12T03:27:18.022 INFO:teuthology.orchestra.run.smithi006.stdout: "age_ref": 0.0, 2024-09-12T03:27:18.022 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:27:18.022 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RELEASED", 2024-09-12T03:27:18.022 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:27:18.023 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:27:18.023 INFO:teuthology.orchestra.run.smithi006.stdout: "timeout": 90.0, 2024-09-12T03:27:18.023 INFO:teuthology.orchestra.run.smithi006.stdout: "expiration": 140.1, 2024-09-12T03:27:18.023 INFO:teuthology.orchestra.run.smithi006.stdout: "members": { 2024-09-12T03:27:18.023 INFO:teuthology.orchestra.run.smithi006.stdout: "file:/": { 2024-09-12T03:27:18.023 INFO:teuthology.orchestra.run.smithi006.stdout: "excluded": false, 2024-09-12T03:27:18.023 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:27:18.023 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RELEASED", 2024-09-12T03:27:18.023 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:27:18.023 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:27:18.023 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:27:18.023 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:27:18.023 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:27:18.023 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:27:18.024 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:27:18.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.975+0000 7fe031dd7740 1 -- 172.21.15.6:0/2407278542 wait complete. 2024-09-12T03:27:18.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.975+0000 7fe031dd7740 20 client.25687 trim_cache size 0 max 16384 2024-09-12T03:27:18.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.985+0000 7fe02f9fe640 1 -- 172.21.15.6:0/3647355505 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe0140a2c40 msgr2=0x7fe0140a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:18.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.985+0000 7fe02f9fe640 1 --2- 172.21.15.6:0/3647355505 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe0140a2c40 0x7fe0140a5100 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe024098340 tx=0x7fe0240a03c0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.985+0000 7fe02f9fe640 1 -- 172.21.15.6:0/3647355505 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe028109830 msgr2=0x7fe0281a7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:18.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.985+0000 7fe02f9fe640 1 --2- 172.21.15.6:0/3647355505 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe028109830 0x7fe0281a7760 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe02400d6d0 tx=0x7fe02400db90 comp rx=0 tx=0).stop 2024-09-12T03:27:18.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.985+0000 7fe02f9fe640 1 -- 172.21.15.6:0/3647355505 shutdown_connections 2024-09-12T03:27:18.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.985+0000 7fe02f9fe640 1 --2- 172.21.15.6:0/3647355505 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe0140a2c40 0x7fe0140a5100 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.985+0000 7fe02f9fe640 1 --2- 172.21.15.6:0/3647355505 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe02811b140 0x7fe0281abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.985+0000 7fe02f9fe640 1 --2- 172.21.15.6:0/3647355505 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe028109830 0x7fe0281a7760 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.985+0000 7fe02f9fe640 1 --2- 172.21.15.6:0/3647355505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe028109450 0x7fe0281a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.985+0000 7fe02f9fe640 1 -- 172.21.15.6:0/3647355505 >> 172.21.15.6:0/3647355505 conn(0x7fe02806e140 msgr2=0x7fe02811c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:18.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.990+0000 7fe02f9fe640 1 -- 172.21.15.6:0/3647355505 shutdown_connections 2024-09-12T03:27:18.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:17.990+0000 7fe02f9fe640 1 -- 172.21.15.6:0/3647355505 wait complete. 2024-09-12T03:27:19.168 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f777655b', total seconds elapsed: 24.4 2024-09-12T03:27:19.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:19 smithi006 ceph-mon[30991]: pgmap v521: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 162 KiB/s rd, 58 MiB/s wr, 6.59k op/s 2024-09-12T03:27:19.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:19 smithi161 ceph-mon[35485]: pgmap v521: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 162 KiB/s rd, 58 MiB/s wr, 6.59k op/s 2024-09-12T03:27:19.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:19 smithi118 ceph-mon[35048]: pgmap v521: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 162 KiB/s rd, 58 MiB/s wr, 6.59k op/s 2024-09-12T03:27:21.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:21 smithi006 ceph-mon[30991]: pgmap v522: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 6.8 KiB/s rd, 56 MiB/s wr, 6.13k op/s 2024-09-12T03:27:21.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:21 smithi161 ceph-mon[35485]: pgmap v522: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 6.8 KiB/s rd, 56 MiB/s wr, 6.13k op/s 2024-09-12T03:27:21.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:21 smithi118 ceph-mon[35048]: pgmap v522: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 6.8 KiB/s rd, 56 MiB/s wr, 6.13k op/s 2024-09-12T03:27:23.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:23 smithi006 ceph-mon[30991]: pgmap v523: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 189 KiB/s rd, 46 MiB/s wr, 4.89k op/s 2024-09-12T03:27:23.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:23 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:27:23.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:23 smithi161 ceph-mon[35485]: pgmap v523: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 189 KiB/s rd, 46 MiB/s wr, 4.89k op/s 2024-09-12T03:27:23.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:23 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:27:23.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:23 smithi118 ceph-mon[35048]: pgmap v523: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 189 KiB/s rd, 46 MiB/s wr, 4.89k op/s 2024-09-12T03:27:23.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:23 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:27:24.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:27:24.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:27:24.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:27:24.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:27:24.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:27:24.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:27:24.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:27:24.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:27:24.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:27:25.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:25 smithi006 ceph-mon[30991]: pgmap v524: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 187 KiB/s rd, 62 MiB/s wr, 6.59k op/s 2024-09-12T03:27:25.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:25 smithi161 ceph-mon[35485]: pgmap v524: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 187 KiB/s rd, 62 MiB/s wr, 6.59k op/s 2024-09-12T03:27:25.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:25 smithi118 ceph-mon[35048]: pgmap v524: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 187 KiB/s rd, 62 MiB/s wr, 6.59k op/s 2024-09-12T03:27:27.322 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:27:27.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:27 smithi006 ceph-mon[30991]: pgmap v525: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 361 KiB/s rd, 105 MiB/s wr, 11.29k op/s 2024-09-12T03:27:27.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:27 smithi161 ceph-mon[35485]: pgmap v525: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 361 KiB/s rd, 105 MiB/s wr, 11.29k op/s 2024-09-12T03:27:27.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:27 smithi118 ceph-mon[35048]: pgmap v525: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 361 KiB/s rd, 105 MiB/s wr, 11.29k op/s 2024-09-12T03:27:27.873 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:29.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.985+0000 7f8d3759e640 1 -- 172.21.15.6:0/1385970044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d38108280 msgr2=0x7f8d38108660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:29.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.985+0000 7f8d3759e640 1 --2- 172.21.15.6:0/1385970044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d38108280 0x7f8d38108660 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8d28009900 tx=0x7f8d2802a440 comp rx=0 tx=0).stop 2024-09-12T03:27:29.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.987+0000 7f8d3759e640 1 -- 172.21.15.6:0/1385970044 shutdown_connections 2024-09-12T03:27:29.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.987+0000 7f8d3759e640 1 --2- 172.21.15.6:0/1385970044 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8d3810d360 0x7f8d38114ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.987+0000 7f8d3759e640 1 --2- 172.21.15.6:0/1385970044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d38108c30 0x7f8d3810cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.987+0000 7f8d3759e640 1 --2- 172.21.15.6:0/1385970044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d38108280 0x7f8d38108660 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.987+0000 7f8d3759e640 1 -- 172.21.15.6:0/1385970044 >> 172.21.15.6:0/1385970044 conn(0x7f8d380779f0 msgr2=0x7f8d38079e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:29.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.990+0000 7f8d3759e640 1 -- 172.21.15.6:0/1385970044 shutdown_connections 2024-09-12T03:27:29.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.992+0000 7f8d3759e640 1 -- 172.21.15.6:0/1385970044 wait complete. 2024-09-12T03:27:29.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.994+0000 7f8d3759e640 1 Processor -- start 2024-09-12T03:27:29.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.994+0000 7f8d3759e640 1 -- start start 2024-09-12T03:27:29.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.994+0000 7f8d3759e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d38108280 0x7f8d38082820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.994+0000 7f8d3759e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8d38108c30 0x7f8d3807b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.994+0000 7f8d3759e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d3810d360 0x7f8d3807bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.994+0000 7f8d3759e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8d38118430 con 0x7f8d38108280 2024-09-12T03:27:29.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.994+0000 7f8d3759e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f8d381182b0 con 0x7f8d38108c30 2024-09-12T03:27:29.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.994+0000 7f8d3759e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8d381185b0 con 0x7f8d3810d360 2024-09-12T03:27:29.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.994+0000 7f8d36d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d3810d360 0x7f8d3807bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:29.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.994+0000 7f8d36d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d3810d360 0x7f8d3807bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:60416/0 (socket says 172.21.15.6:60416) 2024-09-12T03:27:29.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.994+0000 7f8d36d9d640 1 -- 172.21.15.6:0/725045961 learned_addr learned my addr 172.21.15.6:0/725045961 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:29.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.994+0000 7f8d3659c640 1 --2- 172.21.15.6:0/725045961 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d38108280 0x7f8d38082820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:29.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:28.995+0000 7f8d35d9b640 1 --2- 172.21.15.6:0/725045961 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8d38108c30 0x7f8d3807b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:29.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.000+0000 7f8d36d9d640 1 -- 172.21.15.6:0/725045961 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8d38108c30 msgr2=0x7f8d3807b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:29.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.000+0000 7f8d36d9d640 1 --2- 172.21.15.6:0/725045961 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8d38108c30 0x7f8d3807b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.000+0000 7f8d36d9d640 1 -- 172.21.15.6:0/725045961 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d38108280 msgr2=0x7f8d38082820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:29.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.000+0000 7f8d36d9d640 1 --2- 172.21.15.6:0/725045961 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d38108280 0x7f8d38082820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.000+0000 7f8d36d9d640 1 -- 172.21.15.6:0/725045961 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d3807c3b0 con 0x7f8d3810d360 2024-09-12T03:27:29.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.000+0000 7f8d36d9d640 1 --2- 172.21.15.6:0/725045961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d3810d360 0x7f8d3807bdf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000ead0 tx=0x7f8d3000ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:29.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.003+0000 7f8d2f7fe640 1 -- 172.21.15.6:0/725045961 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3000cde0 con 0x7f8d3810d360 2024-09-12T03:27:29.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.003+0000 7f8d2f7fe640 1 -- 172.21.15.6:0/725045961 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d30005020 con 0x7f8d3810d360 2024-09-12T03:27:29.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.003+0000 7f8d3759e640 1 -- 172.21.15.6:0/725045961 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d3807c6a0 con 0x7f8d3810d360 2024-09-12T03:27:29.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.003+0000 7f8d3759e640 1 -- 172.21.15.6:0/725045961 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d38080670 con 0x7f8d3810d360 2024-09-12T03:27:29.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.004+0000 7f8d2f7fe640 1 -- 172.21.15.6:0/725045961 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d300116d0 con 0x7f8d3810d360 2024-09-12T03:27:29.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.011+0000 7f8d3eec1740 1 Processor -- start 2024-09-12T03:27:29.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.012+0000 7f8d3eec1740 1 -- start start 2024-09-12T03:27:29.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.012+0000 7f8d3eec1740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8670 0x555fd18f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.012+0000 7f8d3eec1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f8f90 0x555fd1903310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.012+0000 7f8d3eec1740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555fd1903850 0x555fd1905c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.012+0000 7f8d3eec1740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x555fd185e2e0 con 0x555fd1903850 2024-09-12T03:27:29.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.012+0000 7f8d3eec1740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x555fd1805ce0 con 0x555fd18f8670 2024-09-12T03:27:29.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.012+0000 7f8d3eec1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555fd185a2a0 con 0x555fd18f8f90 2024-09-12T03:27:29.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.013+0000 7f8d3659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f8f90 0x555fd1903310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:29.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.013+0000 7f8d3659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f8f90 0x555fd1903310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:60420/0 (socket says 172.21.15.6:60420) 2024-09-12T03:27:29.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.013+0000 7f8d3659c640 1 -- 172.21.15.6:0/1793109576 learned_addr learned my addr 172.21.15.6:0/1793109576 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:29.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.013+0000 7f8d35d9b640 1 --2- 172.21.15.6:0/1793109576 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555fd1903850 0x555fd1905c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:29.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.014+0000 7f8d35d9b640 1 --2- 172.21.15.6:0/1793109576 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8670 0x555fd18f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:29.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.014+0000 7f8d3659c640 1 -- 172.21.15.6:0/1793109576 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8670 msgr2=0x555fd18f8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:29.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.014+0000 7f8d3659c640 1 --2- 172.21.15.6:0/1793109576 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8670 0x555fd18f8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.014+0000 7f8d3659c640 1 -- 172.21.15.6:0/1793109576 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555fd1903850 msgr2=0x555fd1905c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:29.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.014+0000 7f8d3659c640 1 --2- 172.21.15.6:0/1793109576 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555fd1903850 0x555fd1905c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.014+0000 7f8d3659c640 1 -- 172.21.15.6:0/1793109576 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fd17ea440 con 0x555fd18f8f90 2024-09-12T03:27:29.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.015+0000 7f8d2f7fe640 1 -- 172.21.15.6:0/725045961 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8d300051d0 con 0x7f8d3810d360 2024-09-12T03:27:29.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.016+0000 7f8d2f7fe640 1 --2- 172.21.15.6:0/725045961 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8d240a2b80 0x7f8d240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.016+0000 7f8d2f7fe640 1 -- 172.21.15.6:0/725045961 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f8d300cf200 con 0x7f8d3810d360 2024-09-12T03:27:29.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.016+0000 7f8d3659c640 1 --2- 172.21.15.6:0/1793109576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f8f90 0x555fd1903310 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8d28000c00 tx=0x7f8d28004180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b223a20bba5c927 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:29.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.020+0000 7f8d35d9b640 1 --2- 172.21.15.6:0/725045961 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8d240a2b80 0x7f8d240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:29.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.020+0000 7f8d2cff9640 1 -- 172.21.15.6:0/1793109576 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28035040 con 0x555fd18f8f90 2024-09-12T03:27:29.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.020+0000 7f8d2cff9640 1 -- 172.21.15.6:0/1793109576 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d28005520 con 0x555fd18f8f90 2024-09-12T03:27:29.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.020+0000 7f8d2cff9640 1 -- 172.21.15.6:0/1793109576 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28003850 con 0x555fd18f8f90 2024-09-12T03:27:29.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.021+0000 7f8d3eec1740 1 -- 172.21.15.6:0/1793109576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f8f90 msgr2=0x555fd1903310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:29.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.021+0000 7f8d3eec1740 1 --2- 172.21.15.6:0/1793109576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f8f90 0x555fd1903310 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8d28000c00 tx=0x7f8d28004180 comp rx=0 tx=0).stop 2024-09-12T03:27:29.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.022+0000 7f8d35d9b640 1 --2- 172.21.15.6:0/725045961 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8d240a2b80 0x7f8d240a5040 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8d20006fd0 tx=0x7f8d2000a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:29.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.022+0000 7f8d3eec1740 1 -- 172.21.15.6:0/1793109576 shutdown_connections 2024-09-12T03:27:29.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.022+0000 7f8d3eec1740 1 --2- 172.21.15.6:0/1793109576 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555fd1903850 0x555fd1905c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.022+0000 7f8d3eec1740 1 --2- 172.21.15.6:0/1793109576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f8f90 0x555fd1903310 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.022+0000 7f8d3eec1740 1 --2- 172.21.15.6:0/1793109576 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8670 0x555fd18f8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.022+0000 7f8d3eec1740 1 -- 172.21.15.6:0/1793109576 >> 172.21.15.6:0/1793109576 conn(0x555fd18f4c00 msgr2=0x555fd18f4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:29.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.026+0000 7f8d3eec1740 1 -- 172.21.15.6:0/1793109576 shutdown_connections 2024-09-12T03:27:29.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.026+0000 7f8d3eec1740 1 -- 172.21.15.6:0/1793109576 wait complete. 2024-09-12T03:27:29.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.027+0000 7f8d3eec1740 1 -- start start 2024-09-12T03:27:29.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.027+0000 7f8d3eec1740 1 Processor -- start 2024-09-12T03:27:29.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.027+0000 7f8d3eec1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f8670 0x555fd1903170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.027+0000 7f8d3eec1740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8f90 0x555fd1900420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.027+0000 7f8d3eec1740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555fd1903850 0x555fd1900960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.027+0000 7f8d3eec1740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x555fd1906420 con 0x555fd1903850 2024-09-12T03:27:29.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.027+0000 7f8d3eec1740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x555fd1819050 con 0x555fd18f8f90 2024-09-12T03:27:29.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.027+0000 7f8d3eec1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555fd19065a0 con 0x555fd18f8670 2024-09-12T03:27:29.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.027+0000 7f8d3eec1740 10 client.0 _ll_register_callbacks cb 0x555fd1859160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:29.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.028+0000 7f8d35d9b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8f90 0x555fd1900420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:29.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.028+0000 7f8d35d9b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8f90 0x555fd1900420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:54192/0 (socket says 172.21.15.6:54192) 2024-09-12T03:27:29.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.028+0000 7f8d35d9b640 1 -- 172.21.15.6:0/3372407445 learned_addr learned my addr 172.21.15.6:0/3372407445 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:29.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.028+0000 7f8d3659c640 1 --2- 172.21.15.6:0/3372407445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f8670 0x555fd1903170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:29.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.029+0000 7f8d35d9b640 1 -- 172.21.15.6:0/3372407445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f8670 msgr2=0x555fd1903170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:29.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.029+0000 7f8d35d9b640 1 --2- 172.21.15.6:0/3372407445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f8670 0x555fd1903170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.029+0000 7f8d35d9b640 1 -- 172.21.15.6:0/3372407445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555fd1903850 msgr2=0x555fd1900960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:29.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.029+0000 7f8d35d9b640 1 --2- 172.21.15.6:0/3372407445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555fd1903850 0x555fd1900960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.029+0000 7f8d35d9b640 1 -- 172.21.15.6:0/3372407445 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fd15cf3c0 con 0x555fd18f8f90 2024-09-12T03:27:29.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.029+0000 7f8d35d9b640 1 --2- 172.21.15.6:0/3372407445 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8f90 0x555fd1900420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8d20017710 tx=0x7f8d20006da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f690d015607f5fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:29.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.029+0000 7f8d1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:27:29.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.029+0000 7f8d1dffb640 1 -- 172.21.15.6:0/3372407445 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d20027060 con 0x555fd18f8f90 2024-09-12T03:27:29.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.030+0000 7f8d1dffb640 1 -- 172.21.15.6:0/3372407445 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f8d20016950 con 0x555fd18f8f90 2024-09-12T03:27:29.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.030+0000 7f8d3eec1740 1 -- 172.21.15.6:0/3372407445 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d28005520 con 0x555fd18f8f90 2024-09-12T03:27:29.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.031+0000 7f8d1dffb640 1 -- 172.21.15.6:0/3372407445 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d20016b50 con 0x555fd18f8f90 2024-09-12T03:27:29.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.031+0000 7f8d3eec1740 10 client.25610 fetch_fsmap learned FSMap version 27 2024-09-12T03:27:29.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.031+0000 7f8d3eec1740 1 -- 172.21.15.6:0/3372407445 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555fd1911ec0 con 0x555fd18f8f90 2024-09-12T03:27:29.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.033+0000 7f8d1dffb640 1 -- 172.21.15.6:0/3372407445 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d28005520 con 0x555fd18f8f90 2024-09-12T03:27:29.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.033+0000 7f8d1dffb640 1 -- 172.21.15.6:0/3372407445 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8d20015040 con 0x555fd18f8f90 2024-09-12T03:27:29.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.034+0000 7f8d3eec1740 10 client.25610 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:27:29.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.034+0000 7f8d3eec1740 10 client.25610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:27:29.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.034+0000 7f8d3eec1740 20 client.25610 populate_metadata read hostname 'smithi006' 2024-09-12T03:27:29.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.034+0000 7f8d3eec1740 1 --2- 172.21.15.6:0/3372407445 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x555fd1913770 0x555fd1915c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.034+0000 7f8d3eec1740 4 client.25610 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:29.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.034+0000 7f8d3eec1740 1 -- 172.21.15.6:0/3372407445 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fd186b850 con 0x555fd1913770 2024-09-12T03:27:29.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.036+0000 7f8d3659c640 1 --2- 172.21.15.6:0/3372407445 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x555fd1913770 0x555fd1915c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:29.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.037+0000 7f8d3659c640 1 --2- 172.21.15.6:0/3372407445 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x555fd1913770 0x555fd1915c30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7f0cae9c373e40b server_cookie=ee71f519bd60ac97 in_seq=0 out_seq=0 2024-09-12T03:27:29.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.037+0000 7f8d1dffb640 10 client.25610 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:29.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.045+0000 7f8d1dffb640 1 -- 172.21.15.6:0/3372407445 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555fd186b850 con 0x555fd1913770 2024-09-12T03:27:29.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.045+0000 7f8d1dffb640 10 client.25610 handle_command_reply: tid=0 2024-09-12T03:27:29.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.045+0000 7f8d3eec1740 1 client.25610 shutdown 2024-09-12T03:27:29.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.048+0000 7f8d3eec1740 1 -- 172.21.15.6:0/3372407445 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8f90 msgr2=0x555fd1900420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:29.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.048+0000 7f8d3eec1740 1 --2- 172.21.15.6:0/3372407445 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8f90 0x555fd1900420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8d20017710 tx=0x7f8d20006da0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.048+0000 7f8d3eec1740 1 -- 172.21.15.6:0/3372407445 shutdown_connections 2024-09-12T03:27:29.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.048+0000 7f8d3eec1740 1 --2- 172.21.15.6:0/3372407445 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x555fd1913770 0x555fd1915c30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.048+0000 7f8d3eec1740 1 --2- 172.21.15.6:0/3372407445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555fd1903850 0x555fd1900960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.048+0000 7f8d3eec1740 1 --2- 172.21.15.6:0/3372407445 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8f90 0x555fd1900420 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.048+0000 7f8d3eec1740 1 --2- 172.21.15.6:0/3372407445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f8670 0x555fd1903170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.048+0000 7f8d3eec1740 1 -- 172.21.15.6:0/3372407445 >> 172.21.15.6:0/3372407445 conn(0x555fd18f4c00 msgr2=0x555fd18fa8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:29.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.048+0000 7f8d1dffb640 0 client.25610 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:29.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.049+0000 7f8d3eec1740 1 -- 172.21.15.6:0/3372407445 shutdown_connections 2024-09-12T03:27:29.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.053+0000 7f8d3eec1740 1 -- 172.21.15.6:0/3372407445 wait complete. 2024-09-12T03:27:29.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.053+0000 7f8d3eec1740 20 client.25610 trim_cache size 0 max 16384 2024-09-12T03:27:29.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.092+0000 7f8d3eec1740 1 Processor -- start 2024-09-12T03:27:29.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.093+0000 7f8d3eec1740 1 -- start start 2024-09-12T03:27:29.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.093+0000 7f8d3eec1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f4c00 0x555fd190f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.093+0000 7f8d3eec1740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8670 0x555fd19003e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.093+0000 7f8d3eec1740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555fd18f8f90 0x555fd191b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.093+0000 7f8d3eec1740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8d18010c90 con 0x555fd18f8f90 2024-09-12T03:27:29.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.093+0000 7f8d3eec1740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1800e560 con 0x555fd18f8670 2024-09-12T03:27:29.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.093+0000 7f8d3eec1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1800c3b0 con 0x555fd18f4c00 2024-09-12T03:27:29.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.093+0000 7f8d35d9b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8670 0x555fd19003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:29.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.093+0000 7f8d3659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f4c00 0x555fd190f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:29.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.093+0000 7f8d3659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f4c00 0x555fd190f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:60440/0 (socket says 172.21.15.6:60440) 2024-09-12T03:27:29.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.093+0000 7f8d35d9b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8670 0x555fd19003e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:54204/0 (socket says 172.21.15.6:54204) 2024-09-12T03:27:29.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.093+0000 7f8d3659c640 1 -- 172.21.15.6:0/1017218529 learned_addr learned my addr 172.21.15.6:0/1017218529 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:29.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.094+0000 7f8d35d9b640 1 -- 172.21.15.6:0/1017218529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f4c00 msgr2=0x555fd190f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:29.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.094+0000 7f8d35d9b640 1 --2- 172.21.15.6:0/1017218529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f4c00 0x555fd190f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.094+0000 7f8d35d9b640 1 -- 172.21.15.6:0/1017218529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555fd18f8f90 msgr2=0x555fd191b540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:27:29.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.094+0000 7f8d35d9b640 1 --2- 172.21.15.6:0/1017218529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555fd18f8f90 0x555fd191b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.094+0000 7f8d35d9b640 1 -- 172.21.15.6:0/1017218529 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fd1902d90 con 0x555fd18f8670 2024-09-12T03:27:29.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.094+0000 7f8d35d9b640 1 --2- 172.21.15.6:0/1017218529 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8670 0x555fd19003e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8d2002bf20 tx=0x7f8d20015f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc2df21e68316d20 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:29.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.095+0000 7f8d1ffff640 1 -- 172.21.15.6:0/1017218529 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d20031080 con 0x555fd18f8670 2024-09-12T03:27:29.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.095+0000 7f8d1ffff640 1 -- 172.21.15.6:0/1017218529 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d20016cf0 con 0x555fd18f8670 2024-09-12T03:27:29.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.096+0000 7f8d3eec1740 1 -- 172.21.15.6:0/1017218529 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8670 msgr2=0x555fd19003e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:29.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.096+0000 7f8d3eec1740 1 --2- 172.21.15.6:0/1017218529 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8670 0x555fd19003e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8d2002bf20 tx=0x7f8d20015f10 comp rx=0 tx=0).stop 2024-09-12T03:27:29.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.104+0000 7f8d3eec1740 1 -- 172.21.15.6:0/1017218529 shutdown_connections 2024-09-12T03:27:29.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.104+0000 7f8d3eec1740 1 --2- 172.21.15.6:0/1017218529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555fd18f8f90 0x555fd191b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.104+0000 7f8d3eec1740 1 --2- 172.21.15.6:0/1017218529 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8670 0x555fd19003e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.104+0000 7f8d3eec1740 1 --2- 172.21.15.6:0/1017218529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f4c00 0x555fd190f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.104+0000 7f8d3eec1740 1 -- 172.21.15.6:0/1017218529 >> 172.21.15.6:0/1017218529 conn(0x555fd1900ac0 msgr2=0x555fd18fbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:29.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.112+0000 7f8d3eec1740 1 -- 172.21.15.6:0/1017218529 shutdown_connections 2024-09-12T03:27:29.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.112+0000 7f8d3eec1740 1 -- 172.21.15.6:0/1017218529 wait complete. 2024-09-12T03:27:29.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.113+0000 7f8d3eec1740 1 -- start start 2024-09-12T03:27:29.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.113+0000 7f8d3eec1740 1 Processor -- start 2024-09-12T03:27:29.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.116+0000 7f8d3eec1740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555fd18f4c00 0x555fd1907550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.116+0000 7f8d3eec1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f8670 0x555fd1907a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.116+0000 7f8d3eec1740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8f90 0x555fd1907fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.116+0000 7f8d3eec1740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1800ca20 con 0x555fd18f4c00 2024-09-12T03:27:29.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.116+0000 7f8d3eec1740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1800de90 con 0x555fd18f8f90 2024-09-12T03:27:29.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.116+0000 7f8d3eec1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1800d0f0 con 0x555fd18f8670 2024-09-12T03:27:29.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.116+0000 7f8d3eec1740 10 client.0 _ll_register_callbacks cb 0x7f8d2000abb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:29.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.117+0000 7f8d36d9d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8f90 0x555fd1907fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:29.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.117+0000 7f8d36d9d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8f90 0x555fd1907fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:54206/0 (socket says 172.21.15.6:54206) 2024-09-12T03:27:29.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.117+0000 7f8d36d9d640 1 -- 172.21.15.6:0/125768445 learned_addr learned my addr 172.21.15.6:0/125768445 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:29.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.117+0000 7f8d35d9b640 1 --2- 172.21.15.6:0/125768445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f8670 0x555fd1907a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:29.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.117+0000 7f8d36d9d640 1 -- 172.21.15.6:0/125768445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f8670 msgr2=0x555fd1907a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:29.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.117+0000 7f8d36d9d640 1 --2- 172.21.15.6:0/125768445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f8670 0x555fd1907a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.117+0000 7f8d36d9d640 1 -- 172.21.15.6:0/125768445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555fd18f4c00 msgr2=0x555fd1907550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:29.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.117+0000 7f8d36d9d640 1 --2- 172.21.15.6:0/125768445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555fd18f4c00 0x555fd1907550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.117+0000 7f8d36d9d640 1 -- 172.21.15.6:0/125768445 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fd197d510 con 0x555fd18f8f90 2024-09-12T03:27:29.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.117+0000 7f8d36d9d640 1 --2- 172.21.15.6:0/125768445 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8f90 0x555fd1907fd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8d30098400 tx=0x7f8d3009c1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f11e8449d087e4db server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:29.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.117+0000 7f8d1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:27:29.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.118+0000 7f8d1dffb640 1 -- 172.21.15.6:0/125768445 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3001f040 con 0x555fd18f8f90 2024-09-12T03:27:29.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.118+0000 7f8d3eec1740 1 -- 172.21.15.6:0/125768445 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d20016cf0 con 0x555fd18f8f90 2024-09-12T03:27:29.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.118+0000 7f8d1dffb640 1 -- 172.21.15.6:0/125768445 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f8d300ba280 con 0x555fd18f8f90 2024-09-12T03:27:29.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.118+0000 7f8d1dffb640 1 -- 172.21.15.6:0/125768445 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d300a46a0 con 0x555fd18f8f90 2024-09-12T03:27:29.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.128+0000 7f8d3eec1740 10 client.25616 fetch_fsmap learned FSMap version 27 2024-09-12T03:27:29.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.128+0000 7f8d3eec1740 1 -- 172.21.15.6:0/125768445 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555fd197e770 con 0x555fd18f8f90 2024-09-12T03:27:29.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.129+0000 7f8d1dffb640 1 -- 172.21.15.6:0/125768445 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d20016cf0 con 0x555fd18f8f90 2024-09-12T03:27:29.129 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:29 smithi006 ceph-mon[30991]: pgmap v526: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 361 KiB/s rd, 112 MiB/s wr, 12.18k op/s 2024-09-12T03:27:29.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.129+0000 7f8d1dffb640 1 -- 172.21.15.6:0/125768445 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8d3000f040 con 0x555fd18f8f90 2024-09-12T03:27:29.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.129+0000 7f8d3eec1740 10 client.25616 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:27:29.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.129+0000 7f8d3eec1740 10 client.25616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:27:29.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.129+0000 7f8d3eec1740 20 client.25616 populate_metadata read hostname 'smithi006' 2024-09-12T03:27:29.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.129+0000 7f8d3eec1740 1 --2- 172.21.15.6:0/125768445 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x555fd1903850 0x555fd1980e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.129+0000 7f8d3eec1740 4 client.25616 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:27:29.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.129+0000 7f8d3eec1740 1 -- 172.21.15.6:0/125768445 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555fd1865d60 con 0x555fd1903850 2024-09-12T03:27:29.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.130+0000 7f8d3659c640 1 --2- 172.21.15.6:0/125768445 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x555fd1903850 0x555fd1980e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:29.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.130+0000 7f8d3659c640 1 --2- 172.21.15.6:0/125768445 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x555fd1903850 0x555fd1980e50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=874b8f5dc6369fd9 server_cookie=3c9e3c07fe7633ca in_seq=0 out_seq=0 2024-09-12T03:27:29.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.130+0000 7f8d1dffb640 10 client.25616 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:29.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.131+0000 7f8d1dffb640 1 -- 172.21.15.6:0/125768445 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555fd1865d60 con 0x555fd1903850 2024-09-12T03:27:29.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.131+0000 7f8d1dffb640 10 client.25616 handle_command_reply: tid=0 2024-09-12T03:27:29.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.131+0000 7f8d3eec1740 1 client.25616 shutdown 2024-09-12T03:27:29.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.131+0000 7f8d3eec1740 1 -- 172.21.15.6:0/125768445 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8f90 msgr2=0x555fd1907fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:29.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.131+0000 7f8d3eec1740 1 --2- 172.21.15.6:0/125768445 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8f90 0x555fd1907fd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8d30098400 tx=0x7f8d3009c1f0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.131+0000 7f8d3eec1740 1 -- 172.21.15.6:0/125768445 shutdown_connections 2024-09-12T03:27:29.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.131+0000 7f8d3eec1740 1 --2- 172.21.15.6:0/125768445 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x555fd1903850 0x555fd1980e50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.131+0000 7f8d3eec1740 1 --2- 172.21.15.6:0/125768445 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fd18f8f90 0x555fd1907fd0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.131+0000 7f8d3eec1740 1 --2- 172.21.15.6:0/125768445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fd18f8670 0x555fd1907a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.131+0000 7f8d3eec1740 1 --2- 172.21.15.6:0/125768445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555fd18f4c00 0x555fd1907550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.131+0000 7f8d3eec1740 1 -- 172.21.15.6:0/125768445 >> 172.21.15.6:0/125768445 conn(0x555fd1900ac0 msgr2=0x555fd18f3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:29.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.132+0000 7f8d1dffb640 0 client.25616 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:29.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.132+0000 7f8d3eec1740 1 -- 172.21.15.6:0/125768445 shutdown_connections 2024-09-12T03:27:29.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.132+0000 7f8d3eec1740 1 -- 172.21.15.6:0/125768445 wait complete. 2024-09-12T03:27:29.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.132+0000 7f8d3eec1740 20 client.25616 trim_cache size 0 max 16384 2024-09-12T03:27:29.134 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:27:29.134 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2024-09-12T03:27:29.134 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2024-09-12T03:27:29.134 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:27:29.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.134+0000 7f8d3759e640 1 -- 172.21.15.6:0/725045961 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8d240a2b80 msgr2=0x7f8d240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:29.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.134+0000 7f8d3759e640 1 --2- 172.21.15.6:0/725045961 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8d240a2b80 0x7f8d240a5040 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8d20006fd0 tx=0x7f8d2000a410 comp rx=0 tx=0).stop 2024-09-12T03:27:29.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.134+0000 7f8d3759e640 1 -- 172.21.15.6:0/725045961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d3810d360 msgr2=0x7f8d3807bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:29.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.134+0000 7f8d3759e640 1 --2- 172.21.15.6:0/725045961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d3810d360 0x7f8d3807bdf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000ead0 tx=0x7f8d3000ef90 comp rx=0 tx=0).stop 2024-09-12T03:27:29.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.134+0000 7f8d3759e640 1 -- 172.21.15.6:0/725045961 shutdown_connections 2024-09-12T03:27:29.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.134+0000 7f8d3759e640 1 --2- 172.21.15.6:0/725045961 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f8d240a2b80 0x7f8d240a5040 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.134+0000 7f8d3759e640 1 --2- 172.21.15.6:0/725045961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d3810d360 0x7f8d3807bdf0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.134+0000 7f8d3759e640 1 --2- 172.21.15.6:0/725045961 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8d38108c30 0x7f8d3807b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.134+0000 7f8d3759e640 1 --2- 172.21.15.6:0/725045961 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d38108280 0x7f8d38082820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.134+0000 7f8d3759e640 1 -- 172.21.15.6:0/725045961 >> 172.21.15.6:0/725045961 conn(0x7f8d380779f0 msgr2=0x7f8d38078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:29.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.136+0000 7f8d3759e640 1 -- 172.21.15.6:0/725045961 shutdown_connections 2024-09-12T03:27:29.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:29.138+0000 7f8d3759e640 1 -- 172.21.15.6:0/725045961 wait complete. 2024-09-12T03:27:29.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:29 smithi161 ceph-mon[35485]: pgmap v526: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 361 KiB/s rd, 112 MiB/s wr, 12.18k op/s 2024-09-12T03:27:29.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:29 smithi118 ceph-mon[35048]: pgmap v526: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 361 KiB/s rd, 112 MiB/s wr, 12.18k op/s 2024-09-12T03:27:30.450 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:27:30.450 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:27:31.071 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:31.498 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:31 smithi006 ceph-mon[30991]: pgmap v527: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 360 KiB/s rd, 134 MiB/s wr, 14.66k op/s 2024-09-12T03:27:31.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:31 smithi118 ceph-mon[35048]: pgmap v527: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 360 KiB/s rd, 134 MiB/s wr, 14.66k op/s 2024-09-12T03:27:31.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:31 smithi161 ceph-mon[35485]: pgmap v527: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 360 KiB/s rd, 134 MiB/s wr, 14.66k op/s 2024-09-12T03:27:32.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.945+0000 7f22a0cdc640 1 -- 172.21.15.6:0/2932315947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f229c102570 msgr2=0x7f229c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:32.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.945+0000 7f22a0cdc640 1 --2- 172.21.15.6:0/2932315947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f229c102570 0x7f229c1029b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f228c009860 tx=0x7f228c02a3d0 comp rx=0 tx=0).stop 2024-09-12T03:27:32.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.946+0000 7f22a0cdc640 1 -- 172.21.15.6:0/2932315947 shutdown_connections 2024-09-12T03:27:32.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.946+0000 7f22a0cdc640 1 --2- 172.21.15.6:0/2932315947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f229c102ef0 0x7f229c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:32.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.946+0000 7f22a0cdc640 1 --2- 172.21.15.6:0/2932315947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f229c102570 0x7f229c1029b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:32.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.946+0000 7f22a0cdc640 1 --2- 172.21.15.6:0/2932315947 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f229c100e20 0x7f229c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:32.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.946+0000 7f22a0cdc640 1 -- 172.21.15.6:0/2932315947 >> 172.21.15.6:0/2932315947 conn(0x7f229c0fc7d0 msgr2=0x7f229c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:32.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.952+0000 7f22a0cdc640 1 -- 172.21.15.6:0/2932315947 shutdown_connections 2024-09-12T03:27:32.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.954+0000 7f22a0cdc640 1 -- 172.21.15.6:0/2932315947 wait complete. 2024-09-12T03:27:32.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.956+0000 7f22a0cdc640 1 Processor -- start 2024-09-12T03:27:32.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.956+0000 7f22a0cdc640 1 -- start start 2024-09-12T03:27:32.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.956+0000 7f22a0cdc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f229c100e20 0x7f229c073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:32.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.956+0000 7f22a0cdc640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f229c102570 0x7f229c073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:32.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.956+0000 7f22a0cdc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f229c102ef0 0x7f229c06c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:32.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.956+0000 7f22a0cdc640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f229c110fd0 con 0x7f229c100e20 2024-09-12T03:27:32.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.956+0000 7f22a0cdc640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f229c110e50 con 0x7f229c102570 2024-09-12T03:27:32.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.956+0000 7f22a0cdc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f229c111150 con 0x7f229c102ef0 2024-09-12T03:27:32.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.958+0000 7f2299d9b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f229c102570 0x7f229c073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:32.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.959+0000 7f229ad9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f229c102ef0 0x7f229c06c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:32.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.959+0000 7f229a59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f229c100e20 0x7f229c073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:32.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.959+0000 7f229a59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f229c100e20 0x7f229c073910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48458/0 (socket says 172.21.15.6:48458) 2024-09-12T03:27:32.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.959+0000 7f229a59c640 1 -- 172.21.15.6:0/3481783361 learned_addr learned my addr 172.21.15.6:0/3481783361 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:32.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.961+0000 7f229a59c640 1 -- 172.21.15.6:0/3481783361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f229c102ef0 msgr2=0x7f229c06c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:32.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.961+0000 7f229a59c640 1 --2- 172.21.15.6:0/3481783361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f229c102ef0 0x7f229c06c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:32.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.961+0000 7f229a59c640 1 -- 172.21.15.6:0/3481783361 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f229c102570 msgr2=0x7f229c073e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:32.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.961+0000 7f229a59c640 1 --2- 172.21.15.6:0/3481783361 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f229c102570 0x7f229c073e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:32.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.961+0000 7f229a59c640 1 -- 172.21.15.6:0/3481783361 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229c06d1d0 con 0x7f229c100e20 2024-09-12T03:27:32.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.962+0000 7f229a59c640 1 --2- 172.21.15.6:0/3481783361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f229c100e20 0x7f229c073910 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f228400e9c0 tx=0x7f228400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:32.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.963+0000 7f22937fe640 1 -- 172.21.15.6:0/3481783361 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228400ccc0 con 0x7f229c100e20 2024-09-12T03:27:32.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.963+0000 7f22a0cdc640 1 -- 172.21.15.6:0/3481783361 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f229c06d4c0 con 0x7f229c100e20 2024-09-12T03:27:32.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.963+0000 7f22a0cdc640 1 -- 172.21.15.6:0/3481783361 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f229c1a7640 con 0x7f229c100e20 2024-09-12T03:27:32.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.964+0000 7f22937fe640 1 -- 172.21.15.6:0/3481783361 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2284005500 con 0x7f229c100e20 2024-09-12T03:27:32.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.964+0000 7f22937fe640 1 -- 172.21.15.6:0/3481783361 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284010430 con 0x7f229c100e20 2024-09-12T03:27:32.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.965+0000 7f22a0cdc640 1 -- 172.21.15.6:0/3481783361 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f229c101200 con 0x7f229c100e20 2024-09-12T03:27:32.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.967+0000 7f22937fe640 1 -- 172.21.15.6:0/3481783361 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f22840106b0 con 0x7f229c100e20 2024-09-12T03:27:32.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.968+0000 7f22937fe640 1 --2- 172.21.15.6:0/3481783361 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f22880a2cd0 0x7f22880a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:32.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.969+0000 7f22937fe640 1 -- 172.21.15.6:0/3481783361 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f22840d1030 con 0x7f229c100e20 2024-09-12T03:27:32.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.970+0000 7f2299d9b640 1 --2- 172.21.15.6:0/3481783361 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f22880a2cd0 0x7f22880a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:32.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.974+0000 7f2299d9b640 1 --2- 172.21.15.6:0/3481783361 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f22880a2cd0 0x7f22880a5190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f229c06ab00 tx=0x7f228c005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:32.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:31.983+0000 7f22937fe640 1 -- 172.21.15.6:0/3481783361 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f228409d1f0 con 0x7f229c100e20 2024-09-12T03:27:32.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:32.431+0000 7f22a0cdc640 1 -- 172.21.15.6:0/3481783361 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f229c06a5a0 con 0x7f229c100e20 2024-09-12T03:27:32.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:32.433+0000 7f22937fe640 1 -- 172.21.15.6:0/3481783361 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11908 (secure 0 0 0) 0x7f22840cf090 con 0x7f229c100e20 2024-09-12T03:27:32.436 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:27:32.436 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"btime":"2024-09-12T03:24:26:920603+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:24:26.080047+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24463]},"id":1}]} 2024-09-12T03:27:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 27 2024-09-12T03:27:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:32.436+0000 7f22a0cdc640 1 -- 172.21.15.6:0/3481783361 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f22880a2cd0 msgr2=0x7f22880a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:32.436+0000 7f22a0cdc640 1 --2- 172.21.15.6:0/3481783361 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f22880a2cd0 0x7f22880a5190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f229c06ab00 tx=0x7f228c005c90 comp rx=0 tx=0).stop 2024-09-12T03:27:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:32.436+0000 7f22a0cdc640 1 -- 172.21.15.6:0/3481783361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f229c100e20 msgr2=0x7f229c073910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:32.436+0000 7f22a0cdc640 1 --2- 172.21.15.6:0/3481783361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f229c100e20 0x7f229c073910 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f228400e9c0 tx=0x7f228400ee80 comp rx=0 tx=0).stop 2024-09-12T03:27:32.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:32.441+0000 7f22a0cdc640 1 -- 172.21.15.6:0/3481783361 shutdown_connections 2024-09-12T03:27:32.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:32.441+0000 7f22a0cdc640 1 --2- 172.21.15.6:0/3481783361 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f22880a2cd0 0x7f22880a5190 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:32.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:32.441+0000 7f22a0cdc640 1 --2- 172.21.15.6:0/3481783361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f229c102ef0 0x7f229c06c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:32.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:32.441+0000 7f22a0cdc640 1 --2- 172.21.15.6:0/3481783361 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f229c102570 0x7f229c073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:32.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:32.441+0000 7f22a0cdc640 1 --2- 172.21.15.6:0/3481783361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f229c100e20 0x7f229c073910 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:32.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:32.441+0000 7f22a0cdc640 1 -- 172.21.15.6:0/3481783361 >> 172.21.15.6:0/3481783361 conn(0x7f229c0fc7d0 msgr2=0x7f229c1068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:32.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:32.442+0000 7f22a0cdc640 1 -- 172.21.15.6:0/3481783361 shutdown_connections 2024-09-12T03:27:32.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:32.442+0000 7f22a0cdc640 1 -- 172.21.15.6:0/3481783361 wait complete. 2024-09-12T03:27:32.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:32 smithi118 ceph-mon[35048]: pgmap v528: 97 pgs: 97 active+clean; 13 GiB data, 115 GiB used, 958 GiB / 1.0 TiB avail; 609 KiB/s rd, 152 MiB/s wr, 16.54k op/s 2024-09-12T03:27:32.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:32 smithi006 ceph-mon[30991]: pgmap v528: 97 pgs: 97 active+clean; 13 GiB data, 115 GiB used, 958 GiB / 1.0 TiB avail; 609 KiB/s rd, 152 MiB/s wr, 16.54k op/s 2024-09-12T03:27:32.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:32 smithi161 ceph-mon[35485]: pgmap v528: 97 pgs: 97 active+clean; 13 GiB data, 115 GiB used, 958 GiB / 1.0 TiB avail; 609 KiB/s rd, 152 MiB/s wr, 16.54k op/s 2024-09-12T03:27:33.684 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:33 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3481783361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:33.686 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:27:33.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:33 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3481783361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:33.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:33 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3481783361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:34.201 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:34.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:34 smithi006 ceph-mon[30991]: pgmap v529: 97 pgs: 97 active+clean; 13 GiB data, 115 GiB used, 957 GiB / 1.0 TiB avail; 426 KiB/s rd, 156 MiB/s wr, 16.83k op/s 2024-09-12T03:27:34.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:34 smithi118 ceph-mon[35048]: pgmap v529: 97 pgs: 97 active+clean; 13 GiB data, 115 GiB used, 957 GiB / 1.0 TiB avail; 426 KiB/s rd, 156 MiB/s wr, 16.83k op/s 2024-09-12T03:27:34.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:34 smithi161 ceph-mon[35485]: pgmap v529: 97 pgs: 97 active+clean; 13 GiB data, 115 GiB used, 957 GiB / 1.0 TiB avail; 426 KiB/s rd, 156 MiB/s wr, 16.83k op/s 2024-09-12T03:27:35.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.114+0000 7fa1e5dbb640 1 -- 172.21.15.6:0/362644510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1e010e7b0 msgr2=0x7fa1e0115130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:35.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.114+0000 7fa1e5dbb640 1 --2- 172.21.15.6:0/362644510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1e010e7b0 0x7fa1e0115130 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8009ca0 tx=0x7fa1d802a8f0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.114+0000 7fa1e5dbb640 1 -- 172.21.15.6:0/362644510 shutdown_connections 2024-09-12T03:27:35.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.114+0000 7fa1e5dbb640 1 --2- 172.21.15.6:0/362644510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1e010e7b0 0x7fa1e0115130 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.114+0000 7fa1e5dbb640 1 --2- 172.21.15.6:0/362644510 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1e0109c40 0x7fa1e010e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.114+0000 7fa1e5dbb640 1 --2- 172.21.15.6:0/362644510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1e0109290 0x7fa1e0109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.114+0000 7fa1e5dbb640 1 -- 172.21.15.6:0/362644510 >> 172.21.15.6:0/362644510 conn(0x7fa1e0077c10 msgr2=0x7fa1e007a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:35.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.118+0000 7fa1e5dbb640 1 -- 172.21.15.6:0/362644510 shutdown_connections 2024-09-12T03:27:35.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.118+0000 7fa1e5dbb640 1 -- 172.21.15.6:0/362644510 wait complete. 2024-09-12T03:27:35.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.119+0000 7fa1e5dbb640 1 Processor -- start 2024-09-12T03:27:35.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.119+0000 7fa1e5dbb640 1 -- start start 2024-09-12T03:27:35.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.119+0000 7fa1e5dbb640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1e010e7b0 0x7fa1e0084a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:35.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.119+0000 7fa1e5dbb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1e007da90 0x7fa1e007def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:35.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.119+0000 7fa1e5dbb640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1e007e430 0x7fa1e0082530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:35.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.119+0000 7fa1e5dbb640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e01186d0 con 0x7fa1e007e430 2024-09-12T03:27:35.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.119+0000 7fa1e5dbb640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e0118550 con 0x7fa1e010e7b0 2024-09-12T03:27:35.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.119+0000 7fa1e5dbb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e0118850 con 0x7fa1e007da90 2024-09-12T03:27:35.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.120+0000 7fa1df7fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1e010e7b0 0x7fa1e0084a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:35.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.120+0000 7fa1deffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1e007da90 0x7fa1e007def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:35.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.121+0000 7fa1dffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1e007e430 0x7fa1e0082530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:35.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.121+0000 7fa1dffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1e007e430 0x7fa1e0082530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48478/0 (socket says 172.21.15.6:48478) 2024-09-12T03:27:35.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.121+0000 7fa1dffff640 1 -- 172.21.15.6:0/4290545119 learned_addr learned my addr 172.21.15.6:0/4290545119 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:35.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.122+0000 7fa1deffd640 1 -- 172.21.15.6:0/4290545119 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1e010e7b0 msgr2=0x7fa1e0084a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:35.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.122+0000 7fa1deffd640 1 --2- 172.21.15.6:0/4290545119 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1e010e7b0 0x7fa1e0084a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.122+0000 7fa1deffd640 1 -- 172.21.15.6:0/4290545119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1e007e430 msgr2=0x7fa1e0082530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:35.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.122+0000 7fa1deffd640 1 --2- 172.21.15.6:0/4290545119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1e007e430 0x7fa1e0082530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.122+0000 7fa1deffd640 1 -- 172.21.15.6:0/4290545119 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e0082a70 con 0x7fa1e007da90 2024-09-12T03:27:35.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.122+0000 7fa1deffd640 1 --2- 172.21.15.6:0/4290545119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1e007da90 0x7fa1e007def0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0076400 tx=0x7fa1d400b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:35.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.126+0000 7fa1dcff9640 1 -- 172.21.15.6:0/4290545119 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4015020 con 0x7fa1e007da90 2024-09-12T03:27:35.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.126+0000 7fa1e5dbb640 1 -- 172.21.15.6:0/4290545119 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e0082d60 con 0x7fa1e007da90 2024-09-12T03:27:35.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.126+0000 7fa1e5dbb640 1 -- 172.21.15.6:0/4290545119 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e00832f0 con 0x7fa1e007da90 2024-09-12T03:27:35.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.132+0000 7fa1dcff9640 1 -- 172.21.15.6:0/4290545119 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d4011d10 con 0x7fa1e007da90 2024-09-12T03:27:35.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.132+0000 7fa1dcff9640 1 -- 172.21.15.6:0/4290545119 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d400f2e0 con 0x7fa1e007da90 2024-09-12T03:27:35.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.138+0000 7fa1e8194740 1 Processor -- start 2024-09-12T03:27:35.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.139+0000 7fa1e8194740 1 -- start start 2024-09-12T03:27:35.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.139+0000 7fa1e8194740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498cfb40 0x55a2498ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:35.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.139+0000 7fa1e8194740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2498ce5d0 0x55a2498cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:35.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.139+0000 7fa1e8194740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2498d9370 0x55a2498db760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:35.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.139+0000 7fa1e8194740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a2498342f0 con 0x55a2498ce5d0 2024-09-12T03:27:35.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.139+0000 7fa1e8194740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55a2497dbcf0 con 0x55a2498d9370 2024-09-12T03:27:35.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.139+0000 7fa1e8194740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a2498302b0 con 0x55a2498cfb40 2024-09-12T03:27:35.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.139+0000 7fa1dffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498cfb40 0x55a2498ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:35.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.139+0000 7fa1dffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498cfb40 0x55a2498ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:60494/0 (socket says 172.21.15.6:60494) 2024-09-12T03:27:35.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.139+0000 7fa1dffff640 1 -- 172.21.15.6:0/1724972790 learned_addr learned my addr 172.21.15.6:0/1724972790 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:35.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.140+0000 7fa1df7fe640 1 --2- 172.21.15.6:0/1724972790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2498ce5d0 0x55a2498cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:35.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.140+0000 7fa1dffff640 1 -- 172.21.15.6:0/1724972790 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2498d9370 msgr2=0x55a2498db760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:27:35.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.140+0000 7fa1dffff640 1 --2- 172.21.15.6:0/1724972790 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2498d9370 0x55a2498db760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.140+0000 7fa1dffff640 1 -- 172.21.15.6:0/1724972790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2498ce5d0 msgr2=0x55a2498cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:35.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.140+0000 7fa1dffff640 1 --2- 172.21.15.6:0/1724972790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2498ce5d0 0x55a2498cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.140+0000 7fa1dffff640 1 -- 172.21.15.6:0/1724972790 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2497c03a0 con 0x55a2498cfb40 2024-09-12T03:27:35.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.140+0000 7fa1df7fe640 1 --2- 172.21.15.6:0/1724972790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2498ce5d0 0x55a2498cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:35.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.140+0000 7fa1dffff640 1 --2- 172.21.15.6:0/1724972790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498cfb40 0x55a2498ce090 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa1d802f530 tx=0x7fa1d8031bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=515aa766e7df72d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:35.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.142+0000 7fa1dcff9640 1 -- 172.21.15.6:0/4290545119 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa1d4018020 con 0x7fa1e007da90 2024-09-12T03:27:35.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.144+0000 7fa1dcff9640 1 --2- 172.21.15.6:0/4290545119 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa1c80a2c80 0x7fa1c80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:35.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.144+0000 7fa1df7fe640 1 --2- 172.21.15.6:0/4290545119 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa1c80a2c80 0x7fa1c80a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:35.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.144+0000 7fa1dcff9640 1 -- 172.21.15.6:0/4290545119 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa1d40ce780 con 0x7fa1e007da90 2024-09-12T03:27:35.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.145+0000 7fa1c5ffb640 1 -- 172.21.15.6:0/1724972790 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8003970 con 0x55a2498cfb40 2024-09-12T03:27:35.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.145+0000 7fa1c5ffb640 1 -- 172.21.15.6:0/1724972790 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d8036030 con 0x55a2498cfb40 2024-09-12T03:27:35.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.145+0000 7fa1c5ffb640 1 -- 172.21.15.6:0/1724972790 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8004510 con 0x55a2498cfb40 2024-09-12T03:27:35.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.146+0000 7fa1e8194740 1 -- 172.21.15.6:0/1724972790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498cfb40 msgr2=0x55a2498ce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:35.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.146+0000 7fa1df7fe640 1 --2- 172.21.15.6:0/4290545119 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa1c80a2c80 0x7fa1c80a5140 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55a2498cbac0 tx=0x7fa1d0005d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:35.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.146+0000 7fa1e8194740 1 --2- 172.21.15.6:0/1724972790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498cfb40 0x55a2498ce090 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa1d802f530 tx=0x7fa1d8031bc0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.146+0000 7fa1e8194740 1 -- 172.21.15.6:0/1724972790 shutdown_connections 2024-09-12T03:27:35.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.146+0000 7fa1e8194740 1 --2- 172.21.15.6:0/1724972790 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2498d9370 0x55a2498db760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.146+0000 7fa1e8194740 1 --2- 172.21.15.6:0/1724972790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2498ce5d0 0x55a2498cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.146+0000 7fa1e8194740 1 --2- 172.21.15.6:0/1724972790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498cfb40 0x55a2498ce090 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.146+0000 7fa1e8194740 1 -- 172.21.15.6:0/1724972790 >> 172.21.15.6:0/1724972790 conn(0x55a249897310 msgr2=0x55a2498cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.147+0000 7fa1e8194740 1 -- 172.21.15.6:0/1724972790 shutdown_connections 2024-09-12T03:27:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.147+0000 7fa1e8194740 1 -- 172.21.15.6:0/1724972790 wait complete. 2024-09-12T03:27:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.148+0000 7fa1e8194740 1 -- start start 2024-09-12T03:27:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.148+0000 7fa1e8194740 1 Processor -- start 2024-09-12T03:27:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.149+0000 7fa1e8194740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2498ce5d0 0x55a2498d8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.149+0000 7fa1e8194740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2498cfb40 0x55a2498d6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.149+0000 7fa1e8194740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498d9370 0x55a2498d6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.149+0000 7fa1e8194740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a2498dc010 con 0x55a2498cfb40 2024-09-12T03:27:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.149+0000 7fa1e8194740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55a2497ef060 con 0x55a2498ce5d0 2024-09-12T03:27:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.149+0000 7fa1e8194740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a2498dc190 con 0x55a2498d9370 2024-09-12T03:27:35.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.149+0000 7fa1df7fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2498ce5d0 0x55a2498d8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:35.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.150+0000 7fa1df7fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2498ce5d0 0x55a2498d8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:54242/0 (socket says 172.21.15.6:54242) 2024-09-12T03:27:35.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.150+0000 7fa1df7fe640 1 -- 172.21.15.6:0/1313808047 learned_addr learned my addr 172.21.15.6:0/1313808047 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:35.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.150+0000 7fa1e8194740 10 client.0 _ll_register_callbacks cb 0x55a2495428c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:35.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.151+0000 7fa1deffd640 1 --2- 172.21.15.6:0/1313808047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2498cfb40 0x55a2498d6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:35.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.151+0000 7fa1dffff640 1 --2- 172.21.15.6:0/1313808047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498d9370 0x55a2498d6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:35.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.152+0000 7fa1deffd640 1 -- 172.21.15.6:0/1313808047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498d9370 msgr2=0x55a2498d6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:35.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.152+0000 7fa1deffd640 1 --2- 172.21.15.6:0/1313808047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498d9370 0x55a2498d6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.152+0000 7fa1deffd640 1 -- 172.21.15.6:0/1313808047 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2498ce5d0 msgr2=0x55a2498d8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:35.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.152+0000 7fa1deffd640 1 --2- 172.21.15.6:0/1313808047 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2498ce5d0 0x55a2498d8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.152+0000 7fa1deffd640 1 -- 172.21.15.6:0/1313808047 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2495a53c0 con 0x55a2498cfb40 2024-09-12T03:27:35.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.152+0000 7fa1deffd640 1 --2- 172.21.15.6:0/1313808047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2498cfb40 0x55a2498d6080 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x55a2498cf070 tx=0x7fa1d409aa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08d84e373eb968d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:35.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.152+0000 7fa1beffd640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:27:35.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.153+0000 7fa1beffd640 1 -- 172.21.15.6:0/1313808047 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d409afb0 con 0x55a2498cfb40 2024-09-12T03:27:35.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.154+0000 7fa1beffd640 1 -- 172.21.15.6:0/1313808047 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa1d4097620 con 0x55a2498cfb40 2024-09-12T03:27:35.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.154+0000 7fa1e8194740 1 -- 172.21.15.6:0/1313808047 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1d8036030 con 0x55a2498cfb40 2024-09-12T03:27:35.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.154+0000 7fa1beffd640 1 -- 172.21.15.6:0/1313808047 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa1d409f360 con 0x55a2498cfb40 2024-09-12T03:27:35.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.154+0000 7fa1e8194740 10 client.16086 fetch_fsmap learned FSMap version 27 2024-09-12T03:27:35.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.154+0000 7fa1e8194740 1 -- 172.21.15.6:0/1313808047 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a2498e7b10 con 0x55a2498cfb40 2024-09-12T03:27:35.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.155+0000 7fa1beffd640 1 -- 172.21.15.6:0/1313808047 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d40a3c20 con 0x55a2498cfb40 2024-09-12T03:27:35.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.156+0000 7fa1beffd640 1 -- 172.21.15.6:0/1313808047 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa1d40b8070 con 0x55a2498cfb40 2024-09-12T03:27:35.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.156+0000 7fa1e8194740 10 client.16086 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:27:35.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.156+0000 7fa1e8194740 10 client.16086 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-12T03:27:35.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.156+0000 7fa1e8194740 20 client.16086 populate_metadata read hostname 'smithi006' 2024-09-12T03:27:35.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.156+0000 7fa1e8194740 1 --2- 172.21.15.6:0/1313808047 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55a2498e8fd0 0x55a2498eb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:35.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.156+0000 7fa1e8194740 4 client.16086 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:35.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.157+0000 7fa1e8194740 1 -- 172.21.15.6:0/1313808047 --> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa1d8036030 con 0x55a2498e8fd0 2024-09-12T03:27:35.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.159+0000 7fa1df7fe640 1 --2- 172.21.15.6:0/1313808047 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55a2498e8fd0 0x55a2498eb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:35.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.173+0000 7fa1beffd640 10 client.16086 ms_handle_connect on v2:172.21.15.161:6838/3645555254 2024-09-12T03:27:35.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.173+0000 7fa1df7fe640 1 --2- 172.21.15.6:0/1313808047 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55a2498e8fd0 0x55a2498eb3c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=31d6b9f0c938c916 server_cookie=727695df44fa6ec3 in_seq=0 out_seq=0 2024-09-12T03:27:35.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.177+0000 7fa1beffd640 1 -- 172.21.15.6:0/1313808047 <== mds.2 v2:172.21.15.161:6838/3645555254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa1d8036030 con 0x55a2498e8fd0 2024-09-12T03:27:35.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.177+0000 7fa1beffd640 10 client.16086 handle_command_reply: tid=0 2024-09-12T03:27:35.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.178+0000 7fa1e8194740 1 client.16086 shutdown 2024-09-12T03:27:35.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.180+0000 7fa1e8194740 1 -- 172.21.15.6:0/1313808047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2498cfb40 msgr2=0x55a2498d6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:35.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.180+0000 7fa1e8194740 1 --2- 172.21.15.6:0/1313808047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2498cfb40 0x55a2498d6080 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x55a2498cf070 tx=0x7fa1d409aa00 comp rx=0 tx=0).stop 2024-09-12T03:27:35.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.180+0000 7fa1e8194740 1 -- 172.21.15.6:0/1313808047 shutdown_connections 2024-09-12T03:27:35.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.180+0000 7fa1e8194740 1 --2- 172.21.15.6:0/1313808047 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55a2498e8fd0 0x55a2498eb3c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.180+0000 7fa1e8194740 1 --2- 172.21.15.6:0/1313808047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498d9370 0x55a2498d6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.180+0000 7fa1e8194740 1 --2- 172.21.15.6:0/1313808047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2498cfb40 0x55a2498d6080 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.180+0000 7fa1e8194740 1 --2- 172.21.15.6:0/1313808047 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2498ce5d0 0x55a2498d8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.180+0000 7fa1e8194740 1 -- 172.21.15.6:0/1313808047 >> 172.21.15.6:0/1313808047 conn(0x55a249897310 msgr2=0x55a2498d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:35.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.181+0000 7fa1beffd640 0 client.16086 ms_handle_reset on v2:172.21.15.161:6838/3645555254 2024-09-12T03:27:35.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.181+0000 7fa1e8194740 1 -- 172.21.15.6:0/1313808047 shutdown_connections 2024-09-12T03:27:35.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.181+0000 7fa1e8194740 1 -- 172.21.15.6:0/1313808047 wait complete. 2024-09-12T03:27:35.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.181+0000 7fa1e8194740 20 client.16086 trim_cache size 0 max 16384 2024-09-12T03:27:35.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.215+0000 7fa1e8194740 1 Processor -- start 2024-09-12T03:27:35.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.215+0000 7fa1e8194740 1 -- start start 2024-09-12T03:27:35.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.216+0000 7fa1e8194740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2498ce5d0 0x55a2498cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:35.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.216+0000 7fa1e8194740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2498cfb40 0x55a2498c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:35.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.216+0000 7fa1e8194740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498d9370 0x55a2498d5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:35.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.216+0000 7fa1e8194740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc010b70 con 0x55a2498ce5d0 2024-09-12T03:27:35.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.217+0000 7fa1e8194740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc00e440 con 0x55a2498cfb40 2024-09-12T03:27:35.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.217+0000 7fa1e8194740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc00c340 con 0x55a2498d9370 2024-09-12T03:27:35.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.217+0000 7fa1dffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498d9370 0x55a2498d5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:35.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.217+0000 7fa1dffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498d9370 0x55a2498d5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:60526/0 (socket says 172.21.15.6:60526) 2024-09-12T03:27:35.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.217+0000 7fa1dffff640 1 -- 172.21.15.6:0/4080770912 learned_addr learned my addr 172.21.15.6:0/4080770912 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:35.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.218+0000 7fa1df7fe640 1 --2- 172.21.15.6:0/4080770912 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2498ce5d0 0x55a2498cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:35.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.218+0000 7fa1dffff640 1 -- 172.21.15.6:0/4080770912 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2498cfb40 msgr2=0x55a2498c5ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:27:35.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.218+0000 7fa1dffff640 1 --2- 172.21.15.6:0/4080770912 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2498cfb40 0x55a2498c5ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.218+0000 7fa1dffff640 1 -- 172.21.15.6:0/4080770912 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2498ce5d0 msgr2=0x55a2498cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:35.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.218+0000 7fa1dffff640 1 --2- 172.21.15.6:0/4080770912 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2498ce5d0 0x55a2498cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.218+0000 7fa1dffff640 1 -- 172.21.15.6:0/4080770912 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2498d5030 con 0x55a2498d9370 2024-09-12T03:27:35.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.219+0000 7fa1dffff640 1 --2- 172.21.15.6:0/4080770912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498d9370 0x55a2498d5940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8031c30 tx=0x7fa1d8005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c1829b39c549f82 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:35.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.220+0000 7fa1c57fa640 1 -- 172.21.15.6:0/4080770912 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8053070 con 0x55a2498d9370 2024-09-12T03:27:35.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.220+0000 7fa1c57fa640 1 -- 172.21.15.6:0/4080770912 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d8036040 con 0x55a2498d9370 2024-09-12T03:27:35.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.221+0000 7fa1e8194740 1 -- 172.21.15.6:0/4080770912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498d9370 msgr2=0x55a2498d5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:35.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.221+0000 7fa1e8194740 1 --2- 172.21.15.6:0/4080770912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498d9370 0x55a2498d5940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8031c30 tx=0x7fa1d8005bd0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.223+0000 7fa1e8194740 1 -- 172.21.15.6:0/4080770912 shutdown_connections 2024-09-12T03:27:35.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.223+0000 7fa1e8194740 1 --2- 172.21.15.6:0/4080770912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498d9370 0x55a2498d5940 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.223+0000 7fa1e8194740 1 --2- 172.21.15.6:0/4080770912 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2498cfb40 0x55a2498c5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.223+0000 7fa1e8194740 1 --2- 172.21.15.6:0/4080770912 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2498ce5d0 0x55a2498cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.223+0000 7fa1e8194740 1 -- 172.21.15.6:0/4080770912 >> 172.21.15.6:0/4080770912 conn(0x55a249897310 msgr2=0x55a2498ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:35.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.223+0000 7fa1e8194740 1 -- 172.21.15.6:0/4080770912 shutdown_connections 2024-09-12T03:27:35.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.223+0000 7fa1e8194740 1 -- 172.21.15.6:0/4080770912 wait complete. 2024-09-12T03:27:35.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.223+0000 7fa1e8194740 1 -- start start 2024-09-12T03:27:35.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.224+0000 7fa1e8194740 1 Processor -- start 2024-09-12T03:27:35.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.224+0000 7fa1e8194740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498ce5d0 0x55a2498e17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:35.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.224+0000 7fa1e8194740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2498cfb40 0x55a2498e1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:35.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.224+0000 7fa1e8194740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2498d9370 0x55a2498d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:35.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.224+0000 7fa1e8194740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc00c900 con 0x55a2498d9370 2024-09-12T03:27:35.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.224+0000 7fa1e8194740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc00dd70 con 0x55a2498cfb40 2024-09-12T03:27:35.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.224+0000 7fa1e8194740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc00cfd0 con 0x55a2498ce5d0 2024-09-12T03:27:35.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.224+0000 7fa1e8194740 10 client.0 _ll_register_callbacks cb 0x7fa1d400bef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:35.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.225+0000 7fa1df7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498ce5d0 0x55a2498e17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:35.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.225+0000 7fa1df7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498ce5d0 0x55a2498e17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:60540/0 (socket says 172.21.15.6:60540) 2024-09-12T03:27:35.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.225+0000 7fa1df7fe640 1 -- 172.21.15.6:0/1596564542 learned_addr learned my addr 172.21.15.6:0/1596564542 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:35.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.225+0000 7fa1deffd640 1 --2- 172.21.15.6:0/1596564542 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2498cfb40 0x55a2498e1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:35.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.225+0000 7fa1df7fe640 1 -- 172.21.15.6:0/1596564542 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2498cfb40 msgr2=0x55a2498e1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:35.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.225+0000 7fa1df7fe640 1 --2- 172.21.15.6:0/1596564542 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2498cfb40 0x55a2498e1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.225+0000 7fa1df7fe640 1 -- 172.21.15.6:0/1596564542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2498d9370 msgr2=0x55a2498d2ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:27:35.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.225+0000 7fa1df7fe640 1 --2- 172.21.15.6:0/1596564542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2498d9370 0x55a2498d2ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.225+0000 7fa1df7fe640 1 -- 172.21.15.6:0/1596564542 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2498d32b0 con 0x55a2498ce5d0 2024-09-12T03:27:35.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.226+0000 7fa1df7fe640 1 --2- 172.21.15.6:0/1596564542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498ce5d0 0x55a2498e17b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000f6d0 tx=0x7fa1d0011070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28aefcfc4e0ac1df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:35.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.226+0000 7fa1beffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:27:35.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.226+0000 7fa1beffd640 1 -- 172.21.15.6:0/1596564542 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d001c9b0 con 0x55a2498ce5d0 2024-09-12T03:27:35.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.227+0000 7fa1e8194740 1 -- 172.21.15.6:0/1596564542 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1d8036040 con 0x55a2498ce5d0 2024-09-12T03:27:35.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.227+0000 7fa1beffd640 1 -- 172.21.15.6:0/1596564542 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa1d001cb50 con 0x55a2498ce5d0 2024-09-12T03:27:35.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.227+0000 7fa1beffd640 1 -- 172.21.15.6:0/1596564542 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa1d001cd50 con 0x55a2498ce5d0 2024-09-12T03:27:35.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.228+0000 7fa1e8194740 10 client.25726 fetch_fsmap learned FSMap version 27 2024-09-12T03:27:35.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.228+0000 7fa1e8194740 1 -- 172.21.15.6:0/1596564542 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a249953880 con 0x55a2498ce5d0 2024-09-12T03:27:35.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.228+0000 7fa1beffd640 1 -- 172.21.15.6:0/1596564542 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d8036040 con 0x55a2498ce5d0 2024-09-12T03:27:35.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.229+0000 7fa1beffd640 1 -- 172.21.15.6:0/1596564542 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa1d0025310 con 0x55a2498ce5d0 2024-09-12T03:27:35.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.229+0000 7fa1e8194740 10 client.25726 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:27:35.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.229+0000 7fa1e8194740 10 client.25726 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-12T03:27:35.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.229+0000 7fa1e8194740 20 client.25726 populate_metadata read hostname 'smithi006' 2024-09-12T03:27:35.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.229+0000 7fa1e8194740 1 --2- 172.21.15.6:0/1596564542 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55a249954000 0x55a249956420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:35.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.229+0000 7fa1e8194740 4 client.25726 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:27:35.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.229+0000 7fa1e8194740 1 -- 172.21.15.6:0/1596564542 --> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a249841860 con 0x55a249954000 2024-09-12T03:27:35.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.229+0000 7fa1deffd640 1 --2- 172.21.15.6:0/1596564542 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55a249954000 0x55a249956420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:35.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.230+0000 7fa1deffd640 1 --2- 172.21.15.6:0/1596564542 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55a249954000 0x55a249956420 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=87d261516a9f2cec server_cookie=6e441514b6794ec6 in_seq=0 out_seq=0 2024-09-12T03:27:35.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.230+0000 7fa1beffd640 10 client.25726 ms_handle_connect on v2:172.21.15.161:6838/3645555254 2024-09-12T03:27:35.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.230+0000 7fa1beffd640 1 -- 172.21.15.6:0/1596564542 <== mds.2 v2:172.21.15.161:6838/3645555254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a249841860 con 0x55a249954000 2024-09-12T03:27:35.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.230+0000 7fa1beffd640 10 client.25726 handle_command_reply: tid=0 2024-09-12T03:27:35.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.230+0000 7fa1e8194740 1 client.25726 shutdown 2024-09-12T03:27:35.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.240+0000 7fa1e8194740 1 -- 172.21.15.6:0/1596564542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498ce5d0 msgr2=0x55a2498e17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:35.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.240+0000 7fa1e8194740 1 --2- 172.21.15.6:0/1596564542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498ce5d0 0x55a2498e17b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000f6d0 tx=0x7fa1d0011070 comp rx=0 tx=0).stop 2024-09-12T03:27:35.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.240+0000 7fa1e8194740 1 -- 172.21.15.6:0/1596564542 shutdown_connections 2024-09-12T03:27:35.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.240+0000 7fa1e8194740 1 --2- 172.21.15.6:0/1596564542 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55a249954000 0x55a249956420 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.240+0000 7fa1e8194740 1 --2- 172.21.15.6:0/1596564542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a2498d9370 0x55a2498d2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.240+0000 7fa1e8194740 1 --2- 172.21.15.6:0/1596564542 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a2498cfb40 0x55a2498e1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.240+0000 7fa1e8194740 1 --2- 172.21.15.6:0/1596564542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a2498ce5d0 0x55a2498e17b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.240+0000 7fa1beffd640 0 client.25726 ms_handle_reset on v2:172.21.15.161:6838/3645555254 2024-09-12T03:27:35.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.240+0000 7fa1e8194740 1 -- 172.21.15.6:0/1596564542 >> 172.21.15.6:0/1596564542 conn(0x55a249897310 msgr2=0x55a2498ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:35.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.240+0000 7fa1e8194740 1 -- 172.21.15.6:0/1596564542 shutdown_connections 2024-09-12T03:27:35.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.240+0000 7fa1e8194740 1 -- 172.21.15.6:0/1596564542 wait complete. 2024-09-12T03:27:35.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.241+0000 7fa1e8194740 20 client.25726 trim_cache size 0 max 16384 2024-09-12T03:27:35.242 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-09-12T03:27:35.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.244+0000 7fa1e5dbb640 1 -- 172.21.15.6:0/4290545119 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa1c80a2c80 msgr2=0x7fa1c80a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:35.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.244+0000 7fa1e5dbb640 1 --2- 172.21.15.6:0/4290545119 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa1c80a2c80 0x7fa1c80a5140 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55a2498cbac0 tx=0x7fa1d0005d10 comp rx=0 tx=0).stop 2024-09-12T03:27:35.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.244+0000 7fa1e5dbb640 1 -- 172.21.15.6:0/4290545119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1e007da90 msgr2=0x7fa1e007def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:35.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.244+0000 7fa1e5dbb640 1 --2- 172.21.15.6:0/4290545119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1e007da90 0x7fa1e007def0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0076400 tx=0x7fa1d400b870 comp rx=0 tx=0).stop 2024-09-12T03:27:35.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.244+0000 7fa1e5dbb640 1 -- 172.21.15.6:0/4290545119 shutdown_connections 2024-09-12T03:27:35.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.244+0000 7fa1e5dbb640 1 --2- 172.21.15.6:0/4290545119 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fa1c80a2c80 0x7fa1c80a5140 secure :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55a2498cbac0 tx=0x7fa1d0005d10 comp rx=0 tx=0).stop 2024-09-12T03:27:35.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.244+0000 7fa1e5dbb640 1 --2- 172.21.15.6:0/4290545119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1e007e430 0x7fa1e0082530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.244+0000 7fa1e5dbb640 1 --2- 172.21.15.6:0/4290545119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1e007da90 0x7fa1e007def0 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0076400 tx=0x7fa1d400b870 comp rx=0 tx=0).stop 2024-09-12T03:27:35.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.244+0000 7fa1e5dbb640 1 --2- 172.21.15.6:0/4290545119 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1e010e7b0 0x7fa1e0084a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.244+0000 7fa1e5dbb640 1 -- 172.21.15.6:0/4290545119 >> 172.21.15.6:0/4290545119 conn(0x7fa1e0077c10 msgr2=0x7fa1e0077ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:35.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.244+0000 7fa1e5dbb640 1 -- 172.21.15.6:0/4290545119 shutdown_connections 2024-09-12T03:27:35.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:35.244+0000 7fa1e5dbb640 1 -- 172.21.15.6:0/4290545119 wait complete. 2024-09-12T03:27:36.706 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:27:36.707 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:36 smithi006 ceph-mon[30991]: pgmap v530: 97 pgs: 97 active+clean; 13 GiB data, 116 GiB used, 957 GiB / 1.0 TiB avail; 427 KiB/s rd, 186 MiB/s wr, 19.79k op/s 2024-09-12T03:27:36.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:36 smithi118 ceph-mon[35048]: pgmap v530: 97 pgs: 97 active+clean; 13 GiB data, 116 GiB used, 957 GiB / 1.0 TiB avail; 427 KiB/s rd, 186 MiB/s wr, 19.79k op/s 2024-09-12T03:27:36.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:36 smithi161 ceph-mon[35485]: pgmap v530: 97 pgs: 97 active+clean; 13 GiB data, 116 GiB used, 957 GiB / 1.0 TiB avail; 427 KiB/s rd, 186 MiB/s wr, 19.79k op/s 2024-09-12T03:27:37.198 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:38.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.186+0000 7f53ed2f8640 1 -- 172.21.15.6:0/1013406207 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4040880 con 0x7f53e8106a70 2024-09-12T03:27:38.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.186+0000 7f53f055e640 1 -- 172.21.15.6:0/1013406207 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f53e8106a70 msgr2=0x7f53e8106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:38.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.186+0000 7f53f055e640 1 --2- 172.21.15.6:0/1013406207 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f53e8106a70 0x7f53e8106e50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f53e401a910 tx=0x7f53e403b470 comp rx=0 tx=0).stop 2024-09-12T03:27:38.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.188+0000 7f53f055e640 1 -- 172.21.15.6:0/1013406207 shutdown_connections 2024-09-12T03:27:38.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.188+0000 7f53f055e640 1 --2- 172.21.15.6:0/1013406207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e8101410 0x7f53e810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.188+0000 7f53f055e640 1 --2- 172.21.15.6:0/1013406207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53e8100a70 0x7f53e8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.188+0000 7f53f055e640 1 --2- 172.21.15.6:0/1013406207 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f53e8106a70 0x7f53e8106e50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.188+0000 7f53f055e640 1 -- 172.21.15.6:0/1013406207 >> 172.21.15.6:0/1013406207 conn(0x7f53e80fc790 msgr2=0x7f53e80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:38.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.194+0000 7f53f055e640 1 -- 172.21.15.6:0/1013406207 shutdown_connections 2024-09-12T03:27:38.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.194+0000 7f53f055e640 1 -- 172.21.15.6:0/1013406207 wait complete. 2024-09-12T03:27:38.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.195+0000 7f53f055e640 1 Processor -- start 2024-09-12T03:27:38.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.196+0000 7f53f055e640 1 -- start start 2024-09-12T03:27:38.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.196+0000 7f53f055e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53e8100a70 0x7f53e8073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.196+0000 7f53f055e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f53e8101410 0x7f53e806ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.196+0000 7f53f055e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e8106a70 0x7f53e806f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.196+0000 7f53f055e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f53e806d570 con 0x7f53e8100a70 2024-09-12T03:27:38.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.196+0000 7f53f055e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f53e806d3f0 con 0x7f53e8101410 2024-09-12T03:27:38.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.196+0000 7f53f055e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f53e806d6f0 con 0x7f53e8106a70 2024-09-12T03:27:38.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.196+0000 7f53eeafb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e8106a70 0x7f53e806f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:38.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.196+0000 7f53edaf9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f53e8101410 0x7f53e806ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:38.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.197+0000 7f53edaf9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f53e8101410 0x7f53e806ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:54282/0 (socket says 172.21.15.6:54282) 2024-09-12T03:27:38.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.197+0000 7f53edaf9640 1 -- 172.21.15.6:0/2096388627 learned_addr learned my addr 172.21.15.6:0/2096388627 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:38.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.198+0000 7f53eeafb640 1 -- 172.21.15.6:0/2096388627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f53e8101410 msgr2=0x7f53e806ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:38.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.198+0000 7f53eeafb640 1 --2- 172.21.15.6:0/2096388627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f53e8101410 0x7f53e806ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.198+0000 7f53eeafb640 1 -- 172.21.15.6:0/2096388627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53e8100a70 msgr2=0x7f53e8073b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:27:38.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.198+0000 7f53eeafb640 1 --2- 172.21.15.6:0/2096388627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53e8100a70 0x7f53e8073b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.198+0000 7f53eeafb640 1 -- 172.21.15.6:0/2096388627 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e806f870 con 0x7f53e8106a70 2024-09-12T03:27:38.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.205+0000 7f53eeafb640 1 --2- 172.21.15.6:0/2096388627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e8106a70 0x7f53e806f140 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f53e000be00 tx=0x7f53e000bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:38.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.205+0000 7f53df7fe640 1 -- 172.21.15.6:0/2096388627 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0019070 con 0x7f53e8106a70 2024-09-12T03:27:38.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.205+0000 7f53f055e640 1 -- 172.21.15.6:0/2096388627 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e819a280 con 0x7f53e8106a70 2024-09-12T03:27:38.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.205+0000 7f53f055e640 1 -- 172.21.15.6:0/2096388627 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e819a730 con 0x7f53e8106a70 2024-09-12T03:27:38.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.205+0000 7f53df7fe640 1 -- 172.21.15.6:0/2096388627 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e0014920 con 0x7f53e8106a70 2024-09-12T03:27:38.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.205+0000 7f53df7fe640 1 -- 172.21.15.6:0/2096388627 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e00128e0 con 0x7f53e8106a70 2024-09-12T03:27:38.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.210+0000 7f53f2937740 1 Processor -- start 2024-09-12T03:27:38.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.212+0000 7f53df7fe640 1 -- 172.21.15.6:0/2096388627 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f53e001c070 con 0x7f53e8106a70 2024-09-12T03:27:38.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.213+0000 7f53df7fe640 1 --2- 172.21.15.6:0/2096388627 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f53d40a4ed0 0x7f53d40a7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.213+0000 7f53df7fe640 1 -- 172.21.15.6:0/2096388627 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f53e00ce6e0 con 0x7f53e8106a70 2024-09-12T03:27:38.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.214+0000 7f53f2937740 1 -- start start 2024-09-12T03:27:38.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.214+0000 7f53f2937740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56181ccd3840 0x56181ccd3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.214+0000 7f53f2937740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd2120 0x56181ccd2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.214+0000 7f53f2937740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd2b80 0x56181cce7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.214+0000 7f53f2937740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56181cc382f0 con 0x56181ccd2b80 2024-09-12T03:27:38.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.214+0000 7f53f2937740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x56181cbdfcf0 con 0x56181ccd2120 2024-09-12T03:27:38.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.214+0000 7f53f2937740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56181cc342b0 con 0x56181ccd3840 2024-09-12T03:27:38.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.216+0000 7f53eeafb640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd2b80 0x56181cce7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:38.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.216+0000 7f53eeafb640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd2b80 0x56181cce7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48520/0 (socket says 172.21.15.6:48520) 2024-09-12T03:27:38.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.216+0000 7f53eeafb640 1 -- 172.21.15.6:0/1888173181 learned_addr learned my addr 172.21.15.6:0/1888173181 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:38.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.218+0000 7f53ee2fa640 1 --2- 172.21.15.6:0/1888173181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56181ccd3840 0x56181ccd3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:38.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.222+0000 7f53eeafb640 1 -- 172.21.15.6:0/1888173181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56181ccd3840 msgr2=0x56181ccd3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:38.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.222+0000 7f53eeafb640 1 --2- 172.21.15.6:0/1888173181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56181ccd3840 0x56181ccd3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.222+0000 7f53eeafb640 1 -- 172.21.15.6:0/1888173181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd2120 msgr2=0x56181ccd2560 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:27:38.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.222+0000 7f53edaf9640 1 --2- 172.21.15.6:0/1888173181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd2120 0x56181ccd2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:38.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.223+0000 7f53edaf9640 1 --2- 172.21.15.6:0/2096388627 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f53d40a4ed0 0x7f53d40a7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:38.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.228+0000 7f53eeafb640 1 --2- 172.21.15.6:0/1888173181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd2120 0x56181ccd2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.228+0000 7f53eeafb640 1 -- 172.21.15.6:0/1888173181 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56181cbc43a0 con 0x56181ccd2b80 2024-09-12T03:27:38.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.234+0000 7f53eeafb640 1 --2- 172.21.15.6:0/1888173181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd2b80 0x56181cce7f60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f53e0097420 tx=0x7f53e009a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b00fc112f8807 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:38.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.236+0000 7f53dcff9640 1 -- 172.21.15.6:0/1888173181 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e009f2c0 con 0x56181ccd2b80 2024-09-12T03:27:38.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.238+0000 7f53dcff9640 1 -- 172.21.15.6:0/1888173181 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e00a3b80 con 0x56181ccd2b80 2024-09-12T03:27:38.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.240+0000 7f53f2937740 1 -- 172.21.15.6:0/1888173181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd2b80 msgr2=0x56181cce7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:38.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.240+0000 7f53f2937740 1 --2- 172.21.15.6:0/1888173181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd2b80 0x56181cce7f60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f53e0097420 tx=0x7f53e009a920 comp rx=0 tx=0).stop 2024-09-12T03:27:38.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.241+0000 7f53f2937740 1 -- 172.21.15.6:0/1888173181 shutdown_connections 2024-09-12T03:27:38.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.241+0000 7f53f2937740 1 --2- 172.21.15.6:0/1888173181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd2b80 0x56181cce7f60 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.241+0000 7f53f2937740 1 --2- 172.21.15.6:0/1888173181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd2120 0x56181ccd2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.241+0000 7f53f2937740 1 --2- 172.21.15.6:0/1888173181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56181ccd3840 0x56181ccd3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.241+0000 7f53f2937740 1 -- 172.21.15.6:0/1888173181 >> 172.21.15.6:0/1888173181 conn(0x56181cc9b310 msgr2=0x56181cccec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:38.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.243+0000 7f53edaf9640 1 --2- 172.21.15.6:0/2096388627 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f53d40a4ed0 0x7f53d40a7390 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x56181cccfac0 tx=0x7f53d802b190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:38.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.243+0000 7f53f2937740 1 -- 172.21.15.6:0/1888173181 shutdown_connections 2024-09-12T03:27:38.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.245+0000 7f53f2937740 1 -- 172.21.15.6:0/1888173181 wait complete. 2024-09-12T03:27:38.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.245+0000 7f53f2937740 1 -- start start 2024-09-12T03:27:38.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.245+0000 7f53f2937740 1 Processor -- start 2024-09-12T03:27:38.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.246+0000 7f53f2937740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd2120 0x56181cce2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.246+0000 7f53f2937740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56181ccd2b80 0x56181cce25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.246+0000 7f53f2937740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd3840 0x56181cce7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.246+0000 7f53f2937740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56181ccd9c30 con 0x56181ccd3840 2024-09-12T03:27:38.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.246+0000 7f53f2937740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x56181cbf3060 con 0x56181ccd2120 2024-09-12T03:27:38.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.246+0000 7f53f2937740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56181ccd9db0 con 0x56181ccd2b80 2024-09-12T03:27:38.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.246+0000 7f53f2937740 10 client.0 _ll_register_callbacks cb 0x56181c9468c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:38.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.249+0000 7f53eeafb640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd3840 0x56181cce7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:38.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.250+0000 7f53eeafb640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd3840 0x56181cce7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48524/0 (socket says 172.21.15.6:48524) 2024-09-12T03:27:38.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.250+0000 7f53eeafb640 1 -- 172.21.15.6:0/871039146 learned_addr learned my addr 172.21.15.6:0/871039146 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:38.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.250+0000 7f53eeafb640 1 -- 172.21.15.6:0/871039146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56181ccd2b80 msgr2=0x56181cce25b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:27:38.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.250+0000 7f53eeafb640 1 --2- 172.21.15.6:0/871039146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56181ccd2b80 0x56181cce25b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.250+0000 7f53eeafb640 1 -- 172.21.15.6:0/871039146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd2120 msgr2=0x56181cce2070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:27:38.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.250+0000 7f53eeafb640 1 --2- 172.21.15.6:0/871039146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd2120 0x56181cce2070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.250+0000 7f53eeafb640 1 -- 172.21.15.6:0/871039146 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56181c9a93c0 con 0x56181ccd3840 2024-09-12T03:27:38.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.252+0000 7f53eeafb640 1 --2- 172.21.15.6:0/871039146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd3840 0x56181cce7d80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f53e00a3650 tx=0x7f53e009f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8763b1dd26d7d74d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:38.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.252+0000 7f53d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:27:38.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.252+0000 7f53d1ffb640 1 -- 172.21.15.6:0/871039146 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e00cc380 con 0x56181ccd3840 2024-09-12T03:27:38.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.253+0000 7f53f2937740 1 -- 172.21.15.6:0/871039146 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56181cc45860 con 0x56181ccd3840 2024-09-12T03:27:38.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.253+0000 7f53d1ffb640 1 -- 172.21.15.6:0/871039146 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f53e00b9b50 con 0x56181ccd3840 2024-09-12T03:27:38.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.253+0000 7f53d1ffb640 1 -- 172.21.15.6:0/871039146 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f53e00b9d50 con 0x56181ccd3840 2024-09-12T03:27:38.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.254+0000 7f53f2937740 10 client.16095 fetch_fsmap learned FSMap version 27 2024-09-12T03:27:38.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.254+0000 7f53f2937740 1 -- 172.21.15.6:0/871039146 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56181ccfcdd0 con 0x56181ccd3840 2024-09-12T03:27:38.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.254+0000 7f53d1ffb640 1 -- 172.21.15.6:0/871039146 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56181cc45860 con 0x56181ccd3840 2024-09-12T03:27:38.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.255+0000 7f53d1ffb640 1 -- 172.21.15.6:0/871039146 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f53e009a410 con 0x56181ccd3840 2024-09-12T03:27:38.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.255+0000 7f53f2937740 10 client.16095 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:27:38.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.255+0000 7f53f2937740 10 client.16095 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-12T03:27:38.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.255+0000 7f53f2937740 20 client.16095 populate_metadata read hostname 'smithi006' 2024-09-12T03:27:38.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.255+0000 7f53f2937740 1 --2- 172.21.15.6:0/871039146 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x56181ccfe330 0x56181cd007d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.255+0000 7f53f2937740 4 client.16095 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:38.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.255+0000 7f53f2937740 1 -- 172.21.15.6:0/871039146 --> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56181cc3fd70 con 0x56181ccfe330 2024-09-12T03:27:38.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.259+0000 7f53ee2fa640 1 --2- 172.21.15.6:0/871039146 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x56181ccfe330 0x56181cd007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:38.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.266+0000 7f53ee2fa640 1 --2- 172.21.15.6:0/871039146 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x56181ccfe330 0x56181cd007d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d0dc05eef6b9e82 server_cookie=1a5bd4429c2a010f in_seq=0 out_seq=0 2024-09-12T03:27:38.267 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.267+0000 7f53d1ffb640 10 client.16095 ms_handle_connect on v2:172.21.15.118:6836/3879191727 2024-09-12T03:27:38.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.269+0000 7f53d1ffb640 1 -- 172.21.15.6:0/871039146 <== mds.1 v2:172.21.15.118:6836/3879191727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56181cc3fd70 con 0x56181ccfe330 2024-09-12T03:27:38.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.269+0000 7f53d1ffb640 10 client.16095 handle_command_reply: tid=0 2024-09-12T03:27:38.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.269+0000 7f53f2937740 1 client.16095 shutdown 2024-09-12T03:27:38.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.274+0000 7f53f2937740 1 -- 172.21.15.6:0/871039146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd3840 msgr2=0x56181cce7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:38.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.274+0000 7f53f2937740 1 --2- 172.21.15.6:0/871039146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd3840 0x56181cce7d80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f53e00a3650 tx=0x7f53e009f390 comp rx=0 tx=0).stop 2024-09-12T03:27:38.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.274+0000 7f53f2937740 1 -- 172.21.15.6:0/871039146 shutdown_connections 2024-09-12T03:27:38.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.274+0000 7f53f2937740 1 --2- 172.21.15.6:0/871039146 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x56181ccfe330 0x56181cd007d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.274+0000 7f53f2937740 1 --2- 172.21.15.6:0/871039146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd3840 0x56181cce7d80 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.274+0000 7f53f2937740 1 --2- 172.21.15.6:0/871039146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56181ccd2b80 0x56181cce25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.274+0000 7f53f2937740 1 --2- 172.21.15.6:0/871039146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd2120 0x56181cce2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.274+0000 7f53f2937740 1 -- 172.21.15.6:0/871039146 >> 172.21.15.6:0/871039146 conn(0x56181cc9b310 msgr2=0x56181ccd49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:38.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.274+0000 7f53d1ffb640 0 client.16095 ms_handle_reset on v2:172.21.15.118:6836/3879191727 2024-09-12T03:27:38.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.274+0000 7f53f2937740 1 -- 172.21.15.6:0/871039146 shutdown_connections 2024-09-12T03:27:38.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.274+0000 7f53f2937740 1 -- 172.21.15.6:0/871039146 wait complete. 2024-09-12T03:27:38.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.275+0000 7f53f2937740 20 client.16095 trim_cache size 0 max 16384 2024-09-12T03:27:38.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.305+0000 7f53f2937740 1 Processor -- start 2024-09-12T03:27:38.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.306+0000 7f53f2937740 1 -- start start 2024-09-12T03:27:38.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.306+0000 7f53f2937740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56181ccd2120 0x56181cce7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.307+0000 7f53f2937740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd2b80 0x56181ccd90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.308+0000 7f53f2937740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd3840 0x56181cce2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.308+0000 7f53f2937740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc010db0 con 0x56181ccd2b80 2024-09-12T03:27:38.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.308+0000 7f53f2937740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc00e680 con 0x56181ccd3840 2024-09-12T03:27:38.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.308+0000 7f53f2937740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc00c4d0 con 0x56181ccd2120 2024-09-12T03:27:38.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.309+0000 7f53eeafb640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd3840 0x56181cce2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:38.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.309+0000 7f53eeafb640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd3840 0x56181cce2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:54290/0 (socket says 172.21.15.6:54290) 2024-09-12T03:27:38.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.309+0000 7f53eeafb640 1 -- 172.21.15.6:0/1578287530 learned_addr learned my addr 172.21.15.6:0/1578287530 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:38.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.311+0000 7f53edaf9640 1 --2- 172.21.15.6:0/1578287530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd2b80 0x56181ccd90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:38.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.311+0000 7f53ee2fa640 1 --2- 172.21.15.6:0/1578287530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56181ccd2120 0x56181cce7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:38.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.314+0000 7f53eeafb640 1 -- 172.21.15.6:0/1578287530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56181ccd2120 msgr2=0x56181cce7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:38.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.314+0000 7f53eeafb640 1 --2- 172.21.15.6:0/1578287530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56181ccd2120 0x56181cce7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.314+0000 7f53eeafb640 1 -- 172.21.15.6:0/1578287530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd2b80 msgr2=0x56181ccd90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:38.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.314+0000 7f53eeafb640 1 --2- 172.21.15.6:0/1578287530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd2b80 0x56181ccd90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.314+0000 7f53eeafb640 1 -- 172.21.15.6:0/1578287530 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56181ccc9f90 con 0x56181ccd3840 2024-09-12T03:27:38.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.315+0000 7f53edaf9640 1 --2- 172.21.15.6:0/1578287530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd2b80 0x56181ccd90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:38.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.316+0000 7f53eeafb640 1 --2- 172.21.15.6:0/1578287530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd3840 0x56181cce2070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f53e009acc0 tx=0x7f53e00096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ae71faecc06754e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:38.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.317+0000 7f53d3fff640 1 -- 172.21.15.6:0/1578287530 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e00cc380 con 0x56181ccd3840 2024-09-12T03:27:38.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.317+0000 7f53d3fff640 1 -- 172.21.15.6:0/1578287530 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e00a3900 con 0x56181ccd3840 2024-09-12T03:27:38.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.318+0000 7f53f2937740 1 -- 172.21.15.6:0/1578287530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd3840 msgr2=0x56181cce2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:38.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.318+0000 7f53f2937740 1 --2- 172.21.15.6:0/1578287530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd3840 0x56181cce2070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f53e009acc0 tx=0x7f53e00096c0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.318+0000 7f53f2937740 1 -- 172.21.15.6:0/1578287530 shutdown_connections 2024-09-12T03:27:38.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.318+0000 7f53f2937740 1 --2- 172.21.15.6:0/1578287530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd3840 0x56181cce2070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.318+0000 7f53f2937740 1 --2- 172.21.15.6:0/1578287530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd2b80 0x56181ccd90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.318+0000 7f53f2937740 1 --2- 172.21.15.6:0/1578287530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56181ccd2120 0x56181cce7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.318+0000 7f53f2937740 1 -- 172.21.15.6:0/1578287530 >> 172.21.15.6:0/1578287530 conn(0x56181cc9b310 msgr2=0x56181ccd89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:38.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.319+0000 7f53f2937740 1 -- 172.21.15.6:0/1578287530 shutdown_connections 2024-09-12T03:27:38.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.320+0000 7f53f2937740 1 -- 172.21.15.6:0/1578287530 wait complete. 2024-09-12T03:27:38.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.320+0000 7f53f2937740 1 -- start start 2024-09-12T03:27:38.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.321+0000 7f53f2937740 1 Processor -- start 2024-09-12T03:27:38.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.321+0000 7f53f2937740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd2120 0x56181cd0f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.321+0000 7f53f2937740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd2b80 0x56181cd0f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.321+0000 7f53f2937740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56181ccd3840 0x56181cd66ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.321+0000 7f53f2937740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc00cb40 con 0x56181ccd2120 2024-09-12T03:27:38.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.321+0000 7f53f2937740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc00dfb0 con 0x56181ccd2b80 2024-09-12T03:27:38.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.321+0000 7f53f2937740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc00d210 con 0x56181ccd3840 2024-09-12T03:27:38.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.321+0000 7f53f2937740 10 client.0 _ll_register_callbacks cb 0x7f53e00b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:38.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.322+0000 7f53edaf9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd2b80 0x56181cd0f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:38.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.322+0000 7f53edaf9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd2b80 0x56181cd0f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:54304/0 (socket says 172.21.15.6:54304) 2024-09-12T03:27:38.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.322+0000 7f53edaf9640 1 -- 172.21.15.6:0/870312889 learned_addr learned my addr 172.21.15.6:0/870312889 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:38.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.323+0000 7f53edaf9640 1 -- 172.21.15.6:0/870312889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56181ccd3840 msgr2=0x56181cd66ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:38.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.323+0000 7f53edaf9640 1 --2- 172.21.15.6:0/870312889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56181ccd3840 0x56181cd66ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.323+0000 7f53edaf9640 1 -- 172.21.15.6:0/870312889 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd2120 msgr2=0x56181cd0f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:38.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.323+0000 7f53edaf9640 1 --2- 172.21.15.6:0/870312889 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd2120 0x56181cd0f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.323+0000 7f53edaf9640 1 -- 172.21.15.6:0/870312889 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56181cd67430 con 0x56181ccd2b80 2024-09-12T03:27:38.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.323+0000 7f53edaf9640 1 --2- 172.21.15.6:0/870312889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd2b80 0x56181cd0f6a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f53e80703c0 tx=0x7f53d8004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e22d62ce9ad4d59 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:38.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.323+0000 7f53d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:27:38.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.324+0000 7f53d1ffb640 1 -- 172.21.15.6:0/870312889 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8035d30 con 0x56181ccd2b80 2024-09-12T03:27:38.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.324+0000 7f53f2937740 1 -- 172.21.15.6:0/870312889 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56181cbbc170 con 0x56181ccd2b80 2024-09-12T03:27:38.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.324+0000 7f53d1ffb640 1 -- 172.21.15.6:0/870312889 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f53d8042420 con 0x56181ccd2b80 2024-09-12T03:27:38.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.325+0000 7f53d1ffb640 1 -- 172.21.15.6:0/870312889 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f53d8051070 con 0x56181ccd2b80 2024-09-12T03:27:38.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.325+0000 7f53f2937740 10 client.25640 fetch_fsmap learned FSMap version 27 2024-09-12T03:27:38.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.325+0000 7f53f2937740 1 -- 172.21.15.6:0/870312889 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56181cd68740 con 0x56181ccd2b80 2024-09-12T03:27:38.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.326+0000 7f53d1ffb640 1 -- 172.21.15.6:0/870312889 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56181cbbc170 con 0x56181ccd2b80 2024-09-12T03:27:38.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.326+0000 7f53d1ffb640 1 -- 172.21.15.6:0/870312889 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f53d804a460 con 0x56181ccd2b80 2024-09-12T03:27:38.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.327+0000 7f53f2937740 10 client.25640 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:27:38.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.327+0000 7f53f2937740 10 client.25640 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-12T03:27:38.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.327+0000 7f53f2937740 20 client.25640 populate_metadata read hostname 'smithi006' 2024-09-12T03:27:38.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.327+0000 7f53f2937740 1 --2- 172.21.15.6:0/870312889 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x56181cd68ed0 0x56181cd6b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.327+0000 7f53f2937740 4 client.25640 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:27:38.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.327+0000 7f53f2937740 1 -- 172.21.15.6:0/870312889 --> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56181cc099b0 con 0x56181cd68ed0 2024-09-12T03:27:38.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.330+0000 7f53ee2fa640 1 --2- 172.21.15.6:0/870312889 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x56181cd68ed0 0x56181cd6b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:38.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.331+0000 7f53ee2fa640 1 --2- 172.21.15.6:0/870312889 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x56181cd68ed0 0x56181cd6b330 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b36e38a7450104a server_cookie=2a9f794a6d2ade0e in_seq=0 out_seq=0 2024-09-12T03:27:38.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.333+0000 7f53d1ffb640 10 client.25640 ms_handle_connect on v2:172.21.15.118:6836/3879191727 2024-09-12T03:27:38.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.333+0000 7f53d1ffb640 1 -- 172.21.15.6:0/870312889 <== mds.1 v2:172.21.15.118:6836/3879191727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56181cc099b0 con 0x56181cd68ed0 2024-09-12T03:27:38.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.333+0000 7f53d1ffb640 10 client.25640 handle_command_reply: tid=0 2024-09-12T03:27:38.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.333+0000 7f53f2937740 1 client.25640 shutdown 2024-09-12T03:27:38.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.336+0000 7f53f2937740 1 -- 172.21.15.6:0/870312889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd2b80 msgr2=0x56181cd0f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:38.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.336+0000 7f53f2937740 1 --2- 172.21.15.6:0/870312889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd2b80 0x56181cd0f6a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f53e80703c0 tx=0x7f53d8004700 comp rx=0 tx=0).stop 2024-09-12T03:27:38.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.336+0000 7f53f2937740 1 -- 172.21.15.6:0/870312889 shutdown_connections 2024-09-12T03:27:38.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.336+0000 7f53f2937740 1 --2- 172.21.15.6:0/870312889 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x56181cd68ed0 0x56181cd6b330 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.336+0000 7f53f2937740 1 --2- 172.21.15.6:0/870312889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56181ccd3840 0x56181cd66ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.336+0000 7f53f2937740 1 --2- 172.21.15.6:0/870312889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56181ccd2b80 0x56181cd0f6a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.336+0000 7f53f2937740 1 --2- 172.21.15.6:0/870312889 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56181ccd2120 0x56181cd0f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.336+0000 7f53f2937740 1 -- 172.21.15.6:0/870312889 >> 172.21.15.6:0/870312889 conn(0x56181cc9b310 msgr2=0x56181cccd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:38.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.336+0000 7f53d1ffb640 0 client.25640 ms_handle_reset on v2:172.21.15.118:6836/3879191727 2024-09-12T03:27:38.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.336+0000 7f53f2937740 1 -- 172.21.15.6:0/870312889 shutdown_connections 2024-09-12T03:27:38.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.336+0000 7f53f2937740 1 -- 172.21.15.6:0/870312889 wait complete. 2024-09-12T03:27:38.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.337+0000 7f53f2937740 20 client.25640 trim_cache size 0 max 16384 2024-09-12T03:27:38.338 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-09-12T03:27:38.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.343+0000 7f53f055e640 1 -- 172.21.15.6:0/2096388627 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f53d40a4ed0 msgr2=0x7f53d40a7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:38.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.343+0000 7f53f055e640 1 --2- 172.21.15.6:0/2096388627 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f53d40a4ed0 0x7f53d40a7390 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x56181cccfac0 tx=0x7f53d802b190 comp rx=0 tx=0).stop 2024-09-12T03:27:38.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.343+0000 7f53f055e640 1 -- 172.21.15.6:0/2096388627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e8106a70 msgr2=0x7f53e806f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:38.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.343+0000 7f53f055e640 1 --2- 172.21.15.6:0/2096388627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e8106a70 0x7f53e806f140 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f53e000be00 tx=0x7f53e000bef0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.343+0000 7f53f055e640 1 -- 172.21.15.6:0/2096388627 shutdown_connections 2024-09-12T03:27:38.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.343+0000 7f53f055e640 1 --2- 172.21.15.6:0/2096388627 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f53d40a4ed0 0x7f53d40a7390 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.343+0000 7f53f055e640 1 --2- 172.21.15.6:0/2096388627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e8106a70 0x7f53e806f140 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.343+0000 7f53f055e640 1 --2- 172.21.15.6:0/2096388627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f53e8101410 0x7f53e806ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.343+0000 7f53f055e640 1 --2- 172.21.15.6:0/2096388627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53e8100a70 0x7f53e8073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.343+0000 7f53f055e640 1 -- 172.21.15.6:0/2096388627 >> 172.21.15.6:0/2096388627 conn(0x7f53e80fc790 msgr2=0x7f53e80fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:38.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.344+0000 7f53f055e640 1 -- 172.21.15.6:0/2096388627 shutdown_connections 2024-09-12T03:27:38.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:38.344+0000 7f53f055e640 1 -- 172.21.15.6:0/2096388627 wait complete. 2024-09-12T03:27:38.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:38 smithi006 ceph-mon[30991]: pgmap v531: 97 pgs: 97 active+clean; 13 GiB data, 117 GiB used, 956 GiB / 1.0 TiB avail; 511 KiB/s rd, 166 MiB/s wr, 17.42k op/s 2024-09-12T03:27:38.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:38 smithi118 ceph-mon[35048]: pgmap v531: 97 pgs: 97 active+clean; 13 GiB data, 117 GiB used, 956 GiB / 1.0 TiB avail; 511 KiB/s rd, 166 MiB/s wr, 17.42k op/s 2024-09-12T03:27:38.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:38 smithi161 ceph-mon[35485]: pgmap v531: 97 pgs: 97 active+clean; 13 GiB data, 117 GiB used, 956 GiB / 1.0 TiB avail; 511 KiB/s rd, 166 MiB/s wr, 17.42k op/s 2024-09-12T03:27:39.642 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:27:39.938 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:40.724 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:40 smithi118 ceph-mon[35048]: pgmap v532: 97 pgs: 97 active+clean; 13 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 511 KiB/s rd, 173 MiB/s wr, 17.83k op/s 2024-09-12T03:27:40.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:40 smithi006 ceph-mon[30991]: pgmap v532: 97 pgs: 97 active+clean; 13 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 511 KiB/s rd, 173 MiB/s wr, 17.83k op/s 2024-09-12T03:27:40.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:40 smithi161 ceph-mon[35485]: pgmap v532: 97 pgs: 97 active+clean; 13 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 511 KiB/s rd, 173 MiB/s wr, 17.83k op/s 2024-09-12T03:27:41.031 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-09-12T03:27:41.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.892+0000 7f2ffe08f640 1 -- 172.21.15.6:0/4223957866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ff8101430 msgr2=0x7f2ff806a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:41.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.892+0000 7f2ffe08f640 1 --2- 172.21.15.6:0/4223957866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ff8101430 0x7f2ff806a790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0009ca0 tx=0x7f2ff002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.892+0000 7f2ffe08f640 1 -- 172.21.15.6:0/4223957866 shutdown_connections 2024-09-12T03:27:41.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.892+0000 7f2ffe08f640 1 --2- 172.21.15.6:0/4223957866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ff8101430 0x7f2ff806a790 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.892+0000 7f2ffe08f640 1 --2- 172.21.15.6:0/4223957866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ff8100a90 0x7f2ff8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.892+0000 7f2ffe08f640 1 --2- 172.21.15.6:0/4223957866 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ff8106a90 0x7f2ff8106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.892+0000 7f2ffe08f640 1 -- 172.21.15.6:0/4223957866 >> 172.21.15.6:0/4223957866 conn(0x7f2ff80fc7d0 msgr2=0x7f2ff80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:41.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.896+0000 7f2ffe08f640 1 -- 172.21.15.6:0/4223957866 shutdown_connections 2024-09-12T03:27:41.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.896+0000 7f2ffe08f640 1 -- 172.21.15.6:0/4223957866 wait complete. 2024-09-12T03:27:41.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.897+0000 7f2ffe08f640 1 Processor -- start 2024-09-12T03:27:41.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.897+0000 7f2ffe08f640 1 -- start start 2024-09-12T03:27:41.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.897+0000 7f2ffe08f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ff8100a90 0x7f2ff810fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:41.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.897+0000 7f2ffe08f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ff8106a90 0x7f2ff8110180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:41.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.897+0000 7f2ffe08f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ff81106c0 0x7f2ff819c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:41.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.897+0000 7f2ffe08f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff81141f0 con 0x7f2ff8106a90 2024-09-12T03:27:41.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.897+0000 7f2ffe08f640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff8114070 con 0x7f2ff81106c0 2024-09-12T03:27:41.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.897+0000 7f2ffe08f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff8114370 con 0x7f2ff8100a90 2024-09-12T03:27:41.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.900+0000 7f2ff6ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ff8106a90 0x7f2ff8110180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:41.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.901+0000 7f2ff6ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ff8106a90 0x7f2ff8110180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39752/0 (socket says 172.21.15.6:39752) 2024-09-12T03:27:41.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.901+0000 7f2ff7fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ff81106c0 0x7f2ff819c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:41.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.901+0000 7f2ff6ffd640 1 -- 172.21.15.6:0/498930058 learned_addr learned my addr 172.21.15.6:0/498930058 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:41.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.901+0000 7f2ff77fe640 1 --2- 172.21.15.6:0/498930058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ff8100a90 0x7f2ff810fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:41.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.901+0000 7f2ff6ffd640 1 -- 172.21.15.6:0/498930058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ff8100a90 msgr2=0x7f2ff810fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:41.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.901+0000 7f2ff6ffd640 1 --2- 172.21.15.6:0/498930058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ff8100a90 0x7f2ff810fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.901+0000 7f2ff6ffd640 1 -- 172.21.15.6:0/498930058 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ff81106c0 msgr2=0x7f2ff819c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:41.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.901+0000 7f2ff6ffd640 1 --2- 172.21.15.6:0/498930058 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ff81106c0 0x7f2ff819c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.901+0000 7f2ff6ffd640 1 -- 172.21.15.6:0/498930058 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff819c5f0 con 0x7f2ff8106a90 2024-09-12T03:27:41.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.902+0000 7f2ff6ffd640 1 --2- 172.21.15.6:0/498930058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ff8106a90 0x7f2ff8110180 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800c970 tx=0x7f2fe800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:41.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.905+0000 7f2ff4ff9640 1 -- 172.21.15.6:0/498930058 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8007bb0 con 0x7f2ff8106a90 2024-09-12T03:27:41.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.905+0000 7f2ffe08f640 1 -- 172.21.15.6:0/498930058 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff819c8e0 con 0x7f2ff8106a90 2024-09-12T03:27:41.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.905+0000 7f2ffe08f640 1 -- 172.21.15.6:0/498930058 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff819ce20 con 0x7f2ff8106a90 2024-09-12T03:27:41.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.905+0000 7f2ff4ff9640 1 -- 172.21.15.6:0/498930058 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fe8007d50 con 0x7f2ff8106a90 2024-09-12T03:27:41.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.905+0000 7f2ff4ff9640 1 -- 172.21.15.6:0/498930058 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe800f450 con 0x7f2ff8106a90 2024-09-12T03:27:41.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.909+0000 7f3000468740 1 Processor -- start 2024-09-12T03:27:41.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.909+0000 7f3000468740 1 -- start start 2024-09-12T03:27:41.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.910+0000 7f3000468740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcedb40 0x55896fcec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:41.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.910+0000 7f3000468740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fcec5d0 0x55896fceca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:41.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.910+0000 7f3000468740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fd08380 0x55896fd0a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:41.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.912+0000 7f2ff7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcedb40 0x55896fcec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:41.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.912+0000 7f2ff7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcedb40 0x55896fcec090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:42970/0 (socket says 172.21.15.6:42970) 2024-09-12T03:27:41.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.912+0000 7f2ff77fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fcec5d0 0x55896fceca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:41.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.912+0000 7f2ff4ff9640 1 -- 172.21.15.6:0/498930058 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2fe80046c0 con 0x7f2ff8106a90 2024-09-12T03:27:41.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.913+0000 7f2ff4ff9640 1 --2- 172.21.15.6:0/498930058 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2fe00a2da0 0x7f2fe00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:41.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.913+0000 7f2ff4ff9640 1 -- 172.21.15.6:0/498930058 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f2fe80d0cc0 con 0x7f2ff8106a90 2024-09-12T03:27:41.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.913+0000 7f2ff77fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fcec5d0 0x55896fceca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:49290/0 (socket says 172.21.15.6:49290) 2024-09-12T03:27:41.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.913+0000 7f2ff7fff640 1 -- 172.21.15.6:0/1127817710 learned_addr learned my addr 172.21.15.6:0/1127817710 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:41.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.913+0000 7f3000468740 1 -- 172.21.15.6:0/1127817710 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55896fc522f0 con 0x55896fd08380 2024-09-12T03:27:41.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.913+0000 7f3000468740 1 -- 172.21.15.6:0/1127817710 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55896fbf9cf0 con 0x55896fcec5d0 2024-09-12T03:27:41.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.914+0000 7f3000468740 1 -- 172.21.15.6:0/1127817710 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55896fc4e2b0 con 0x55896fcedb40 2024-09-12T03:27:41.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.914+0000 7f2ff6ffd640 1 --2- 172.21.15.6:0/1127817710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fd08380 0x55896fd0a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:41.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.914+0000 7f2ff6ffd640 1 -- 172.21.15.6:0/1127817710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcedb40 msgr2=0x55896fcec090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:41.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.914+0000 7f2ff6ffd640 1 --2- 172.21.15.6:0/1127817710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcedb40 0x55896fcec090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.914+0000 7f2ff6ffd640 1 -- 172.21.15.6:0/1127817710 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fcec5d0 msgr2=0x55896fceca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:41.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.914+0000 7f2ff6ffd640 1 --2- 172.21.15.6:0/1127817710 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fcec5d0 0x55896fceca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.914+0000 7f2ff6ffd640 1 -- 172.21.15.6:0/1127817710 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55896fbde3a0 con 0x55896fd08380 2024-09-12T03:27:41.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.915+0000 7f2ff7fff640 1 --2- 172.21.15.6:0/1127817710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcedb40 0x55896fcec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:41.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.915+0000 7f2ff77fe640 1 --2- 172.21.15.6:0/1127817710 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fcec5d0 0x55896fceca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:41.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.915+0000 7f2ff6ffd640 1 --2- 172.21.15.6:0/1127817710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fd08380 0x55896fd0a770 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800c940 tx=0x7f2fe809cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=379e07617175c308 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:41.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.915+0000 7f2ff7fff640 1 --2- 172.21.15.6:0/498930058 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2fe00a2da0 0x7f2fe00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:41.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.916+0000 7f2fddffb640 1 -- 172.21.15.6:0/1127817710 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe80b6070 con 0x55896fd08380 2024-09-12T03:27:41.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.916+0000 7f2fddffb640 1 -- 172.21.15.6:0/1127817710 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fe80a6160 con 0x55896fd08380 2024-09-12T03:27:41.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.916+0000 7f2fddffb640 1 -- 172.21.15.6:0/1127817710 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe801d040 con 0x55896fd08380 2024-09-12T03:27:41.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.917+0000 7f2ff7fff640 1 --2- 172.21.15.6:0/498930058 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2fe00a2da0 0x7f2fe00a5260 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0005e30 tx=0x7f2ff00095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:41.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.917+0000 7f3000468740 1 -- 172.21.15.6:0/1127817710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fd08380 msgr2=0x55896fd0a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:41.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.917+0000 7f3000468740 1 --2- 172.21.15.6:0/1127817710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fd08380 0x55896fd0a770 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800c940 tx=0x7f2fe809cf70 comp rx=0 tx=0).stop 2024-09-12T03:27:41.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.918+0000 7f3000468740 1 -- 172.21.15.6:0/1127817710 shutdown_connections 2024-09-12T03:27:41.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.918+0000 7f3000468740 1 --2- 172.21.15.6:0/1127817710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fd08380 0x55896fd0a770 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.918+0000 7f3000468740 1 --2- 172.21.15.6:0/1127817710 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fcec5d0 0x55896fceca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.918+0000 7f3000468740 1 --2- 172.21.15.6:0/1127817710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcedb40 0x55896fcec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.918+0000 7f3000468740 1 -- 172.21.15.6:0/1127817710 >> 172.21.15.6:0/1127817710 conn(0x55896fcb5310 msgr2=0x55896fce8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:41.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.919+0000 7f3000468740 1 -- 172.21.15.6:0/1127817710 shutdown_connections 2024-09-12T03:27:41.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.919+0000 7f3000468740 1 -- 172.21.15.6:0/1127817710 wait complete. 2024-09-12T03:27:41.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.920+0000 7f3000468740 1 -- start start 2024-09-12T03:27:41.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.920+0000 7f3000468740 1 Processor -- start 2024-09-12T03:27:41.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.921+0000 7f3000468740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fcec5d0 0x55896fcf95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:41.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.921+0000 7f3000468740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcedb40 0x55896fcf9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:41.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.921+0000 7f3000468740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fd08380 0x55896fcf48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:41.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.921+0000 7f3000468740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55896fcef6e0 con 0x55896fcec5d0 2024-09-12T03:27:41.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.921+0000 7f3000468740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55896fc0d060 con 0x55896fd08380 2024-09-12T03:27:41.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.921+0000 7f3000468740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55896fcef860 con 0x55896fcedb40 2024-09-12T03:27:41.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.921+0000 7f3000468740 10 client.0 _ll_register_callbacks cb 0x55896f9608c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:41.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.921+0000 7f2ff7fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fd08380 0x55896fcf48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:41.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.921+0000 7f2ff7fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fd08380 0x55896fcf48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:49296/0 (socket says 172.21.15.6:49296) 2024-09-12T03:27:41.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.921+0000 7f2ff7fff640 1 -- 172.21.15.6:0/3357247174 learned_addr learned my addr 172.21.15.6:0/3357247174 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:41.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.922+0000 7f2ff6ffd640 1 --2- 172.21.15.6:0/3357247174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcedb40 0x55896fcf9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:41.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.923+0000 7f2ff77fe640 1 --2- 172.21.15.6:0/3357247174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fcec5d0 0x55896fcf95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:41.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.923+0000 7f2ff7fff640 1 -- 172.21.15.6:0/3357247174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcedb40 msgr2=0x55896fcf9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:41.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.923+0000 7f2ff7fff640 1 --2- 172.21.15.6:0/3357247174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcedb40 0x55896fcf9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.923+0000 7f2ff7fff640 1 -- 172.21.15.6:0/3357247174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fcec5d0 msgr2=0x55896fcf95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:41.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.923+0000 7f2ff7fff640 1 --2- 172.21.15.6:0/3357247174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fcec5d0 0x55896fcf95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.923+0000 7f2ff7fff640 1 -- 172.21.15.6:0/3357247174 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55896f9c33c0 con 0x55896fd08380 2024-09-12T03:27:41.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.924+0000 7f2ff7fff640 1 --2- 172.21.15.6:0/3357247174 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fd08380 0x55896fcf48d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2ff00042c0 tx=0x7f2ff002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=824d83fcd4f4adb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:41.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.924+0000 7f2fd6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:27:41.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.925+0000 7f2fd6ffd640 1 -- 172.21.15.6:0/3357247174 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff004d070 con 0x55896fd08380 2024-09-12T03:27:41.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.925+0000 7f3000468740 1 -- 172.21.15.6:0/3357247174 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fe80a6160 con 0x55896fd08380 2024-09-12T03:27:41.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.925+0000 7f2fd6ffd640 1 -- 172.21.15.6:0/3357247174 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f2ff0030b40 con 0x55896fd08380 2024-09-12T03:27:41.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.925+0000 7f2fd6ffd640 1 -- 172.21.15.6:0/3357247174 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2ff0030d40 con 0x55896fd08380 2024-09-12T03:27:41.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.925+0000 7f3000468740 10 client.25646 fetch_fsmap learned FSMap version 27 2024-09-12T03:27:41.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.925+0000 7f3000468740 1 -- 172.21.15.6:0/3357247174 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55896fd031f0 con 0x55896fd08380 2024-09-12T03:27:41.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.929+0000 7f2fd6ffd640 1 -- 172.21.15.6:0/3357247174 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ff0036070 con 0x55896fd08380 2024-09-12T03:27:41.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.929+0000 7f2fd6ffd640 1 -- 172.21.15.6:0/3357247174 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2ff0042040 con 0x55896fd08380 2024-09-12T03:27:41.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.929+0000 7f3000468740 10 client.25646 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:27:41.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.929+0000 7f3000468740 10 client.25646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:27:41.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.929+0000 7f3000468740 20 client.25646 populate_metadata read hostname 'smithi006' 2024-09-12T03:27:41.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.929+0000 7f3000468740 1 --2- 172.21.15.6:0/3357247174 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55896fd04600 0x55896fd06a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:41.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.929+0000 7f3000468740 4 client.25646 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:41.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.929+0000 7f3000468740 1 -- 172.21.15.6:0/3357247174 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2fe80a6160 con 0x55896fd04600 2024-09-12T03:27:41.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.931+0000 7f2ff77fe640 1 --2- 172.21.15.6:0/3357247174 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55896fd04600 0x55896fd06a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:41.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.933+0000 7f2ff77fe640 1 --2- 172.21.15.6:0/3357247174 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55896fd04600 0x55896fd06a80 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8470dfc18b2305d4 server_cookie=eab398b63f13097d in_seq=0 out_seq=0 2024-09-12T03:27:41.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.933+0000 7f2fd6ffd640 10 client.25646 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:41.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.934+0000 7f2fd6ffd640 1 -- 172.21.15.6:0/3357247174 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f2fe80a6160 con 0x55896fd04600 2024-09-12T03:27:41.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.934+0000 7f2fd6ffd640 10 client.25646 handle_command_reply: tid=0 2024-09-12T03:27:41.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.934+0000 7f3000468740 1 client.25646 shutdown 2024-09-12T03:27:41.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.935+0000 7f3000468740 1 -- 172.21.15.6:0/3357247174 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fd08380 msgr2=0x55896fcf48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:41.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.935+0000 7f3000468740 1 --2- 172.21.15.6:0/3357247174 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fd08380 0x55896fcf48d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2ff00042c0 tx=0x7f2ff002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.935+0000 7f3000468740 1 -- 172.21.15.6:0/3357247174 shutdown_connections 2024-09-12T03:27:41.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.935+0000 7f3000468740 1 --2- 172.21.15.6:0/3357247174 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55896fd04600 0x55896fd06a80 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.935+0000 7f3000468740 1 --2- 172.21.15.6:0/3357247174 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fd08380 0x55896fcf48d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.935+0000 7f2fd6ffd640 0 client.25646 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:41.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.935+0000 7f3000468740 1 --2- 172.21.15.6:0/3357247174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcedb40 0x55896fcf9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.935+0000 7f3000468740 1 --2- 172.21.15.6:0/3357247174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fcec5d0 0x55896fcf95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.935+0000 7f3000468740 1 -- 172.21.15.6:0/3357247174 >> 172.21.15.6:0/3357247174 conn(0x55896fcb5310 msgr2=0x55896fcee990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:41.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.940+0000 7f3000468740 1 -- 172.21.15.6:0/3357247174 shutdown_connections 2024-09-12T03:27:41.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.941+0000 7f3000468740 1 -- 172.21.15.6:0/3357247174 wait complete. 2024-09-12T03:27:41.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.941+0000 7f3000468740 20 client.25646 trim_cache size 0 max 16384 2024-09-12T03:27:41.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.971+0000 7f3000468740 1 Processor -- start 2024-09-12T03:27:41.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.980+0000 7f3000468740 1 -- start start 2024-09-12T03:27:41.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.981+0000 7f3000468740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcec5d0 0x55896fcf48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:41.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.981+0000 7f3000468740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fcedb40 0x55896fd31cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:41.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.981+0000 7f3000468740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fd04600 0x55896fd32230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:41.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.981+0000 7f3000468740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe4010c80 con 0x55896fcedb40 2024-09-12T03:27:41.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.981+0000 7f3000468740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe400e550 con 0x55896fd04600 2024-09-12T03:27:41.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.981+0000 7f3000468740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe400c3a0 con 0x55896fcec5d0 2024-09-12T03:27:41.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.981+0000 7f2ff6ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fcedb40 0x55896fd31cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:41.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.981+0000 7f2ff6ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fcedb40 0x55896fd31cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39772/0 (socket says 172.21.15.6:39772) 2024-09-12T03:27:41.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.981+0000 7f2ff6ffd640 1 -- 172.21.15.6:0/91443210 learned_addr learned my addr 172.21.15.6:0/91443210 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:41.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.982+0000 7f2ff6ffd640 1 -- 172.21.15.6:0/91443210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcec5d0 msgr2=0x55896fcf48a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:27:41.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.982+0000 7f2ff6ffd640 1 --2- 172.21.15.6:0/91443210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcec5d0 0x55896fcf48a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.982+0000 7f2ff6ffd640 1 -- 172.21.15.6:0/91443210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fd04600 msgr2=0x55896fd32230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:27:41.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.982+0000 7f2ff6ffd640 1 --2- 172.21.15.6:0/91443210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fd04600 0x55896fd32230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.982+0000 7f2ff6ffd640 1 -- 172.21.15.6:0/91443210 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55896fcf9e60 con 0x55896fcedb40 2024-09-12T03:27:41.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.982+0000 7f2ff6ffd640 1 --2- 172.21.15.6:0/91443210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fcedb40 0x55896fd31cf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8007590 tx=0x7f2fe8099790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43c32038701994a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:41.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.983+0000 7f2fdd7fa640 1 -- 172.21.15.6:0/91443210 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe801d040 con 0x55896fcedb40 2024-09-12T03:27:41.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.983+0000 7f2fdd7fa640 1 -- 172.21.15.6:0/91443210 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fe8099b60 con 0x55896fcedb40 2024-09-12T03:27:41.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.983+0000 7f2fdd7fa640 1 -- 172.21.15.6:0/91443210 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe80b6040 con 0x55896fcedb40 2024-09-12T03:27:41.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.984+0000 7f3000468740 1 -- 172.21.15.6:0/91443210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fcedb40 msgr2=0x55896fd31cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:41.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.984+0000 7f3000468740 1 --2- 172.21.15.6:0/91443210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fcedb40 0x55896fd31cf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8007590 tx=0x7f2fe8099790 comp rx=0 tx=0).stop 2024-09-12T03:27:41.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.988+0000 7f3000468740 1 -- 172.21.15.6:0/91443210 shutdown_connections 2024-09-12T03:27:41.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.988+0000 7f3000468740 1 --2- 172.21.15.6:0/91443210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fd04600 0x55896fd32230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.988+0000 7f3000468740 1 --2- 172.21.15.6:0/91443210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fcedb40 0x55896fd31cf0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.988+0000 7f3000468740 1 --2- 172.21.15.6:0/91443210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcec5d0 0x55896fcf48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.988+0000 7f3000468740 1 -- 172.21.15.6:0/91443210 >> 172.21.15.6:0/91443210 conn(0x55896fcb5310 msgr2=0x55896fcf4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:41.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.990+0000 7f3000468740 1 -- 172.21.15.6:0/91443210 shutdown_connections 2024-09-12T03:27:41.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.990+0000 7f3000468740 1 -- 172.21.15.6:0/91443210 wait complete. 2024-09-12T03:27:41.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.991+0000 7f3000468740 1 -- start start 2024-09-12T03:27:41.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.991+0000 7f3000468740 1 Processor -- start 2024-09-12T03:27:41.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.991+0000 7f3000468740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcec5d0 0x55896fcf23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:41.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.991+0000 7f3000468740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fcedb40 0x55896fcfc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:41.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.992+0000 7f3000468740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fd04600 0x55896fcfc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:41.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.992+0000 7f3000468740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe400ca10 con 0x55896fd04600 2024-09-12T03:27:41.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.992+0000 7f3000468740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe400de80 con 0x55896fcedb40 2024-09-12T03:27:41.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.992+0000 7f3000468740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe400d0e0 con 0x55896fcec5d0 2024-09-12T03:27:41.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.992+0000 7f2ff6ffd640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fcedb40 0x55896fcfc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:41.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.992+0000 7f2ff6ffd640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fcedb40 0x55896fcfc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:49310/0 (socket says 172.21.15.6:49310) 2024-09-12T03:27:41.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.992+0000 7f2ff6ffd640 1 -- 172.21.15.6:0/2859866036 learned_addr learned my addr 172.21.15.6:0/2859866036 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:41.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.992+0000 7f3000468740 10 client.0 _ll_register_callbacks cb 0x7f2fec005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:41.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.993+0000 7f2ff7fff640 1 --2- 172.21.15.6:0/2859866036 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fd04600 0x55896fcfc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:41.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.993+0000 7f2ff77fe640 1 --2- 172.21.15.6:0/2859866036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcec5d0 0x55896fcf23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:41.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.993+0000 7f2ff6ffd640 1 -- 172.21.15.6:0/2859866036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcec5d0 msgr2=0x55896fcf23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:41.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.993+0000 7f2ff6ffd640 1 --2- 172.21.15.6:0/2859866036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcec5d0 0x55896fcf23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.993+0000 7f2ff6ffd640 1 -- 172.21.15.6:0/2859866036 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fd04600 msgr2=0x55896fcfc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:41.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.993+0000 7f2ff6ffd640 1 --2- 172.21.15.6:0/2859866036 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fd04600 0x55896fcfc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.993+0000 7f2ff6ffd640 1 -- 172.21.15.6:0/2859866036 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55896fcfcf50 con 0x55896fcedb40 2024-09-12T03:27:41.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.994+0000 7f2ff77fe640 1 --2- 172.21.15.6:0/2859866036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcec5d0 0x55896fcf23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:41.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.995+0000 7f2ff6ffd640 1 --2- 172.21.15.6:0/2859866036 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fcedb40 0x55896fcfc210 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80a5c30 tx=0x7f2fe809cb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6cd00cafead8880 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:41.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.995+0000 7f2fd6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:27:41.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.995+0000 7f2fd6ffd640 1 -- 172.21.15.6:0/2859866036 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe80ba070 con 0x55896fcedb40 2024-09-12T03:27:41.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.995+0000 7f2fd6ffd640 1 -- 172.21.15.6:0/2859866036 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f2fe809c3b0 con 0x55896fcedb40 2024-09-12T03:27:41.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.996+0000 7f3000468740 1 -- 172.21.15.6:0/2859866036 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fe8099b60 con 0x55896fcedb40 2024-09-12T03:27:41.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.998+0000 7f2fd6ffd640 1 -- 172.21.15.6:0/2859866036 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2fe801d070 con 0x55896fcedb40 2024-09-12T03:27:41.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.998+0000 7f3000468740 10 client.25649 fetch_fsmap learned FSMap version 27 2024-09-12T03:27:41.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.998+0000 7f3000468740 1 -- 172.21.15.6:0/2859866036 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55896fd93c40 con 0x55896fcedb40 2024-09-12T03:27:41.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:40.999+0000 7f2fd6ffd640 1 -- 172.21.15.6:0/2859866036 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fe80a6160 con 0x55896fcedb40 2024-09-12T03:27:41.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.000+0000 7f2fd6ffd640 1 -- 172.21.15.6:0/2859866036 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2fe80bd340 con 0x55896fcedb40 2024-09-12T03:27:41.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.000+0000 7f3000468740 10 client.25649 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:27:41.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.000+0000 7f3000468740 10 client.25649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:27:41.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.000+0000 7f3000468740 20 client.25649 populate_metadata read hostname 'smithi006' 2024-09-12T03:27:41.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.000+0000 7f3000468740 1 --2- 172.21.15.6:0/2859866036 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55896fd08380 0x55896fd95c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:41.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.000+0000 7f3000468740 4 client.25649 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:27:41.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.000+0000 7f3000468740 1 -- 172.21.15.6:0/2859866036 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2fe8099b60 con 0x55896fd08380 2024-09-12T03:27:41.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.002+0000 7f2ff77fe640 1 --2- 172.21.15.6:0/2859866036 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55896fd08380 0x55896fd95c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:41.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.012+0000 7f2fd6ffd640 10 client.25649 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:41.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.012+0000 7f2ff77fe640 1 --2- 172.21.15.6:0/2859866036 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55896fd08380 0x55896fd95c50 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3ebbe02e463dfbc server_cookie=cc3a6d2a6ec3afc3 in_seq=0 out_seq=0 2024-09-12T03:27:41.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.019+0000 7f2fd6ffd640 1 -- 172.21.15.6:0/2859866036 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2fe8099b60 con 0x55896fd08380 2024-09-12T03:27:41.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.019+0000 7f2fd6ffd640 10 client.25649 handle_command_reply: tid=0 2024-09-12T03:27:41.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.019+0000 7f3000468740 1 client.25649 shutdown 2024-09-12T03:27:41.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.019+0000 7f3000468740 1 -- 172.21.15.6:0/2859866036 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fcedb40 msgr2=0x55896fcfc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:41.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.019+0000 7f3000468740 1 --2- 172.21.15.6:0/2859866036 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fcedb40 0x55896fcfc210 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80a5c30 tx=0x7f2fe809cb70 comp rx=0 tx=0).stop 2024-09-12T03:27:41.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.020+0000 7f3000468740 1 -- 172.21.15.6:0/2859866036 shutdown_connections 2024-09-12T03:27:41.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.020+0000 7f3000468740 1 --2- 172.21.15.6:0/2859866036 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55896fd08380 0x55896fd95c50 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.020+0000 7f3000468740 1 --2- 172.21.15.6:0/2859866036 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55896fd04600 0x55896fcfc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.020+0000 7f2fd6ffd640 0 client.25649 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:41.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.020+0000 7f3000468740 1 --2- 172.21.15.6:0/2859866036 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55896fcedb40 0x55896fcfc210 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.020+0000 7f3000468740 1 --2- 172.21.15.6:0/2859866036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55896fcec5d0 0x55896fcf23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.020+0000 7f3000468740 1 -- 172.21.15.6:0/2859866036 >> 172.21.15.6:0/2859866036 conn(0x55896fcb5310 msgr2=0x55896fce7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:41.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.020+0000 7f3000468740 1 -- 172.21.15.6:0/2859866036 shutdown_connections 2024-09-12T03:27:41.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.020+0000 7f3000468740 1 -- 172.21.15.6:0/2859866036 wait complete. 2024-09-12T03:27:41.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.021+0000 7f3000468740 20 client.25649 trim_cache size 0 max 16384 2024-09-12T03:27:41.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.022+0000 7f2ffe08f640 1 -- 172.21.15.6:0/498930058 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2fe00a2da0 msgr2=0x7f2fe00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.022+0000 7f2ffe08f640 1 --2- 172.21.15.6:0/498930058 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2fe00a2da0 0x7f2fe00a5260 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0005e30 tx=0x7f2ff00095c0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.022+0000 7f2ffe08f640 1 -- 172.21.15.6:0/498930058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ff8106a90 msgr2=0x7f2ff8110180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.022+0000 7f2ffe08f640 1 --2- 172.21.15.6:0/498930058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ff8106a90 0x7f2ff8110180 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800c970 tx=0x7f2fe800ce30 comp rx=0 tx=0).stop 2024-09-12T03:27:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.022+0000 7f2ffe08f640 1 -- 172.21.15.6:0/498930058 shutdown_connections 2024-09-12T03:27:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.022+0000 7f2ffe08f640 1 --2- 172.21.15.6:0/498930058 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2fe00a2da0 0x7f2fe00a5260 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.022+0000 7f2ffe08f640 1 --2- 172.21.15.6:0/498930058 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ff81106c0 0x7f2ff819c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.022+0000 7f2ffe08f640 1 --2- 172.21.15.6:0/498930058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ff8106a90 0x7f2ff8110180 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.022+0000 7f2ffe08f640 1 --2- 172.21.15.6:0/498930058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ff8100a90 0x7f2ff810fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.022+0000 7f2ffe08f640 1 -- 172.21.15.6:0/498930058 >> 172.21.15.6:0/498930058 conn(0x7f2ff80fc7d0 msgr2=0x7f2ff810f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.022+0000 7f2ffe08f640 1 -- 172.21.15.6:0/498930058 shutdown_connections 2024-09-12T03:27:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:41.022+0000 7f2ffe08f640 1 -- 172.21.15.6:0/498930058 wait complete. 2024-09-12T03:27:42.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:42 smithi118 ceph-mon[35048]: pgmap v533: 97 pgs: 97 active+clean; 13 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 510 KiB/s rd, 176 MiB/s wr, 17.80k op/s 2024-09-12T03:27:42.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:42 smithi006 ceph-mon[30991]: pgmap v533: 97 pgs: 97 active+clean; 13 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 510 KiB/s rd, 176 MiB/s wr, 17.80k op/s 2024-09-12T03:27:42.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:42 smithi161 ceph-mon[35485]: pgmap v533: 97 pgs: 97 active+clean; 13 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 510 KiB/s rd, 176 MiB/s wr, 17.80k op/s 2024-09-12T03:27:43.261 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:27:43.262 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:27:43.805 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:44.509 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:44 smithi006 ceph-mon[30991]: pgmap v534: 97 pgs: 97 active+clean; 13 GiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 431 KiB/s rd, 163 MiB/s wr, 16.39k op/s 2024-09-12T03:27:44.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.642+0000 7f156e7a2640 1 -- 172.21.15.6:0/2957411049 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1568101ea0 msgr2=0x7f156810d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:44.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.642+0000 7f156e7a2640 1 --2- 172.21.15.6:0/2957411049 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1568101ea0 0x7f156810d9b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1560009be0 tx=0x7f156002a6c0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.642+0000 7f156e7a2640 1 -- 172.21.15.6:0/2957411049 shutdown_connections 2024-09-12T03:27:44.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.642+0000 7f156e7a2640 1 --2- 172.21.15.6:0/2957411049 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1568101ea0 0x7f156810d9b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.642+0000 7f156e7a2640 1 --2- 172.21.15.6:0/2957411049 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1568101500 0x7f1568101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.642+0000 7f156e7a2640 1 --2- 172.21.15.6:0/2957411049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1568106be0 0x7f1568106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.642+0000 7f156e7a2640 1 -- 172.21.15.6:0/2957411049 >> 172.21.15.6:0/2957411049 conn(0x7f15680fba90 msgr2=0x7f15680fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:44.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.642+0000 7f156e7a2640 1 -- 172.21.15.6:0/2957411049 shutdown_connections 2024-09-12T03:27:44.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.642+0000 7f156e7a2640 1 -- 172.21.15.6:0/2957411049 wait complete. 2024-09-12T03:27:44.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.643+0000 7f156e7a2640 1 Processor -- start 2024-09-12T03:27:44.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.643+0000 7f156e7a2640 1 -- start start 2024-09-12T03:27:44.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.643+0000 7f156e7a2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1568101500 0x7f1568198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.643+0000 7f156e7a2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1568101ea0 0x7f1568198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.643+0000 7f156e7a2640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1568106be0 0x7f156819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.644+0000 7f156e7a2640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1568110f00 con 0x7f1568101ea0 2024-09-12T03:27:44.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.644+0000 7f156e7a2640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f1568110d80 con 0x7f1568106be0 2024-09-12T03:27:44.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.644+0000 7f1567fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1568101500 0x7f1568198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:44.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.644+0000 7f1567fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1568101500 0x7f1568198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:42998/0 (socket says 172.21.15.6:42998) 2024-09-12T03:27:44.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.644+0000 7f1567fff640 1 -- 172.21.15.6:0/448214992 learned_addr learned my addr 172.21.15.6:0/448214992 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:44.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.644+0000 7f156e7a2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1568111080 con 0x7f1568101500 2024-09-12T03:27:44.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.644+0000 7f15677fe640 1 --2- 172.21.15.6:0/448214992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1568101ea0 0x7f1568198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:44.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.644+0000 7f156cd3f640 1 --2- 172.21.15.6:0/448214992 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1568106be0 0x7f156819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:44.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.644+0000 7f1567fff640 1 -- 172.21.15.6:0/448214992 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1568106be0 msgr2=0x7f156819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:44.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.644+0000 7f1567fff640 1 --2- 172.21.15.6:0/448214992 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1568106be0 0x7f156819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.644+0000 7f1567fff640 1 -- 172.21.15.6:0/448214992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1568101ea0 msgr2=0x7f1568198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:44.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.644+0000 7f1567fff640 1 --2- 172.21.15.6:0/448214992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1568101ea0 0x7f1568198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.644+0000 7f1567fff640 1 -- 172.21.15.6:0/448214992 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156819d4f0 con 0x7f1568101500 2024-09-12T03:27:44.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.644+0000 7f1567fff640 1 --2- 172.21.15.6:0/448214992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1568101500 0x7f1568198540 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f155800e9c0 tx=0x7f155800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:44.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.645+0000 7f15657fa640 1 -- 172.21.15.6:0/448214992 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155800ccc0 con 0x7f1568101500 2024-09-12T03:27:44.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.645+0000 7f15657fa640 1 -- 172.21.15.6:0/448214992 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1558005500 con 0x7f1568101500 2024-09-12T03:27:44.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.645+0000 7f156e7a2640 1 -- 172.21.15.6:0/448214992 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f156819d7e0 con 0x7f1568101500 2024-09-12T03:27:44.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.645+0000 7f15657fa640 1 -- 172.21.15.6:0/448214992 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558010430 con 0x7f1568101500 2024-09-12T03:27:44.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.645+0000 7f156e7a2640 1 -- 172.21.15.6:0/448214992 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15681a50c0 con 0x7f1568101500 2024-09-12T03:27:44.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.648+0000 7f1570b7b740 1 Processor -- start 2024-09-12T03:27:44.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.648+0000 7f1570b7b740 1 -- start start 2024-09-12T03:27:44.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.648+0000 7f1570b7b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1ec350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.648+0000 7f1570b7b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1ec890 0x55eb2e1f6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.648+0000 7f1570b7b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb2e1f7150 0x55eb2e1f9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.648+0000 7f1570b7b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55eb2e14d280 con 0x55eb2e1ebf70 2024-09-12T03:27:44.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.648+0000 7f1570b7b740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55eb2e0f4b10 con 0x55eb2e1f7150 2024-09-12T03:27:44.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.648+0000 7f1570b7b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eb2e149240 con 0x55eb2e1ec890 2024-09-12T03:27:44.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.648+0000 7f156cd3f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1ec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:44.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.648+0000 7f156cd3f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1ec350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39802/0 (socket says 172.21.15.6:39802) 2024-09-12T03:27:44.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.648+0000 7f156cd3f640 1 -- 172.21.15.6:0/3460697934 learned_addr learned my addr 172.21.15.6:0/3460697934 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:44.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.648+0000 7f15677fe640 1 --2- 172.21.15.6:0/3460697934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb2e1f7150 0x55eb2e1f9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:44.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.649+0000 7f156cd3f640 1 -- 172.21.15.6:0/3460697934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1ec890 msgr2=0x55eb2e1f6c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:27:44.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.649+0000 7f156cd3f640 1 --2- 172.21.15.6:0/3460697934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1ec890 0x55eb2e1f6c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.649+0000 7f156cd3f640 1 -- 172.21.15.6:0/3460697934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb2e1f7150 msgr2=0x55eb2e1f9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:44.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.649+0000 7f156cd3f640 1 --2- 172.21.15.6:0/3460697934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb2e1f7150 0x55eb2e1f9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.649+0000 7f156cd3f640 1 -- 172.21.15.6:0/3460697934 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb2e1011f0 con 0x55eb2e1ebf70 2024-09-12T03:27:44.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.649+0000 7f15657fa640 1 -- 172.21.15.6:0/448214992 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f15580046c0 con 0x7f1568101500 2024-09-12T03:27:44.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.649+0000 7f156cd3f640 1 --2- 172.21.15.6:0/3460697934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1ec350 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1560033000 tx=0x7f1560009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a658be6d7f1bcbf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:44.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.649+0000 7f154e7fc640 1 -- 172.21.15.6:0/3460697934 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560036040 con 0x55eb2e1ebf70 2024-09-12T03:27:44.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.650+0000 7f154e7fc640 1 -- 172.21.15.6:0/3460697934 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1560037040 con 0x55eb2e1ebf70 2024-09-12T03:27:44.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.650+0000 7f15657fa640 1 --2- 172.21.15.6:0/448214992 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f15500a2c00 0x7f15500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.650+0000 7f1567fff640 1 --2- 172.21.15.6:0/448214992 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f15500a2c00 0x7f15500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:44.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.650+0000 7f15657fa640 1 -- 172.21.15.6:0/448214992 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f15580cf280 con 0x7f1568101500 2024-09-12T03:27:44.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.651+0000 7f1570b7b740 1 -- 172.21.15.6:0/3460697934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1ebf70 msgr2=0x55eb2e1ec350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:44.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.651+0000 7f1570b7b740 1 --2- 172.21.15.6:0/3460697934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1ec350 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1560033000 tx=0x7f1560009300 comp rx=0 tx=0).stop 2024-09-12T03:27:44.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.651+0000 7f1570b7b740 1 -- 172.21.15.6:0/3460697934 shutdown_connections 2024-09-12T03:27:44.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.651+0000 7f1570b7b740 1 --2- 172.21.15.6:0/3460697934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb2e1f7150 0x55eb2e1f9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.651+0000 7f1570b7b740 1 --2- 172.21.15.6:0/3460697934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1ec890 0x55eb2e1f6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.651+0000 7f1570b7b740 1 --2- 172.21.15.6:0/3460697934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1ec350 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.651+0000 7f1570b7b740 1 -- 172.21.15.6:0/3460697934 >> 172.21.15.6:0/3460697934 conn(0x55eb2e1e8500 msgr2=0x55eb2e1e88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:44.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.651+0000 7f1570b7b740 1 -- 172.21.15.6:0/3460697934 shutdown_connections 2024-09-12T03:27:44.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.651+0000 7f1570b7b740 1 -- 172.21.15.6:0/3460697934 wait complete. 2024-09-12T03:27:44.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.651+0000 7f1570b7b740 1 -- start start 2024-09-12T03:27:44.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.651+0000 7f1567fff640 1 --2- 172.21.15.6:0/448214992 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f15500a2c00 0x7f15500a50c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f15580ccdb0 tx=0x7f1558090ff0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:44.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.651+0000 7f1570b7b740 1 Processor -- start 2024-09-12T03:27:44.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.652+0000 7f1570b7b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.652+0000 7f1570b7b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1ec890 0x55eb2e1f0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.652+0000 7f1570b7b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb2e1f7150 0x55eb2e1f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.652+0000 7f1570b7b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55eb2e1e3d70 con 0x55eb2e1ec890 2024-09-12T03:27:44.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.652+0000 7f1570b7b740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55eb2e107f10 con 0x55eb2e1f7150 2024-09-12T03:27:44.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.652+0000 7f1570b7b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eb2e1e3ef0 con 0x55eb2e1ebf70 2024-09-12T03:27:44.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.652+0000 7f1570b7b740 10 client.0 _ll_register_callbacks cb 0x55eb2e0d0a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:44.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.652+0000 7f15677fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1ec890 0x55eb2e1f0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:44.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.652+0000 7f156cd3f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1f03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:44.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.652+0000 7f156cd3f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1f03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:43000/0 (socket says 172.21.15.6:43000) 2024-09-12T03:27:44.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.652+0000 7f156cd3f640 1 -- 172.21.15.6:0/2422570486 learned_addr learned my addr 172.21.15.6:0/2422570486 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:44.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.652+0000 7f156cd3f640 1 --2- 172.21.15.6:0/2422570486 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb2e1f7150 0x55eb2e1f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:44.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.653+0000 7f156cd3f640 1 -- 172.21.15.6:0/2422570486 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb2e1f7150 msgr2=0x55eb2e1f4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:44.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.653+0000 7f156cd3f640 1 --2- 172.21.15.6:0/2422570486 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb2e1f7150 0x55eb2e1f4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.653+0000 7f156cd3f640 1 -- 172.21.15.6:0/2422570486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1ec890 msgr2=0x55eb2e1f0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:44.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.653+0000 7f156cd3f640 1 --2- 172.21.15.6:0/2422570486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1ec890 0x55eb2e1f0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.653+0000 7f156cd3f640 1 -- 172.21.15.6:0/2422570486 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb2debe6a0 con 0x55eb2e1ebf70 2024-09-12T03:27:44.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.653+0000 7f15677fe640 1 --2- 172.21.15.6:0/2422570486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1ec890 0x55eb2e1f0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.653+0000 7f156cd3f640 1 --2- 172.21.15.6:0/2422570486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1f03f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1560030eb0 tx=0x7f1560045040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6becd80b65060cc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.653+0000 7f15477fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:27:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.653+0000 7f15477fe640 1 -- 172.21.15.6:0/2422570486 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560036030 con 0x55eb2e1ebf70 2024-09-12T03:27:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.653+0000 7f15477fe640 1 -- 172.21.15.6:0/2422570486 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f1560051070 con 0x55eb2e1ebf70 2024-09-12T03:27:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.653+0000 7f1570b7b740 1 -- 172.21.15.6:0/2422570486 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eb2e15a7f0 con 0x55eb2e1ebf70 2024-09-12T03:27:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.654+0000 7f15477fe640 1 -- 172.21.15.6:0/2422570486 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1560043b00 con 0x55eb2e1ebf70 2024-09-12T03:27:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.654+0000 7f1570b7b740 10 client.25750 fetch_fsmap learned FSMap version 27 2024-09-12T03:27:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.654+0000 7f1570b7b740 1 -- 172.21.15.6:0/2422570486 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eb2e1e3d70 con 0x55eb2e1ebf70 2024-09-12T03:27:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.654+0000 7f15477fe640 1 -- 172.21.15.6:0/2422570486 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1560043ca0 con 0x55eb2e1ebf70 2024-09-12T03:27:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.654+0000 7f15477fe640 1 -- 172.21.15.6:0/2422570486 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1560054070 con 0x55eb2e1ebf70 2024-09-12T03:27:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.654+0000 7f1570b7b740 10 client.25750 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:27:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.654+0000 7f1570b7b740 10 client.25750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:27:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.654+0000 7f1570b7b740 20 client.25750 populate_metadata read hostname 'smithi006' 2024-09-12T03:27:44.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.654+0000 7f1570b7b740 1 --2- 172.21.15.6:0/2422570486 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55eb2e202d30 0x55eb2e205150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.654+0000 7f1570b7b740 4 client.25750 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:44.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.654+0000 7f1570b7b740 1 -- 172.21.15.6:0/2422570486 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb2e15a7f0 con 0x55eb2e202d30 2024-09-12T03:27:44.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.654+0000 7f15677fe640 1 --2- 172.21.15.6:0/2422570486 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55eb2e202d30 0x55eb2e205150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:44.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.656+0000 7f15677fe640 1 --2- 172.21.15.6:0/2422570486 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55eb2e202d30 0x55eb2e205150 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20515c6e79d1a63e server_cookie=5627d8764b4a94ba in_seq=0 out_seq=0 2024-09-12T03:27:44.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.656+0000 7f15477fe640 10 client.25750 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:44.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.657+0000 7f15477fe640 1 -- 172.21.15.6:0/2422570486 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb2e15a7f0 con 0x55eb2e202d30 2024-09-12T03:27:44.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.657+0000 7f15477fe640 10 client.25750 handle_command_reply: tid=0 2024-09-12T03:27:44.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.657+0000 7f1570b7b740 1 client.25750 shutdown 2024-09-12T03:27:44.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.657+0000 7f1570b7b740 1 -- 172.21.15.6:0/2422570486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1ebf70 msgr2=0x55eb2e1f03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:44.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.658+0000 7f1570b7b740 1 --2- 172.21.15.6:0/2422570486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1f03f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1560030eb0 tx=0x7f1560045040 comp rx=0 tx=0).stop 2024-09-12T03:27:44.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.658+0000 7f1570b7b740 1 -- 172.21.15.6:0/2422570486 shutdown_connections 2024-09-12T03:27:44.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.658+0000 7f1570b7b740 1 --2- 172.21.15.6:0/2422570486 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55eb2e202d30 0x55eb2e205150 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.658+0000 7f1570b7b740 1 --2- 172.21.15.6:0/2422570486 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb2e1f7150 0x55eb2e1f4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.658+0000 7f1570b7b740 1 --2- 172.21.15.6:0/2422570486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1ec890 0x55eb2e1f0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.658+0000 7f1570b7b740 1 --2- 172.21.15.6:0/2422570486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1f03f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.658+0000 7f15477fe640 0 client.25750 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:44.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.658+0000 7f1570b7b740 1 -- 172.21.15.6:0/2422570486 >> 172.21.15.6:0/2422570486 conn(0x55eb2e1e8500 msgr2=0x55eb2e1ee260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:44.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.658+0000 7f1570b7b740 1 -- 172.21.15.6:0/2422570486 shutdown_connections 2024-09-12T03:27:44.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.658+0000 7f1570b7b740 1 -- 172.21.15.6:0/2422570486 wait complete. 2024-09-12T03:27:44.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.659+0000 7f1570b7b740 20 client.25750 trim_cache size 0 max 16384 2024-09-12T03:27:44.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.680+0000 7f1570b7b740 1 Processor -- start 2024-09-12T03:27:44.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.681+0000 7f1570b7b740 1 -- start start 2024-09-12T03:27:44.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.681+0000 7f1570b7b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1f4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.681+0000 7f1570b7b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb2e1ec890 0x55eb2e1f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.681+0000 7f1570b7b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1f7150 0x55eb2e20aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.681+0000 7f1570b7b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f15681adb70 con 0x55eb2e1f7150 2024-09-12T03:27:44.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.681+0000 7f1570b7b740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f15681ab440 con 0x55eb2e1ec890 2024-09-12T03:27:44.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.681+0000 7f1570b7b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f15681a92b0 con 0x55eb2e1ebf70 2024-09-12T03:27:44.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.681+0000 7f156cd3f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1f7150 0x55eb2e20aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:44.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.681+0000 7f156cd3f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1f4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:44.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.681+0000 7f156cd3f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1f4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:43010/0 (socket says 172.21.15.6:43010) 2024-09-12T03:27:44.745 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:27:44.745 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2024-09-12T03:27:44.745 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "838db3dd-096f-4ad3-9d4d-00a5fbaa8642", 2024-09-12T03:27:44.746 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2024-09-12T03:27:44.746 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:27:44.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.681+0000 7f156cd3f640 1 -- 172.21.15.6:0/746019215 learned_addr learned my addr 172.21.15.6:0/746019215 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:44.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.681+0000 7f15677fe640 1 --2- 172.21.15.6:0/746019215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb2e1ec890 0x55eb2e1f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:44.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.681+0000 7f156cd3f640 1 -- 172.21.15.6:0/746019215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb2e1ec890 msgr2=0x55eb2e1f4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:44.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.681+0000 7f156cd3f640 1 --2- 172.21.15.6:0/746019215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb2e1ec890 0x55eb2e1f4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.681+0000 7f156cd3f640 1 -- 172.21.15.6:0/746019215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1f7150 msgr2=0x55eb2e20aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:44.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.681+0000 7f156cd3f640 1 --2- 172.21.15.6:0/746019215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1f7150 0x55eb2e20aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.681+0000 7f156cd3f640 1 -- 172.21.15.6:0/746019215 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb2e1e7fa0 con 0x55eb2e1ebf70 2024-09-12T03:27:44.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.682+0000 7f156cd3f640 1 --2- 172.21.15.6:0/746019215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1f4aa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f156002abe0 tx=0x7f1560000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd414a493de63721 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:44.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.682+0000 7f154dffb640 1 -- 172.21.15.6:0/746019215 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560036070 con 0x55eb2e1ebf70 2024-09-12T03:27:44.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.682+0000 7f154dffb640 1 -- 172.21.15.6:0/746019215 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f156006b040 con 0x55eb2e1ebf70 2024-09-12T03:27:44.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.682+0000 7f154dffb640 1 -- 172.21.15.6:0/746019215 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156006a070 con 0x55eb2e1ebf70 2024-09-12T03:27:44.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.683+0000 7f1570b7b740 1 -- 172.21.15.6:0/746019215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1ebf70 msgr2=0x55eb2e1f4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:44.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.683+0000 7f1570b7b740 1 --2- 172.21.15.6:0/746019215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1f4aa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f156002abe0 tx=0x7f1560000f40 comp rx=0 tx=0).stop 2024-09-12T03:27:44.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.683+0000 7f1570b7b740 1 -- 172.21.15.6:0/746019215 shutdown_connections 2024-09-12T03:27:44.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.683+0000 7f1570b7b740 1 --2- 172.21.15.6:0/746019215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1f7150 0x55eb2e20aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.683+0000 7f1570b7b740 1 --2- 172.21.15.6:0/746019215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb2e1ec890 0x55eb2e1f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.683+0000 7f1570b7b740 1 --2- 172.21.15.6:0/746019215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1f4aa0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.683+0000 7f1570b7b740 1 -- 172.21.15.6:0/746019215 >> 172.21.15.6:0/746019215 conn(0x55eb2e1e8500 msgr2=0x55eb2e1e3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:44.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.683+0000 7f1570b7b740 1 -- 172.21.15.6:0/746019215 shutdown_connections 2024-09-12T03:27:44.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.683+0000 7f1570b7b740 1 -- 172.21.15.6:0/746019215 wait complete. 2024-09-12T03:27:44.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.683+0000 7f1570b7b740 1 -- start start 2024-09-12T03:27:44.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.683+0000 7f1570b7b740 1 Processor -- start 2024-09-12T03:27:44.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.684+0000 7f1570b7b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1feb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.684+0000 7f1570b7b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1ec890 0x55eb2e1ff060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.684+0000 7f1570b7b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1f7150 0x55eb2e1e5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.684+0000 7f1570b7b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f15681a9900 con 0x55eb2e1ec890 2024-09-12T03:27:44.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.684+0000 7f1570b7b740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f15681aad70 con 0x55eb2e1ebf70 2024-09-12T03:27:44.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.684+0000 7f156cd3f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1feb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:44.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.684+0000 7f156cd3f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1feb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:49374/0 (socket says 172.21.15.6:49374) 2024-09-12T03:27:44.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.684+0000 7f156cd3f640 1 -- 172.21.15.6:0/75895701 learned_addr learned my addr 172.21.15.6:0/75895701 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:44.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.684+0000 7f1570b7b740 1 -- 172.21.15.6:0/75895701 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f15681a9fd0 con 0x55eb2e1f7150 2024-09-12T03:27:44.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.684+0000 7f156cd3f640 1 --2- 172.21.15.6:0/75895701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1f7150 0x55eb2e1e5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:44.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.684+0000 7f1570b7b740 10 client.0 _ll_register_callbacks cb 0x7f155c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:44.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.684+0000 7f15677fe640 1 --2- 172.21.15.6:0/75895701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1ec890 0x55eb2e1ff060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:44.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.684+0000 7f156cd3f640 1 -- 172.21.15.6:0/75895701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb2e1ebf70 msgr2=0x55eb2e1feb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:44.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.685+0000 7f156cd3f640 1 --2- 172.21.15.6:0/75895701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1feb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.685+0000 7f156cd3f640 1 -- 172.21.15.6:0/75895701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1ec890 msgr2=0x55eb2e1ff060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:44.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.685+0000 7f156cd3f640 1 --2- 172.21.15.6:0/75895701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1ec890 0x55eb2e1ff060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.685+0000 7f156cd3f640 1 -- 172.21.15.6:0/75895701 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb2e1ff930 con 0x55eb2e1f7150 2024-09-12T03:27:44.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.685+0000 7f15677fe640 1 --2- 172.21.15.6:0/75895701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1ec890 0x55eb2e1ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:44.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.685+0000 7f156cd3f640 1 --2- 172.21.15.6:0/75895701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1f7150 0x55eb2e1e5e10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1560005f00 tx=0x7f1560009960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc32980c4a722df1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:44.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.685+0000 7f15477fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:27:44.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.685+0000 7f15477fe640 1 -- 172.21.15.6:0/75895701 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560036070 con 0x55eb2e1f7150 2024-09-12T03:27:44.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.685+0000 7f15477fe640 1 -- 172.21.15.6:0/75895701 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f15600305a0 con 0x55eb2e1f7150 2024-09-12T03:27:44.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.685+0000 7f1570b7b740 1 -- 172.21.15.6:0/75895701 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f156006b040 con 0x55eb2e1f7150 2024-09-12T03:27:44.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.686+0000 7f15477fe640 1 -- 172.21.15.6:0/75895701 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1560051040 con 0x55eb2e1f7150 2024-09-12T03:27:44.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.686+0000 7f1570b7b740 10 client.25756 fetch_fsmap learned FSMap version 27 2024-09-12T03:27:44.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.686+0000 7f1570b7b740 1 -- 172.21.15.6:0/75895701 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eb2e26d0b0 con 0x55eb2e1f7150 2024-09-12T03:27:44.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.686+0000 7f15477fe640 1 -- 172.21.15.6:0/75895701 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1560037040 con 0x55eb2e1f7150 2024-09-12T03:27:44.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.686+0000 7f15477fe640 1 -- 172.21.15.6:0/75895701 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f156006f040 con 0x55eb2e1f7150 2024-09-12T03:27:44.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.686+0000 7f1570b7b740 10 client.25756 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:27:44.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.686+0000 7f1570b7b740 10 client.25756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:27:44.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.686+0000 7f1570b7b740 20 client.25756 populate_metadata read hostname 'smithi006' 2024-09-12T03:27:44.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.686+0000 7f1570b7b740 1 --2- 172.21.15.6:0/75895701 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55eb2e202d30 0x55eb2e26f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.686+0000 7f1570b7b740 4 client.25756 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:27:44.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.686+0000 7f1570b7b740 1 -- 172.21.15.6:0/75895701 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f156006b040 con 0x55eb2e202d30 2024-09-12T03:27:44.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.687+0000 7f15677fe640 1 --2- 172.21.15.6:0/75895701 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55eb2e202d30 0x55eb2e26f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:44.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.687+0000 7f15677fe640 1 --2- 172.21.15.6:0/75895701 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55eb2e202d30 0x55eb2e26f6f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5df443faa7667e87 server_cookie=fc7477c692f9499e in_seq=0 out_seq=0 2024-09-12T03:27:44.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.687+0000 7f15477fe640 10 client.25756 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:44.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.688+0000 7f15477fe640 1 -- 172.21.15.6:0/75895701 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f156006b040 con 0x55eb2e202d30 2024-09-12T03:27:44.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.688+0000 7f15477fe640 10 client.25756 handle_command_reply: tid=0 2024-09-12T03:27:44.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.688+0000 7f1570b7b740 1 client.25756 shutdown 2024-09-12T03:27:44.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.689+0000 7f1570b7b740 1 -- 172.21.15.6:0/75895701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1f7150 msgr2=0x55eb2e1e5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:44.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.689+0000 7f1570b7b740 1 --2- 172.21.15.6:0/75895701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1f7150 0x55eb2e1e5e10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1560005f00 tx=0x7f1560009960 comp rx=0 tx=0).stop 2024-09-12T03:27:44.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.689+0000 7f1570b7b740 1 -- 172.21.15.6:0/75895701 shutdown_connections 2024-09-12T03:27:44.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.689+0000 7f1570b7b740 1 --2- 172.21.15.6:0/75895701 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x55eb2e202d30 0x55eb2e26f6f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.689+0000 7f1570b7b740 1 --2- 172.21.15.6:0/75895701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2e1f7150 0x55eb2e1e5e10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.689+0000 7f1570b7b740 1 --2- 172.21.15.6:0/75895701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eb2e1ec890 0x55eb2e1ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.689+0000 7f1570b7b740 1 --2- 172.21.15.6:0/75895701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb2e1ebf70 0x55eb2e1feb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.689+0000 7f1570b7b740 1 -- 172.21.15.6:0/75895701 >> 172.21.15.6:0/75895701 conn(0x55eb2e1e8500 msgr2=0x55eb2e1f5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.689+0000 7f15477fe640 0 client.25756 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.689+0000 7f1570b7b740 1 -- 172.21.15.6:0/75895701 shutdown_connections 2024-09-12T03:27:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.689+0000 7f1570b7b740 1 -- 172.21.15.6:0/75895701 wait complete. 2024-09-12T03:27:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.689+0000 7f1570b7b740 20 client.25756 trim_cache size 0 max 16384 2024-09-12T03:27:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.690+0000 7f156e7a2640 1 -- 172.21.15.6:0/448214992 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f15500a2c00 msgr2=0x7f15500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.690+0000 7f156e7a2640 1 --2- 172.21.15.6:0/448214992 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f15500a2c00 0x7f15500a50c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f15580ccdb0 tx=0x7f1558090ff0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.690+0000 7f156e7a2640 1 -- 172.21.15.6:0/448214992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1568101500 msgr2=0x7f1568198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.690+0000 7f156e7a2640 1 --2- 172.21.15.6:0/448214992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1568101500 0x7f1568198540 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f155800e9c0 tx=0x7f155800ee80 comp rx=0 tx=0).stop 2024-09-12T03:27:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.690+0000 7f156e7a2640 1 -- 172.21.15.6:0/448214992 shutdown_connections 2024-09-12T03:27:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.690+0000 7f156e7a2640 1 --2- 172.21.15.6:0/448214992 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f15500a2c00 0x7f15500a50c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.690+0000 7f156e7a2640 1 --2- 172.21.15.6:0/448214992 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1568106be0 0x7f156819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.690+0000 7f156e7a2640 1 --2- 172.21.15.6:0/448214992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1568101ea0 0x7f1568198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.690+0000 7f156e7a2640 1 --2- 172.21.15.6:0/448214992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1568101500 0x7f1568198540 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.690+0000 7f156e7a2640 1 -- 172.21.15.6:0/448214992 >> 172.21.15.6:0/448214992 conn(0x7f15680fba90 msgr2=0x7f15681005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:44.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.690+0000 7f156e7a2640 1 -- 172.21.15.6:0/448214992 shutdown_connections 2024-09-12T03:27:44.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:44.690+0000 7f156e7a2640 1 -- 172.21.15.6:0/448214992 wait complete. 2024-09-12T03:27:44.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:44 smithi161 ceph-mon[35485]: pgmap v534: 97 pgs: 97 active+clean; 13 GiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 431 KiB/s rd, 163 MiB/s wr, 16.39k op/s 2024-09-12T03:27:44.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:44 smithi118 ceph-mon[35048]: pgmap v534: 97 pgs: 97 active+clean; 13 GiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 431 KiB/s rd, 163 MiB/s wr, 16.39k op/s 2024-09-12T03:27:45.736 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:27:45.736 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:45 smithi006 ceph-mon[30991]: scrub queued for path: / 2024-09-12T03:27:45.736 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:45 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:45.736 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:45 smithi006 ceph-mon[30991]: scrub summary: active paths [/] 2024-09-12T03:27:45.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:45 smithi161 ceph-mon[35485]: scrub queued for path: / 2024-09-12T03:27:45.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:45 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:45.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:45 smithi161 ceph-mon[35485]: scrub summary: active paths [/] 2024-09-12T03:27:45.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:45 smithi118 ceph-mon[35048]: scrub queued for path: / 2024-09-12T03:27:45.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:45 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:45.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:45 smithi118 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-12T03:27:46.282 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:46.785 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:46 smithi006 ceph-mon[30991]: pgmap v535: 97 pgs: 97 active+clean; 13 GiB data, 120 GiB used, 953 GiB / 1.0 TiB avail; 431 KiB/s rd, 196 MiB/s wr, 19.92k op/s 2024-09-12T03:27:46.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:46 smithi161 ceph-mon[35485]: pgmap v535: 97 pgs: 97 active+clean; 13 GiB data, 120 GiB used, 953 GiB / 1.0 TiB avail; 431 KiB/s rd, 196 MiB/s wr, 19.92k op/s 2024-09-12T03:27:46.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:46 smithi118 ceph-mon[35048]: pgmap v535: 97 pgs: 97 active+clean; 13 GiB data, 120 GiB used, 953 GiB / 1.0 TiB avail; 431 KiB/s rd, 196 MiB/s wr, 19.92k op/s 2024-09-12T03:27:47.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.405+0000 7fc10df73640 1 -- 172.21.15.6:0/1041959337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc1000b7570 msgr2=0x7fc1000b9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:47.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.405+0000 7fc10df73640 1 --2- 172.21.15.6:0/1041959337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc1000b7570 0x7fc1000b9960 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc108067960 tx=0x7fc10808e420 comp rx=0 tx=0).stop 2024-09-12T03:27:47.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.406+0000 7fc10df73640 1 -- 172.21.15.6:0/1041959337 shutdown_connections 2024-09-12T03:27:47.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.406+0000 7fc10df73640 1 --2- 172.21.15.6:0/1041959337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc1000b7570 0x7fc1000b9960 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.406+0000 7fc10df73640 1 --2- 172.21.15.6:0/1041959337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1000a5170 0x7fc1000b7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.406+0000 7fc10df73640 1 --2- 172.21.15.6:0/1041959337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1000a4850 0x7fc1000a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.406+0000 7fc10df73640 1 -- 172.21.15.6:0/1041959337 >> 172.21.15.6:0/1041959337 conn(0x7fc10001a110 msgr2=0x7fc10001a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:47.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.411+0000 7fc10df73640 1 -- 172.21.15.6:0/1041959337 shutdown_connections 2024-09-12T03:27:47.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.414+0000 7fc10df73640 1 -- 172.21.15.6:0/1041959337 wait complete. 2024-09-12T03:27:47.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.415+0000 7fc10df73640 1 Processor -- start 2024-09-12T03:27:47.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.415+0000 7fc10df73640 1 -- start start 2024-09-12T03:27:47.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.416+0000 7fc10df73640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1000a4850 0x7fc1000b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.416+0000 7fc10df73640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc1000a5170 0x7fc1000b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.416+0000 7fc10df73640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1000b7570 0x7fc1000b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.416+0000 7fc10df73640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc1000ad260 con 0x7fc1000b7570 2024-09-12T03:27:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.416+0000 7fc10df73640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fc1000ad0e0 con 0x7fc1000a5170 2024-09-12T03:27:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.416+0000 7fc10df73640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc1000ad3e0 con 0x7fc1000a4850 2024-09-12T03:27:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.418+0000 7fc1077fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1000a4850 0x7fc1000b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.418+0000 7fc1077fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1000a4850 0x7fc1000b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:43020/0 (socket says 172.21.15.6:43020) 2024-09-12T03:27:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.418+0000 7fc1077fe640 1 -- 172.21.15.6:0/4229541415 learned_addr learned my addr 172.21.15.6:0/4229541415 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.419+0000 7fc107fff640 1 --2- 172.21.15.6:0/4229541415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1000b7570 0x7fc1000b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.422+0000 7fc106ffd640 1 --2- 172.21.15.6:0/4229541415 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc1000a5170 0x7fc1000b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.422+0000 7fc1077fe640 1 -- 172.21.15.6:0/4229541415 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc1000a5170 msgr2=0x7fc1000b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.422+0000 7fc1077fe640 1 --2- 172.21.15.6:0/4229541415 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc1000a5170 0x7fc1000b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.422+0000 7fc1077fe640 1 -- 172.21.15.6:0/4229541415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1000b7570 msgr2=0x7fc1000b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.422+0000 7fc1077fe640 1 --2- 172.21.15.6:0/4229541415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1000b7570 0x7fc1000b11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.422+0000 7fc1077fe640 1 -- 172.21.15.6:0/4229541415 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1000b1900 con 0x7fc1000a4850 2024-09-12T03:27:47.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.425+0000 7fc1077fe640 1 --2- 172.21.15.6:0/4229541415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1000a4850 0x7fc1000b6c10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00b450 tx=0x7fc0fc00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:47.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.430+0000 7fc104ff9640 1 -- 172.21.15.6:0/4229541415 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc015020 con 0x7fc1000a4850 2024-09-12T03:27:47.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.430+0000 7fc104ff9640 1 -- 172.21.15.6:0/4229541415 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0fc011d70 con 0x7fc1000a4850 2024-09-12T03:27:47.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.430+0000 7fc104ff9640 1 -- 172.21.15.6:0/4229541415 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc00f340 con 0x7fc1000a4850 2024-09-12T03:27:47.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.430+0000 7fc10df73640 1 -- 172.21.15.6:0/4229541415 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc10000b810 con 0x7fc1000a4850 2024-09-12T03:27:47.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.430+0000 7fc10df73640 1 -- 172.21.15.6:0/4229541415 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc10000bd00 con 0x7fc1000a4850 2024-09-12T03:27:47.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.435+0000 7fc11034c740 1 Processor -- start 2024-09-12T03:27:47.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.437+0000 7fc104ff9640 1 -- 172.21.15.6:0/4229541415 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc0fc018020 con 0x7fc1000a4850 2024-09-12T03:27:47.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.437+0000 7fc11034c740 1 -- start start 2024-09-12T03:27:47.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.438+0000 7fc11034c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e4d52a670 0x562e4d52aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.438+0000 7fc11034c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52af90 0x562e4d535310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.438+0000 7fc11034c740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d535850 0x562e4d537c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.438+0000 7fc11034c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x562e4d4902e0 con 0x562e4d52a670 2024-09-12T03:27:47.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.438+0000 7fc1077fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52af90 0x562e4d535310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:47.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.438+0000 7fc1077fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52af90 0x562e4d535310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:43034/0 (socket says 172.21.15.6:43034) 2024-09-12T03:27:47.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.438+0000 7fc1077fe640 1 -- 172.21.15.6:0/3727943397 learned_addr learned my addr 172.21.15.6:0/3727943397 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:47.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.438+0000 7fc11034c740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x562e4d437ce0 con 0x562e4d535850 2024-09-12T03:27:47.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.438+0000 7fc11034c740 1 -- 172.21.15.6:0/3727943397 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562e4d48c2a0 con 0x562e4d52af90 2024-09-12T03:27:47.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.438+0000 7fc106ffd640 1 --2- 172.21.15.6:0/3727943397 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d535850 0x562e4d537c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:47.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.439+0000 7fc1077fe640 1 -- 172.21.15.6:0/3727943397 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d535850 msgr2=0x562e4d537c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:47.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.439+0000 7fc1077fe640 1 --2- 172.21.15.6:0/3727943397 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d535850 0x562e4d537c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.439+0000 7fc1077fe640 1 -- 172.21.15.6:0/3727943397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e4d52a670 msgr2=0x562e4d52aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:47.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.439+0000 7fc1077fe640 1 --2- 172.21.15.6:0/3727943397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e4d52a670 0x562e4d52aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.439+0000 7fc1077fe640 1 -- 172.21.15.6:0/3727943397 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e4d41c440 con 0x562e4d52af90 2024-09-12T03:27:47.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.440+0000 7fc1077fe640 1 --2- 172.21.15.6:0/3727943397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52af90 0x562e4d535310 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0d4c20 tx=0x7fc0fc0d6670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8370d314bac9ce6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:47.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.440+0000 7fc0f1ffb640 1 -- 172.21.15.6:0/3727943397 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc0dd070 con 0x562e4d52af90 2024-09-12T03:27:47.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.440+0000 7fc0f1ffb640 1 -- 172.21.15.6:0/3727943397 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0fc0d07b0 con 0x562e4d52af90 2024-09-12T03:27:47.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.440+0000 7fc0f1ffb640 1 -- 172.21.15.6:0/3727943397 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc0d9bd0 con 0x562e4d52af90 2024-09-12T03:27:47.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.441+0000 7fc11034c740 1 -- 172.21.15.6:0/3727943397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52af90 msgr2=0x562e4d535310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:47.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.441+0000 7fc11034c740 1 --2- 172.21.15.6:0/3727943397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52af90 0x562e4d535310 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0d4c20 tx=0x7fc0fc0d6670 comp rx=0 tx=0).stop 2024-09-12T03:27:47.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.442+0000 7fc11034c740 1 -- 172.21.15.6:0/3727943397 shutdown_connections 2024-09-12T03:27:47.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.442+0000 7fc11034c740 1 --2- 172.21.15.6:0/3727943397 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d535850 0x562e4d537c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.442+0000 7fc11034c740 1 --2- 172.21.15.6:0/3727943397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52af90 0x562e4d535310 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.442+0000 7fc11034c740 1 --2- 172.21.15.6:0/3727943397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e4d52a670 0x562e4d52aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.442+0000 7fc11034c740 1 -- 172.21.15.6:0/3727943397 >> 172.21.15.6:0/3727943397 conn(0x562e4d526c00 msgr2=0x562e4d526fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:47.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.442+0000 7fc11034c740 1 -- 172.21.15.6:0/3727943397 shutdown_connections 2024-09-12T03:27:47.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.442+0000 7fc11034c740 1 -- 172.21.15.6:0/3727943397 wait complete. 2024-09-12T03:27:47.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.443+0000 7fc11034c740 1 -- start start 2024-09-12T03:27:47.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.444+0000 7fc11034c740 1 Processor -- start 2024-09-12T03:27:47.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.448+0000 7fc11034c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52a670 0x562e4d535150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.448+0000 7fc11034c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e4d52af90 0x562e4d532400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.449+0000 7fc11034c740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d535850 0x562e4d532940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.449+0000 7fc107fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52a670 0x562e4d535150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:47.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.449+0000 7fc107fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52a670 0x562e4d535150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:43038/0 (socket says 172.21.15.6:43038) 2024-09-12T03:27:47.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.449+0000 7fc107fff640 1 -- 172.21.15.6:0/3371955393 learned_addr learned my addr 172.21.15.6:0/3371955393 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:47.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.449+0000 7fc11034c740 1 -- 172.21.15.6:0/3371955393 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x562e4d538470 con 0x562e4d52af90 2024-09-12T03:27:47.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.449+0000 7fc106ffd640 1 --2- 172.21.15.6:0/3371955393 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d535850 0x562e4d532940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:47.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.449+0000 7fc11034c740 1 -- 172.21.15.6:0/3371955393 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x562e4d44b050 con 0x562e4d535850 2024-09-12T03:27:47.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.449+0000 7fc11034c740 1 -- 172.21.15.6:0/3371955393 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562e4d5385f0 con 0x562e4d52a670 2024-09-12T03:27:47.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.450+0000 7fc11034c740 10 client.0 _ll_register_callbacks cb 0x562e4d48b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:47.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.450+0000 7fc107fff640 1 -- 172.21.15.6:0/3371955393 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d535850 msgr2=0x562e4d532940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:47.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.450+0000 7fc107fff640 1 --2- 172.21.15.6:0/3371955393 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d535850 0x562e4d532940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.450+0000 7fc107fff640 1 -- 172.21.15.6:0/3371955393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e4d52af90 msgr2=0x562e4d532400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:47.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.450+0000 7fc107fff640 1 --2- 172.21.15.6:0/3371955393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e4d52af90 0x562e4d532400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.450+0000 7fc107fff640 1 -- 172.21.15.6:0/3371955393 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e4d2013c0 con 0x562e4d52a670 2024-09-12T03:27:47.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.451+0000 7fc107fff640 1 --2- 172.21.15.6:0/3371955393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52a670 0x562e4d535150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc108051790 tx=0x7fc10806e750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fce34a3b68c8c47 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:47.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.451+0000 7fc104ff9640 1 --2- 172.21.15.6:0/4229541415 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc0f80a2d10 0x7fc0f80a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.452+0000 7fc104ff9640 1 -- 172.21.15.6:0/4229541415 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fc0fc0ce8b0 con 0x7fc1000a4850 2024-09-12T03:27:47.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.452+0000 7fc0e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:27:47.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.452+0000 7fc0e6ffd640 1 -- 172.21.15.6:0/3371955393 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1080a5020 con 0x562e4d52a670 2024-09-12T03:27:47.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.452+0000 7fc11034c740 1 -- 172.21.15.6:0/3371955393 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0fc0d07b0 con 0x562e4d52a670 2024-09-12T03:27:47.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.453+0000 7fc0e6ffd640 1 -- 172.21.15.6:0/3371955393 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fc10806ec10 con 0x562e4d52a670 2024-09-12T03:27:47.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.453+0000 7fc0e6ffd640 1 -- 172.21.15.6:0/3371955393 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc10806ee30 con 0x562e4d52a670 2024-09-12T03:27:47.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.453+0000 7fc11034c740 10 client.25765 fetch_fsmap learned FSMap version 27 2024-09-12T03:27:47.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.453+0000 7fc11034c740 1 -- 172.21.15.6:0/3371955393 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562e4d543ef0 con 0x562e4d52a670 2024-09-12T03:27:47.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.453+0000 7fc107fff640 1 --2- 172.21.15.6:0/4229541415 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc0f80a2d10 0x7fc0f80a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:47.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.454+0000 7fc0e6ffd640 1 -- 172.21.15.6:0/3371955393 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0fc0d07b0 con 0x562e4d52a670 2024-09-12T03:27:47.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.454+0000 7fc0e6ffd640 1 -- 172.21.15.6:0/3371955393 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc1080adc10 con 0x562e4d52a670 2024-09-12T03:27:47.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.454+0000 7fc11034c740 10 client.25765 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:27:47.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.454+0000 7fc11034c740 10 client.25765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:27:47.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.454+0000 7fc11034c740 20 client.25765 populate_metadata read hostname 'smithi006' 2024-09-12T03:27:47.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.454+0000 7fc11034c740 1 --2- 172.21.15.6:0/3371955393 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x562e4d5456d0 0x562e4d547b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.454+0000 7fc11034c740 4 client.25765 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:47.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.454+0000 7fc11034c740 1 -- 172.21.15.6:0/3371955393 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e4d49d850 con 0x562e4d5456d0 2024-09-12T03:27:47.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.454+0000 7fc1077fe640 1 --2- 172.21.15.6:0/3371955393 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x562e4d5456d0 0x562e4d547b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:47.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.456+0000 7fc1077fe640 1 --2- 172.21.15.6:0/3371955393 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x562e4d5456d0 0x562e4d547b70 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=238779b0986f2643 server_cookie=91280f9a7592f299 in_seq=0 out_seq=0 2024-09-12T03:27:47.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.456+0000 7fc0e6ffd640 10 client.25765 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:47.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.457+0000 7fc0e6ffd640 1 -- 172.21.15.6:0/3371955393 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e4d49d850 con 0x562e4d5456d0 2024-09-12T03:27:47.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.457+0000 7fc0e6ffd640 10 client.25765 handle_command_reply: tid=0 2024-09-12T03:27:47.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.457+0000 7fc11034c740 1 client.25765 shutdown 2024-09-12T03:27:47.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.458+0000 7fc11034c740 1 -- 172.21.15.6:0/3371955393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52a670 msgr2=0x562e4d535150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:47.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.458+0000 7fc11034c740 1 --2- 172.21.15.6:0/3371955393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52a670 0x562e4d535150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc108051790 tx=0x7fc10806e750 comp rx=0 tx=0).stop 2024-09-12T03:27:47.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.459+0000 7fc11034c740 1 -- 172.21.15.6:0/3371955393 shutdown_connections 2024-09-12T03:27:47.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.459+0000 7fc11034c740 1 --2- 172.21.15.6:0/3371955393 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x562e4d5456d0 0x562e4d547b70 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.459+0000 7fc11034c740 1 --2- 172.21.15.6:0/3371955393 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d535850 0x562e4d532940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.459+0000 7fc11034c740 1 --2- 172.21.15.6:0/3371955393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e4d52af90 0x562e4d532400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.459+0000 7fc11034c740 1 --2- 172.21.15.6:0/3371955393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52a670 0x562e4d535150 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.459+0000 7fc11034c740 1 -- 172.21.15.6:0/3371955393 >> 172.21.15.6:0/3371955393 conn(0x562e4d526c00 msgr2=0x562e4d52c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:47.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.459+0000 7fc0e6ffd640 0 client.25765 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:47.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.459+0000 7fc11034c740 1 -- 172.21.15.6:0/3371955393 shutdown_connections 2024-09-12T03:27:47.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.459+0000 7fc107fff640 1 --2- 172.21.15.6:0/4229541415 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc0f80a2d10 0x7fc0f80a51d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc1080a7a30 tx=0x7fc1080a7a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:47.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.459+0000 7fc11034c740 1 -- 172.21.15.6:0/3371955393 wait complete. 2024-09-12T03:27:47.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.459+0000 7fc11034c740 20 client.25765 trim_cache size 0 max 16384 2024-09-12T03:27:47.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.498+0000 7fc11034c740 1 Processor -- start 2024-09-12T03:27:47.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.499+0000 7fc11034c740 1 -- start start 2024-09-12T03:27:47.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.499+0000 7fc11034c740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d52a670 0x562e4d5417b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.499+0000 7fc11034c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52af90 0x562e4d530cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.500+0000 7fc1077fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d52a670 0x562e4d5417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:47.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.500+0000 7fc1077fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d52a670 0x562e4d5417b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:49412/0 (socket says 172.21.15.6:49412) 2024-09-12T03:27:47.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.500+0000 7fc106ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52af90 0x562e4d530cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:47.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.500+0000 7fc106ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52af90 0x562e4d530cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:43054/0 (socket says 172.21.15.6:43054) 2024-09-12T03:27:47.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.500+0000 7fc11034c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e4d535850 0x562e4d531210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.500+0000 7fc1077fe640 1 -- 172.21.15.6:0/2432290669 learned_addr learned my addr 172.21.15.6:0/2432290669 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:47.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.501+0000 7fc11034c740 1 -- 172.21.15.6:0/2432290669 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc100174670 con 0x562e4d535850 2024-09-12T03:27:47.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.501+0000 7fc11034c740 1 -- 172.21.15.6:0/2432290669 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fc100014550 con 0x562e4d52a670 2024-09-12T03:27:47.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.501+0000 7fc11034c740 1 -- 172.21.15.6:0/2432290669 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc1000a5af0 con 0x562e4d52af90 2024-09-12T03:27:47.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.502+0000 7fc106ffd640 1 -- 172.21.15.6:0/2432290669 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d52a670 msgr2=0x562e4d5417b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:47.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.502+0000 7fc106ffd640 1 --2- 172.21.15.6:0/2432290669 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d52a670 0x562e4d5417b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.502+0000 7fc106ffd640 1 -- 172.21.15.6:0/2432290669 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e4d535850 msgr2=0x562e4d531210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:47.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.502+0000 7fc106ffd640 1 --2- 172.21.15.6:0/2432290669 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e4d535850 0x562e4d531210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.502+0000 7fc106ffd640 1 -- 172.21.15.6:0/2432290669 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e4d5354c0 con 0x562e4d52af90 2024-09-12T03:27:47.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.502+0000 7fc106ffd640 1 --2- 172.21.15.6:0/2432290669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52af90 0x562e4d530cd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x562e4d527f80 tx=0x7fc0f4013830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aff8f6c0c43ed87c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:47.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.503+0000 7fc0f17fa640 1 -- 172.21.15.6:0/2432290669 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f401c440 con 0x562e4d52af90 2024-09-12T03:27:47.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.503+0000 7fc0f17fa640 1 -- 172.21.15.6:0/2432290669 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0f4011950 con 0x562e4d52af90 2024-09-12T03:27:47.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.503+0000 7fc0f17fa640 1 -- 172.21.15.6:0/2432290669 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f401ba60 con 0x562e4d52af90 2024-09-12T03:27:47.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.504+0000 7fc11034c740 1 -- 172.21.15.6:0/2432290669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52af90 msgr2=0x562e4d530cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:47.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.504+0000 7fc11034c740 1 --2- 172.21.15.6:0/2432290669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52af90 0x562e4d530cd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x562e4d527f80 tx=0x7fc0f4013830 comp rx=0 tx=0).stop 2024-09-12T03:27:47.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.506+0000 7fc11034c740 1 -- 172.21.15.6:0/2432290669 shutdown_connections 2024-09-12T03:27:47.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.506+0000 7fc11034c740 1 --2- 172.21.15.6:0/2432290669 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e4d535850 0x562e4d531210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.506+0000 7fc11034c740 1 --2- 172.21.15.6:0/2432290669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52af90 0x562e4d530cd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.506+0000 7fc11034c740 1 --2- 172.21.15.6:0/2432290669 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d52a670 0x562e4d5417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.506+0000 7fc11034c740 1 -- 172.21.15.6:0/2432290669 >> 172.21.15.6:0/2432290669 conn(0x562e4d526c00 msgr2=0x562e4d5323c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:47.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.506+0000 7fc11034c740 1 -- 172.21.15.6:0/2432290669 shutdown_connections 2024-09-12T03:27:47.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.506+0000 7fc11034c740 1 -- 172.21.15.6:0/2432290669 wait complete. 2024-09-12T03:27:47.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.507+0000 7fc11034c740 1 -- start start 2024-09-12T03:27:47.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.507+0000 7fc11034c740 1 Processor -- start 2024-09-12T03:27:47.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.508+0000 7fc11034c740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d52a670 0x562e4d53e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.511+0000 7fc11034c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52af90 0x562e4d53a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.511+0000 7fc11034c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e4d535850 0x562e4d53ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.511+0000 7fc11034c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc100012a10 con 0x562e4d535850 2024-09-12T03:27:47.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.511+0000 7fc11034c740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fc100013e80 con 0x562e4d52a670 2024-09-12T03:27:47.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.511+0000 7fc11034c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc1000130e0 con 0x562e4d52af90 2024-09-12T03:27:47.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.511+0000 7fc11034c740 10 client.0 _ll_register_callbacks cb 0x7fc108050890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:47.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.512+0000 7fc107fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e4d535850 0x562e4d53ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:47.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.512+0000 7fc107fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e4d535850 0x562e4d53ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39896/0 (socket says 172.21.15.6:39896) 2024-09-12T03:27:47.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.512+0000 7fc107fff640 1 -- 172.21.15.6:0/2813331085 learned_addr learned my addr 172.21.15.6:0/2813331085 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:47.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.512+0000 7fc106ffd640 1 --2- 172.21.15.6:0/2813331085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52af90 0x562e4d53a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:47.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.512+0000 7fc1077fe640 1 --2- 172.21.15.6:0/2813331085 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d52a670 0x562e4d53e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:47.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.516+0000 7fc1077fe640 1 -- 172.21.15.6:0/2813331085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52af90 msgr2=0x562e4d53a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:47.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.516+0000 7fc1077fe640 1 --2- 172.21.15.6:0/2813331085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52af90 0x562e4d53a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.516+0000 7fc1077fe640 1 -- 172.21.15.6:0/2813331085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e4d535850 msgr2=0x562e4d53ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:47.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.516+0000 7fc1077fe640 1 --2- 172.21.15.6:0/2813331085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e4d535850 0x562e4d53ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.516+0000 7fc1077fe640 1 -- 172.21.15.6:0/2813331085 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e4d53b6a0 con 0x562e4d52a670 2024-09-12T03:27:47.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.516+0000 7fc107fff640 1 --2- 172.21.15.6:0/2813331085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e4d535850 0x562e4d53ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:47.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.516+0000 7fc1077fe640 1 --2- 172.21.15.6:0/2813331085 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d52a670 0x562e4d53e110 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc09b070 tx=0x7fc0fc0107f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46a68e6b27073aff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:47.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.516+0000 7fc0e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:27:47.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.517+0000 7fc0e6ffd640 1 -- 172.21.15.6:0/2813331085 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc0dd040 con 0x562e4d52a670 2024-09-12T03:27:47.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.517+0000 7fc11034c740 1 -- 172.21.15.6:0/2813331085 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0f4011950 con 0x562e4d52a670 2024-09-12T03:27:47.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.520+0000 7fc0e6ffd640 1 -- 172.21.15.6:0/2813331085 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fc0fc0993e0 con 0x562e4d52a670 2024-09-12T03:27:47.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.528+0000 7fc0e6ffd640 1 -- 172.21.15.6:0/2813331085 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc0fc097380 con 0x562e4d52a670 2024-09-12T03:27:47.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.528+0000 7fc11034c740 10 client.25685 fetch_fsmap learned FSMap version 27 2024-09-12T03:27:47.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.528+0000 7fc11034c740 1 -- 172.21.15.6:0/2813331085 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562e4d5af500 con 0x562e4d52a670 2024-09-12T03:27:47.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.529+0000 7fc0e6ffd640 1 -- 172.21.15.6:0/2813331085 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0f4011950 con 0x562e4d52a670 2024-09-12T03:27:47.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.529+0000 7fc0e6ffd640 1 -- 172.21.15.6:0/2813331085 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc0fc022060 con 0x562e4d52a670 2024-09-12T03:27:47.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.529+0000 7fc11034c740 10 client.25685 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:27:47.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.529+0000 7fc11034c740 10 client.25685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:27:47.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.529+0000 7fc11034c740 20 client.25685 populate_metadata read hostname 'smithi006' 2024-09-12T03:27:47.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.529+0000 7fc11034c740 1 --2- 172.21.15.6:0/2813331085 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x562e4d5af790 0x562e4d5b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.529+0000 7fc11034c740 4 client.25685 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:27:47.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.529+0000 7fc11034c740 1 -- 172.21.15.6:0/2813331085 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562e4d497d60 con 0x562e4d5af790 2024-09-12T03:27:47.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.537+0000 7fc106ffd640 1 --2- 172.21.15.6:0/2813331085 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x562e4d5af790 0x562e4d5b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:47.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.542+0000 7fc106ffd640 1 --2- 172.21.15.6:0/2813331085 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x562e4d5af790 0x562e4d5b1b80 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37ca6e5436f9de74 server_cookie=1c1589f823d469cb in_seq=0 out_seq=0 2024-09-12T03:27:47.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.542+0000 7fc0e6ffd640 10 client.25685 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:47.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.542+0000 7fc0e6ffd640 1 -- 172.21.15.6:0/2813331085 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x562e4d497d60 con 0x562e4d5af790 2024-09-12T03:27:47.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.542+0000 7fc0e6ffd640 10 client.25685 handle_command_reply: tid=0 2024-09-12T03:27:47.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.542+0000 7fc11034c740 1 client.25685 shutdown 2024-09-12T03:27:47.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.543+0000 7fc11034c740 1 -- 172.21.15.6:0/2813331085 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d52a670 msgr2=0x562e4d53e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:47.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.543+0000 7fc11034c740 1 --2- 172.21.15.6:0/2813331085 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d52a670 0x562e4d53e110 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc09b070 tx=0x7fc0fc0107f0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.543+0000 7fc11034c740 1 -- 172.21.15.6:0/2813331085 shutdown_connections 2024-09-12T03:27:47.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.543+0000 7fc11034c740 1 --2- 172.21.15.6:0/2813331085 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x562e4d5af790 0x562e4d5b1b80 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.543+0000 7fc11034c740 1 --2- 172.21.15.6:0/2813331085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e4d535850 0x562e4d53ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.543+0000 7fc11034c740 1 --2- 172.21.15.6:0/2813331085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e4d52af90 0x562e4d53a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.543+0000 7fc11034c740 1 --2- 172.21.15.6:0/2813331085 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562e4d52a670 0x562e4d53e110 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.543+0000 7fc11034c740 1 -- 172.21.15.6:0/2813331085 >> 172.21.15.6:0/2813331085 conn(0x562e4d526c00 msgr2=0x562e4d5308a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:47.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.544+0000 7fc0e6ffd640 0 client.25685 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:27:47.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.544+0000 7fc11034c740 1 -- 172.21.15.6:0/2813331085 shutdown_connections 2024-09-12T03:27:47.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.544+0000 7fc11034c740 1 -- 172.21.15.6:0/2813331085 wait complete. 2024-09-12T03:27:47.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.545+0000 7fc11034c740 20 client.25685 trim_cache size 0 max 16384 2024-09-12T03:27:47.550 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:27:47.550 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "scrub active (155 inodes in the stack)", 2024-09-12T03:27:47.550 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2024-09-12T03:27:47.550 INFO:teuthology.orchestra.run.smithi006.stdout: "838db3dd-096f-4ad3-9d4d-00a5fbaa8642": { 2024-09-12T03:27:47.550 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-09-12T03:27:47.550 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "838db3dd-096f-4ad3-9d4d-00a5fbaa8642", 2024-09-12T03:27:47.551 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2024-09-12T03:27:47.551 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:27:47.551 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:27:47.551 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:27:47.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.553+0000 7fc10df73640 1 -- 172.21.15.6:0/4229541415 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc0f80a2d10 msgr2=0x7fc0f80a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:47.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.553+0000 7fc10df73640 1 --2- 172.21.15.6:0/4229541415 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc0f80a2d10 0x7fc0f80a51d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc1080a7a30 tx=0x7fc1080a7a60 comp rx=0 tx=0).stop 2024-09-12T03:27:47.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.554+0000 7fc10df73640 1 -- 172.21.15.6:0/4229541415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1000a4850 msgr2=0x7fc1000b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:47.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.554+0000 7fc10df73640 1 --2- 172.21.15.6:0/4229541415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1000a4850 0x7fc1000b6c10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00b450 tx=0x7fc0fc00b910 comp rx=0 tx=0).stop 2024-09-12T03:27:47.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.554+0000 7fc10df73640 1 -- 172.21.15.6:0/4229541415 shutdown_connections 2024-09-12T03:27:47.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.554+0000 7fc10df73640 1 --2- 172.21.15.6:0/4229541415 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fc0f80a2d10 0x7fc0f80a51d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.554+0000 7fc10df73640 1 --2- 172.21.15.6:0/4229541415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1000b7570 0x7fc1000b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.554+0000 7fc10df73640 1 --2- 172.21.15.6:0/4229541415 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc1000a5170 0x7fc1000b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.554+0000 7fc10df73640 1 --2- 172.21.15.6:0/4229541415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1000a4850 0x7fc1000b6c10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.554+0000 7fc10df73640 1 -- 172.21.15.6:0/4229541415 >> 172.21.15.6:0/4229541415 conn(0x7fc10001a110 msgr2=0x7fc1000a3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:47.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.554+0000 7fc10df73640 1 -- 172.21.15.6:0/4229541415 shutdown_connections 2024-09-12T03:27:47.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:47.554+0000 7fc10df73640 1 -- 172.21.15.6:0/4229541415 wait complete. 2024-09-12T03:27:48.695 INFO:tasks.cephfs.filesystem:scrub status for tag:838db3dd-096f-4ad3-9d4d-00a5fbaa8642 - {'path': '/', 'tag': '838db3dd-096f-4ad3-9d4d-00a5fbaa8642', 'options': 'recursive,force'} 2024-09-12T03:27:48.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:48 smithi006 ceph-mon[30991]: pgmap v536: 97 pgs: 97 active+clean; 13 GiB data, 121 GiB used, 952 GiB / 1.0 TiB avail; 603 KiB/s rd, 172 MiB/s wr, 17.52k op/s 2024-09-12T03:27:48.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:48 smithi161 ceph-mon[35485]: pgmap v536: 97 pgs: 97 active+clean; 13 GiB data, 121 GiB used, 952 GiB / 1.0 TiB avail; 603 KiB/s rd, 172 MiB/s wr, 17.52k op/s 2024-09-12T03:27:48.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:48 smithi118 ceph-mon[35048]: pgmap v536: 97 pgs: 97 active+clean; 13 GiB data, 121 GiB used, 952 GiB / 1.0 TiB avail; 603 KiB/s rd, 172 MiB/s wr, 17.52k op/s 2024-09-12T03:27:50.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:50 smithi006 ceph-mon[30991]: pgmap v537: 97 pgs: 97 active+clean; 13 GiB data, 121 GiB used, 952 GiB / 1.0 TiB avail; 343 KiB/s rd, 151 MiB/s wr, 15.43k op/s 2024-09-12T03:27:50.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:50 smithi161 ceph-mon[35485]: pgmap v537: 97 pgs: 97 active+clean; 13 GiB data, 121 GiB used, 952 GiB / 1.0 TiB avail; 343 KiB/s rd, 151 MiB/s wr, 15.43k op/s 2024-09-12T03:27:50.977 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:50 smithi118 ceph-mon[35048]: pgmap v537: 97 pgs: 97 active+clean; 13 GiB data, 121 GiB used, 952 GiB / 1.0 TiB avail; 343 KiB/s rd, 151 MiB/s wr, 15.43k op/s 2024-09-12T03:27:52.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:52 smithi006 ceph-mon[30991]: pgmap v538: 97 pgs: 97 active+clean; 13 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 351 KiB/s rd, 169 MiB/s wr, 17.52k op/s 2024-09-12T03:27:52.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:52 smithi161 ceph-mon[35485]: pgmap v538: 97 pgs: 97 active+clean; 13 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 351 KiB/s rd, 169 MiB/s wr, 17.52k op/s 2024-09-12T03:27:52.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:52 smithi118 ceph-mon[35048]: pgmap v538: 97 pgs: 97 active+clean; 13 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 351 KiB/s rd, 169 MiB/s wr, 17.52k op/s 2024-09-12T03:27:54.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:54 smithi006 ceph-mon[30991]: pgmap v539: 97 pgs: 97 active+clean; 13 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 351 KiB/s rd, 165 MiB/s wr, 17.26k op/s 2024-09-12T03:27:54.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:54 smithi161 ceph-mon[35485]: pgmap v539: 97 pgs: 97 active+clean; 13 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 351 KiB/s rd, 165 MiB/s wr, 17.26k op/s 2024-09-12T03:27:54.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:54 smithi118 ceph-mon[35048]: pgmap v539: 97 pgs: 97 active+clean; 13 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 351 KiB/s rd, 165 MiB/s wr, 17.26k op/s 2024-09-12T03:27:56.498 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.6 2024-09-12T03:27:56.498 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:27:56.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:56 smithi006 ceph-mon[30991]: pgmap v540: 97 pgs: 97 active+clean; 13 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 182 KiB/s rd, 189 MiB/s wr, 20.43k op/s 2024-09-12T03:27:56.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:56 smithi161 ceph-mon[35485]: pgmap v540: 97 pgs: 97 active+clean; 13 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 182 KiB/s rd, 189 MiB/s wr, 20.43k op/s 2024-09-12T03:27:56.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:56 smithi118 ceph-mon[35048]: pgmap v540: 97 pgs: 97 active+clean; 13 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 182 KiB/s rd, 189 MiB/s wr, 20.43k op/s 2024-09-12T03:27:57.050 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:57.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.929+0000 7f28cfd9d640 1 -- 172.21.15.6:0/443043563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28c8101420 msgr2=0x7f28c810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:57.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.930+0000 7f28cfd9d640 1 --2- 172.21.15.6:0/443043563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28c8101420 0x7f28c810d950 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009c10 tx=0x7f28c002e880 comp rx=0 tx=0).stop 2024-09-12T03:27:57.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.930+0000 7f28cfd9d640 1 -- 172.21.15.6:0/443043563 shutdown_connections 2024-09-12T03:27:57.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.930+0000 7f28cfd9d640 1 --2- 172.21.15.6:0/443043563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28c8101420 0x7f28c810d950 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:57.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.930+0000 7f28cfd9d640 1 --2- 172.21.15.6:0/443043563 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28c8100a80 0x7f28c8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:57.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.930+0000 7f28cfd9d640 1 --2- 172.21.15.6:0/443043563 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28c8106a80 0x7f28c8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:57.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.930+0000 7f28cfd9d640 1 -- 172.21.15.6:0/443043563 >> 172.21.15.6:0/443043563 conn(0x7f28c80fc7a0 msgr2=0x7f28c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:57.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.931+0000 7f28cfd9d640 1 -- 172.21.15.6:0/443043563 shutdown_connections 2024-09-12T03:27:57.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.935+0000 7f28cfd9d640 1 -- 172.21.15.6:0/443043563 wait complete. 2024-09-12T03:27:57.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.935+0000 7f28cfd9d640 1 Processor -- start 2024-09-12T03:27:57.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.936+0000 7f28cfd9d640 1 -- start start 2024-09-12T03:27:57.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.937+0000 7f28cfd9d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28c8100a80 0x7f28c8073940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:57.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.937+0000 7f28cfd9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28c8101420 0x7f28c8073e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:57.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.937+0000 7f28cfd9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28c8106a80 0x7f28c806cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:57.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.937+0000 7f28cfd9d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f28c810fd70 con 0x7f28c8101420 2024-09-12T03:27:57.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.937+0000 7f28cfd9d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f28c810fbf0 con 0x7f28c8100a80 2024-09-12T03:27:57.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.937+0000 7f28cfd9d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f28c810fef0 con 0x7f28c8106a80 2024-09-12T03:27:57.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.939+0000 7f28cdb39640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28c8100a80 0x7f28c8073940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:57.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.939+0000 7f28ce33a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28c8106a80 0x7f28c806cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:57.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.939+0000 7f28ce33a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28c8106a80 0x7f28c806cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:34616/0 (socket says 172.21.15.6:34616) 2024-09-12T03:27:57.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.939+0000 7f28ce33a640 1 -- 172.21.15.6:0/1263957326 learned_addr learned my addr 172.21.15.6:0/1263957326 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:27:57.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.939+0000 7f28cd338640 1 --2- 172.21.15.6:0/1263957326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28c8101420 0x7f28c8073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:57.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.940+0000 7f28cd338640 1 -- 172.21.15.6:0/1263957326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28c8106a80 msgr2=0x7f28c806cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:57.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.940+0000 7f28cd338640 1 --2- 172.21.15.6:0/1263957326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28c8106a80 0x7f28c806cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:57.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.940+0000 7f28cd338640 1 -- 172.21.15.6:0/1263957326 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28c8100a80 msgr2=0x7f28c8073940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:57.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.940+0000 7f28cd338640 1 --2- 172.21.15.6:0/1263957326 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28c8100a80 0x7f28c8073940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:57.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.940+0000 7f28cd338640 1 -- 172.21.15.6:0/1263957326 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c806d470 con 0x7f28c8101420 2024-09-12T03:27:57.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.940+0000 7f28cdb39640 1 --2- 172.21.15.6:0/1263957326 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28c8100a80 0x7f28c8073940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:57.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.941+0000 7f28cd338640 1 --2- 172.21.15.6:0/1263957326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28c8101420 0x7f28c8073e80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f28b800d6d0 tx=0x7f28b800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:57.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.943+0000 7f28beffd640 1 -- 172.21.15.6:0/1263957326 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8017070 con 0x7f28c8101420 2024-09-12T03:27:57.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.943+0000 7f28beffd640 1 -- 172.21.15.6:0/1263957326 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28b8009820 con 0x7f28c8101420 2024-09-12T03:27:57.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.943+0000 7f28beffd640 1 -- 172.21.15.6:0/1263957326 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8004d00 con 0x7f28c8101420 2024-09-12T03:27:57.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.943+0000 7f28cfd9d640 1 -- 172.21.15.6:0/1263957326 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c806d700 con 0x7f28c8101420 2024-09-12T03:27:57.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.943+0000 7f28cfd9d640 1 -- 172.21.15.6:0/1263957326 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c80716a0 con 0x7f28c8101420 2024-09-12T03:27:57.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.948+0000 7f28cfd9d640 1 -- 172.21.15.6:0/1263957326 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28ac005290 con 0x7f28c8101420 2024-09-12T03:27:57.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.955+0000 7f28beffd640 1 -- 172.21.15.6:0/1263957326 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f28b8005220 con 0x7f28c8101420 2024-09-12T03:27:57.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.957+0000 7f28beffd640 1 --2- 172.21.15.6:0/1263957326 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f28b40a2d50 0x7f28b40a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:57.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.957+0000 7f28beffd640 1 -- 172.21.15.6:0/1263957326 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f28b80cee20 con 0x7f28c8101420 2024-09-12T03:27:57.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.962+0000 7f28cdb39640 1 --2- 172.21.15.6:0/1263957326 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f28b40a2d50 0x7f28b40a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:57.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.969+0000 7f28cdb39640 1 --2- 172.21.15.6:0/1263957326 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f28b40a2d50 0x7f28b40a5210 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f28c4005ab0 tx=0x7f28c4005f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:57.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:57.970+0000 7f28beffd640 1 -- 172.21.15.6:0/1263957326 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f28b809be50 con 0x7f28c8101420 2024-09-12T03:27:58.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:58.406+0000 7f28cfd9d640 1 -- 172.21.15.6:0/1263957326 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28ac005580 con 0x7f28c8101420 2024-09-12T03:27:58.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:58.411+0000 7f28beffd640 1 -- 172.21.15.6:0/1263957326 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11908 (secure 0 0 0) 0x7f28b809b750 con 0x7f28c8101420 2024-09-12T03:27:58.411 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:27:58.411 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"btime":"2024-09-12T03:24:26:920603+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:24:26.080047+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24463]},"id":1}]} 2024-09-12T03:27:58.412 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 27 2024-09-12T03:27:58.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:58.415+0000 7f28cfd9d640 1 -- 172.21.15.6:0/1263957326 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f28b40a2d50 msgr2=0x7f28b40a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:58.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:58.415+0000 7f28cfd9d640 1 --2- 172.21.15.6:0/1263957326 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f28b40a2d50 0x7f28b40a5210 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f28c4005ab0 tx=0x7f28c4005f70 comp rx=0 tx=0).stop 2024-09-12T03:27:58.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:58.415+0000 7f28cfd9d640 1 -- 172.21.15.6:0/1263957326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28c8101420 msgr2=0x7f28c8073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:58.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:58.415+0000 7f28cfd9d640 1 --2- 172.21.15.6:0/1263957326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28c8101420 0x7f28c8073e80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f28b800d6d0 tx=0x7f28b800db90 comp rx=0 tx=0).stop 2024-09-12T03:27:58.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:58.415+0000 7f28cfd9d640 1 -- 172.21.15.6:0/1263957326 shutdown_connections 2024-09-12T03:27:58.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:58.415+0000 7f28cfd9d640 1 --2- 172.21.15.6:0/1263957326 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f28b40a2d50 0x7f28b40a5210 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:58.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:58.415+0000 7f28cfd9d640 1 --2- 172.21.15.6:0/1263957326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28c8106a80 0x7f28c806cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:58.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:58.415+0000 7f28cfd9d640 1 --2- 172.21.15.6:0/1263957326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28c8101420 0x7f28c8073e80 secure :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f28b800d6d0 tx=0x7f28b800db90 comp rx=0 tx=0).stop 2024-09-12T03:27:58.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:58.415+0000 7f28cfd9d640 1 --2- 172.21.15.6:0/1263957326 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28c8100a80 0x7f28c8073940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:58.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:58.415+0000 7f28cfd9d640 1 -- 172.21.15.6:0/1263957326 >> 172.21.15.6:0/1263957326 conn(0x7f28c80fc7a0 msgr2=0x7f28c80fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:58.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:58.415+0000 7f28cfd9d640 1 -- 172.21.15.6:0/1263957326 shutdown_connections 2024-09-12T03:27:58.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:27:58.416+0000 7f28cfd9d640 1 -- 172.21.15.6:0/1263957326 wait complete. 2024-09-12T03:27:58.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:58 smithi006 ceph-mon[30991]: pgmap v541: 97 pgs: 97 active+clean; 13 GiB data, 124 GiB used, 949 GiB / 1.0 TiB avail; 202 KiB/s rd, 155 MiB/s wr, 16.91k op/s 2024-09-12T03:27:58.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:27:58 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1263957326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:58.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:58 smithi161 ceph-mon[35485]: pgmap v541: 97 pgs: 97 active+clean; 13 GiB data, 124 GiB used, 949 GiB / 1.0 TiB avail; 202 KiB/s rd, 155 MiB/s wr, 16.91k op/s 2024-09-12T03:27:58.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:27:58 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1263957326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:58.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:58 smithi118 ceph-mon[35048]: pgmap v541: 97 pgs: 97 active+clean; 13 GiB data, 124 GiB used, 949 GiB / 1.0 TiB avail; 202 KiB/s rd, 155 MiB/s wr, 16.91k op/s 2024-09-12T03:27:58.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:27:58 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1263957326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:59.462 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0' 2024-09-12T03:27:59.462 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0 2024-09-12T03:28:00.023 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:00.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.732+0000 7f0a13d33640 1 -- 172.21.15.6:0/4213605355 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a0c1027a0 msgr2=0x7f0a0c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:00.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.732+0000 7f0a13d33640 1 --2- 172.21.15.6:0/4213605355 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a0c1027a0 0x7f0a0c1067f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009860 tx=0x7f09fc02e750 comp rx=0 tx=0).stop 2024-09-12T03:28:00.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.733+0000 7f0a13d33640 1 -- 172.21.15.6:0/4213605355 shutdown_connections 2024-09-12T03:28:00.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.733+0000 7f0a13d33640 1 --2- 172.21.15.6:0/4213605355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a0c106f20 0x7f0a0c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.733+0000 7f0a13d33640 1 --2- 172.21.15.6:0/4213605355 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a0c1027a0 0x7f0a0c1067f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.733+0000 7f0a13d33640 1 --2- 172.21.15.6:0/4213605355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a0c101df0 0x7f0a0c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.733+0000 7f0a13d33640 1 -- 172.21.15.6:0/4213605355 >> 172.21.15.6:0/4213605355 conn(0x7f0a0c0fd8f0 msgr2=0x7f0a0c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:00.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.734+0000 7f0a13d33640 1 -- 172.21.15.6:0/4213605355 shutdown_connections 2024-09-12T03:28:00.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.736+0000 7f0a13d33640 1 -- 172.21.15.6:0/4213605355 wait complete. 2024-09-12T03:28:00.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.740+0000 7f0a13d33640 1 Processor -- start 2024-09-12T03:28:00.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.741+0000 7f0a13d33640 1 -- start start 2024-09-12T03:28:00.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.741+0000 7f0a13d33640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a0c101df0 0x7f0a0c075940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:00.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.741+0000 7f0a13d33640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a0c1027a0 0x7f0a0c075e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:00.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.741+0000 7f0a13d33640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a0c106f20 0x7f0a0c06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:00.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.742+0000 7f0a112ce640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a0c1027a0 0x7f0a0c075e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:00.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.742+0000 7f0a112ce640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a0c1027a0 0x7f0a0c075e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:60064/0 (socket says 172.21.15.6:60064) 2024-09-12T03:28:00.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.742+0000 7f0a112ce640 1 -- 172.21.15.6:0/3863541931 learned_addr learned my addr 172.21.15.6:0/3863541931 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:00.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.742+0000 7f0a13d33640 1 -- 172.21.15.6:0/3863541931 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c06e610 con 0x7f0a0c106f20 2024-09-12T03:28:00.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.742+0000 7f0a13d33640 1 -- 172.21.15.6:0/3863541931 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c06e490 con 0x7f0a0c1027a0 2024-09-12T03:28:00.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.742+0000 7f0a13d33640 1 -- 172.21.15.6:0/3863541931 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c06e790 con 0x7f0a0c101df0 2024-09-12T03:28:00.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.744+0000 7f0a11acf640 1 --2- 172.21.15.6:0/3863541931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a0c101df0 0x7f0a0c075940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:00.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.746+0000 7f0a112ce640 1 -- 172.21.15.6:0/3863541931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a0c101df0 msgr2=0x7f0a0c075940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:00.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.746+0000 7f0a112ce640 1 --2- 172.21.15.6:0/3863541931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a0c101df0 0x7f0a0c075940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.746+0000 7f0a112ce640 1 -- 172.21.15.6:0/3863541931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a0c106f20 msgr2=0x7f0a0c06f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:00.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.746+0000 7f0a112ce640 1 --2- 172.21.15.6:0/3863541931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a0c106f20 0x7f0a0c06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.746+0000 7f0a112ce640 1 -- 172.21.15.6:0/3863541931 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0c070290 con 0x7f0a0c1027a0 2024-09-12T03:28:00.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.747+0000 7f0a112ce640 1 --2- 172.21.15.6:0/3863541931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a0c1027a0 0x7f0a0c075e80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f09fc005d00 tx=0x7f09fc034940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:00.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.747+0000 7f0a02ffd640 1 -- 172.21.15.6:0/3863541931 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc004440 con 0x7f0a0c1027a0 2024-09-12T03:28:00.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.747+0000 7f0a13d33640 1 -- 172.21.15.6:0/3863541931 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a0c070520 con 0x7f0a0c1027a0 2024-09-12T03:28:00.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.747+0000 7f0a13d33640 1 -- 172.21.15.6:0/3863541931 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a0c1a2b40 con 0x7f0a0c1027a0 2024-09-12T03:28:00.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.751+0000 7f0a1610c740 1 Processor -- start 2024-09-12T03:28:00.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.752+0000 7f0a1610c740 1 -- start start 2024-09-12T03:28:00.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.752+0000 7f0a1610c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6e900 0x5560edf6ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:00.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.754+0000 7f0a1610c740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5560edf6f220 0x5560edf8a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:00.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.754+0000 7f0a1610c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5560edf8ac30 0x5560edf8d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:00.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.754+0000 7f0a1610c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5560eded42a0 con 0x5560edf8ac30 2024-09-12T03:28:00.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.754+0000 7f0a1610c740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5560ede7bb10 con 0x5560edf6f220 2024-09-12T03:28:00.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.754+0000 7f0a1610c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5560eded0260 con 0x5560edf6e900 2024-09-12T03:28:00.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.754+0000 7f0a122d0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6e900 0x5560edf6ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:00.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.754+0000 7f0a122d0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6e900 0x5560edf6ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:60956/0 (socket says 172.21.15.6:60956) 2024-09-12T03:28:00.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.754+0000 7f0a122d0640 1 -- 172.21.15.6:0/779687161 learned_addr learned my addr 172.21.15.6:0/779687161 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:00.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.755+0000 7f0a122d0640 1 -- 172.21.15.6:0/779687161 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5560edf6f220 msgr2=0x5560edf8a6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:28:00.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.755+0000 7f0a122d0640 1 --2- 172.21.15.6:0/779687161 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5560edf6f220 0x5560edf8a6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.755+0000 7f0a122d0640 1 -- 172.21.15.6:0/779687161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5560edf8ac30 msgr2=0x5560edf8d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:00.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.755+0000 7f0a122d0640 1 --2- 172.21.15.6:0/779687161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5560edf8ac30 0x5560edf8d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.755+0000 7f0a122d0640 1 -- 172.21.15.6:0/779687161 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560edc45b90 con 0x5560edf6e900 2024-09-12T03:28:00.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.756+0000 7f0a122d0640 1 --2- 172.21.15.6:0/779687161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6e900 0x5560edf6ece0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0a0402bd10 tx=0x7f0a0402d420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f821b366f8985b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:00.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.758+0000 7f09f77fe640 1 -- 172.21.15.6:0/779687161 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0402dd40 con 0x5560edf6e900 2024-09-12T03:28:00.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.758+0000 7f09f77fe640 1 -- 172.21.15.6:0/779687161 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a04039070 con 0x5560edf6e900 2024-09-12T03:28:00.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.758+0000 7f09f77fe640 1 -- 172.21.15.6:0/779687161 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a04033790 con 0x5560edf6e900 2024-09-12T03:28:00.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.759+0000 7f0a1610c740 1 -- 172.21.15.6:0/779687161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6e900 msgr2=0x5560edf6ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:00.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.759+0000 7f0a1610c740 1 --2- 172.21.15.6:0/779687161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6e900 0x5560edf6ece0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0a0402bd10 tx=0x7f0a0402d420 comp rx=0 tx=0).stop 2024-09-12T03:28:00.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.759+0000 7f0a1610c740 1 -- 172.21.15.6:0/779687161 shutdown_connections 2024-09-12T03:28:00.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.759+0000 7f0a1610c740 1 --2- 172.21.15.6:0/779687161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5560edf8ac30 0x5560edf8d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.759+0000 7f0a1610c740 1 --2- 172.21.15.6:0/779687161 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5560edf6f220 0x5560edf8a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.759+0000 7f0a1610c740 1 --2- 172.21.15.6:0/779687161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6e900 0x5560edf6ece0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.759+0000 7f0a1610c740 1 -- 172.21.15.6:0/779687161 >> 172.21.15.6:0/779687161 conn(0x5560edf6afd0 msgr2=0x5560edf6b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:00.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.760+0000 7f0a02ffd640 1 -- 172.21.15.6:0/3863541931 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09fc043070 con 0x7f0a0c1027a0 2024-09-12T03:28:00.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.760+0000 7f0a02ffd640 1 -- 172.21.15.6:0/3863541931 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc0046c0 con 0x7f0a0c1027a0 2024-09-12T03:28:00.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.760+0000 7f0a02ffd640 1 -- 172.21.15.6:0/3863541931 <== mon.2 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f09fc004920 con 0x7f0a0c1027a0 2024-09-12T03:28:00.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.761+0000 7f0a02ffd640 1 --2- 172.21.15.6:0/3863541931 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f09f80a2cf0 0x7f09f80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:00.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.762+0000 7f0a02ffd640 1 -- 172.21.15.6:0/3863541931 <== mon.2 v2:172.21.15.118:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f09fc0f1530 con 0x7f0a0c1027a0 2024-09-12T03:28:00.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.762+0000 7f0a11acf640 1 --2- 172.21.15.6:0/3863541931 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f09f80a2cf0 0x7f09f80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:00.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.762+0000 7f0a1610c740 1 -- 172.21.15.6:0/779687161 shutdown_connections 2024-09-12T03:28:00.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.762+0000 7f0a1610c740 1 -- 172.21.15.6:0/779687161 wait complete. 2024-09-12T03:28:00.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.763+0000 7f0a1610c740 1 -- start start 2024-09-12T03:28:00.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.763+0000 7f0a1610c740 1 Processor -- start 2024-09-12T03:28:00.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.763+0000 7f0a1610c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6e900 0x5560edf7be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:00.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.764+0000 7f0a1610c740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5560edf8ac30 0x5560edf7c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:00.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.764+0000 7f0a1610c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5560edf77160 0x5560edf775c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:00.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.764+0000 7f0a11acf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6e900 0x5560edf7be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:00.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.764+0000 7f0a11acf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6e900 0x5560edf7be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:60962/0 (socket says 172.21.15.6:60962) 2024-09-12T03:28:00.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.764+0000 7f0a112ce640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5560edf8ac30 0x5560edf7c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:00.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.764+0000 7f0a11acf640 1 -- 172.21.15.6:0/3749877397 learned_addr learned my addr 172.21.15.6:0/3749877397 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:00.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.764+0000 7f0a1610c740 1 -- 172.21.15.6:0/3749877397 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5560edf71830 con 0x5560edf77160 2024-09-12T03:28:00.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.764+0000 7f0a1610c740 1 -- 172.21.15.6:0/3749877397 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5560ede5c6e0 con 0x5560edf8ac30 2024-09-12T03:28:00.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.764+0000 7f0a1610c740 1 -- 172.21.15.6:0/3749877397 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5560edf719b0 con 0x5560edf6e900 2024-09-12T03:28:00.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.764+0000 7f0a1610c740 10 client.0 _ll_register_callbacks cb 0x5560ede80a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:00.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.765+0000 7f0a11acf640 1 -- 172.21.15.6:0/3749877397 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5560edf8ac30 msgr2=0x5560edf7c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:00.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.765+0000 7f0a11acf640 1 --2- 172.21.15.6:0/3749877397 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5560edf8ac30 0x5560edf7c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.765+0000 7f0a11acf640 1 -- 172.21.15.6:0/3749877397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5560edf77160 msgr2=0x5560edf775c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:00.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.765+0000 7f0a11acf640 1 --2- 172.21.15.6:0/3749877397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5560edf77160 0x5560edf775c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.765+0000 7f0a11acf640 1 -- 172.21.15.6:0/3749877397 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560edf77df0 con 0x5560edf6e900 2024-09-12T03:28:00.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.765+0000 7f0a11acf640 1 --2- 172.21.15.6:0/3749877397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6e900 0x5560edf7be40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0a080213b0 tx=0x7f0a080217b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d39ff1ac04efed9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:00.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.766+0000 7f09f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:28:00.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.766+0000 7f09f57fa640 1 -- 172.21.15.6:0/3749877397 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a08005050 con 0x5560edf6e900 2024-09-12T03:28:00.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.766+0000 7f0a1610c740 1 -- 172.21.15.6:0/3749877397 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a04039070 con 0x5560edf6e900 2024-09-12T03:28:00.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.766+0000 7f09f57fa640 1 -- 172.21.15.6:0/3749877397 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f0a080051f0 con 0x5560edf6e900 2024-09-12T03:28:00.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.768+0000 7f09f57fa640 1 -- 172.21.15.6:0/3749877397 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0a080053f0 con 0x5560edf6e900 2024-09-12T03:28:00.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.768+0000 7f0a1610c740 10 client.25786 fetch_fsmap learned FSMap version 27 2024-09-12T03:28:00.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.768+0000 7f0a1610c740 1 -- 172.21.15.6:0/3749877397 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5560edf86190 con 0x5560edf6e900 2024-09-12T03:28:00.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.772+0000 7f09f57fa640 1 -- 172.21.15.6:0/3749877397 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a08005590 con 0x5560edf6e900 2024-09-12T03:28:00.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.773+0000 7f09f57fa640 1 -- 172.21.15.6:0/3749877397 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0a0801f070 con 0x5560edf6e900 2024-09-12T03:28:00.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.773+0000 7f0a1610c740 10 client.25786 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:28:00.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.773+0000 7f0a1610c740 10 client.25786 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:28:00.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.773+0000 7f0a1610c740 20 client.25786 populate_metadata read hostname 'smithi006' 2024-09-12T03:28:00.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.773+0000 7f0a1610c740 1 --2- 172.21.15.6:0/3749877397 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5560edf87a30 0x5560edf89ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:00.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.773+0000 7f0a1610c740 4 client.25786 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:00.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.773+0000 7f0a1610c740 1 -- 172.21.15.6:0/3749877397 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0a04039070 con 0x5560edf87a30 2024-09-12T03:28:00.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.773+0000 7f0a11acf640 1 --2- 172.21.15.6:0/3863541931 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f09f80a2cf0 0x7f09f80a51b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0a08032000 tx=0x7f0a08028000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:00.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.773+0000 7f0a112ce640 1 --2- 172.21.15.6:0/3749877397 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5560edf87a30 0x5560edf89ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:00.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.775+0000 7f0a112ce640 1 --2- 172.21.15.6:0/3749877397 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5560edf87a30 0x5560edf89ef0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff8831040e77510d server_cookie=8430693bfbda727b in_seq=0 out_seq=0 2024-09-12T03:28:00.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.777+0000 7f09f57fa640 10 client.25786 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:00.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.779+0000 7f09f57fa640 1 -- 172.21.15.6:0/3749877397 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f0a04039070 con 0x5560edf87a30 2024-09-12T03:28:00.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.779+0000 7f09f57fa640 10 client.25786 handle_command_reply: tid=0 2024-09-12T03:28:00.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.780+0000 7f0a1610c740 1 client.25786 shutdown 2024-09-12T03:28:00.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.780+0000 7f0a1610c740 1 -- 172.21.15.6:0/3749877397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6e900 msgr2=0x5560edf7be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:00.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.780+0000 7f0a1610c740 1 --2- 172.21.15.6:0/3749877397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6e900 0x5560edf7be40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0a080213b0 tx=0x7f0a080217b0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.780+0000 7f0a1610c740 1 -- 172.21.15.6:0/3749877397 shutdown_connections 2024-09-12T03:28:00.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.780+0000 7f0a1610c740 1 --2- 172.21.15.6:0/3749877397 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5560edf87a30 0x5560edf89ef0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.780+0000 7f0a1610c740 1 --2- 172.21.15.6:0/3749877397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5560edf77160 0x5560edf775c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.780+0000 7f0a1610c740 1 --2- 172.21.15.6:0/3749877397 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5560edf8ac30 0x5560edf7c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.780+0000 7f0a1610c740 1 --2- 172.21.15.6:0/3749877397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6e900 0x5560edf7be40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.780+0000 7f0a1610c740 1 -- 172.21.15.6:0/3749877397 >> 172.21.15.6:0/3749877397 conn(0x5560edf6afd0 msgr2=0x5560edf6a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:00.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.780+0000 7f09f57fa640 0 client.25786 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:00.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.780+0000 7f0a1610c740 1 -- 172.21.15.6:0/3749877397 shutdown_connections 2024-09-12T03:28:00.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.780+0000 7f0a1610c740 1 -- 172.21.15.6:0/3749877397 wait complete. 2024-09-12T03:28:00.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.780+0000 7f0a1610c740 20 client.25786 trim_cache size 0 max 16384 2024-09-12T03:28:00.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.827+0000 7f0a1610c740 1 Processor -- start 2024-09-12T03:28:00.828 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:00 smithi006 ceph-mon[30991]: pgmap v542: 97 pgs: 97 active+clean; 13 GiB data, 124 GiB used, 949 GiB / 1.0 TiB avail; 30 KiB/s rd, 144 MiB/s wr, 16.09k op/s 2024-09-12T03:28:00.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.828+0000 7f0a1610c740 1 -- start start 2024-09-12T03:28:00.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.828+0000 7f0a1610c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5560edf6afd0 0x5560edf6a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:00.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.828+0000 7f0a1610c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6e900 0x5560edf925f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:00.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.828+0000 7f0a1610c740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5560edf77160 0x5560edf92b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:00.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.828+0000 7f0a1610c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c1a4c20 con 0x5560edf6afd0 2024-09-12T03:28:00.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.828+0000 7f0a1610c740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c1a52f0 con 0x5560edf77160 2024-09-12T03:28:00.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.828+0000 7f0a1610c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c070ef0 con 0x5560edf6e900 2024-09-12T03:28:00.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.829+0000 7f0a122d0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5560edf77160 0x5560edf92b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:00.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.829+0000 7f0a122d0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5560edf77160 0x5560edf92b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:60084/0 (socket says 172.21.15.6:60084) 2024-09-12T03:28:00.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.829+0000 7f0a122d0640 1 -- 172.21.15.6:0/945749619 learned_addr learned my addr 172.21.15.6:0/945749619 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:00.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.829+0000 7f0a112ce640 1 --2- 172.21.15.6:0/945749619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6e900 0x5560edf925f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:00.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.831+0000 7f0a11acf640 1 --2- 172.21.15.6:0/945749619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5560edf6afd0 0x5560edf6a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:00.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.831+0000 7f0a122d0640 1 -- 172.21.15.6:0/945749619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6e900 msgr2=0x5560edf925f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:00.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.831+0000 7f0a122d0640 1 --2- 172.21.15.6:0/945749619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6e900 0x5560edf925f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.831+0000 7f0a122d0640 1 -- 172.21.15.6:0/945749619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5560edf6afd0 msgr2=0x5560edf6a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:00.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.831+0000 7f0a122d0640 1 --2- 172.21.15.6:0/945749619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5560edf6afd0 0x5560edf6a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.831+0000 7f0a122d0640 1 -- 172.21.15.6:0/945749619 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560edf72070 con 0x5560edf77160 2024-09-12T03:28:00.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.832+0000 7f0a122d0640 1 --2- 172.21.15.6:0/945749619 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5560edf77160 0x5560edf92b30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0a0402ba80 tx=0x7f0a0402bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c0762f301c6aa4c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:00.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.834+0000 7f09f6ffd640 1 -- 172.21.15.6:0/945749619 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a04043070 con 0x5560edf77160 2024-09-12T03:28:00.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.835+0000 7f09f6ffd640 1 -- 172.21.15.6:0/945749619 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a04039040 con 0x5560edf77160 2024-09-12T03:28:00.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.836+0000 7f0a1610c740 1 -- 172.21.15.6:0/945749619 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5560edf77160 msgr2=0x5560edf92b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:00.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.836+0000 7f0a1610c740 1 --2- 172.21.15.6:0/945749619 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5560edf77160 0x5560edf92b30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0a0402ba80 tx=0x7f0a0402bab0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.836+0000 7f0a1610c740 1 -- 172.21.15.6:0/945749619 shutdown_connections 2024-09-12T03:28:00.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.836+0000 7f0a1610c740 1 --2- 172.21.15.6:0/945749619 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5560edf77160 0x5560edf92b30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.836+0000 7f0a1610c740 1 --2- 172.21.15.6:0/945749619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6e900 0x5560edf925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.836+0000 7f0a1610c740 1 --2- 172.21.15.6:0/945749619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5560edf6afd0 0x5560edf6a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.836+0000 7f0a1610c740 1 -- 172.21.15.6:0/945749619 >> 172.21.15.6:0/945749619 conn(0x5560edf76d80 msgr2=0x5560edf69af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:00.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.838+0000 7f0a1610c740 1 -- 172.21.15.6:0/945749619 shutdown_connections 2024-09-12T03:28:00.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.838+0000 7f0a1610c740 1 -- 172.21.15.6:0/945749619 wait complete. 2024-09-12T03:28:00.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.838+0000 7f0a1610c740 1 -- start start 2024-09-12T03:28:00.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.838+0000 7f0a1610c740 1 Processor -- start 2024-09-12T03:28:00.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.839+0000 7f0a1610c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6afd0 0x5560edf838f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:00.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.839+0000 7f0a1610c740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5560edf6e900 0x5560edf7eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:00.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.839+0000 7f0a1610c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5560edf8ac30 0x5560edf7f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:00.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.839+0000 7f0a1610c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c19ae60 con 0x5560edf8ac30 2024-09-12T03:28:00.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.839+0000 7f0a1610c740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c19c2b0 con 0x5560edf6e900 2024-09-12T03:28:00.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.839+0000 7f0a1610c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c19b510 con 0x5560edf6afd0 2024-09-12T03:28:00.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.839+0000 7f0a1610c740 10 client.0 _ll_register_callbacks cb 0x7f0a08025bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:00.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.839+0000 7f0a11acf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6afd0 0x5560edf838f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:00.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.839+0000 7f0a11acf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6afd0 0x5560edf838f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:60978/0 (socket says 172.21.15.6:60978) 2024-09-12T03:28:00.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.839+0000 7f0a11acf640 1 -- 172.21.15.6:0/576956655 learned_addr learned my addr 172.21.15.6:0/576956655 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:00.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.839+0000 7f0a112ce640 1 --2- 172.21.15.6:0/576956655 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5560edf6e900 0x5560edf7eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:00.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.840+0000 7f0a11acf640 1 -- 172.21.15.6:0/576956655 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5560edf6e900 msgr2=0x5560edf7eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:00.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.840+0000 7f0a11acf640 1 --2- 172.21.15.6:0/576956655 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5560edf6e900 0x5560edf7eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.840+0000 7f0a11acf640 1 -- 172.21.15.6:0/576956655 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5560edf8ac30 msgr2=0x5560edf7f0c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:28:00.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.840+0000 7f0a11acf640 1 --2- 172.21.15.6:0/576956655 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5560edf8ac30 0x5560edf7f0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.840+0000 7f0a11acf640 1 -- 172.21.15.6:0/576956655 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560edf45670 con 0x5560edf6afd0 2024-09-12T03:28:00.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.842+0000 7f0a11acf640 1 --2- 172.21.15.6:0/576956655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6afd0 0x5560edf838f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0a0801a830 tx=0x7f0a0801a860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bdbf32b1b03eb4e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:00.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.842+0000 7f09f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:28:00.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.842+0000 7f09f4ff9640 1 -- 172.21.15.6:0/576956655 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0803dcf0 con 0x5560edf6afd0 2024-09-12T03:28:00.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.842+0000 7f0a1610c740 1 -- 172.21.15.6:0/576956655 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5560edee19d0 con 0x5560edf6afd0 2024-09-12T03:28:00.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.843+0000 7f09f4ff9640 1 -- 172.21.15.6:0/576956655 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f0a0803d420 con 0x5560edf6afd0 2024-09-12T03:28:00.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.843+0000 7f09f4ff9640 1 -- 172.21.15.6:0/576956655 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0a0803d620 con 0x5560edf6afd0 2024-09-12T03:28:00.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.843+0000 7f0a1610c740 10 client.25792 fetch_fsmap learned FSMap version 27 2024-09-12T03:28:00.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.843+0000 7f0a1610c740 1 -- 172.21.15.6:0/576956655 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5560edff52f0 con 0x5560edf6afd0 2024-09-12T03:28:00.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.845+0000 7f09f4ff9640 1 -- 172.21.15.6:0/576956655 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5560edee19d0 con 0x5560edf6afd0 2024-09-12T03:28:00.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.845+0000 7f09f4ff9640 1 -- 172.21.15.6:0/576956655 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0a0801f050 con 0x5560edf6afd0 2024-09-12T03:28:00.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.845+0000 7f0a1610c740 10 client.25792 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:28:00.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.845+0000 7f0a1610c740 10 client.25792 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:28:00.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.845+0000 7f0a1610c740 20 client.25792 populate_metadata read hostname 'smithi006' 2024-09-12T03:28:00.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.845+0000 7f0a1610c740 1 --2- 172.21.15.6:0/576956655 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5560edff5ae0 0x5560edff7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:00.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.845+0000 7f0a1610c740 4 client.25792 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}] 2024-09-12T03:28:00.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.845+0000 7f0a1610c740 1 -- 172.21.15.6:0/576956655 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}) -- 0x5560ededbd20 con 0x5560edff5ae0 2024-09-12T03:28:00.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.846+0000 7f0a112ce640 1 --2- 172.21.15.6:0/576956655 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5560edff5ae0 0x5560edff7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:00.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.849+0000 7f0a112ce640 1 --2- 172.21.15.6:0/576956655 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5560edff5ae0 0x5560edff7f40 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ae61c0b86e06841 server_cookie=c4065f756cb3d061 in_seq=0 out_seq=0 2024-09-12T03:28:00.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:00.849+0000 7f09f4ff9640 10 client.25792 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:00.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:00 smithi161 ceph-mon[35485]: pgmap v542: 97 pgs: 97 active+clean; 13 GiB data, 124 GiB used, 949 GiB / 1.0 TiB avail; 30 KiB/s rd, 144 MiB/s wr, 16.09k op/s 2024-09-12T03:28:00.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:00 smithi118 ceph-mon[35048]: pgmap v542: 97 pgs: 97 active+clean; 13 GiB data, 124 GiB used, 949 GiB / 1.0 TiB avail; 30 KiB/s rd, 144 MiB/s wr, 16.09k op/s 2024-09-12T03:28:02.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:02 smithi006 ceph-mon[30991]: pgmap v543: 97 pgs: 97 active+clean; 13 GiB data, 125 GiB used, 948 GiB / 1.0 TiB avail; 30 KiB/s rd, 180 MiB/s wr, 20.84k op/s 2024-09-12T03:28:02.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:02 smithi161 ceph-mon[35485]: pgmap v543: 97 pgs: 97 active+clean; 13 GiB data, 125 GiB used, 948 GiB / 1.0 TiB avail; 30 KiB/s rd, 180 MiB/s wr, 20.84k op/s 2024-09-12T03:28:02.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:02 smithi118 ceph-mon[35048]: pgmap v543: 97 pgs: 97 active+clean; 13 GiB data, 125 GiB used, 948 GiB / 1.0 TiB avail; 30 KiB/s rd, 180 MiB/s wr, 20.84k op/s 2024-09-12T03:28:04.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:04 smithi006 ceph-mon[30991]: pgmap v544: 97 pgs: 97 active+clean; 13 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 22 KiB/s rd, 153 MiB/s wr, 18.36k op/s 2024-09-12T03:28:04.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:04 smithi161 ceph-mon[35485]: pgmap v544: 97 pgs: 97 active+clean; 13 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 22 KiB/s rd, 153 MiB/s wr, 18.36k op/s 2024-09-12T03:28:04.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:04 smithi118 ceph-mon[35048]: pgmap v544: 97 pgs: 97 active+clean; 13 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 22 KiB/s rd, 153 MiB/s wr, 18.36k op/s 2024-09-12T03:28:06.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:06 smithi006 ceph-mon[30991]: pgmap v545: 97 pgs: 97 active+clean; 13 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 22 KiB/s rd, 165 MiB/s wr, 20.77k op/s 2024-09-12T03:28:06.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:06 smithi161 ceph-mon[35485]: pgmap v545: 97 pgs: 97 active+clean; 13 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 22 KiB/s rd, 165 MiB/s wr, 20.77k op/s 2024-09-12T03:28:06.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:06 smithi118 ceph-mon[35048]: pgmap v545: 97 pgs: 97 active+clean; 13 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 22 KiB/s rd, 165 MiB/s wr, 20.77k op/s 2024-09-12T03:28:08.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:08 smithi118 ceph-mon[35048]: pgmap v546: 97 pgs: 97 active+clean; 13 GiB data, 123 GiB used, 949 GiB / 1.0 TiB avail; 45 KiB/s rd, 135 MiB/s wr, 17.97k op/s 2024-09-12T03:28:09.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:08 smithi006 ceph-mon[30991]: pgmap v546: 97 pgs: 97 active+clean; 13 GiB data, 123 GiB used, 949 GiB / 1.0 TiB avail; 45 KiB/s rd, 135 MiB/s wr, 17.97k op/s 2024-09-12T03:28:09.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:08 smithi161 ceph-mon[35485]: pgmap v546: 97 pgs: 97 active+clean; 13 GiB data, 123 GiB used, 949 GiB / 1.0 TiB avail; 45 KiB/s rd, 135 MiB/s wr, 17.97k op/s 2024-09-12T03:28:10.964 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:10 smithi118 ceph-mon[35048]: pgmap v547: 97 pgs: 97 active+clean; 13 GiB data, 124 GiB used, 949 GiB / 1.0 TiB avail; 25 KiB/s rd, 134 MiB/s wr, 18.37k op/s 2024-09-12T03:28:11.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:10 smithi006 ceph-mon[30991]: pgmap v547: 97 pgs: 97 active+clean; 13 GiB data, 124 GiB used, 949 GiB / 1.0 TiB avail; 25 KiB/s rd, 134 MiB/s wr, 18.37k op/s 2024-09-12T03:28:11.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:10 smithi161 ceph-mon[35485]: pgmap v547: 97 pgs: 97 active+clean; 13 GiB data, 124 GiB used, 949 GiB / 1.0 TiB avail; 25 KiB/s rd, 134 MiB/s wr, 18.37k op/s 2024-09-12T03:28:12.915 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:12 smithi161 ceph-mon[35485]: pgmap v548: 97 pgs: 97 active+clean; 13 GiB data, 125 GiB used, 948 GiB / 1.0 TiB avail; 26 KiB/s rd, 154 MiB/s wr, 22.22k op/s 2024-09-12T03:28:12.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:12 smithi118 ceph-mon[35048]: pgmap v548: 97 pgs: 97 active+clean; 13 GiB data, 125 GiB used, 948 GiB / 1.0 TiB avail; 26 KiB/s rd, 154 MiB/s wr, 22.22k op/s 2024-09-12T03:28:13.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:12 smithi006 ceph-mon[30991]: pgmap v548: 97 pgs: 97 active+clean; 13 GiB data, 125 GiB used, 948 GiB / 1.0 TiB avail; 26 KiB/s rd, 154 MiB/s wr, 22.22k op/s 2024-09-12T03:28:14.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:14 smithi118 ceph-mon[35048]: pgmap v549: 97 pgs: 97 active+clean; 13 GiB data, 125 GiB used, 948 GiB / 1.0 TiB avail; 34 KiB/s rd, 125 MiB/s wr, 18.85k op/s 2024-09-12T03:28:15.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:14 smithi006 ceph-mon[30991]: pgmap v549: 97 pgs: 97 active+clean; 13 GiB data, 125 GiB used, 948 GiB / 1.0 TiB avail; 34 KiB/s rd, 125 MiB/s wr, 18.85k op/s 2024-09-12T03:28:15.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:14 smithi161 ceph-mon[35485]: pgmap v549: 97 pgs: 97 active+clean; 13 GiB data, 125 GiB used, 948 GiB / 1.0 TiB avail; 34 KiB/s rd, 125 MiB/s wr, 18.85k op/s 2024-09-12T03:28:16.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:16 smithi118 ceph-mon[35048]: pgmap v550: 97 pgs: 97 active+clean; 13 GiB data, 125 GiB used, 948 GiB / 1.0 TiB avail; 33 KiB/s rd, 127 MiB/s wr, 20.11k op/s 2024-09-12T03:28:17.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:16 smithi006 ceph-mon[30991]: pgmap v550: 97 pgs: 97 active+clean; 13 GiB data, 125 GiB used, 948 GiB / 1.0 TiB avail; 33 KiB/s rd, 127 MiB/s wr, 20.11k op/s 2024-09-12T03:28:17.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:16 smithi161 ceph-mon[35485]: pgmap v550: 97 pgs: 97 active+clean; 13 GiB data, 125 GiB used, 948 GiB / 1.0 TiB avail; 33 KiB/s rd, 127 MiB/s wr, 20.11k op/s 2024-09-12T03:28:18.696 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:28:18.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:18 smithi118 ceph-mon[35048]: pgmap v551: 97 pgs: 97 active+clean; 13 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 34 KiB/s rd, 110 MiB/s wr, 18.71k op/s 2024-09-12T03:28:18.964 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:19.086 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:18 smithi006 ceph-mon[30991]: pgmap v551: 97 pgs: 97 active+clean; 13 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 34 KiB/s rd, 110 MiB/s wr, 18.71k op/s 2024-09-12T03:28:19.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:18 smithi161 ceph-mon[35485]: pgmap v551: 97 pgs: 97 active+clean; 13 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 34 KiB/s rd, 110 MiB/s wr, 18.71k op/s 2024-09-12T03:28:19.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.617+0000 7f09f4ff9640 1 -- 172.21.15.6:0/576956655 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5560ededbd20 con 0x5560edff5ae0 2024-09-12T03:28:19.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.617+0000 7f09f4ff9640 10 client.25792 handle_command_reply: tid=0 2024-09-12T03:28:19.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.617+0000 7f0a1610c740 1 client.25792 shutdown 2024-09-12T03:28:19.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.617+0000 7f0a1610c740 1 -- 172.21.15.6:0/576956655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6afd0 msgr2=0x5560edf838f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:19.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.617+0000 7f0a1610c740 1 --2- 172.21.15.6:0/576956655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6afd0 0x5560edf838f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0a0801a830 tx=0x7f0a0801a860 comp rx=0 tx=0).stop 2024-09-12T03:28:19.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.617+0000 7f0a1610c740 1 -- 172.21.15.6:0/576956655 shutdown_connections 2024-09-12T03:28:19.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.617+0000 7f0a1610c740 1 --2- 172.21.15.6:0/576956655 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5560edff5ae0 0x5560edff7f40 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.617+0000 7f0a1610c740 1 --2- 172.21.15.6:0/576956655 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5560edf8ac30 0x5560edf7f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.617+0000 7f0a1610c740 1 --2- 172.21.15.6:0/576956655 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5560edf6e900 0x5560edf7eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.617+0000 7f0a1610c740 1 --2- 172.21.15.6:0/576956655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560edf6afd0 0x5560edf838f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.617+0000 7f09f4ff9640 0 client.25792 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:19.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.617+0000 7f0a1610c740 1 -- 172.21.15.6:0/576956655 >> 172.21.15.6:0/576956655 conn(0x5560edf76d80 msgr2=0x5560edf6a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:19.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.618+0000 7f0a1610c740 1 -- 172.21.15.6:0/576956655 shutdown_connections 2024-09-12T03:28:19.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.618+0000 7f0a1610c740 1 -- 172.21.15.6:0/576956655 wait complete. 2024-09-12T03:28:19.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.618+0000 7f0a1610c740 20 client.25792 trim_cache size 0 max 16384 2024-09-12T03:28:19.620 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:28:19.620 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 27, 2024-09-12T03:28:19.620 INFO:teuthology.orchestra.run.smithi006.stdout: "leader": 24494, 2024-09-12T03:28:19.620 INFO:teuthology.orchestra.run.smithi006.stdout: "set_version": 26, 2024-09-12T03:28:19.620 INFO:teuthology.orchestra.run.smithi006.stdout: "sets": { 2024-09-12T03:28:19.620 INFO:teuthology.orchestra.run.smithi006.stdout: "7867f508": { 2024-09-12T03:28:19.620 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 26, 2024-09-12T03:28:19.620 INFO:teuthology.orchestra.run.smithi006.stdout: "age_ref": 0.0, 2024-09-12T03:28:19.620 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:28:19.621 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "QUIESCED", 2024-09-12T03:28:19.621 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:28:19.621 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:28:19.621 INFO:teuthology.orchestra.run.smithi006.stdout: "timeout": 90.0, 2024-09-12T03:28:19.621 INFO:teuthology.orchestra.run.smithi006.stdout: "expiration": 136.6, 2024-09-12T03:28:19.621 INFO:teuthology.orchestra.run.smithi006.stdout: "members": { 2024-09-12T03:28:19.621 INFO:teuthology.orchestra.run.smithi006.stdout: "file:/": { 2024-09-12T03:28:19.621 INFO:teuthology.orchestra.run.smithi006.stdout: "excluded": false, 2024-09-12T03:28:19.621 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:28:19.621 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "QUIESCED", 2024-09-12T03:28:19.621 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:28:19.621 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:28:19.622 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:28:19.622 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:28:19.622 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:28:19.622 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:28:19.622 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:28:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.618+0000 7f0a00ff9640 1 -- 172.21.15.6:0/3863541931 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f09f80a2cf0 msgr2=0x7f09f80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.618+0000 7f0a00ff9640 1 --2- 172.21.15.6:0/3863541931 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f09f80a2cf0 0x7f09f80a51b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0a08032000 tx=0x7f0a08028000 comp rx=0 tx=0).stop 2024-09-12T03:28:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.618+0000 7f0a00ff9640 1 -- 172.21.15.6:0/3863541931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a0c1027a0 msgr2=0x7f0a0c075e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.618+0000 7f0a00ff9640 1 --2- 172.21.15.6:0/3863541931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a0c1027a0 0x7f0a0c075e80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f09fc005d00 tx=0x7f09fc034940 comp rx=0 tx=0).stop 2024-09-12T03:28:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.618+0000 7f0a00ff9640 1 -- 172.21.15.6:0/3863541931 shutdown_connections 2024-09-12T03:28:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.618+0000 7f0a00ff9640 1 --2- 172.21.15.6:0/3863541931 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f09f80a2cf0 0x7f09f80a51b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.618+0000 7f0a00ff9640 1 --2- 172.21.15.6:0/3863541931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a0c106f20 0x7f0a0c06f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.618+0000 7f0a00ff9640 1 --2- 172.21.15.6:0/3863541931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a0c1027a0 0x7f0a0c075e80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.618+0000 7f0a00ff9640 1 --2- 172.21.15.6:0/3863541931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a0c101df0 0x7f0a0c075940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.618+0000 7f0a00ff9640 1 -- 172.21.15.6:0/3863541931 >> 172.21.15.6:0/3863541931 conn(0x7f0a0c0fd8f0 msgr2=0x7f0a0c0fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.619+0000 7f0a00ff9640 1 -- 172.21.15.6:0/3863541931 shutdown_connections 2024-09-12T03:28:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:19.619+0000 7f0a00ff9640 1 -- 172.21.15.6:0/3863541931 wait complete. 2024-09-12T03:28:20.067 INFO:tasks.workunit.client.0.smithi006.stdout:Syncing()...28 sec 2024-09-12T03:28:20.067 INFO:tasks.workunit.client.0.smithi006.stdout:FFSB benchmark finished at: Thu Sep 12 03:28:20 2024 2024-09-12T03:28:20.067 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-09-12T03:28:20.068 INFO:tasks.workunit.client.0.smithi006.stdout:Results: 2024-09-12T03:28:20.068 INFO:tasks.workunit.client.0.smithi006.stdout:Benchmark took 333.88 sec 2024-09-12T03:28:20.068 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-09-12T03:28:20.068 INFO:tasks.workunit.client.0.smithi006.stdout:Total Results 2024-09-12T03:28:20.068 INFO:tasks.workunit.client.0.smithi006.stdout:=============== 2024-09-12T03:28:20.068 INFO:tasks.workunit.client.0.smithi006.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-09-12T03:28:20.068 INFO:tasks.workunit.client.0.smithi006.stdout: ======= ============ ========= ======= =========== ========== 2024-09-12T03:28:20.068 INFO:tasks.workunit.client.0.smithi006.stdout: write : 23939840 71702.77 100.000% 100.000% 280MB/sec 2024-09-12T03:28:20.068 INFO:tasks.workunit.client.0.smithi006.stdout:- 2024-09-12T03:28:20.068 INFO:tasks.workunit.client.0.smithi006.stdout:71702.77 Transactions per Second 2024-09-12T03:28:20.068 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-09-12T03:28:20.069 INFO:tasks.workunit.client.0.smithi006.stdout:Throughput Results 2024-09-12T03:28:20.069 INFO:tasks.workunit.client.0.smithi006.stdout:=================== 2024-09-12T03:28:20.069 INFO:tasks.workunit.client.0.smithi006.stdout:Write Throughput: 280MB/sec 2024-09-12T03:28:20.157 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-09-12T03:28:20.157 INFO:tasks.workunit.client.0.smithi006.stdout:System Call Latency statistics in millisecs 2024-09-12T03:28:20.157 INFO:tasks.workunit.client.0.smithi006.stdout:===== 2024-09-12T03:28:20.157 INFO:tasks.workunit.client.0.smithi006.stdout: Min Avg Max Total Calls 2024-09-12T03:28:20.157 INFO:tasks.workunit.client.0.smithi006.stdout: ======== ======== ======== ============ 2024-09-12T03:28:20.157 INFO:tasks.workunit.client.0.smithi006.stdout:[ open] 0.005000 4.109157 539.585999 18703 2024-09-12T03:28:20.157 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-12T03:28:20.157 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-12T03:28:20.158 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-12T03:28:20.158 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[3] 500.000000 - 1000.000000 : 6 2024-09-12T03:28:20.158 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[4] 200.000000 - 500.000000 : 103 2024-09-12T03:28:20.158 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[5] 100.000000 - 200.000000 : 196 2024-09-12T03:28:20.158 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[6] 50.000000 - 100.000000 : 129 2024-09-12T03:28:20.158 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[7] 20.000000 - 50.000000 : 127 2024-09-12T03:28:20.158 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[8] 10.000000 - 20.000000 : 80 2024-09-12T03:28:20.158 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[9] 5.000000 - 10.000000 : 59 2024-09-12T03:28:20.158 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[10] 2.000000 - 5.000000 : 55 2024-09-12T03:28:20.158 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[11] 1.000000 - 2.000000 : 18 2024-09-12T03:28:20.158 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[12] 0.500000 - 1.000000 : 8 2024-09-12T03:28:20.158 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[13] 0.200000 - 0.500000 : 3 2024-09-12T03:28:20.158 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[14] 0.100000 - 0.200000 : 17 2024-09-12T03:28:20.158 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[15] 0.050000 - 0.100000 : 119 2024-09-12T03:28:20.159 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[16] 0.020000 - 0.050000 : 7669 2024-09-12T03:28:20.159 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[17] 0.010000 - 0.020000 : 9522 2024-09-12T03:28:20.159 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[18] 0.000000 - 0.010000 : 592 2024-09-12T03:28:20.159 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-09-12T03:28:20.159 INFO:tasks.workunit.client.0.smithi006.stdout:[ write] 0.001000 0.399457 23890.976562 23939840 2024-09-12T03:28:20.159 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[0] 5000.000000 - 10000.000000 : 225 2024-09-12T03:28:20.159 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[1] 2000.000000 - 5000.000000 : 577 2024-09-12T03:28:20.159 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[2] 1000.000000 - 2000.000000 : 706 2024-09-12T03:28:20.159 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[3] 500.000000 - 1000.000000 : 640 2024-09-12T03:28:20.159 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[4] 200.000000 - 500.000000 : 1006 2024-09-12T03:28:20.159 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[5] 100.000000 - 200.000000 : 1414 2024-09-12T03:28:20.159 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[6] 50.000000 - 100.000000 : 13530 2024-09-12T03:28:20.159 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[7] 20.000000 - 50.000000 : 17717 2024-09-12T03:28:20.159 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[8] 10.000000 - 20.000000 : 1594 2024-09-12T03:28:20.160 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[9] 5.000000 - 10.000000 : 1281 2024-09-12T03:28:20.160 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[10] 2.000000 - 5.000000 : 2252 2024-09-12T03:28:20.160 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[11] 1.000000 - 2.000000 : 1867 2024-09-12T03:28:20.160 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[12] 0.500000 - 1.000000 : 1477 2024-09-12T03:28:20.160 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[13] 0.200000 - 0.500000 : 1594 2024-09-12T03:28:20.160 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[14] 0.100000 - 0.200000 : 2452 2024-09-12T03:28:20.160 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[15] 0.050000 - 0.100000 : 8632 2024-09-12T03:28:20.160 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[16] 0.020000 - 0.050000 : 43766 2024-09-12T03:28:20.160 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[17] 0.010000 - 0.020000 : 133672 2024-09-12T03:28:20.160 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[18] 0.000000 - 0.010000 : 23705290 2024-09-12T03:28:20.160 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-09-12T03:28:20.160 INFO:tasks.workunit.client.0.smithi006.stdout:[ lseek] 0.000000 0.000805 92.403999 23939840 2024-09-12T03:28:20.160 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-12T03:28:20.160 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-12T03:28:20.161 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-12T03:28:20.161 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-12T03:28:20.161 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-12T03:28:20.161 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-09-12T03:28:20.161 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[6] 50.000000 - 100.000000 : 2 2024-09-12T03:28:20.161 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[7] 20.000000 - 50.000000 : 72 2024-09-12T03:28:20.161 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[8] 10.000000 - 20.000000 : 74 2024-09-12T03:28:20.161 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[9] 5.000000 - 10.000000 : 64 2024-09-12T03:28:20.161 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[10] 2.000000 - 5.000000 : 142 2024-09-12T03:28:20.161 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[11] 1.000000 - 2.000000 : 135 2024-09-12T03:28:20.161 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[12] 0.500000 - 1.000000 : 173 2024-09-12T03:28:20.161 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[13] 0.200000 - 0.500000 : 185 2024-09-12T03:28:20.161 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[14] 0.100000 - 0.200000 : 264 2024-09-12T03:28:20.161 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[15] 0.050000 - 0.100000 : 733 2024-09-12T03:28:20.162 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[16] 0.020000 - 0.050000 : 4216 2024-09-12T03:28:20.162 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[17] 0.010000 - 0.020000 : 11472 2024-09-12T03:28:20.162 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[18] 0.000000 - 0.010000 : 23922308 2024-09-12T03:28:20.162 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-09-12T03:28:20.162 INFO:tasks.workunit.client.0.smithi006.stdout:[ close] 0.001000 0.006691 21.013000 18703 2024-09-12T03:28:20.162 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-12T03:28:20.162 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-12T03:28:20.162 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-12T03:28:20.162 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-12T03:28:20.162 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-12T03:28:20.162 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-09-12T03:28:20.162 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-09-12T03:28:20.162 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[7] 20.000000 - 50.000000 : 1 2024-09-12T03:28:20.162 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2024-09-12T03:28:20.163 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[9] 5.000000 - 10.000000 : 0 2024-09-12T03:28:20.163 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[10] 2.000000 - 5.000000 : 1 2024-09-12T03:28:20.163 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[11] 1.000000 - 2.000000 : 2 2024-09-12T03:28:20.163 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[12] 0.500000 - 1.000000 : 2 2024-09-12T03:28:20.163 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[13] 0.200000 - 0.500000 : 3 2024-09-12T03:28:20.163 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[14] 0.100000 - 0.200000 : 1 2024-09-12T03:28:20.163 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[15] 0.050000 - 0.100000 : 6 2024-09-12T03:28:20.163 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[16] 0.020000 - 0.050000 : 61 2024-09-12T03:28:20.163 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[17] 0.010000 - 0.020000 : 169 2024-09-12T03:28:20.163 INFO:tasks.workunit.client.0.smithi006.stdout: msec_range[18] 0.000000 - 0.010000 : 18457 2024-09-12T03:28:20.163 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-09-12T03:28:20.163 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-09-12T03:28:20.163 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-09-12T03:28:20.163 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-09-12T03:28:20.163 INFO:tasks.workunit.client.0.smithi006.stdout:6.6% User Time 2024-09-12T03:28:20.164 INFO:tasks.workunit.client.0.smithi006.stdout:21.6% System Time 2024-09-12T03:28:20.164 INFO:tasks.workunit.client.0.smithi006.stdout:28.2% CPU Utilization 2024-09-12T03:28:20.175 INFO:tasks.workunit.client.0.smithi006.stderr:+ cd .. 2024-09-12T03:28:20.179 INFO:tasks.workunit.client.0.smithi006.stderr:+ rm -r tmp ffsb 2024-09-12T03:28:20.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:20 smithi118 ceph-mon[35048]: pgmap v552: 97 pgs: 97 active+clean; 13 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 10 KiB/s rd, 97 MiB/s wr, 16.80k op/s 2024-09-12T03:28:20.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:20 smithi118 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-12T03:28:20.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:20 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:20.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:20 smithi118 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-12T03:28:20.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:20 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:20.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:20 smithi118 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:21.022 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:20 smithi006 ceph-mon[30991]: pgmap v552: 97 pgs: 97 active+clean; 13 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 10 KiB/s rd, 97 MiB/s wr, 16.80k op/s 2024-09-12T03:28:21.022 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:20 smithi006 ceph-mon[30991]: scrub summary: active paths [/] 2024-09-12T03:28:21.023 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:20 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:21.023 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:20 smithi006 ceph-mon[30991]: scrub summary: active paths [/] 2024-09-12T03:28:21.023 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:20 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:21.023 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:20 smithi006 ceph-mon[30991]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:21.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:20 smithi161 ceph-mon[35485]: pgmap v552: 97 pgs: 97 active+clean; 13 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 10 KiB/s rd, 97 MiB/s wr, 16.80k op/s 2024-09-12T03:28:21.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:20 smithi161 ceph-mon[35485]: scrub summary: active paths [/] 2024-09-12T03:28:21.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:20 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:21.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:20 smithi161 ceph-mon[35485]: scrub summary: active paths [/] 2024-09-12T03:28:21.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:20 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:21.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:20 smithi161 ceph-mon[35485]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:21.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.412+0000 7fe370542640 1 -- 172.21.15.6:0/3760747571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe368102570 msgr2=0x7fe3681029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:21.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.412+0000 7fe370542640 1 --2- 172.21.15.6:0/3760747571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe368102570 0x7fe3681029b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe358009900 tx=0x7fe35802a480 comp rx=0 tx=0).stop 2024-09-12T03:28:21.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.412+0000 7fe370542640 1 -- 172.21.15.6:0/3760747571 shutdown_connections 2024-09-12T03:28:21.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.412+0000 7fe370542640 1 --2- 172.21.15.6:0/3760747571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe368102ef0 0x7fe36810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.412+0000 7fe370542640 1 --2- 172.21.15.6:0/3760747571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe368102570 0x7fe3681029b0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.412+0000 7fe370542640 1 --2- 172.21.15.6:0/3760747571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe368100e20 0x7fe368101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.412+0000 7fe370542640 1 -- 172.21.15.6:0/3760747571 >> 172.21.15.6:0/3760747571 conn(0x7fe3680fc7d0 msgr2=0x7fe3680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:21.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.412+0000 7fe370542640 1 -- 172.21.15.6:0/3760747571 shutdown_connections 2024-09-12T03:28:21.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.412+0000 7fe370542640 1 -- 172.21.15.6:0/3760747571 wait complete. 2024-09-12T03:28:21.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.413+0000 7fe370542640 1 Processor -- start 2024-09-12T03:28:21.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.413+0000 7fe370542640 1 -- start start 2024-09-12T03:28:21.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.413+0000 7fe370542640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe368100e20 0x7fe3681a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.413+0000 7fe370542640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe368102570 0x7fe3681a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.413+0000 7fe370542640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe368102ef0 0x7fe36819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.413+0000 7fe370542640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe368110f80 con 0x7fe368102570 2024-09-12T03:28:21.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.413+0000 7fe370542640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe368110e00 con 0x7fe368102ef0 2024-09-12T03:28:21.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.413+0000 7fe370542640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe368111100 con 0x7fe368100e20 2024-09-12T03:28:21.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.413+0000 7fe36dadd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe368102570 0x7fe3681a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.413+0000 7fe36dadd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe368102570 0x7fe3681a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53150/0 (socket says 172.21.15.6:53150) 2024-09-12T03:28:21.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.413+0000 7fe36e2de640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe368100e20 0x7fe3681a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.413+0000 7fe36dadd640 1 -- 172.21.15.6:0/1601086006 learned_addr learned my addr 172.21.15.6:0/1601086006 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:21.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.413+0000 7fe36eadf640 1 --2- 172.21.15.6:0/1601086006 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe368102ef0 0x7fe36819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.413+0000 7fe36dadd640 1 -- 172.21.15.6:0/1601086006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe368100e20 msgr2=0x7fe3681a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:21.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.413+0000 7fe36dadd640 1 --2- 172.21.15.6:0/1601086006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe368100e20 0x7fe3681a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.413+0000 7fe36dadd640 1 -- 172.21.15.6:0/1601086006 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe368102ef0 msgr2=0x7fe36819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:21.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.413+0000 7fe36dadd640 1 --2- 172.21.15.6:0/1601086006 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe368102ef0 0x7fe36819a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.413+0000 7fe36dadd640 1 -- 172.21.15.6:0/1601086006 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36819acc0 con 0x7fe368102570 2024-09-12T03:28:21.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.413+0000 7fe36e2de640 1 --2- 172.21.15.6:0/1601086006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe368100e20 0x7fe3681a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:21.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.414+0000 7fe36dadd640 1 --2- 172.21.15.6:0/1601086006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe368102570 0x7fe3681a0910 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe358006010 tx=0x7fe358003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:21.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.414+0000 7fe35f7fe640 1 -- 172.21.15.6:0/1601086006 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35803f070 con 0x7fe368102570 2024-09-12T03:28:21.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.414+0000 7fe370542640 1 -- 172.21.15.6:0/1601086006 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe36819af50 con 0x7fe368102570 2024-09-12T03:28:21.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.414+0000 7fe35f7fe640 1 -- 172.21.15.6:0/1601086006 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe358031c30 con 0x7fe368102570 2024-09-12T03:28:21.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.414+0000 7fe35f7fe640 1 -- 172.21.15.6:0/1601086006 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe358035030 con 0x7fe368102570 2024-09-12T03:28:21.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.414+0000 7fe370542640 1 -- 172.21.15.6:0/1601086006 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3681a72a0 con 0x7fe368102570 2024-09-12T03:28:21.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.416+0000 7fe35f7fe640 1 -- 172.21.15.6:0/1601086006 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe358042070 con 0x7fe368102570 2024-09-12T03:28:21.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.417+0000 7fe37291b740 1 Processor -- start 2024-09-12T03:28:21.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.417+0000 7fe37291b740 1 -- start start 2024-09-12T03:28:21.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.417+0000 7fe37291b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e734a0 0x557284e73880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.417+0000 7fe37291b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557284e73dc0 0x557284e7e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.417+0000 7fe37291b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e7e680 0x557284e80a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.417+0000 7fe35f7fe640 1 --2- 172.21.15.6:0/1601086006 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe3540a2b80 0x7fe3540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.417+0000 7fe36eadf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e734a0 0x557284e73880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.417+0000 7fe37291b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x557284dd92e0 con 0x557284e7e680 2024-09-12T03:28:21.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.417+0000 7fe37291b740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x557284d80ce0 con 0x557284e73dc0 2024-09-12T03:28:21.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.417+0000 7fe37291b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557284dd52a0 con 0x557284e734a0 2024-09-12T03:28:21.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.417+0000 7fe36eadf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e734a0 0x557284e73880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:35370/0 (socket says 172.21.15.6:35370) 2024-09-12T03:28:21.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.417+0000 7fe36eadf640 1 -- 172.21.15.6:0/1673163233 learned_addr learned my addr 172.21.15.6:0/1673163233 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:21.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.417+0000 7fe36dadd640 1 --2- 172.21.15.6:0/1673163233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e7e680 0x557284e80a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.417+0000 7fe36e2de640 1 --2- 172.21.15.6:0/1673163233 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557284e73dc0 0x557284e7e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.417+0000 7fe36eadf640 1 --2- 172.21.15.6:0/1601086006 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe3540a2b80 0x7fe3540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.418+0000 7fe35f7fe640 1 -- 172.21.15.6:0/1601086006 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe3580ef710 con 0x7fe368102570 2024-09-12T03:28:21.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.418+0000 7fe36eadf640 1 -- 172.21.15.6:0/1673163233 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557284e73dc0 msgr2=0x557284e7e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:21.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.418+0000 7fe36eadf640 1 --2- 172.21.15.6:0/1673163233 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557284e73dc0 0x557284e7e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.418+0000 7fe36eadf640 1 -- 172.21.15.6:0/1673163233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e7e680 msgr2=0x557284e80a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:21.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.418+0000 7fe36eadf640 1 --2- 172.21.15.6:0/1673163233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e7e680 0x557284e80a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.418+0000 7fe36eadf640 1 -- 172.21.15.6:0/1673163233 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557284d65440 con 0x557284e734a0 2024-09-12T03:28:21.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.418+0000 7fe36eadf640 1 --2- 172.21.15.6:0/1673163233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e734a0 0x557284e73880 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe360009370 tx=0x7fe3600117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2da88f6a741ff55 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:21.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.418+0000 7fe36eadf640 1 --2- 172.21.15.6:0/1601086006 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe3540a2b80 0x7fe3540a5040 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe360011cb0 tx=0x7fe360016000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:21.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.418+0000 7fe35cff9640 1 -- 172.21.15.6:0/1673163233 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360009c40 con 0x557284e734a0 2024-09-12T03:28:21.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.418+0000 7fe35cff9640 1 -- 172.21.15.6:0/1673163233 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe360003e50 con 0x557284e734a0 2024-09-12T03:28:21.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.418+0000 7fe35cff9640 1 -- 172.21.15.6:0/1673163233 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360004030 con 0x557284e734a0 2024-09-12T03:28:21.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.419+0000 7fe37291b740 1 -- 172.21.15.6:0/1673163233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e734a0 msgr2=0x557284e73880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:21.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.419+0000 7fe37291b740 1 --2- 172.21.15.6:0/1673163233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e734a0 0x557284e73880 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe360009370 tx=0x7fe3600117b0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.419+0000 7fe37291b740 1 -- 172.21.15.6:0/1673163233 shutdown_connections 2024-09-12T03:28:21.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.419+0000 7fe37291b740 1 --2- 172.21.15.6:0/1673163233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e7e680 0x557284e80a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.419+0000 7fe37291b740 1 --2- 172.21.15.6:0/1673163233 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557284e73dc0 0x557284e7e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.419+0000 7fe37291b740 1 --2- 172.21.15.6:0/1673163233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e734a0 0x557284e73880 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.419+0000 7fe37291b740 1 -- 172.21.15.6:0/1673163233 >> 172.21.15.6:0/1673163233 conn(0x557284e6fa30 msgr2=0x557284e6fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:21.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.419+0000 7fe37291b740 1 -- 172.21.15.6:0/1673163233 shutdown_connections 2024-09-12T03:28:21.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.419+0000 7fe37291b740 1 -- 172.21.15.6:0/1673163233 wait complete. 2024-09-12T03:28:21.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.420+0000 7fe37291b740 1 -- start start 2024-09-12T03:28:21.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.420+0000 7fe37291b740 1 Processor -- start 2024-09-12T03:28:21.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.420+0000 7fe37291b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557284e734a0 0x557284e7e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.420+0000 7fe37291b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e73dc0 0x557284e7b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.420+0000 7fe37291b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e7e680 0x557284e7b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.420+0000 7fe37291b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x557284e81250 con 0x557284e7e680 2024-09-12T03:28:21.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.420+0000 7fe37291b740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x557284d94050 con 0x557284e734a0 2024-09-12T03:28:21.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.420+0000 7fe37291b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557284e813d0 con 0x557284e73dc0 2024-09-12T03:28:21.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.420+0000 7fe37291b740 10 client.0 _ll_register_callbacks cb 0x557284dd4160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:21.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.420+0000 7fe36dadd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e73dc0 0x557284e7b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.420+0000 7fe36e2de640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557284e734a0 0x557284e7e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.420+0000 7fe36eadf640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e7e680 0x557284e7b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.420+0000 7fe36dadd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e73dc0 0x557284e7b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:35376/0 (socket says 172.21.15.6:35376) 2024-09-12T03:28:21.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.420+0000 7fe36dadd640 1 -- 172.21.15.6:0/3101374211 learned_addr learned my addr 172.21.15.6:0/3101374211 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:21.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.420+0000 7fe36eadf640 1 --2- 172.21.15.6:0/3101374211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e7e680 0x557284e7b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53166/0 (socket says 172.21.15.6:53166) 2024-09-12T03:28:21.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.421+0000 7fe36dadd640 1 -- 172.21.15.6:0/3101374211 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557284e734a0 msgr2=0x557284e7e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:21.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.421+0000 7fe36dadd640 1 --2- 172.21.15.6:0/3101374211 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557284e734a0 0x557284e7e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.421+0000 7fe36dadd640 1 -- 172.21.15.6:0/3101374211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e7e680 msgr2=0x557284e7b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:21.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.421+0000 7fe36dadd640 1 --2- 172.21.15.6:0/3101374211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e7e680 0x557284e7b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.421+0000 7fe36dadd640 1 -- 172.21.15.6:0/3101374211 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557284b4a3c0 con 0x557284e73dc0 2024-09-12T03:28:21.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.421+0000 7fe36dadd640 1 --2- 172.21.15.6:0/3101374211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e73dc0 0x557284e7b330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe3580b8970 tx=0x7fe3580870c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=422d3672c3ef9f13 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:21.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.421+0000 7fe351ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:28:21.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.421+0000 7fe351ffb640 1 -- 172.21.15.6:0/3101374211 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe358049070 con 0x557284e73dc0 2024-09-12T03:28:21.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.421+0000 7fe37291b740 1 -- 172.21.15.6:0/3101374211 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe360003e50 con 0x557284e73dc0 2024-09-12T03:28:21.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.421+0000 7fe351ffb640 1 -- 172.21.15.6:0/3101374211 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe3580b8610 con 0x557284e73dc0 2024-09-12T03:28:21.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.421+0000 7fe351ffb640 1 -- 172.21.15.6:0/3101374211 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe3580bb900 con 0x557284e73dc0 2024-09-12T03:28:21.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.421+0000 7fe37291b740 10 client.25804 fetch_fsmap learned FSMap version 27 2024-09-12T03:28:21.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.421+0000 7fe37291b740 1 -- 172.21.15.6:0/3101374211 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557284e812b0 con 0x557284e73dc0 2024-09-12T03:28:21.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.421+0000 7fe351ffb640 1 -- 172.21.15.6:0/3101374211 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe360003e50 con 0x557284e73dc0 2024-09-12T03:28:21.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.422+0000 7fe351ffb640 1 -- 172.21.15.6:0/3101374211 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe3580e4620 con 0x557284e73dc0 2024-09-12T03:28:21.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.422+0000 7fe37291b740 10 client.25804 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:28:21.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.422+0000 7fe37291b740 10 client.25804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:28:21.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.422+0000 7fe37291b740 20 client.25804 populate_metadata read hostname 'smithi006' 2024-09-12T03:28:21.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.422+0000 7fe37291b740 1 --2- 172.21.15.6:0/3101374211 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557284e8e0e0 0x557284e90500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.422+0000 7fe37291b740 4 client.25804 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:21.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.422+0000 7fe37291b740 1 -- 172.21.15.6:0/3101374211 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557284de6850 con 0x557284e8e0e0 2024-09-12T03:28:21.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.422+0000 7fe36e2de640 1 --2- 172.21.15.6:0/3101374211 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557284e8e0e0 0x557284e90500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.422+0000 7fe351ffb640 10 client.25804 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:21.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.422+0000 7fe36e2de640 1 --2- 172.21.15.6:0/3101374211 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557284e8e0e0 0x557284e90500 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b03f03cfb12d0a74 server_cookie=5e0f7d7d1db3279 in_seq=0 out_seq=0 2024-09-12T03:28:21.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.423+0000 7fe351ffb640 1 -- 172.21.15.6:0/3101374211 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557284de6850 con 0x557284e8e0e0 2024-09-12T03:28:21.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.423+0000 7fe351ffb640 10 client.25804 handle_command_reply: tid=0 2024-09-12T03:28:21.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.423+0000 7fe37291b740 1 client.25804 shutdown 2024-09-12T03:28:21.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.423+0000 7fe37291b740 1 -- 172.21.15.6:0/3101374211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e73dc0 msgr2=0x557284e7b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:21.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.423+0000 7fe37291b740 1 --2- 172.21.15.6:0/3101374211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e73dc0 0x557284e7b330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe3580b8970 tx=0x7fe3580870c0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.423+0000 7fe37291b740 1 -- 172.21.15.6:0/3101374211 shutdown_connections 2024-09-12T03:28:21.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.423+0000 7fe37291b740 1 --2- 172.21.15.6:0/3101374211 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557284e8e0e0 0x557284e90500 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.423+0000 7fe37291b740 1 --2- 172.21.15.6:0/3101374211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e7e680 0x557284e7b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.423+0000 7fe37291b740 1 --2- 172.21.15.6:0/3101374211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e73dc0 0x557284e7b330 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.423+0000 7fe351ffb640 0 client.25804 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:21.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.423+0000 7fe37291b740 1 --2- 172.21.15.6:0/3101374211 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557284e734a0 0x557284e7e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.423+0000 7fe37291b740 1 -- 172.21.15.6:0/3101374211 >> 172.21.15.6:0/3101374211 conn(0x557284e6fa30 msgr2=0x557284e75760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:21.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.424+0000 7fe37291b740 1 -- 172.21.15.6:0/3101374211 shutdown_connections 2024-09-12T03:28:21.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.424+0000 7fe37291b740 1 -- 172.21.15.6:0/3101374211 wait complete. 2024-09-12T03:28:21.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.424+0000 7fe37291b740 20 client.25804 trim_cache size 0 max 16384 2024-09-12T03:28:21.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.442+0000 7fe37291b740 1 Processor -- start 2024-09-12T03:28:21.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.442+0000 7fe37291b740 1 -- start start 2024-09-12T03:28:21.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.442+0000 7fe37291b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557284e734a0 0x557284e8a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.442+0000 7fe37291b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e73dc0 0x557284e95e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.442+0000 7fe37291b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e7e680 0x557284e96350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.442+0000 7fe37291b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681b1f30 con 0x557284e7e680 2024-09-12T03:28:21.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.442+0000 7fe37291b740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681af800 con 0x557284e734a0 2024-09-12T03:28:21.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.442+0000 7fe37291b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681ad650 con 0x557284e73dc0 2024-09-12T03:28:21.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.442+0000 7fe36eadf640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e7e680 0x557284e96350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.442+0000 7fe36dadd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e73dc0 0x557284e95e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.442+0000 7fe36eadf640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e7e680 0x557284e96350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53174/0 (socket says 172.21.15.6:53174) 2024-09-12T03:28:21.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.442+0000 7fe36eadf640 1 -- 172.21.15.6:0/1467173715 learned_addr learned my addr 172.21.15.6:0/1467173715 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:21.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.442+0000 7fe36e2de640 1 --2- 172.21.15.6:0/1467173715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557284e734a0 0x557284e8a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.443+0000 7fe36eadf640 1 -- 172.21.15.6:0/1467173715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e73dc0 msgr2=0x557284e95e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:21.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.443+0000 7fe36eadf640 1 --2- 172.21.15.6:0/1467173715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e73dc0 0x557284e95e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.443+0000 7fe36eadf640 1 -- 172.21.15.6:0/1467173715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557284e734a0 msgr2=0x557284e8a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:21.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.443+0000 7fe36eadf640 1 --2- 172.21.15.6:0/1467173715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557284e734a0 0x557284e8a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.443+0000 7fe36eadf640 1 -- 172.21.15.6:0/1467173715 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557284e90b80 con 0x557284e7e680 2024-09-12T03:28:21.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.443+0000 7fe36dadd640 1 --2- 172.21.15.6:0/1467173715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e73dc0 0x557284e95e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:21.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.443+0000 7fe36eadf640 1 --2- 172.21.15.6:0/1467173715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e7e680 0x557284e96350 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe360014dc0 tx=0x7fe36000bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4066240ae1f91330 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:21.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.443+0000 7fe353fff640 1 -- 172.21.15.6:0/1467173715 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36000bd40 con 0x557284e7e680 2024-09-12T03:28:21.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.443+0000 7fe353fff640 1 -- 172.21.15.6:0/1467173715 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe36002f070 con 0x557284e7e680 2024-09-12T03:28:21.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.443+0000 7fe353fff640 1 -- 172.21.15.6:0/1467173715 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3600050b0 con 0x557284e7e680 2024-09-12T03:28:21.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.444+0000 7fe37291b740 1 -- 172.21.15.6:0/1467173715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e7e680 msgr2=0x557284e96350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:21.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.444+0000 7fe37291b740 1 --2- 172.21.15.6:0/1467173715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e7e680 0x557284e96350 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe360014dc0 tx=0x7fe36000bbf0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.444+0000 7fe37291b740 1 -- 172.21.15.6:0/1467173715 shutdown_connections 2024-09-12T03:28:21.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.444+0000 7fe37291b740 1 --2- 172.21.15.6:0/1467173715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e7e680 0x557284e96350 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.444+0000 7fe37291b740 1 --2- 172.21.15.6:0/1467173715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e73dc0 0x557284e95e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.444+0000 7fe37291b740 1 --2- 172.21.15.6:0/1467173715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557284e734a0 0x557284e8a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.444+0000 7fe37291b740 1 -- 172.21.15.6:0/1467173715 >> 172.21.15.6:0/1467173715 conn(0x557284e6fa30 msgr2=0x557284e76900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:21.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.444+0000 7fe37291b740 1 -- 172.21.15.6:0/1467173715 shutdown_connections 2024-09-12T03:28:21.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.444+0000 7fe37291b740 1 -- 172.21.15.6:0/1467173715 wait complete. 2024-09-12T03:28:21.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.444+0000 7fe37291b740 1 -- start start 2024-09-12T03:28:21.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.444+0000 7fe37291b740 1 Processor -- start 2024-09-12T03:28:21.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.445+0000 7fe37291b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e734a0 0x557284e82380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.445+0000 7fe37291b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557284e73dc0 0x557284e828c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.445+0000 7fe37291b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e7e680 0x557284e82e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.445+0000 7fe37291b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681adcc0 con 0x557284e734a0 2024-09-12T03:28:21.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.445+0000 7fe37291b740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681af130 con 0x557284e73dc0 2024-09-12T03:28:21.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.445+0000 7fe37291b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681ae390 con 0x557284e7e680 2024-09-12T03:28:21.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.445+0000 7fe37291b740 10 client.0 _ll_register_callbacks cb 0x7fe3580b8240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:21.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.445+0000 7fe36e2de640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e734a0 0x557284e82380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.445+0000 7fe36eadf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e7e680 0x557284e82e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.445+0000 7fe36e2de640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e734a0 0x557284e82380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53190/0 (socket says 172.21.15.6:53190) 2024-09-12T03:28:21.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.445+0000 7fe36eadf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e7e680 0x557284e82e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:35386/0 (socket says 172.21.15.6:35386) 2024-09-12T03:28:21.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.445+0000 7fe36e2de640 1 -- 172.21.15.6:0/177394246 learned_addr learned my addr 172.21.15.6:0/177394246 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:21.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.445+0000 7fe36eadf640 1 -- 172.21.15.6:0/177394246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557284e73dc0 msgr2=0x557284e828c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:21.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.445+0000 7fe36eadf640 1 --2- 172.21.15.6:0/177394246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557284e73dc0 0x557284e828c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.445+0000 7fe36eadf640 1 -- 172.21.15.6:0/177394246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e734a0 msgr2=0x557284e82380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:21.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.445+0000 7fe36eadf640 1 --2- 172.21.15.6:0/177394246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e734a0 0x557284e82380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.445+0000 7fe36eadf640 1 -- 172.21.15.6:0/177394246 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557284ef7de0 con 0x557284e7e680 2024-09-12T03:28:21.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.445+0000 7fe36eadf640 1 --2- 172.21.15.6:0/177394246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e7e680 0x557284e82e00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe360014700 tx=0x7fe36000bd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a8290e279e225d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:21.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.445+0000 7fe351ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:28:21.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.445+0000 7fe351ffb640 1 -- 172.21.15.6:0/177394246 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360014a60 con 0x557284e7e680 2024-09-12T03:28:21.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.446+0000 7fe351ffb640 1 -- 172.21.15.6:0/177394246 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe360003970 con 0x557284e7e680 2024-09-12T03:28:21.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.446+0000 7fe37291b740 1 -- 172.21.15.6:0/177394246 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe36002f070 con 0x557284e7e680 2024-09-12T03:28:21.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.446+0000 7fe351ffb640 1 -- 172.21.15.6:0/177394246 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe360003b70 con 0x557284e7e680 2024-09-12T03:28:21.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.446+0000 7fe37291b740 10 client.25810 fetch_fsmap learned FSMap version 27 2024-09-12T03:28:21.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.446+0000 7fe37291b740 1 -- 172.21.15.6:0/177394246 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557284ef90d0 con 0x557284e7e680 2024-09-12T03:28:21.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.446+0000 7fe351ffb640 1 -- 172.21.15.6:0/177394246 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe36002f070 con 0x557284e7e680 2024-09-12T03:28:21.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.446+0000 7fe351ffb640 1 -- 172.21.15.6:0/177394246 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe360034470 con 0x557284e7e680 2024-09-12T03:28:21.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.446+0000 7fe37291b740 10 client.25810 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:28:21.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.446+0000 7fe37291b740 10 client.25810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:28:21.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.446+0000 7fe37291b740 20 client.25810 populate_metadata read hostname 'smithi006' 2024-09-12T03:28:21.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.446+0000 7fe37291b740 1 --2- 172.21.15.6:0/177394246 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557284ef9600 0x557284efb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.446+0000 7fe37291b740 4 client.25810 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:28:21.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.446+0000 7fe37291b740 1 -- 172.21.15.6:0/177394246 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557284de0d60 con 0x557284ef9600 2024-09-12T03:28:21.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.447+0000 7fe36e2de640 1 --2- 172.21.15.6:0/177394246 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557284ef9600 0x557284efb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.447+0000 7fe36e2de640 1 --2- 172.21.15.6:0/177394246 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557284ef9600 0x557284efb9f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67bde0d326f0a8a6 server_cookie=d2105f8619a096b2 in_seq=0 out_seq=0 2024-09-12T03:28:21.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.447+0000 7fe351ffb640 10 client.25810 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:21.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.447+0000 7fe351ffb640 1 -- 172.21.15.6:0/177394246 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557284de0d60 con 0x557284ef9600 2024-09-12T03:28:21.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.447+0000 7fe351ffb640 10 client.25810 handle_command_reply: tid=0 2024-09-12T03:28:21.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.447+0000 7fe37291b740 1 client.25810 shutdown 2024-09-12T03:28:21.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.447+0000 7fe37291b740 1 -- 172.21.15.6:0/177394246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e7e680 msgr2=0x557284e82e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:21.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.447+0000 7fe37291b740 1 --2- 172.21.15.6:0/177394246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e7e680 0x557284e82e00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe360014700 tx=0x7fe36000bd10 comp rx=0 tx=0).stop 2024-09-12T03:28:21.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.447+0000 7fe37291b740 1 -- 172.21.15.6:0/177394246 shutdown_connections 2024-09-12T03:28:21.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.447+0000 7fe37291b740 1 --2- 172.21.15.6:0/177394246 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557284ef9600 0x557284efb9f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.447+0000 7fe37291b740 1 --2- 172.21.15.6:0/177394246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557284e7e680 0x557284e82e00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.447+0000 7fe37291b740 1 --2- 172.21.15.6:0/177394246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557284e73dc0 0x557284e828c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.447+0000 7fe351ffb640 0 client.25810 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:21.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.447+0000 7fe37291b740 1 --2- 172.21.15.6:0/177394246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557284e734a0 0x557284e82380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.447+0000 7fe37291b740 1 -- 172.21.15.6:0/177394246 >> 172.21.15.6:0/177394246 conn(0x557284e6fa30 msgr2=0x557284e7cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:21.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.447+0000 7fe37291b740 1 -- 172.21.15.6:0/177394246 shutdown_connections 2024-09-12T03:28:21.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.447+0000 7fe37291b740 1 -- 172.21.15.6:0/177394246 wait complete. 2024-09-12T03:28:21.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.448+0000 7fe37291b740 20 client.25810 trim_cache size 0 max 16384 2024-09-12T03:28:21.453 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:28:21.453 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:28:21.453 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2024-09-12T03:28:21.454 INFO:teuthology.orchestra.run.smithi006.stdout: "838db3dd-096f-4ad3-9d4d-00a5fbaa8642": { 2024-09-12T03:28:21.454 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-09-12T03:28:21.454 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "838db3dd-096f-4ad3-9d4d-00a5fbaa8642", 2024-09-12T03:28:21.454 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2024-09-12T03:28:21.454 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:28:21.454 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:28:21.454 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:28:21.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.448+0000 7fe370542640 1 -- 172.21.15.6:0/1601086006 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe3540a2b80 msgr2=0x7fe3540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:21.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.448+0000 7fe370542640 1 --2- 172.21.15.6:0/1601086006 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe3540a2b80 0x7fe3540a5040 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe360011cb0 tx=0x7fe360016000 comp rx=0 tx=0).stop 2024-09-12T03:28:21.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.448+0000 7fe370542640 1 -- 172.21.15.6:0/1601086006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe368102570 msgr2=0x7fe3681a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:21.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.448+0000 7fe370542640 1 --2- 172.21.15.6:0/1601086006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe368102570 0x7fe3681a0910 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe358006010 tx=0x7fe358003c10 comp rx=0 tx=0).stop 2024-09-12T03:28:21.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.448+0000 7fe370542640 1 -- 172.21.15.6:0/1601086006 shutdown_connections 2024-09-12T03:28:21.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.448+0000 7fe370542640 1 --2- 172.21.15.6:0/1601086006 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fe3540a2b80 0x7fe3540a5040 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.448+0000 7fe370542640 1 --2- 172.21.15.6:0/1601086006 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe368102ef0 0x7fe36819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.448+0000 7fe370542640 1 --2- 172.21.15.6:0/1601086006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe368102570 0x7fe3681a0910 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.448+0000 7fe370542640 1 --2- 172.21.15.6:0/1601086006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe368100e20 0x7fe3681a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.448+0000 7fe370542640 1 -- 172.21.15.6:0/1601086006 >> 172.21.15.6:0/1601086006 conn(0x7fe3680fc7d0 msgr2=0x7fe36806a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:21.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.448+0000 7fe370542640 1 -- 172.21.15.6:0/1601086006 shutdown_connections 2024-09-12T03:28:21.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:21.448+0000 7fe370542640 1 -- 172.21.15.6:0/1601086006 wait complete. 2024-09-12T03:28:22.134 INFO:tasks.cephfs.filesystem:scrub status for tag:838db3dd-096f-4ad3-9d4d-00a5fbaa8642 - {'path': '/', 'tag': '838db3dd-096f-4ad3-9d4d-00a5fbaa8642', 'options': 'recursive,force'} 2024-09-12T03:28:22.710 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7867f508', quiesce took 26.2 seconds. Will release after: -9.599999999999998 2024-09-12T03:28:22.710 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:28:22.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:22 smithi118 ceph-mon[35048]: pgmap v553: 97 pgs: 97 active+clean; 13 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 11 KiB/s rd, 101 MiB/s wr, 17.92k op/s 2024-09-12T03:28:22.970 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:23.091 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:22 smithi006 ceph-mon[30991]: pgmap v553: 97 pgs: 97 active+clean; 13 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 11 KiB/s rd, 101 MiB/s wr, 17.92k op/s 2024-09-12T03:28:23.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:22 smithi161 ceph-mon[35485]: pgmap v553: 97 pgs: 97 active+clean; 13 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 11 KiB/s rd, 101 MiB/s wr, 17.92k op/s 2024-09-12T03:28:23.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.429+0000 7fdefb5cd640 1 -- 172.21.15.6:0/2304280765 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdef410c170 msgr2=0x7fdef410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:23.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.429+0000 7fdefb5cd640 1 --2- 172.21.15.6:0/2304280765 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdef410c170 0x7fdef410e610 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009ca0 tx=0x7fdeec02a950 comp rx=0 tx=0).stop 2024-09-12T03:28:23.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.429+0000 7fdefb5cd640 1 -- 172.21.15.6:0/2304280765 shutdown_connections 2024-09-12T03:28:23.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.429+0000 7fdefb5cd640 1 --2- 172.21.15.6:0/2304280765 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdef410c170 0x7fdef410e610 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.429+0000 7fdefb5cd640 1 --2- 172.21.15.6:0/2304280765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdef4103660 0x7fdef410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.429+0000 7fdefb5cd640 1 --2- 172.21.15.6:0/2304280765 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdef4102d40 0x7fdef4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.429+0000 7fdefb5cd640 1 -- 172.21.15.6:0/2304280765 >> 172.21.15.6:0/2304280765 conn(0x7fdef40fc7a0 msgr2=0x7fdef40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:23.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.429+0000 7fdefb5cd640 1 -- 172.21.15.6:0/2304280765 shutdown_connections 2024-09-12T03:28:23.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.429+0000 7fdefb5cd640 1 -- 172.21.15.6:0/2304280765 wait complete. 2024-09-12T03:28:23.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.430+0000 7fdefb5cd640 1 Processor -- start 2024-09-12T03:28:23.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.430+0000 7fdefb5cd640 1 -- start start 2024-09-12T03:28:23.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.430+0000 7fdefb5cd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdef4102d40 0x7fdef4198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:23.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.430+0000 7fdefb5cd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdef4103660 0x7fdef4198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:23.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.430+0000 7fdefb5cd640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdef410c170 0x7fdef419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:23.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.430+0000 7fdefb5cd640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdef4111fd0 con 0x7fdef4103660 2024-09-12T03:28:23.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.430+0000 7fdefb5cd640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fdef4111e50 con 0x7fdef410c170 2024-09-12T03:28:23.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.430+0000 7fdefb5cd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdef4112150 con 0x7fdef4102d40 2024-09-12T03:28:23.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.430+0000 7fdef8b68640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdef4103660 0x7fdef4198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:23.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.430+0000 7fdef8b68640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdef4103660 0x7fdef4198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53212/0 (socket says 172.21.15.6:53212) 2024-09-12T03:28:23.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.430+0000 7fdef8b68640 1 -- 172.21.15.6:0/616688872 learned_addr learned my addr 172.21.15.6:0/616688872 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:23.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.430+0000 7fdef9369640 1 --2- 172.21.15.6:0/616688872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdef4102d40 0x7fdef4198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:23.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.430+0000 7fdef9b6a640 1 --2- 172.21.15.6:0/616688872 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdef410c170 0x7fdef419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.430+0000 7fdef8b68640 1 -- 172.21.15.6:0/616688872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdef4102d40 msgr2=0x7fdef4198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.430+0000 7fdef8b68640 1 --2- 172.21.15.6:0/616688872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdef4102d40 0x7fdef4198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.430+0000 7fdef8b68640 1 -- 172.21.15.6:0/616688872 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdef410c170 msgr2=0x7fdef419ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.430+0000 7fdef8b68640 1 --2- 172.21.15.6:0/616688872 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdef410c170 0x7fdef419ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.430+0000 7fdef8b68640 1 -- 172.21.15.6:0/616688872 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef419d4e0 con 0x7fdef4103660 2024-09-12T03:28:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.430+0000 7fdef9b6a640 1 --2- 172.21.15.6:0/616688872 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdef410c170 0x7fdef419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.431+0000 7fdef8b68640 1 --2- 172.21.15.6:0/616688872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdef4103660 0x7fdef4198a70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdee400e9f0 tx=0x7fdee400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.431+0000 7fdeea7fc640 1 -- 172.21.15.6:0/616688872 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee400cab0 con 0x7fdef4103660 2024-09-12T03:28:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.431+0000 7fdeea7fc640 1 -- 172.21.15.6:0/616688872 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdee4005500 con 0x7fdef4103660 2024-09-12T03:28:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.431+0000 7fdefb5cd640 1 -- 172.21.15.6:0/616688872 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef419d7d0 con 0x7fdef4103660 2024-09-12T03:28:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.431+0000 7fdeea7fc640 1 -- 172.21.15.6:0/616688872 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee4010430 con 0x7fdef4103660 2024-09-12T03:28:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.431+0000 7fdefb5cd640 1 -- 172.21.15.6:0/616688872 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef41a50b0 con 0x7fdef4103660 2024-09-12T03:28:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.432+0000 7fdefb5cd640 1 -- 172.21.15.6:0/616688872 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdef4103120 con 0x7fdef4103660 2024-09-12T03:28:23.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.433+0000 7fdeea7fc640 1 -- 172.21.15.6:0/616688872 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fdee4004f90 con 0x7fdef4103660 2024-09-12T03:28:23.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.434+0000 7fdeea7fc640 1 --2- 172.21.15.6:0/616688872 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fdee00a2b80 0x7fdee00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:23.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.434+0000 7fdef9369640 1 --2- 172.21.15.6:0/616688872 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fdee00a2b80 0x7fdee00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:23.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.434+0000 7fdeea7fc640 1 -- 172.21.15.6:0/616688872 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fdee40cf890 con 0x7fdef4103660 2024-09-12T03:28:23.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.434+0000 7fdef9369640 1 --2- 172.21.15.6:0/616688872 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fdee00a2b80 0x7fdee00a5040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009990 tx=0x7fdef0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:23.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.437+0000 7fdeea7fc640 1 -- 172.21.15.6:0/616688872 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdee40541b0 con 0x7fdef4103660 2024-09-12T03:28:23.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.653+0000 7fdefb5cd640 1 -- 172.21.15.6:0/616688872 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdef4107210 con 0x7fdef4103660 2024-09-12T03:28:23.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.654+0000 7fdeea7fc640 1 -- 172.21.15.6:0/616688872 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11908 (secure 0 0 0) 0x7fdee409c820 con 0x7fdef4103660 2024-09-12T03:28:23.654 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:28:23.655 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"btime":"2024-09-12T03:24:26:920603+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:24:26.080047+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24463]},"id":1}]} 2024-09-12T03:28:23.655 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 27 2024-09-12T03:28:23.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.656+0000 7fdefb5cd640 1 -- 172.21.15.6:0/616688872 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fdee00a2b80 msgr2=0x7fdee00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:23.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.656+0000 7fdefb5cd640 1 --2- 172.21.15.6:0/616688872 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fdee00a2b80 0x7fdee00a5040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009990 tx=0x7fdef0009110 comp rx=0 tx=0).stop 2024-09-12T03:28:23.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.656+0000 7fdefb5cd640 1 -- 172.21.15.6:0/616688872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdef4103660 msgr2=0x7fdef4198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:23.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.656+0000 7fdefb5cd640 1 --2- 172.21.15.6:0/616688872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdef4103660 0x7fdef4198a70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdee400e9f0 tx=0x7fdee400edf0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.656+0000 7fdefb5cd640 1 -- 172.21.15.6:0/616688872 shutdown_connections 2024-09-12T03:28:23.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.656+0000 7fdefb5cd640 1 --2- 172.21.15.6:0/616688872 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fdee00a2b80 0x7fdee00a5040 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.656+0000 7fdefb5cd640 1 --2- 172.21.15.6:0/616688872 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdef410c170 0x7fdef419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.656+0000 7fdefb5cd640 1 --2- 172.21.15.6:0/616688872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdef4103660 0x7fdef4198a70 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.656+0000 7fdefb5cd640 1 --2- 172.21.15.6:0/616688872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdef4102d40 0x7fdef4198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.656+0000 7fdefb5cd640 1 -- 172.21.15.6:0/616688872 >> 172.21.15.6:0/616688872 conn(0x7fdef40fc7a0 msgr2=0x7fdef40fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:23.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.656+0000 7fdefb5cd640 1 -- 172.21.15.6:0/616688872 shutdown_connections 2024-09-12T03:28:23.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:23.656+0000 7fdefb5cd640 1 -- 172.21.15.6:0/616688872 wait complete. 2024-09-12T03:28:23.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:23 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/616688872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:24.107 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:23 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/616688872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:24.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:23 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/616688872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:24.312 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 7867f508 --release --await' 2024-09-12T03:28:24.312 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 7867f508 --release --await 2024-09-12T03:28:24.592 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:24.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:24 smithi118 ceph-mon[35048]: pgmap v554: 97 pgs: 97 active+clean; 13 GiB data, 126 GiB used, 946 GiB / 1.0 TiB avail; 16 KiB/s rd, 70 MiB/s wr, 12.82k op/s 2024-09-12T03:28:24.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:28:24.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.963 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:24 smithi118 ceph-mon[35048]: Adjusting osd_memory_target on smithi161 to 176.0M 2024-09-12T03:28:24.964 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:24 smithi118 ceph-mon[35048]: Adjusting osd_memory_target on smithi118 to 176.0M 2024-09-12T03:28:24.964 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:24 smithi118 ceph-mon[35048]: Unable to set osd_memory_target on smithi161 to 184593203: error parsing value: Value '184593203' is below minimum 939524096 2024-09-12T03:28:24.964 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:24 smithi118 ceph-mon[35048]: Unable to set osd_memory_target on smithi118 to 184567398: error parsing value: Value '184567398' is below minimum 939524096 2024-09-12T03:28:24.964 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.964 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.964 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.964 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.964 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:28:24.964 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:28:24.964 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:28:24.965 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:24 smithi006 ceph-mon[30991]: pgmap v554: 97 pgs: 97 active+clean; 13 GiB data, 126 GiB used, 946 GiB / 1.0 TiB avail; 16 KiB/s rd, 70 MiB/s wr, 12.82k op/s 2024-09-12T03:28:24.965 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:28:24.965 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.965 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.965 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.965 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.965 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.965 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.965 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.966 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.966 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:24 smithi006 ceph-mon[30991]: Adjusting osd_memory_target on smithi161 to 176.0M 2024-09-12T03:28:24.966 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:24 smithi006 ceph-mon[30991]: Adjusting osd_memory_target on smithi118 to 176.0M 2024-09-12T03:28:24.966 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:24 smithi006 ceph-mon[30991]: Unable to set osd_memory_target on smithi161 to 184593203: error parsing value: Value '184593203' is below minimum 939524096 2024-09-12T03:28:24.966 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:24 smithi006 ceph-mon[30991]: Unable to set osd_memory_target on smithi118 to 184567398: error parsing value: Value '184567398' is below minimum 939524096 2024-09-12T03:28:24.966 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.966 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.966 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.966 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:24.966 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:28:24.966 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:28:24.966 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:28:25.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.052+0000 7fcdda4a8640 1 -- 172.21.15.6:0/457831834 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdd41027a0 msgr2=0x7fcdd41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:25.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.052+0000 7fcdda4a8640 1 --2- 172.21.15.6:0/457831834 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdd41027a0 0x7fcdd41067f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009900 tx=0x7fcdbc02a3f0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.052+0000 7fcdda4a8640 1 -- 172.21.15.6:0/457831834 shutdown_connections 2024-09-12T03:28:25.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.052+0000 7fcdda4a8640 1 --2- 172.21.15.6:0/457831834 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcdd4106f20 0x7fcdd410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.052+0000 7fcdda4a8640 1 --2- 172.21.15.6:0/457831834 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdd41027a0 0x7fcdd41067f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.052+0000 7fcdda4a8640 1 --2- 172.21.15.6:0/457831834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdd4101df0 0x7fcdd41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.052+0000 7fcdda4a8640 1 -- 172.21.15.6:0/457831834 >> 172.21.15.6:0/457831834 conn(0x7fcdd40fd8f0 msgr2=0x7fcdd40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:25.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.052+0000 7fcdda4a8640 1 -- 172.21.15.6:0/457831834 shutdown_connections 2024-09-12T03:28:25.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.052+0000 7fcdda4a8640 1 -- 172.21.15.6:0/457831834 wait complete. 2024-09-12T03:28:25.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.052+0000 7fcdda4a8640 1 Processor -- start 2024-09-12T03:28:25.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.053+0000 7fcdda4a8640 1 -- start start 2024-09-12T03:28:25.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.053+0000 7fcdda4a8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdd4101df0 0x7fcdd41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.053+0000 7fcdda4a8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdd41027a0 0x7fcdd4198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.053+0000 7fcdda4a8640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcdd4106f20 0x7fcdd419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.053+0000 7fcdda4a8640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd4111f80 con 0x7fcdd41027a0 2024-09-12T03:28:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.053+0000 7fcdda4a8640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd4111e00 con 0x7fcdd4106f20 2024-09-12T03:28:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.053+0000 7fcdda4a8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd4112100 con 0x7fcdd4101df0 2024-09-12T03:28:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.053+0000 7fcdcbfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdd41027a0 0x7fcdd4198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.053+0000 7fcdd3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdd4101df0 0x7fcdd41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.053+0000 7fcdcbfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdd41027a0 0x7fcdd4198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53238/0 (socket says 172.21.15.6:53238) 2024-09-12T03:28:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.053+0000 7fcdcbfff640 1 -- 172.21.15.6:0/4047276001 learned_addr learned my addr 172.21.15.6:0/4047276001 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.053+0000 7fcdd8a45640 1 --2- 172.21.15.6:0/4047276001 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcdd4106f20 0x7fcdd419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.053+0000 7fcdcbfff640 1 -- 172.21.15.6:0/4047276001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdd4101df0 msgr2=0x7fcdd41983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.053+0000 7fcdcbfff640 1 --2- 172.21.15.6:0/4047276001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdd4101df0 0x7fcdd41983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.053+0000 7fcdcbfff640 1 -- 172.21.15.6:0/4047276001 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcdd4106f20 msgr2=0x7fcdd419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.053+0000 7fcdcbfff640 1 --2- 172.21.15.6:0/4047276001 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcdd4106f20 0x7fcdd419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.053+0000 7fcdcbfff640 1 -- 172.21.15.6:0/4047276001 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd419d370 con 0x7fcdd41027a0 2024-09-12T03:28:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.053+0000 7fcdd8a45640 1 --2- 172.21.15.6:0/4047276001 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcdd4106f20 0x7fcdd419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.054+0000 7fcdcbfff640 1 --2- 172.21.15.6:0/4047276001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdd41027a0 0x7fcdd4198900 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc006010 tx=0x7fcdbc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.054+0000 7fcdd1ffb640 1 -- 172.21.15.6:0/4047276001 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc03f030 con 0x7fcdd41027a0 2024-09-12T03:28:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.054+0000 7fcdd1ffb640 1 -- 172.21.15.6:0/4047276001 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdbc003ce0 con 0x7fcdd41027a0 2024-09-12T03:28:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.054+0000 7fcdda4a8640 1 -- 172.21.15.6:0/4047276001 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd419d600 con 0x7fcdd41027a0 2024-09-12T03:28:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.054+0000 7fcdd1ffb640 1 -- 172.21.15.6:0/4047276001 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc035070 con 0x7fcdd41027a0 2024-09-12T03:28:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.054+0000 7fcdda4a8640 1 -- 172.21.15.6:0/4047276001 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd41a4f40 con 0x7fcdd41027a0 2024-09-12T03:28:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.056+0000 7fcdd1ffb640 1 -- 172.21.15.6:0/4047276001 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcdbc042030 con 0x7fcdd41027a0 2024-09-12T03:28:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.056+0000 7fcddc881740 1 Processor -- start 2024-09-12T03:28:25.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.057+0000 7fcddc881740 1 -- start start 2024-09-12T03:28:25.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.057+0000 7fcddc881740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84e6060 0x5644b84e6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.057+0000 7fcddc881740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5644b84e6980 0x5644b84f0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.057+0000 7fcddc881740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84f1240 0x5644b84f3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.057+0000 7fcddc881740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5644b844b620 con 0x5644b84f1240 2024-09-12T03:28:25.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.057+0000 7fcddc881740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5644b83f2e50 con 0x5644b84e6980 2024-09-12T03:28:25.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.057+0000 7fcddc881740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5644b84475e0 con 0x5644b84e6060 2024-09-12T03:28:25.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.057+0000 7fcdcbfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84f1240 0x5644b84f3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.057+0000 7fcdd8a45640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84e6060 0x5644b84e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.057+0000 7fcdcbfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84f1240 0x5644b84f3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53250/0 (socket says 172.21.15.6:53250) 2024-09-12T03:28:25.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.057+0000 7fcdcbfff640 1 -- 172.21.15.6:0/1775318009 learned_addr learned my addr 172.21.15.6:0/1775318009 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:25.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.057+0000 7fcdd8a45640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84e6060 0x5644b84e6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:35450/0 (socket says 172.21.15.6:35450) 2024-09-12T03:28:25.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.057+0000 7fcdd1ffb640 1 --2- 172.21.15.6:0/4047276001 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcdc00a2c00 0x7fcdc00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.057+0000 7fcdd3fff640 1 --2- 172.21.15.6:0/1775318009 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5644b84e6980 0x5644b84f0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.057+0000 7fcdd8a45640 1 --2- 172.21.15.6:0/4047276001 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcdc00a2c00 0x7fcdc00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.057+0000 7fcdd3fff640 1 -- 172.21.15.6:0/1775318009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84e6060 msgr2=0x5644b84e6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:25.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.057+0000 7fcdd3fff640 1 --2- 172.21.15.6:0/1775318009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84e6060 0x5644b84e6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.057+0000 7fcdd3fff640 1 -- 172.21.15.6:0/1775318009 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84f1240 msgr2=0x5644b84f3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:25.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.057+0000 7fcdd3fff640 1 --2- 172.21.15.6:0/1775318009 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84f1240 0x5644b84f3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.057+0000 7fcdd3fff640 1 -- 172.21.15.6:0/1775318009 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b83e3d40 con 0x5644b84e6980 2024-09-12T03:28:25.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.057+0000 7fcdd1ffb640 1 -- 172.21.15.6:0/4047276001 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fcdbc0ef5a0 con 0x7fcdd41027a0 2024-09-12T03:28:25.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.058+0000 7fcdd8a45640 1 --2- 172.21.15.6:0/1775318009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84e6060 0x5644b84e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:25.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.058+0000 7fcdd3fff640 1 --2- 172.21.15.6:0/1775318009 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5644b84e6980 0x5644b84f0d00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4005fd0 tx=0x7fcdc4012b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33ff9d9cd58b9f05 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:25.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.058+0000 7fcdd8a45640 1 --2- 172.21.15.6:0/4047276001 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcdc00a2c00 0x7fcdc00a50c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc012940 tx=0x7fcdcc004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:25.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.058+0000 7fcdca7fc640 1 -- 172.21.15.6:0/1775318009 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4020070 con 0x5644b84e6980 2024-09-12T03:28:25.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.058+0000 7fcdca7fc640 1 -- 172.21.15.6:0/1775318009 <== mon.2 v2:172.21.15.118:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc400be20 con 0x5644b84e6980 2024-09-12T03:28:25.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.058+0000 7fcdca7fc640 1 -- 172.21.15.6:0/1775318009 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc401bd50 con 0x5644b84e6980 2024-09-12T03:28:25.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.059+0000 7fcddc881740 1 -- 172.21.15.6:0/1775318009 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5644b84e6980 msgr2=0x5644b84f0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:25.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.059+0000 7fcddc881740 1 --2- 172.21.15.6:0/1775318009 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5644b84e6980 0x5644b84f0d00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4005fd0 tx=0x7fcdc4012b30 comp rx=0 tx=0).stop 2024-09-12T03:28:25.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.059+0000 7fcddc881740 1 -- 172.21.15.6:0/1775318009 shutdown_connections 2024-09-12T03:28:25.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.059+0000 7fcddc881740 1 --2- 172.21.15.6:0/1775318009 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84f1240 0x5644b84f3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.059+0000 7fcddc881740 1 --2- 172.21.15.6:0/1775318009 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5644b84e6980 0x5644b84f0d00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.059+0000 7fcddc881740 1 --2- 172.21.15.6:0/1775318009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84e6060 0x5644b84e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.059+0000 7fcddc881740 1 -- 172.21.15.6:0/1775318009 >> 172.21.15.6:0/1775318009 conn(0x5644b84e25f0 msgr2=0x5644b84e29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:25.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.059+0000 7fcddc881740 1 -- 172.21.15.6:0/1775318009 shutdown_connections 2024-09-12T03:28:25.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.059+0000 7fcddc881740 1 -- 172.21.15.6:0/1775318009 wait complete. 2024-09-12T03:28:25.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.059+0000 7fcddc881740 1 -- start start 2024-09-12T03:28:25.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.059+0000 7fcddc881740 1 Processor -- start 2024-09-12T03:28:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.060+0000 7fcddc881740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84e6060 0x5644b84ee170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.060+0000 7fcddc881740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84e6980 0x5644b84ee6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.060+0000 7fcddc881740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5644b84f1240 0x5644b84eebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.060+0000 7fcddc881740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5644b84f3e10 con 0x5644b84e6060 2024-09-12T03:28:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.060+0000 7fcddc881740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5644b83d3f10 con 0x5644b84f1240 2024-09-12T03:28:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.060+0000 7fcddc881740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5644b84f3f90 con 0x5644b84e6980 2024-09-12T03:28:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.060+0000 7fcddc881740 10 client.0 _ll_register_callbacks cb 0x5644b83f7db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.060+0000 7fcdd3fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84e6060 0x5644b84ee170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.060+0000 7fcdd3fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84e6060 0x5644b84ee170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53252/0 (socket says 172.21.15.6:53252) 2024-09-12T03:28:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.060+0000 7fcdd3fff640 1 -- 172.21.15.6:0/3555068920 learned_addr learned my addr 172.21.15.6:0/3555068920 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.060+0000 7fcdcbfff640 1 --2- 172.21.15.6:0/3555068920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84e6980 0x5644b84ee6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.060+0000 7fcdd8a45640 1 --2- 172.21.15.6:0/3555068920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5644b84f1240 0x5644b84eebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.060+0000 7fcdd3fff640 1 -- 172.21.15.6:0/3555068920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84e6980 msgr2=0x5644b84ee6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.060+0000 7fcdd3fff640 1 --2- 172.21.15.6:0/3555068920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84e6980 0x5644b84ee6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.060+0000 7fcdd3fff640 1 -- 172.21.15.6:0/3555068920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5644b84f1240 msgr2=0x5644b84eebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.060+0000 7fcdd3fff640 1 --2- 172.21.15.6:0/3555068920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5644b84f1240 0x5644b84eebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.060+0000 7fcdd3fff640 1 -- 172.21.15.6:0/3555068920 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b815a580 con 0x5644b84e6060 2024-09-12T03:28:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.060+0000 7fcdd3fff640 1 --2- 172.21.15.6:0/3555068920 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84e6060 0x5644b84ee170 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fcdc401e040 tx=0x7fcdc401b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910262c2233c325d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.060+0000 7fcdb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:28:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.060+0000 7fcdb37fe640 1 -- 172.21.15.6:0/3555068920 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4020070 con 0x5644b84e6060 2024-09-12T03:28:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.061+0000 7fcdb37fe640 1 -- 172.21.15.6:0/3555068920 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fcdc401bca0 con 0x5644b84e6060 2024-09-12T03:28:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.061+0000 7fcddc881740 1 -- 172.21.15.6:0/3555068920 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdc400be20 con 0x5644b84e6060 2024-09-12T03:28:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.061+0000 7fcdb37fe640 1 -- 172.21.15.6:0/3555068920 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcdc400d920 con 0x5644b84e6060 2024-09-12T03:28:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.061+0000 7fcddc881740 10 client.16182 fetch_fsmap learned FSMap version 27 2024-09-12T03:28:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.061+0000 7fcddc881740 1 -- 172.21.15.6:0/3555068920 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5644b8224960 con 0x5644b84e6060 2024-09-12T03:28:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.061+0000 7fcdb37fe640 1 -- 172.21.15.6:0/3555068920 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc400be20 con 0x5644b84e6060 2024-09-12T03:28:25.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.061+0000 7fcdb37fe640 1 -- 172.21.15.6:0/3555068920 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fcdc402c030 con 0x5644b84e6060 2024-09-12T03:28:25.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.061+0000 7fcddc881740 10 client.16182 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:28:25.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.061+0000 7fcddc881740 10 client.16182 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:28:25.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.061+0000 7fcddc881740 20 client.16182 populate_metadata read hostname 'smithi006' 2024-09-12T03:28:25.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.061+0000 7fcddc881740 1 --2- 172.21.15.6:0/3555068920 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5644b8500da0 0x5644b8503190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.061+0000 7fcddc881740 4 client.16182 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:25.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.061+0000 7fcddc881740 1 -- 172.21.15.6:0/3555068920 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644b8458b90 con 0x5644b8500da0 2024-09-12T03:28:25.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.061+0000 7fcdcbfff640 1 --2- 172.21.15.6:0/3555068920 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5644b8500da0 0x5644b8503190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.062+0000 7fcdb37fe640 10 client.16182 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:25.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.062+0000 7fcdcbfff640 1 --2- 172.21.15.6:0/3555068920 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5644b8500da0 0x5644b8503190 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f3a4d74fc7df690 server_cookie=9b29178a60ef183c in_seq=0 out_seq=0 2024-09-12T03:28:25.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.062+0000 7fcdb37fe640 1 -- 172.21.15.6:0/3555068920 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644b8458b90 con 0x5644b8500da0 2024-09-12T03:28:25.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.062+0000 7fcdb37fe640 10 client.16182 handle_command_reply: tid=0 2024-09-12T03:28:25.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.062+0000 7fcddc881740 1 client.16182 shutdown 2024-09-12T03:28:25.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.063+0000 7fcddc881740 1 -- 172.21.15.6:0/3555068920 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84e6060 msgr2=0x5644b84ee170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:25.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.063+0000 7fcddc881740 1 --2- 172.21.15.6:0/3555068920 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84e6060 0x5644b84ee170 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fcdc401e040 tx=0x7fcdc401b6c0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.063+0000 7fcddc881740 1 -- 172.21.15.6:0/3555068920 shutdown_connections 2024-09-12T03:28:25.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.063+0000 7fcddc881740 1 --2- 172.21.15.6:0/3555068920 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5644b8500da0 0x5644b8503190 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.063+0000 7fcddc881740 1 --2- 172.21.15.6:0/3555068920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5644b84f1240 0x5644b84eebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.063+0000 7fcddc881740 1 --2- 172.21.15.6:0/3555068920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84e6980 0x5644b84ee6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.063+0000 7fcdb37fe640 0 client.16182 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:25.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.063+0000 7fcddc881740 1 --2- 172.21.15.6:0/3555068920 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84e6060 0x5644b84ee170 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.063+0000 7fcddc881740 1 -- 172.21.15.6:0/3555068920 >> 172.21.15.6:0/3555068920 conn(0x5644b84e25f0 msgr2=0x5644b84e82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:25.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.063+0000 7fcddc881740 1 -- 172.21.15.6:0/3555068920 shutdown_connections 2024-09-12T03:28:25.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.063+0000 7fcddc881740 1 -- 172.21.15.6:0/3555068920 wait complete. 2024-09-12T03:28:25.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.063+0000 7fcddc881740 20 client.16182 trim_cache size 0 max 16384 2024-09-12T03:28:25.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.080+0000 7fcddc881740 1 Processor -- start 2024-09-12T03:28:25.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.081+0000 7fcddc881740 1 -- start start 2024-09-12T03:28:25.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.081+0000 7fcddc881740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5644b84e6060 0x5644b85087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.081+0000 7fcddc881740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84e6980 0x5644b8508d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.081+0000 7fcddc881740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84f1240 0x5644b84eb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.081+0000 7fcddc881740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd41ada60 con 0x5644b84e6980 2024-09-12T03:28:25.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.081+0000 7fcddc881740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd41ab330 con 0x5644b84e6060 2024-09-12T03:28:25.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.081+0000 7fcddc881740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd41a91a0 con 0x5644b84f1240 2024-09-12T03:28:25.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.081+0000 7fcdcbfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84e6980 0x5644b8508d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.081+0000 7fcdcbfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84e6980 0x5644b8508d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53260/0 (socket says 172.21.15.6:53260) 2024-09-12T03:28:25.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.081+0000 7fcdcbfff640 1 -- 172.21.15.6:0/881521130 learned_addr learned my addr 172.21.15.6:0/881521130 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:25.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.081+0000 7fcdd3fff640 1 --2- 172.21.15.6:0/881521130 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5644b84e6060 0x5644b85087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.081+0000 7fcdd8a45640 1 --2- 172.21.15.6:0/881521130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84f1240 0x5644b84eb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.081+0000 7fcdcbfff640 1 -- 172.21.15.6:0/881521130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84f1240 msgr2=0x5644b84eb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:25.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.081+0000 7fcdcbfff640 1 --2- 172.21.15.6:0/881521130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84f1240 0x5644b84eb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.081+0000 7fcdcbfff640 1 -- 172.21.15.6:0/881521130 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5644b84e6060 msgr2=0x5644b85087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:25.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.081+0000 7fcdcbfff640 1 --2- 172.21.15.6:0/881521130 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5644b84e6060 0x5644b85087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.081+0000 7fcdcbfff640 1 -- 172.21.15.6:0/881521130 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b84e67b0 con 0x5644b84e6980 2024-09-12T03:28:25.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.081+0000 7fcdd8a45640 1 --2- 172.21.15.6:0/881521130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84f1240 0x5644b84eb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:25.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.081+0000 7fcdcbfff640 1 --2- 172.21.15.6:0/881521130 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84e6980 0x5644b8508d30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc000c00 tx=0x7fcdbc0dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9e489038e60e0d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:25.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.082+0000 7fcdc9ffb640 1 -- 172.21.15.6:0/881521130 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc049040 con 0x5644b84e6980 2024-09-12T03:28:25.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.082+0000 7fcdc9ffb640 1 -- 172.21.15.6:0/881521130 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdbc03b580 con 0x5644b84e6980 2024-09-12T03:28:25.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.082+0000 7fcdc9ffb640 1 -- 172.21.15.6:0/881521130 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0bbef0 con 0x5644b84e6980 2024-09-12T03:28:25.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.082+0000 7fcddc881740 1 -- 172.21.15.6:0/881521130 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84e6980 msgr2=0x5644b8508d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:25.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.082+0000 7fcddc881740 1 --2- 172.21.15.6:0/881521130 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84e6980 0x5644b8508d30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc000c00 tx=0x7fcdbc0dc040 comp rx=0 tx=0).stop 2024-09-12T03:28:25.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.082+0000 7fcddc881740 1 -- 172.21.15.6:0/881521130 shutdown_connections 2024-09-12T03:28:25.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.082+0000 7fcddc881740 1 --2- 172.21.15.6:0/881521130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84f1240 0x5644b84eb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.082+0000 7fcddc881740 1 --2- 172.21.15.6:0/881521130 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84e6980 0x5644b8508d30 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.082+0000 7fcddc881740 1 --2- 172.21.15.6:0/881521130 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5644b84e6060 0x5644b85087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.082+0000 7fcddc881740 1 -- 172.21.15.6:0/881521130 >> 172.21.15.6:0/881521130 conn(0x5644b84e25f0 msgr2=0x5644b84ef9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:25.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.082+0000 7fcddc881740 1 -- 172.21.15.6:0/881521130 shutdown_connections 2024-09-12T03:28:25.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.082+0000 7fcddc881740 1 -- 172.21.15.6:0/881521130 wait complete. 2024-09-12T03:28:25.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.083+0000 7fcddc881740 1 -- start start 2024-09-12T03:28:25.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.083+0000 7fcddc881740 1 Processor -- start 2024-09-12T03:28:25.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.083+0000 7fcddc881740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5644b84e6060 0x5644b84f5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.083+0000 7fcddc881740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84e6980 0x5644b84f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.083+0000 7fcddc881740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84f1240 0x5644b856a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.083+0000 7fcddc881740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd41a97f0 con 0x5644b84e6980 2024-09-12T03:28:25.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.083+0000 7fcddc881740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd41aac60 con 0x5644b84e6060 2024-09-12T03:28:25.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.083+0000 7fcddc881740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd41a9ec0 con 0x5644b84f1240 2024-09-12T03:28:25.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.083+0000 7fcddc881740 10 client.0 _ll_register_callbacks cb 0x7fcdcc007600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:25.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.083+0000 7fcdcbfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84e6980 0x5644b84f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.083+0000 7fcdcbfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84e6980 0x5644b84f57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53270/0 (socket says 172.21.15.6:53270) 2024-09-12T03:28:25.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.083+0000 7fcdcbfff640 1 -- 172.21.15.6:0/1406455490 learned_addr learned my addr 172.21.15.6:0/1406455490 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:25.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.084+0000 7fcdd3fff640 1 --2- 172.21.15.6:0/1406455490 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5644b84e6060 0x5644b84f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.084+0000 7fcdd8a45640 1 --2- 172.21.15.6:0/1406455490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84f1240 0x5644b856a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.084+0000 7fcdcbfff640 1 -- 172.21.15.6:0/1406455490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84f1240 msgr2=0x5644b856a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:25.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.084+0000 7fcdcbfff640 1 --2- 172.21.15.6:0/1406455490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84f1240 0x5644b856a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.084+0000 7fcdcbfff640 1 -- 172.21.15.6:0/1406455490 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5644b84e6060 msgr2=0x5644b84f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:25.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.084+0000 7fcdcbfff640 1 --2- 172.21.15.6:0/1406455490 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5644b84e6060 0x5644b84f5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.084+0000 7fcdcbfff640 1 -- 172.21.15.6:0/1406455490 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b84f6030 con 0x5644b84e6980 2024-09-12T03:28:25.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.084+0000 7fcdcbfff640 1 --2- 172.21.15.6:0/1406455490 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84e6980 0x5644b84f57a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc005ab0 tx=0x7fcdbc0045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592650e7607b3b76 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:25.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.084+0000 7fcdb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:28:25.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.084+0000 7fcdb37fe640 1 -- 172.21.15.6:0/1406455490 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc049040 con 0x5644b84e6980 2024-09-12T03:28:25.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.084+0000 7fcdb37fe640 1 -- 172.21.15.6:0/1406455490 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fcdbc0ba1f0 con 0x5644b84e6980 2024-09-12T03:28:25.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.084+0000 7fcddc881740 1 -- 172.21.15.6:0/1406455490 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5644b84530a0 con 0x5644b84e6980 2024-09-12T03:28:25.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.084+0000 7fcdb37fe640 1 -- 172.21.15.6:0/1406455490 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcdbc0bbcc0 con 0x5644b84e6980 2024-09-12T03:28:25.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.084+0000 7fcddc881740 10 client.16188 fetch_fsmap learned FSMap version 27 2024-09-12T03:28:25.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.084+0000 7fcddc881740 1 -- 172.21.15.6:0/1406455490 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5644b856be10 con 0x5644b84e6980 2024-09-12T03:28:25.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.085+0000 7fcdb37fe640 1 -- 172.21.15.6:0/1406455490 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5644b84530a0 con 0x5644b84e6980 2024-09-12T03:28:25.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.085+0000 7fcdb37fe640 1 -- 172.21.15.6:0/1406455490 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fcdbc0ef0e0 con 0x5644b84e6980 2024-09-12T03:28:25.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.085+0000 7fcddc881740 10 client.16188 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:28:25.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.085+0000 7fcddc881740 10 client.16188 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:28:25.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.085+0000 7fcddc881740 20 client.16188 populate_metadata read hostname 'smithi006' 2024-09-12T03:28:25.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.085+0000 7fcddc881740 1 --2- 172.21.15.6:0/1406455490 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5644b856c530 0x5644b856e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.085+0000 7fcddc881740 4 client.16188 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7867f508", "release": true, "await": true}] 2024-09-12T03:28:25.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.085+0000 7fcddc881740 1 -- 172.21.15.6:0/1406455490 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7867f508", "release": true, "await": true}) -- 0x5644b83cf5a0 con 0x5644b856c530 2024-09-12T03:28:25.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.085+0000 7fcdd3fff640 1 --2- 172.21.15.6:0/1406455490 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5644b856c530 0x5644b856e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.085+0000 7fcdb37fe640 10 client.16188 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:25.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.085+0000 7fcdd3fff640 1 --2- 172.21.15.6:0/1406455490 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5644b856c530 0x5644b856e950 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20bbb69f04b6cbd6 server_cookie=2d67d74e4331d86b in_seq=0 out_seq=0 2024-09-12T03:28:25.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.108+0000 7fcdb37fe640 1 -- 172.21.15.6:0/1406455490 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5644b83cf5a0 con 0x5644b856c530 2024-09-12T03:28:25.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.108+0000 7fcdb37fe640 10 client.16188 handle_command_reply: tid=0 2024-09-12T03:28:25.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.108+0000 7fcddc881740 1 client.16188 shutdown 2024-09-12T03:28:25.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.108+0000 7fcddc881740 1 -- 172.21.15.6:0/1406455490 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84e6980 msgr2=0x5644b84f57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:25.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.108+0000 7fcddc881740 1 --2- 172.21.15.6:0/1406455490 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84e6980 0x5644b84f57a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc005ab0 tx=0x7fcdbc0045d0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.108+0000 7fcddc881740 1 -- 172.21.15.6:0/1406455490 shutdown_connections 2024-09-12T03:28:25.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.108+0000 7fcddc881740 1 --2- 172.21.15.6:0/1406455490 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5644b856c530 0x5644b856e950 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.108+0000 7fcddc881740 1 --2- 172.21.15.6:0/1406455490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b84f1240 0x5644b856a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.108+0000 7fcddc881740 1 --2- 172.21.15.6:0/1406455490 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644b84e6980 0x5644b84f57a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.108+0000 7fcddc881740 1 --2- 172.21.15.6:0/1406455490 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5644b84e6060 0x5644b84f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.108+0000 7fcdb37fe640 0 client.16188 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:25.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.108+0000 7fcddc881740 1 -- 172.21.15.6:0/1406455490 >> 172.21.15.6:0/1406455490 conn(0x5644b84e25f0 msgr2=0x5644b84e0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:25.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.109+0000 7fcddc881740 1 -- 172.21.15.6:0/1406455490 shutdown_connections 2024-09-12T03:28:25.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.109+0000 7fcddc881740 1 -- 172.21.15.6:0/1406455490 wait complete. 2024-09-12T03:28:25.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.109+0000 7fcddc881740 20 client.16188 trim_cache size 0 max 16384 2024-09-12T03:28:25.110 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 27, 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi006.stdout: "leader": 24494, 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi006.stdout: "set_version": 28, 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi006.stdout: "sets": { 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi006.stdout: "7867f508": { 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 28, 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi006.stdout: "age_ref": 0.0, 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RELEASED", 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi006.stdout: "timeout": 90.0, 2024-09-12T03:28:25.112 INFO:teuthology.orchestra.run.smithi006.stdout: "expiration": 136.6, 2024-09-12T03:28:25.112 INFO:teuthology.orchestra.run.smithi006.stdout: "members": { 2024-09-12T03:28:25.112 INFO:teuthology.orchestra.run.smithi006.stdout: "file:/": { 2024-09-12T03:28:25.112 INFO:teuthology.orchestra.run.smithi006.stdout: "excluded": false, 2024-09-12T03:28:25.112 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-09-12T03:28:25.112 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RELEASED", 2024-09-12T03:28:25.112 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-09-12T03:28:25.112 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:28:25.112 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:28:25.112 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:28:25.112 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:28:25.112 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:28:25.113 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:28:25.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.109+0000 7fcdda4a8640 1 -- 172.21.15.6:0/4047276001 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcdc00a2c00 msgr2=0x7fcdc00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:25.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.109+0000 7fcdda4a8640 1 --2- 172.21.15.6:0/4047276001 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcdc00a2c00 0x7fcdc00a50c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc012940 tx=0x7fcdcc004ab0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.109+0000 7fcdda4a8640 1 -- 172.21.15.6:0/4047276001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdd41027a0 msgr2=0x7fcdd4198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:25.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.109+0000 7fcdda4a8640 1 --2- 172.21.15.6:0/4047276001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdd41027a0 0x7fcdd4198900 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc006010 tx=0x7fcdbc0042f0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.109+0000 7fcdda4a8640 1 -- 172.21.15.6:0/4047276001 shutdown_connections 2024-09-12T03:28:25.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.109+0000 7fcdda4a8640 1 --2- 172.21.15.6:0/4047276001 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fcdc00a2c00 0x7fcdc00a50c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.109+0000 7fcdda4a8640 1 --2- 172.21.15.6:0/4047276001 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcdd4106f20 0x7fcdd419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.109+0000 7fcdda4a8640 1 --2- 172.21.15.6:0/4047276001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdd41027a0 0x7fcdd4198900 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.109+0000 7fcdda4a8640 1 --2- 172.21.15.6:0/4047276001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdd4101df0 0x7fcdd41983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.109+0000 7fcdda4a8640 1 -- 172.21.15.6:0/4047276001 >> 172.21.15.6:0/4047276001 conn(0x7fcdd40fd8f0 msgr2=0x7fcdd40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:25.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.109+0000 7fcdda4a8640 1 -- 172.21.15.6:0/4047276001 shutdown_connections 2024-09-12T03:28:25.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:25.109+0000 7fcdda4a8640 1 -- 172.21.15.6:0/4047276001 wait complete. 2024-09-12T03:28:25.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:24 smithi161 ceph-mon[35485]: pgmap v554: 97 pgs: 97 active+clean; 13 GiB data, 126 GiB used, 946 GiB / 1.0 TiB avail; 16 KiB/s rd, 70 MiB/s wr, 12.82k op/s 2024-09-12T03:28:25.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:28:25.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:25.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:25.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:25.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:25.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:25.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:25.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:25.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:25.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:24 smithi161 ceph-mon[35485]: Adjusting osd_memory_target on smithi161 to 176.0M 2024-09-12T03:28:25.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:24 smithi161 ceph-mon[35485]: Adjusting osd_memory_target on smithi118 to 176.0M 2024-09-12T03:28:25.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:24 smithi161 ceph-mon[35485]: Unable to set osd_memory_target on smithi161 to 184593203: error parsing value: Value '184593203' is below minimum 939524096 2024-09-12T03:28:25.115 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:24 smithi161 ceph-mon[35485]: Unable to set osd_memory_target on smithi118 to 184567398: error parsing value: Value '184567398' is below minimum 939524096 2024-09-12T03:28:25.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:25.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:25.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:25.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:28:25.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:28:25.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:28:25.116 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:28:26.050 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:25 smithi006 ceph-mon[30991]: Adjusting osd_memory_target on smithi006 to 177.0M 2024-09-12T03:28:26.050 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:25 smithi006 ceph-mon[30991]: Unable to set osd_memory_target on smithi006 to 185687040: error parsing value: Value '185687040' is below minimum 939524096 2024-09-12T03:28:26.095 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7867f508', total seconds elapsed: 29.6 2024-09-12T03:28:26.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:25 smithi161 ceph-mon[35485]: Adjusting osd_memory_target on smithi006 to 177.0M 2024-09-12T03:28:26.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:25 smithi161 ceph-mon[35485]: Unable to set osd_memory_target on smithi006 to 185687040: error parsing value: Value '185687040' is below minimum 939524096 2024-09-12T03:28:26.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:25 smithi118 ceph-mon[35048]: Adjusting osd_memory_target on smithi006 to 177.0M 2024-09-12T03:28:26.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:25 smithi118 ceph-mon[35048]: Unable to set osd_memory_target on smithi006 to 185687040: error parsing value: Value '185687040' is below minimum 939524096 2024-09-12T03:28:27.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:26 smithi006 ceph-mon[30991]: pgmap v555: 97 pgs: 97 active+clean; 13 GiB data, 126 GiB used, 946 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 57 MiB/s wr, 10.70k op/s 2024-09-12T03:28:27.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:26 smithi006 ceph-mon[30991]: scrub summary: idle 2024-09-12T03:28:27.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:26 smithi006 ceph-mon[30991]: scrub summary: idle 2024-09-12T03:28:27.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:26 smithi161 ceph-mon[35485]: pgmap v555: 97 pgs: 97 active+clean; 13 GiB data, 126 GiB used, 946 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 57 MiB/s wr, 10.70k op/s 2024-09-12T03:28:27.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:26 smithi161 ceph-mon[35485]: scrub summary: idle 2024-09-12T03:28:27.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:26 smithi161 ceph-mon[35485]: scrub summary: idle 2024-09-12T03:28:27.114 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-12T03:28:27.114 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-12T03:28:27.185 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-09-12T03:28:27.185 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-12T03:28:27.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:26 smithi118 ceph-mon[35048]: pgmap v555: 97 pgs: 97 active+clean; 13 GiB data, 126 GiB used, 946 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 57 MiB/s wr, 10.70k op/s 2024-09-12T03:28:27.368 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:26 smithi118 ceph-mon[35048]: scrub summary: idle 2024-09-12T03:28:27.369 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:26 smithi118 ceph-mon[35048]: scrub summary: idle 2024-09-12T03:28:27.625 DEBUG:teuthology.parallel:result is None 2024-09-12T03:28:27.626 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T03:28:27.678 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T03:28:27.678 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-12T03:28:28.152 INFO:tasks.quiescer:joining Quiescers 2024-09-12T03:28:28.152 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-12T03:28:28.152 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:28:28.154 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:27 smithi161 ceph-mon[35485]: scrub summary: idle 2024-09-12T03:28:28.154 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:27 smithi006 ceph-mon[30991]: scrub summary: idle 2024-09-12T03:28:28.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:27 smithi118 ceph-mon[35048]: scrub summary: idle 2024-09-12T03:28:28.532 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:29.032 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:28 smithi006 ceph-mon[30991]: pgmap v556: 97 pgs: 97 active+clean; 12 GiB data, 126 GiB used, 946 GiB / 1.0 TiB avail; 96 KiB/s rd, 41 MiB/s wr, 7.66k op/s 2024-09-12T03:28:29.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:28 smithi161 ceph-mon[35485]: pgmap v556: 97 pgs: 97 active+clean; 12 GiB data, 126 GiB used, 946 GiB / 1.0 TiB avail; 96 KiB/s rd, 41 MiB/s wr, 7.66k op/s 2024-09-12T03:28:29.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.154+0000 7efc531f3640 1 -- 172.21.15.6:0/2172817261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc4c102570 msgr2=0x7efc4c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:29.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.154+0000 7efc531f3640 1 --2- 172.21.15.6:0/2172817261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc4c102570 0x7efc4c1029b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7efc38009900 tx=0x7efc3802a480 comp rx=0 tx=0).stop 2024-09-12T03:28:29.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.156+0000 7efc531f3640 1 -- 172.21.15.6:0/2172817261 shutdown_connections 2024-09-12T03:28:29.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.156+0000 7efc531f3640 1 --2- 172.21.15.6:0/2172817261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc4c102ef0 0x7efc4c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:29.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.156+0000 7efc531f3640 1 --2- 172.21.15.6:0/2172817261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc4c102570 0x7efc4c1029b0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:29.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.156+0000 7efc531f3640 1 --2- 172.21.15.6:0/2172817261 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efc4c100e20 0x7efc4c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:29.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.156+0000 7efc531f3640 1 -- 172.21.15.6:0/2172817261 >> 172.21.15.6:0/2172817261 conn(0x7efc4c0fc7d0 msgr2=0x7efc4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:29.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.157+0000 7efc531f3640 1 -- 172.21.15.6:0/2172817261 shutdown_connections 2024-09-12T03:28:29.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.157+0000 7efc531f3640 1 -- 172.21.15.6:0/2172817261 wait complete. 2024-09-12T03:28:29.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.158+0000 7efc531f3640 1 Processor -- start 2024-09-12T03:28:29.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.158+0000 7efc531f3640 1 -- start start 2024-09-12T03:28:29.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.158+0000 7efc531f3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc4c100e20 0x7efc4c10d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:29.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.158+0000 7efc531f3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc4c102570 0x7efc4c109670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:29.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.158+0000 7efc531f3640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efc4c102ef0 0x7efc4c109bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:29.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.158+0000 7efc531f3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c110fc0 con 0x7efc4c102570 2024-09-12T03:28:29.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.158+0000 7efc531f3640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c110e40 con 0x7efc4c102ef0 2024-09-12T03:28:29.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.158+0000 7efc531f3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c111140 con 0x7efc4c100e20 2024-09-12T03:28:29.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.158+0000 7efc4bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc4c102570 0x7efc4c109670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:29.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.158+0000 7efc4bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc4c102570 0x7efc4c109670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52976/0 (socket says 172.21.15.6:52976) 2024-09-12T03:28:29.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.158+0000 7efc4bfff640 1 -- 172.21.15.6:0/2271230169 learned_addr learned my addr 172.21.15.6:0/2271230169 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:29.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.158+0000 7efc50f8f640 1 --2- 172.21.15.6:0/2271230169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc4c100e20 0x7efc4c10d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:29.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.158+0000 7efc51790640 1 --2- 172.21.15.6:0/2271230169 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efc4c102ef0 0x7efc4c109bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:29.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.159+0000 7efc50f8f640 1 -- 172.21.15.6:0/2271230169 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efc4c102ef0 msgr2=0x7efc4c109bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:29.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.159+0000 7efc50f8f640 1 --2- 172.21.15.6:0/2271230169 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efc4c102ef0 0x7efc4c109bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:29.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.159+0000 7efc50f8f640 1 -- 172.21.15.6:0/2271230169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc4c102570 msgr2=0x7efc4c109670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:29.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.159+0000 7efc50f8f640 1 --2- 172.21.15.6:0/2271230169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc4c102570 0x7efc4c109670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:29.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.159+0000 7efc50f8f640 1 -- 172.21.15.6:0/2271230169 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4c10a1a0 con 0x7efc4c100e20 2024-09-12T03:28:29.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.160+0000 7efc50f8f640 1 --2- 172.21.15.6:0/2271230169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc4c100e20 0x7efc4c10d5d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efc4000b420 tx=0x7efc4000b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:29.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.160+0000 7efc49ffb640 1 -- 172.21.15.6:0/2271230169 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40011830 con 0x7efc4c100e20 2024-09-12T03:28:29.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.160+0000 7efc49ffb640 1 -- 172.21.15.6:0/2271230169 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc40016070 con 0x7efc4c100e20 2024-09-12T03:28:29.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.160+0000 7efc49ffb640 1 -- 172.21.15.6:0/2271230169 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4000f2e0 con 0x7efc4c100e20 2024-09-12T03:28:29.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.161+0000 7efc531f3640 1 -- 172.21.15.6:0/2271230169 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4c10a490 con 0x7efc4c100e20 2024-09-12T03:28:29.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.161+0000 7efc531f3640 1 -- 172.21.15.6:0/2271230169 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc4c1ab440 con 0x7efc4c100e20 2024-09-12T03:28:29.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.163+0000 7efc531f3640 1 -- 172.21.15.6:0/2271230169 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc4c1021b0 con 0x7efc4c100e20 2024-09-12T03:28:29.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.164+0000 7efc49ffb640 1 -- 172.21.15.6:0/2271230169 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7efc400119d0 con 0x7efc4c100e20 2024-09-12T03:28:29.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.165+0000 7efc49ffb640 1 --2- 172.21.15.6:0/2271230169 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efc3c0a2c80 0x7efc3c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:29.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.165+0000 7efc49ffb640 1 -- 172.21.15.6:0/2271230169 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7efc400ce620 con 0x7efc4c100e20 2024-09-12T03:28:29.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.165+0000 7efc4bfff640 1 --2- 172.21.15.6:0/2271230169 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efc3c0a2c80 0x7efc3c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:29.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.166+0000 7efc4bfff640 1 --2- 172.21.15.6:0/2271230169 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efc3c0a2c80 0x7efc3c0a5140 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efc380096c0 tx=0x7efc38000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:29.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.171+0000 7efc49ffb640 1 -- 172.21.15.6:0/2271230169 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc4009b550 con 0x7efc4c100e20 2024-09-12T03:28:29.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:28 smithi118 ceph-mon[35048]: pgmap v556: 97 pgs: 97 active+clean; 12 GiB data, 126 GiB used, 946 GiB / 1.0 TiB avail; 96 KiB/s rd, 41 MiB/s wr, 7.66k op/s 2024-09-12T03:28:29.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.479+0000 7efc531f3640 1 -- 172.21.15.6:0/2271230169 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc4c1ab910 con 0x7efc4c100e20 2024-09-12T03:28:29.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.480+0000 7efc49ffb640 1 -- 172.21.15.6:0/2271230169 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11909 (secure 0 0 0) 0x7efc4009ae50 con 0x7efc4c100e20 2024-09-12T03:28:29.482 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:28:29.482 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":28,"btime":"2024-09-12T03:28:28:730461+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:28:27.748823+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24463,24464,24494]},"id":1}]} 2024-09-12T03:28:29.482 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 28 2024-09-12T03:28:29.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.484+0000 7efc531f3640 1 -- 172.21.15.6:0/2271230169 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efc3c0a2c80 msgr2=0x7efc3c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:29.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.484+0000 7efc531f3640 1 --2- 172.21.15.6:0/2271230169 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efc3c0a2c80 0x7efc3c0a5140 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efc380096c0 tx=0x7efc38000bc0 comp rx=0 tx=0).stop 2024-09-12T03:28:29.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.484+0000 7efc531f3640 1 -- 172.21.15.6:0/2271230169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc4c100e20 msgr2=0x7efc4c10d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:29.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.484+0000 7efc531f3640 1 --2- 172.21.15.6:0/2271230169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc4c100e20 0x7efc4c10d5d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efc4000b420 tx=0x7efc4000b8e0 comp rx=0 tx=0).stop 2024-09-12T03:28:29.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.484+0000 7efc531f3640 1 -- 172.21.15.6:0/2271230169 shutdown_connections 2024-09-12T03:28:29.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.484+0000 7efc531f3640 1 --2- 172.21.15.6:0/2271230169 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7efc3c0a2c80 0x7efc3c0a5140 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:29.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.484+0000 7efc531f3640 1 --2- 172.21.15.6:0/2271230169 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efc4c102ef0 0x7efc4c109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:29.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.484+0000 7efc531f3640 1 --2- 172.21.15.6:0/2271230169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc4c102570 0x7efc4c109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:29.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.484+0000 7efc531f3640 1 --2- 172.21.15.6:0/2271230169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc4c100e20 0x7efc4c10d5d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:29.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.484+0000 7efc531f3640 1 -- 172.21.15.6:0/2271230169 >> 172.21.15.6:0/2271230169 conn(0x7efc4c0fc7d0 msgr2=0x7efc4c103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:29.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.484+0000 7efc531f3640 1 -- 172.21.15.6:0/2271230169 shutdown_connections 2024-09-12T03:28:29.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:29.484+0000 7efc531f3640 1 -- 172.21.15.6:0/2271230169 wait complete. 2024-09-12T03:28:30.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:29 smithi006 ceph-mon[30991]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:28:30.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:29 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2271230169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:30.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:29 smithi161 ceph-mon[35485]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:28:30.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:29 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2271230169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:30.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:29 smithi118 ceph-mon[35048]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:28:30.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:29 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2271230169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:30.492 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --cancel --all' 2024-09-12T03:28:30.492 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --cancel --all 2024-09-12T03:28:30.848 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:31.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:30 smithi006 ceph-mon[30991]: pgmap v557: 97 pgs: 97 active+clean; 12 GiB data, 126 GiB used, 946 GiB / 1.0 TiB avail; 97 KiB/s rd, 22 MiB/s wr, 4.12k op/s 2024-09-12T03:28:31.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:30 smithi161 ceph-mon[35485]: pgmap v557: 97 pgs: 97 active+clean; 12 GiB data, 126 GiB used, 946 GiB / 1.0 TiB avail; 97 KiB/s rd, 22 MiB/s wr, 4.12k op/s 2024-09-12T03:28:31.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:30 smithi118 ceph-mon[35048]: pgmap v557: 97 pgs: 97 active+clean; 12 GiB data, 126 GiB used, 946 GiB / 1.0 TiB avail; 97 KiB/s rd, 22 MiB/s wr, 4.12k op/s 2024-09-12T03:28:31.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.509+0000 7f517d8e9640 1 -- 172.21.15.6:0/2918356770 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5178103740 msgr2=0x7f5178107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:31.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.509+0000 7f517d8e9640 1 --2- 172.21.15.6:0/2918356770 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5178103740 0x7f5178107b80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f516c009930 tx=0x7f516c02a420 comp rx=0 tx=0).stop 2024-09-12T03:28:31.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.509+0000 7f517d8e9640 1 -- 172.21.15.6:0/2918356770 shutdown_connections 2024-09-12T03:28:31.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.509+0000 7f517d8e9640 1 --2- 172.21.15.6:0/2918356770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5178108290 0x7f517810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.509+0000 7f517d8e9640 1 --2- 172.21.15.6:0/2918356770 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5178103740 0x7f5178107b80 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.509+0000 7f517d8e9640 1 --2- 172.21.15.6:0/2918356770 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5178102d90 0x7f5178103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.509+0000 7f517d8e9640 1 -- 172.21.15.6:0/2918356770 >> 172.21.15.6:0/2918356770 conn(0x7f51780fc7d0 msgr2=0x7f51780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:31.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.509+0000 7f517d8e9640 1 -- 172.21.15.6:0/2918356770 shutdown_connections 2024-09-12T03:28:31.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.509+0000 7f517d8e9640 1 -- 172.21.15.6:0/2918356770 wait complete. 2024-09-12T03:28:31.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.509+0000 7f517d8e9640 1 Processor -- start 2024-09-12T03:28:31.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.510+0000 7f517d8e9640 1 -- start start 2024-09-12T03:28:31.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.510+0000 7f517d8e9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5178102d90 0x7f51781985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.510+0000 7f517d8e9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5178103740 0x7f5178198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.510+0000 7f517d8e9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5178108290 0x7f517819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.510+0000 7f517d8e9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5178112050 con 0x7f5178103740 2024-09-12T03:28:31.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.510+0000 7f517d8e9640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f5178111ed0 con 0x7f5178108290 2024-09-12T03:28:31.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.510+0000 7f517d8e9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f51781121d0 con 0x7f5178102d90 2024-09-12T03:28:31.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.510+0000 7f51767fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5178103740 0x7f5178198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.510+0000 7f51767fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5178103740 0x7f5178198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53006/0 (socket says 172.21.15.6:53006) 2024-09-12T03:28:31.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.510+0000 7f51767fc640 1 -- 172.21.15.6:0/2023519181 learned_addr learned my addr 172.21.15.6:0/2023519181 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:31.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.510+0000 7f51777fe640 1 --2- 172.21.15.6:0/2023519181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5178108290 0x7f517819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.510+0000 7f5176ffd640 1 --2- 172.21.15.6:0/2023519181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5178102d90 0x7f51781985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.510+0000 7f51767fc640 1 -- 172.21.15.6:0/2023519181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5178102d90 msgr2=0x7f51781985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:31.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.510+0000 7f51767fc640 1 --2- 172.21.15.6:0/2023519181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5178102d90 0x7f51781985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.510+0000 7f51767fc640 1 -- 172.21.15.6:0/2023519181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5178108290 msgr2=0x7f517819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:31.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.510+0000 7f51767fc640 1 --2- 172.21.15.6:0/2023519181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5178108290 0x7f517819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.510+0000 7f51767fc640 1 -- 172.21.15.6:0/2023519181 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517819d560 con 0x7f5178103740 2024-09-12T03:28:31.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.510+0000 7f51767fc640 1 --2- 172.21.15.6:0/2023519181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5178103740 0x7f5178198af0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f516c005940 tx=0x7f516c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:31.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.511+0000 7f517c8e7640 1 -- 172.21.15.6:0/2023519181 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c03f030 con 0x7f5178103740 2024-09-12T03:28:31.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.511+0000 7f517c8e7640 1 -- 172.21.15.6:0/2023519181 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f516c0044c0 con 0x7f5178103740 2024-09-12T03:28:31.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.511+0000 7f517d8e9640 1 -- 172.21.15.6:0/2023519181 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f517819d7f0 con 0x7f5178103740 2024-09-12T03:28:31.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.511+0000 7f517c8e7640 1 -- 172.21.15.6:0/2023519181 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c035070 con 0x7f5178103740 2024-09-12T03:28:31.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.511+0000 7f517d8e9640 1 -- 172.21.15.6:0/2023519181 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51781a5130 con 0x7f5178103740 2024-09-12T03:28:31.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.513+0000 7f517c8e7640 1 -- 172.21.15.6:0/2023519181 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f516c042030 con 0x7f5178103740 2024-09-12T03:28:31.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.513+0000 7f517fcc2740 1 Processor -- start 2024-09-12T03:28:31.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.513+0000 7f517fcc2740 1 -- start start 2024-09-12T03:28:31.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.513+0000 7f517fcc2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059f2a00 0x5625059f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.513+0000 7f517fcc2740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059f3320 0x5625059fd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.513+0000 7f517fcc2740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059fdbe0 0x5625059fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.514+0000 7f517fcc2740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x562505957200 con 0x5625059f3320 2024-09-12T03:28:31.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.514+0000 7f517fcc2740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5625058fddc0 con 0x5625059fdbe0 2024-09-12T03:28:31.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.514+0000 7f517fcc2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5625059531c0 con 0x5625059f2a00 2024-09-12T03:28:31.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.514+0000 7f5176ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059f3320 0x5625059fd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.514+0000 7f51777fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059f2a00 0x5625059f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.514+0000 7f51777fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059f2a00 0x5625059f2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:55588/0 (socket says 172.21.15.6:55588) 2024-09-12T03:28:31.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.514+0000 7f5176ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059f3320 0x5625059fd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53014/0 (socket says 172.21.15.6:53014) 2024-09-12T03:28:31.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.514+0000 7f51777fe640 1 -- 172.21.15.6:0/1838555975 learned_addr learned my addr 172.21.15.6:0/1838555975 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:31.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.514+0000 7f51767fc640 1 --2- 172.21.15.6:0/1838555975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059fdbe0 0x5625059fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.514+0000 7f517c8e7640 1 --2- 172.21.15.6:0/2023519181 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f51600a2b80 0x7f51600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.514+0000 7f5176ffd640 1 -- 172.21.15.6:0/1838555975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059f2a00 msgr2=0x5625059f2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:31.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.514+0000 7f5176ffd640 1 --2- 172.21.15.6:0/1838555975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059f2a00 0x5625059f2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.514+0000 7f5176ffd640 1 -- 172.21.15.6:0/1838555975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059fdbe0 msgr2=0x5625059fffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:31.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.514+0000 7f5176ffd640 1 --2- 172.21.15.6:0/1838555975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059fdbe0 0x5625059fffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.514+0000 7f5176ffd640 1 -- 172.21.15.6:0/1838555975 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625056c8620 con 0x5625059f3320 2024-09-12T03:28:31.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.514+0000 7f51767fc640 1 --2- 172.21.15.6:0/1838555975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059fdbe0 0x5625059fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:31.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.514+0000 7f51777fe640 1 --2- 172.21.15.6:0/2023519181 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f51600a2b80 0x7f51600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.514+0000 7f517c8e7640 1 -- 172.21.15.6:0/2023519181 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f516c0ef370 con 0x7f5178103740 2024-09-12T03:28:31.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.514+0000 7f5176ffd640 1 --2- 172.21.15.6:0/1838555975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059f3320 0x5625059fd6a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5168009fa0 tx=0x7f51680118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe3b682be3a7c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:31.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.514+0000 7f515dffb640 1 -- 172.21.15.6:0/1838555975 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5168003c30 con 0x5625059f3320 2024-09-12T03:28:31.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.515+0000 7f515dffb640 1 -- 172.21.15.6:0/1838555975 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5168003dd0 con 0x5625059f3320 2024-09-12T03:28:31.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.515+0000 7f515dffb640 1 -- 172.21.15.6:0/1838555975 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516800fad0 con 0x5625059f3320 2024-09-12T03:28:31.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.515+0000 7f517fcc2740 1 -- 172.21.15.6:0/1838555975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059f3320 msgr2=0x5625059fd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:31.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.515+0000 7f51777fe640 1 --2- 172.21.15.6:0/2023519181 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f51600a2b80 0x7f51600a5040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f51700045e0 tx=0x7f51700046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:31.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.515+0000 7f517fcc2740 1 --2- 172.21.15.6:0/1838555975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059f3320 0x5625059fd6a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5168009fa0 tx=0x7f51680118d0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.515+0000 7f517fcc2740 1 -- 172.21.15.6:0/1838555975 shutdown_connections 2024-09-12T03:28:31.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.515+0000 7f517fcc2740 1 --2- 172.21.15.6:0/1838555975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059fdbe0 0x5625059fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.515+0000 7f517fcc2740 1 --2- 172.21.15.6:0/1838555975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059f3320 0x5625059fd6a0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.515+0000 7f517fcc2740 1 --2- 172.21.15.6:0/1838555975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059f2a00 0x5625059f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.515+0000 7f517fcc2740 1 -- 172.21.15.6:0/1838555975 >> 172.21.15.6:0/1838555975 conn(0x5625059eef90 msgr2=0x5625059ef370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:31.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.515+0000 7f517fcc2740 1 -- 172.21.15.6:0/1838555975 shutdown_connections 2024-09-12T03:28:31.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.515+0000 7f517fcc2740 1 -- 172.21.15.6:0/1838555975 wait complete. 2024-09-12T03:28:31.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.516+0000 7f517fcc2740 1 -- start start 2024-09-12T03:28:31.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.516+0000 7f517fcc2740 1 Processor -- start 2024-09-12T03:28:31.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.516+0000 7f517fcc2740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059f2a00 0x5625059fd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.516+0000 7f517fcc2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059f3320 0x5625059fa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.516+0000 7f517fcc2740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059fdbe0 0x5625059faca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.516+0000 7f517fcc2740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x562505a04bc0 con 0x5625059fdbe0 2024-09-12T03:28:31.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.516+0000 7f517fcc2740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5625058f1c50 con 0x5625059f2a00 2024-09-12T03:28:31.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.516+0000 7f517fcc2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562505a04d40 con 0x5625059f3320 2024-09-12T03:28:31.577 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:28:31.577 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 29, 2024-09-12T03:28:31.577 INFO:teuthology.orchestra.run.smithi006.stdout: "leader": 24494, 2024-09-12T03:28:31.577 INFO:teuthology.orchestra.run.smithi006.stdout: "set_version": 28, 2024-09-12T03:28:31.578 INFO:teuthology.orchestra.run.smithi006.stdout: "sets": {} 2024-09-12T03:28:31.578 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:28:31.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.517+0000 7f51767fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059f3320 0x5625059fa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.517+0000 7f51777fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059fdbe0 0x5625059faca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.517+0000 7f51767fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059f3320 0x5625059fa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:55602/0 (socket says 172.21.15.6:55602) 2024-09-12T03:28:31.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.517+0000 7f5176ffd640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059f2a00 0x5625059fd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.517+0000 7f51767fc640 1 -- 172.21.15.6:0/2855823670 learned_addr learned my addr 172.21.15.6:0/2855823670 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:31.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.517+0000 7f517fcc2740 10 client.0 _ll_register_callbacks cb 0x562505903ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:31.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.517+0000 7f51767fc640 1 -- 172.21.15.6:0/2855823670 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059f2a00 msgr2=0x5625059fd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:31.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.517+0000 7f51767fc640 1 --2- 172.21.15.6:0/2855823670 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059f2a00 0x5625059fd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.517+0000 7f51767fc640 1 -- 172.21.15.6:0/2855823670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059fdbe0 msgr2=0x5625059faca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:31.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.517+0000 7f51767fc640 1 --2- 172.21.15.6:0/2855823670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059fdbe0 0x5625059faca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.517+0000 7f51767fc640 1 -- 172.21.15.6:0/2855823670 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625059fb4d0 con 0x5625059f3320 2024-09-12T03:28:31.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.517+0000 7f51767fc640 1 --2- 172.21.15.6:0/2855823670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059f3320 0x5625059fa760 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f516c000c00 tx=0x7f516c009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e820136e033aa809 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:31.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.517+0000 7f5156ffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-12T03:28:31.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.517+0000 7f5156ffd640 1 -- 172.21.15.6:0/2855823670 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c049070 con 0x5625059f3320 2024-09-12T03:28:31.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.517+0000 7f517fcc2740 1 -- 172.21.15.6:0/2855823670 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5168003dd0 con 0x5625059f3320 2024-09-12T03:28:31.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.517+0000 7f5156ffd640 1 -- 172.21.15.6:0/2855823670 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f516c0b8320 con 0x5625059f3320 2024-09-12T03:28:31.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.517+0000 7f5156ffd640 1 -- 172.21.15.6:0/2855823670 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f516c0bb7f0 con 0x5625059f3320 2024-09-12T03:28:31.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.518+0000 7f517fcc2740 10 client.25852 fetch_fsmap learned FSMap version 29 2024-09-12T03:28:31.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.518+0000 7f517fcc2740 1 -- 172.21.15.6:0/2855823670 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562505a04c70 con 0x5625059f3320 2024-09-12T03:28:31.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.518+0000 7f5156ffd640 1 -- 172.21.15.6:0/2855823670 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5168003dd0 con 0x5625059f3320 2024-09-12T03:28:31.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.518+0000 7f5156ffd640 1 -- 172.21.15.6:0/2855823670 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f516c0c43e0 con 0x5625059f3320 2024-09-12T03:28:31.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.518+0000 7f517fcc2740 10 client.25852 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T03:28:31.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.518+0000 7f517fcc2740 10 client.25852 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:28:31.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.518+0000 7f517fcc2740 20 client.25852 populate_metadata read hostname 'smithi006' 2024-09-12T03:28:31.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.518+0000 7f517fcc2740 1 --2- 172.21.15.6:0/2855823670 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x562505a11dc0 0x562505a14200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.518+0000 7f517fcc2740 4 client.25852 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:31.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.518+0000 7f517fcc2740 1 -- 172.21.15.6:0/2855823670 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562505964770 con 0x562505a11dc0 2024-09-12T03:28:31.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.518+0000 7f5176ffd640 1 --2- 172.21.15.6:0/2855823670 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x562505a11dc0 0x562505a14200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.519+0000 7f5156ffd640 10 client.25852 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:31.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.519+0000 7f5176ffd640 1 --2- 172.21.15.6:0/2855823670 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x562505a11dc0 0x562505a14200 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0181ef09ed0b46d server_cookie=13f49c5870b6d134 in_seq=0 out_seq=0 2024-09-12T03:28:31.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.520+0000 7f5156ffd640 1 -- 172.21.15.6:0/2855823670 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562505964770 con 0x562505a11dc0 2024-09-12T03:28:31.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.520+0000 7f5156ffd640 10 client.25852 handle_command_reply: tid=0 2024-09-12T03:28:31.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.520+0000 7f517fcc2740 1 client.25852 shutdown 2024-09-12T03:28:31.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.520+0000 7f517fcc2740 1 -- 172.21.15.6:0/2855823670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059f3320 msgr2=0x5625059fa760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:31.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.520+0000 7f517fcc2740 1 --2- 172.21.15.6:0/2855823670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059f3320 0x5625059fa760 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f516c000c00 tx=0x7f516c009680 comp rx=0 tx=0).stop 2024-09-12T03:28:31.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.520+0000 7f517fcc2740 1 -- 172.21.15.6:0/2855823670 shutdown_connections 2024-09-12T03:28:31.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.520+0000 7f517fcc2740 1 --2- 172.21.15.6:0/2855823670 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x562505a11dc0 0x562505a14200 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.520+0000 7f517fcc2740 1 --2- 172.21.15.6:0/2855823670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059fdbe0 0x5625059faca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.520+0000 7f517fcc2740 1 --2- 172.21.15.6:0/2855823670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059f3320 0x5625059fa760 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.520+0000 7f517fcc2740 1 --2- 172.21.15.6:0/2855823670 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059f2a00 0x5625059fd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.520+0000 7f5156ffd640 0 client.25852 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:31.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.520+0000 7f517fcc2740 1 -- 172.21.15.6:0/2855823670 >> 172.21.15.6:0/2855823670 conn(0x5625059eef90 msgr2=0x5625059ee1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:31.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.520+0000 7f517fcc2740 1 -- 172.21.15.6:0/2855823670 shutdown_connections 2024-09-12T03:28:31.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.520+0000 7f517fcc2740 1 -- 172.21.15.6:0/2855823670 wait complete. 2024-09-12T03:28:31.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.520+0000 7f517fcc2740 20 client.25852 trim_cache size 0 max 16384 2024-09-12T03:28:31.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.538+0000 7f517fcc2740 1 Processor -- start 2024-09-12T03:28:31.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.538+0000 7f517fcc2740 1 -- start start 2024-09-12T03:28:31.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.539+0000 7f517fcc2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059eef90 0x5625059f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.539+0000 7f517fcc2740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059f2a00 0x562505a0dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.539+0000 7f517fcc2740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059f3320 0x5625059f9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.539+0000 7f517fcc2740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f51781adce0 con 0x5625059f2a00 2024-09-12T03:28:31.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.539+0000 7f517fcc2740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f51781ab550 con 0x5625059f3320 2024-09-12T03:28:31.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.539+0000 7f517fcc2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f51781a93c0 con 0x5625059eef90 2024-09-12T03:28:31.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.539+0000 7f51767fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059f2a00 0x562505a0dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.539+0000 7f5176ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059eef90 0x5625059f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.539+0000 7f51777fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059f3320 0x5625059f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.539+0000 7f5176ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059eef90 0x5625059f5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:55606/0 (socket says 172.21.15.6:55606) 2024-09-12T03:28:31.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.539+0000 7f5176ffd640 1 -- 172.21.15.6:0/2881370725 learned_addr learned my addr 172.21.15.6:0/2881370725 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:31.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.539+0000 7f5176ffd640 1 -- 172.21.15.6:0/2881370725 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059f3320 msgr2=0x5625059f9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:31.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.539+0000 7f5176ffd640 1 --2- 172.21.15.6:0/2881370725 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059f3320 0x5625059f9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.539+0000 7f5176ffd640 1 -- 172.21.15.6:0/2881370725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059f2a00 msgr2=0x562505a0dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:31.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.539+0000 7f5176ffd640 1 --2- 172.21.15.6:0/2881370725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059f2a00 0x562505a0dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.539+0000 7f5176ffd640 1 -- 172.21.15.6:0/2881370725 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625059eedc0 con 0x5625059eef90 2024-09-12T03:28:31.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.539+0000 7f5176ffd640 1 --2- 172.21.15.6:0/2881370725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059eef90 0x5625059f5cb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5168009fa0 tx=0x7f5168009330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=716a7b8745ffc840 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:31.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.540+0000 7f515d7fa640 1 -- 172.21.15.6:0/2881370725 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5168033070 con 0x5625059eef90 2024-09-12T03:28:31.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.540+0000 7f515d7fa640 1 -- 172.21.15.6:0/2881370725 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5168009b50 con 0x5625059eef90 2024-09-12T03:28:31.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.540+0000 7f515d7fa640 1 -- 172.21.15.6:0/2881370725 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51680052a0 con 0x5625059eef90 2024-09-12T03:28:31.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.540+0000 7f517fcc2740 1 -- 172.21.15.6:0/2881370725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059eef90 msgr2=0x5625059f5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:31.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.540+0000 7f517fcc2740 1 --2- 172.21.15.6:0/2881370725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059eef90 0x5625059f5cb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5168009fa0 tx=0x7f5168009330 comp rx=0 tx=0).stop 2024-09-12T03:28:31.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.541+0000 7f517fcc2740 1 -- 172.21.15.6:0/2881370725 shutdown_connections 2024-09-12T03:28:31.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.541+0000 7f517fcc2740 1 --2- 172.21.15.6:0/2881370725 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059f3320 0x5625059f9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.541+0000 7f517fcc2740 1 --2- 172.21.15.6:0/2881370725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059f2a00 0x562505a0dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.541+0000 7f517fcc2740 1 --2- 172.21.15.6:0/2881370725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059eef90 0x5625059f5cb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.541+0000 7f517fcc2740 1 -- 172.21.15.6:0/2881370725 >> 172.21.15.6:0/2881370725 conn(0x5625059fae00 msgr2=0x5625059ee1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:31.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.541+0000 7f517fcc2740 1 -- 172.21.15.6:0/2881370725 shutdown_connections 2024-09-12T03:28:31.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.541+0000 7f517fcc2740 1 -- 172.21.15.6:0/2881370725 wait complete. 2024-09-12T03:28:31.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.541+0000 7f517fcc2740 1 -- start start 2024-09-12T03:28:31.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.541+0000 7f517fcc2740 1 Processor -- start 2024-09-12T03:28:31.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.541+0000 7f517fcc2740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059eef90 0x562505a18370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.541+0000 7f517fcc2740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059f2a00 0x562505a188b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.541+0000 7f517fcc2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059f3320 0x562505a19810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.542+0000 7f517fcc2740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f51781a9a10 con 0x5625059f2a00 2024-09-12T03:28:31.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.542+0000 7f517fcc2740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f51781aae80 con 0x5625059eef90 2024-09-12T03:28:31.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.542+0000 7f517fcc2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f51781aa0e0 con 0x5625059f3320 2024-09-12T03:28:31.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.542+0000 7f51767fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059f2a00 0x562505a188b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.542+0000 7f5176ffd640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059eef90 0x562505a18370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.542+0000 7f517fcc2740 10 client.0 _ll_register_callbacks cb 0x7f516c005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:31.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.542+0000 7f5176ffd640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059eef90 0x562505a18370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.6:34106/0 (socket says 172.21.15.6:34106) 2024-09-12T03:28:31.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.542+0000 7f5176ffd640 1 -- 172.21.15.6:0/1487952018 learned_addr learned my addr 172.21.15.6:0/1487952018 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:31.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.542+0000 7f51777fe640 1 --2- 172.21.15.6:0/1487952018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059f3320 0x562505a19810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.542+0000 7f5176ffd640 1 -- 172.21.15.6:0/1487952018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059f3320 msgr2=0x562505a19810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:31.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.542+0000 7f5176ffd640 1 --2- 172.21.15.6:0/1487952018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059f3320 0x562505a19810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.542+0000 7f5176ffd640 1 -- 172.21.15.6:0/1487952018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059f2a00 msgr2=0x562505a188b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:31.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.542+0000 7f5176ffd640 1 --2- 172.21.15.6:0/1487952018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059f2a00 0x562505a188b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.542+0000 7f5176ffd640 1 -- 172.21.15.6:0/1487952018 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562505a04aa0 con 0x5625059eef90 2024-09-12T03:28:31.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.542+0000 7f51767fc640 1 --2- 172.21.15.6:0/1487952018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059f2a00 0x562505a188b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:31.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.542+0000 7f5176ffd640 1 --2- 172.21.15.6:0/1487952018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059eef90 0x562505a18370 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5168009560 tx=0x7f5168003a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e7565ade0f7cf31 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:31.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.542+0000 7f5156ffd640 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-09-12T03:28:31.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.542+0000 7f5156ffd640 1 -- 172.21.15.6:0/1487952018 <== mon.2 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516803c060 con 0x5625059eef90 2024-09-12T03:28:31.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.542+0000 7f5156ffd640 1 -- 172.21.15.6:0/1487952018 <== mon.2 v2:172.21.15.118:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f516800f7a0 con 0x5625059eef90 2024-09-12T03:28:31.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.542+0000 7f517fcc2740 1 -- 172.21.15.6:0/1487952018 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5168009b50 con 0x5625059eef90 2024-09-12T03:28:31.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.543+0000 7f5156ffd640 1 -- 172.21.15.6:0/1487952018 <== mon.2 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5168033040 con 0x5625059eef90 2024-09-12T03:28:31.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.543+0000 7f517fcc2740 10 client.25757 fetch_fsmap learned FSMap version 29 2024-09-12T03:28:31.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.543+0000 7f517fcc2740 1 -- 172.21.15.6:0/1487952018 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562505a7d3d0 con 0x5625059eef90 2024-09-12T03:28:31.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.543+0000 7f5156ffd640 1 -- 172.21.15.6:0/1487952018 <== mon.2 v2:172.21.15.118:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5168009b50 con 0x5625059eef90 2024-09-12T03:28:31.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.543+0000 7f5156ffd640 1 -- 172.21.15.6:0/1487952018 <== mon.2 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5168036070 con 0x5625059eef90 2024-09-12T03:28:31.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.543+0000 7f517fcc2740 10 client.25757 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T03:28:31.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.543+0000 7f517fcc2740 10 client.25757 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:28:31.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.543+0000 7f517fcc2740 20 client.25757 populate_metadata read hostname 'smithi006' 2024-09-12T03:28:31.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.543+0000 7f517fcc2740 1 --2- 172.21.15.6:0/1487952018 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5625059fdbe0 0x562505a7f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.543+0000 7f517fcc2740 4 client.25757 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-12T03:28:31.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.543+0000 7f517fcc2740 1 -- 172.21.15.6:0/1487952018 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x56250595ec80 con 0x5625059fdbe0 2024-09-12T03:28:31.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.543+0000 7f51767fc640 1 --2- 172.21.15.6:0/1487952018 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5625059fdbe0 0x562505a7f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.544+0000 7f5156ffd640 10 client.25757 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:31.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.544+0000 7f51767fc640 1 --2- 172.21.15.6:0/1487952018 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5625059fdbe0 0x562505a7f870 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5485b670db067d8 server_cookie=69ca9e3c438c261a in_seq=0 out_seq=0 2024-09-12T03:28:31.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.544+0000 7f5156ffd640 1 -- 172.21.15.6:0/1487952018 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x56250595ec80 con 0x5625059fdbe0 2024-09-12T03:28:31.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.544+0000 7f5156ffd640 10 client.25757 handle_command_reply: tid=0 2024-09-12T03:28:31.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.544+0000 7f517fcc2740 1 client.25757 shutdown 2024-09-12T03:28:31.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.544+0000 7f517fcc2740 1 -- 172.21.15.6:0/1487952018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059eef90 msgr2=0x562505a18370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:31.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.544+0000 7f517fcc2740 1 --2- 172.21.15.6:0/1487952018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059eef90 0x562505a18370 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5168009560 tx=0x7f5168003a30 comp rx=0 tx=0).stop 2024-09-12T03:28:31.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.544+0000 7f517fcc2740 1 -- 172.21.15.6:0/1487952018 shutdown_connections 2024-09-12T03:28:31.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.544+0000 7f517fcc2740 1 --2- 172.21.15.6:0/1487952018 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x5625059fdbe0 0x562505a7f870 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.544+0000 7f517fcc2740 1 --2- 172.21.15.6:0/1487952018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625059f3320 0x562505a19810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.544+0000 7f517fcc2740 1 --2- 172.21.15.6:0/1487952018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625059f2a00 0x562505a188b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.544+0000 7f517fcc2740 1 --2- 172.21.15.6:0/1487952018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625059eef90 0x562505a18370 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.544+0000 7f5156ffd640 0 client.25757 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:31.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.544+0000 7f517fcc2740 1 -- 172.21.15.6:0/1487952018 >> 172.21.15.6:0/1487952018 conn(0x5625059fae00 msgr2=0x5625059fbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:31.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.544+0000 7f517fcc2740 1 -- 172.21.15.6:0/1487952018 shutdown_connections 2024-09-12T03:28:31.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.544+0000 7f517fcc2740 1 -- 172.21.15.6:0/1487952018 wait complete. 2024-09-12T03:28:31.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.544+0000 7f517fcc2740 20 client.25757 trim_cache size 0 max 16384 2024-09-12T03:28:31.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.545+0000 7f517d8e9640 1 -- 172.21.15.6:0/2023519181 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f51600a2b80 msgr2=0x7f51600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:31.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.545+0000 7f517d8e9640 1 --2- 172.21.15.6:0/2023519181 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f51600a2b80 0x7f51600a5040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f51700045e0 tx=0x7f51700046c0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.545+0000 7f517d8e9640 1 -- 172.21.15.6:0/2023519181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5178103740 msgr2=0x7f5178198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:31.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.545+0000 7f517d8e9640 1 --2- 172.21.15.6:0/2023519181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5178103740 0x7f5178198af0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f516c005940 tx=0x7f516c0040c0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.545+0000 7f517d8e9640 1 -- 172.21.15.6:0/2023519181 shutdown_connections 2024-09-12T03:28:31.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.545+0000 7f517d8e9640 1 --2- 172.21.15.6:0/2023519181 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f51600a2b80 0x7f51600a5040 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.545+0000 7f517d8e9640 1 --2- 172.21.15.6:0/2023519181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5178108290 0x7f517819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.545+0000 7f517d8e9640 1 --2- 172.21.15.6:0/2023519181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5178103740 0x7f5178198af0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.545+0000 7f517d8e9640 1 --2- 172.21.15.6:0/2023519181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5178102d90 0x7f51781985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.545+0000 7f517d8e9640 1 -- 172.21.15.6:0/2023519181 >> 172.21.15.6:0/2023519181 conn(0x7f51780fc7d0 msgr2=0x7f5178105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:31.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.545+0000 7f517d8e9640 1 -- 172.21.15.6:0/2023519181 shutdown_connections 2024-09-12T03:28:31.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:31.545+0000 7f517d8e9640 1 -- 172.21.15.6:0/2023519181 wait complete. 2024-09-12T03:28:32.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:31 smithi006 ceph-mon[30991]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:28:32.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:31 smithi161 ceph-mon[35485]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:28:32.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:31 smithi118 ceph-mon[35048]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:28:32.421 INFO:tasks.quiescer:done joining Quiescers 2024-09-12T03:28:32.422 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-12T03:28:32.433 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-12T03:28:33.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:32 smithi006 ceph-mon[30991]: pgmap v558: 97 pgs: 97 active+clean; 3.0 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 97 KiB/s rd, 17 MiB/s wr, 3.51k op/s 2024-09-12T03:28:33.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:32 smithi161 ceph-mon[35485]: pgmap v558: 97 pgs: 97 active+clean; 3.0 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 97 KiB/s rd, 17 MiB/s wr, 3.51k op/s 2024-09-12T03:28:33.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:32 smithi118 ceph-mon[35048]: pgmap v558: 97 pgs: 97 active+clean; 3.0 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 97 KiB/s rd, 17 MiB/s wr, 3.51k op/s 2024-09-12T03:28:35.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:34 smithi006 ceph-mon[30991]: pgmap v559: 97 pgs: 97 active+clean; 952 MiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 168 KiB/s rd, 1.8 MiB/s wr, 613 op/s 2024-09-12T03:28:35.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:34 smithi161 ceph-mon[35485]: pgmap v559: 97 pgs: 97 active+clean; 952 MiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 168 KiB/s rd, 1.8 MiB/s wr, 613 op/s 2024-09-12T03:28:35.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:34 smithi118 ceph-mon[35048]: pgmap v559: 97 pgs: 97 active+clean; 952 MiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 168 KiB/s rd, 1.8 MiB/s wr, 613 op/s 2024-09-12T03:28:37.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:36 smithi006 ceph-mon[30991]: pgmap v560: 97 pgs: 97 active+clean; 224 MiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 162 KiB/s rd, 150 KiB/s wr, 296 op/s 2024-09-12T03:28:37.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:36 smithi161 ceph-mon[35485]: pgmap v560: 97 pgs: 97 active+clean; 224 MiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 162 KiB/s rd, 150 KiB/s wr, 296 op/s 2024-09-12T03:28:37.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:36 smithi118 ceph-mon[35048]: pgmap v560: 97 pgs: 97 active+clean; 224 MiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 162 KiB/s rd, 150 KiB/s wr, 296 op/s 2024-09-12T03:28:39.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:38 smithi006 ceph-mon[30991]: pgmap v561: 97 pgs: 97 active+clean; 159 MiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 164 KiB/s rd, 150 KiB/s wr, 303 op/s 2024-09-12T03:28:39.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:38 smithi161 ceph-mon[35485]: pgmap v561: 97 pgs: 97 active+clean; 159 MiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 164 KiB/s rd, 150 KiB/s wr, 303 op/s 2024-09-12T03:28:39.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:38 smithi118 ceph-mon[35048]: pgmap v561: 97 pgs: 97 active+clean; 159 MiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 164 KiB/s rd, 150 KiB/s wr, 303 op/s 2024-09-12T03:28:41.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:40 smithi006 ceph-mon[30991]: pgmap v562: 97 pgs: 97 active+clean; 159 MiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 76 KiB/s rd, 34 KiB/s wr, 287 op/s 2024-09-12T03:28:41.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:40 smithi161 ceph-mon[35485]: pgmap v562: 97 pgs: 97 active+clean; 159 MiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 76 KiB/s rd, 34 KiB/s wr, 287 op/s 2024-09-12T03:28:41.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:40 smithi118 ceph-mon[35048]: pgmap v562: 97 pgs: 97 active+clean; 159 MiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 76 KiB/s rd, 34 KiB/s wr, 287 op/s 2024-09-12T03:28:43.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:42 smithi006 ceph-mon[30991]: pgmap v563: 97 pgs: 97 active+clean; 159 MiB data, 116 GiB used, 957 GiB / 1.0 TiB avail; 75 KiB/s rd, 33 KiB/s wr, 273 op/s 2024-09-12T03:28:43.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:42 smithi161 ceph-mon[35485]: pgmap v563: 97 pgs: 97 active+clean; 159 MiB data, 116 GiB used, 957 GiB / 1.0 TiB avail; 75 KiB/s rd, 33 KiB/s wr, 273 op/s 2024-09-12T03:28:43.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:42 smithi118 ceph-mon[35048]: pgmap v563: 97 pgs: 97 active+clean; 159 MiB data, 116 GiB used, 957 GiB / 1.0 TiB avail; 75 KiB/s rd, 33 KiB/s wr, 273 op/s 2024-09-12T03:28:45.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:44 smithi006 ceph-mon[30991]: pgmap v564: 97 pgs: 97 active+clean; 159 MiB data, 116 GiB used, 957 GiB / 1.0 TiB avail; 76 KiB/s rd, 33 KiB/s wr, 83 op/s 2024-09-12T03:28:45.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:44 smithi161 ceph-mon[35485]: pgmap v564: 97 pgs: 97 active+clean; 159 MiB data, 116 GiB used, 957 GiB / 1.0 TiB avail; 76 KiB/s rd, 33 KiB/s wr, 83 op/s 2024-09-12T03:28:45.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:44 smithi118 ceph-mon[35048]: pgmap v564: 97 pgs: 97 active+clean; 159 MiB data, 116 GiB used, 957 GiB / 1.0 TiB avail; 76 KiB/s rd, 33 KiB/s wr, 83 op/s 2024-09-12T03:28:47.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:46 smithi006 ceph-mon[30991]: pgmap v565: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.8 KiB/s rd, 938 B/s wr, 31 op/s 2024-09-12T03:28:47.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:46 smithi161 ceph-mon[35485]: pgmap v565: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.8 KiB/s rd, 938 B/s wr, 31 op/s 2024-09-12T03:28:47.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:46 smithi118 ceph-mon[35048]: pgmap v565: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.8 KiB/s rd, 938 B/s wr, 31 op/s 2024-09-12T03:28:49.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:48 smithi006 ceph-mon[30991]: pgmap v566: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-12T03:28:49.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:48 smithi161 ceph-mon[35485]: pgmap v566: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-12T03:28:49.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:48 smithi118 ceph-mon[35048]: pgmap v566: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-12T03:28:51.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:50 smithi006 ceph-mon[30991]: pgmap v567: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T03:28:51.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:50 smithi161 ceph-mon[35485]: pgmap v567: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T03:28:51.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:50 smithi118 ceph-mon[35048]: pgmap v567: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T03:28:52.135 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:28:52.392 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:52.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.875+0000 7eff8f308640 1 -- 172.21.15.6:0/442183389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff88103740 msgr2=0x7eff88107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:52.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.875+0000 7eff8f308640 1 --2- 172.21.15.6:0/442183389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff88103740 0x7eff88107b80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009930 tx=0x7eff7c02a420 comp rx=0 tx=0).stop 2024-09-12T03:28:52.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.875+0000 7eff8f308640 1 -- 172.21.15.6:0/442183389 shutdown_connections 2024-09-12T03:28:52.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.875+0000 7eff8f308640 1 --2- 172.21.15.6:0/442183389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff88108290 0x7eff8810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.875+0000 7eff8f308640 1 --2- 172.21.15.6:0/442183389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff88103740 0x7eff88107b80 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.875+0000 7eff8f308640 1 --2- 172.21.15.6:0/442183389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff88102d90 0x7eff88103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.875+0000 7eff8f308640 1 -- 172.21.15.6:0/442183389 >> 172.21.15.6:0/442183389 conn(0x7eff880fc7d0 msgr2=0x7eff880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:52.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.875+0000 7eff8f308640 1 -- 172.21.15.6:0/442183389 shutdown_connections 2024-09-12T03:28:52.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.875+0000 7eff8f308640 1 -- 172.21.15.6:0/442183389 wait complete. 2024-09-12T03:28:52.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.876+0000 7eff8f308640 1 Processor -- start 2024-09-12T03:28:52.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.876+0000 7eff8f308640 1 -- start start 2024-09-12T03:28:52.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff8f308640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff88102d90 0x7eff88198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff8f308640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff88103740 0x7eff88198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff8f308640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff88108290 0x7eff8819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff8f308640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7eff88112090 con 0x7eff88103740 2024-09-12T03:28:52.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff8f308640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7eff88111f10 con 0x7eff88108290 2024-09-12T03:28:52.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff8f308640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff88112210 con 0x7eff88102d90 2024-09-12T03:28:52.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff8c8a3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff88103740 0x7eff88198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff8c8a3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff88103740 0x7eff88198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35920/0 (socket says 172.21.15.6:35920) 2024-09-12T03:28:52.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff8c8a3640 1 -- 172.21.15.6:0/3395181910 learned_addr learned my addr 172.21.15.6:0/3395181910 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:52.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff8d0a4640 1 --2- 172.21.15.6:0/3395181910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff88102d90 0x7eff88198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff8c8a3640 1 -- 172.21.15.6:0/3395181910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff88102d90 msgr2=0x7eff88198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:52.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff8d8a5640 1 --2- 172.21.15.6:0/3395181910 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff88108290 0x7eff8819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff8c8a3640 1 --2- 172.21.15.6:0/3395181910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff88102d90 0x7eff88198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff8c8a3640 1 -- 172.21.15.6:0/3395181910 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff88108290 msgr2=0x7eff8819cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:52.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff8c8a3640 1 --2- 172.21.15.6:0/3395181910 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff88108290 0x7eff8819cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff8c8a3640 1 -- 172.21.15.6:0/3395181910 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8819d600 con 0x7eff88103740 2024-09-12T03:28:52.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff8d8a5640 1 --2- 172.21.15.6:0/3395181910 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff88108290 0x7eff8819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:52.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff8c8a3640 1 --2- 172.21.15.6:0/3395181910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff88103740 0x7eff88198b90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7eff7c005940 tx=0x7eff7c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:52.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff867fc640 1 -- 172.21.15.6:0/3395181910 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c03f030 con 0x7eff88103740 2024-09-12T03:28:52.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff867fc640 1 -- 172.21.15.6:0/3395181910 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff7c0044c0 con 0x7eff88103740 2024-09-12T03:28:52.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff8f308640 1 -- 172.21.15.6:0/3395181910 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8819d890 con 0x7eff88103740 2024-09-12T03:28:52.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.877+0000 7eff867fc640 1 -- 172.21.15.6:0/3395181910 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c035070 con 0x7eff88103740 2024-09-12T03:28:52.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.878+0000 7eff8f308640 1 -- 172.21.15.6:0/3395181910 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff881a51d0 con 0x7eff88103740 2024-09-12T03:28:52.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.880+0000 7eff867fc640 1 -- 172.21.15.6:0/3395181910 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7eff7c042030 con 0x7eff88103740 2024-09-12T03:28:52.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.880+0000 7eff916e1740 1 Processor -- start 2024-09-12T03:28:52.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.880+0000 7eff916e1740 1 -- start start 2024-09-12T03:28:52.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.880+0000 7eff916e1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56321be3c380 0x56321be3c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.880+0000 7eff916e1740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3cca0 0x56321be4b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.880+0000 7eff916e1740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 0x56321be4de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.880+0000 7eff916e1740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56321bda22e0 con 0x56321be3cca0 2024-09-12T03:28:52.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.880+0000 7eff916e1740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x56321bd49ce0 con 0x56321be4ba90 2024-09-12T03:28:52.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.880+0000 7eff916e1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56321bd9e2a0 con 0x56321be3c380 2024-09-12T03:28:52.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.881+0000 7eff8d0a4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3cca0 0x56321be4b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.881+0000 7eff8d0a4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3cca0 0x56321be4b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35936/0 (socket says 172.21.15.6:35936) 2024-09-12T03:28:52.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.881+0000 7eff8d0a4640 1 -- 172.21.15.6:0/4049250513 learned_addr learned my addr 172.21.15.6:0/4049250513 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:52.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.881+0000 7eff867fc640 1 --2- 172.21.15.6:0/3395181910 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7eff700a2b80 0x7eff700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.881+0000 7eff8d8a5640 1 --2- 172.21.15.6:0/4049250513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56321be3c380 0x56321be3c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.881+0000 7eff8c8a3640 1 --2- 172.21.15.6:0/4049250513 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 0x56321be4de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.881+0000 7eff8d8a5640 1 --2- 172.21.15.6:0/3395181910 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7eff700a2b80 0x7eff700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.881+0000 7eff8d0a4640 1 -- 172.21.15.6:0/4049250513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56321be3c380 msgr2=0x56321be3c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:52.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.881+0000 7eff8d0a4640 1 --2- 172.21.15.6:0/4049250513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56321be3c380 0x56321be3c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.881+0000 7eff8d0a4640 1 -- 172.21.15.6:0/4049250513 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 msgr2=0x56321be4de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:52.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.881+0000 7eff8d0a4640 1 --2- 172.21.15.6:0/4049250513 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 0x56321be4de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.881+0000 7eff8d0a4640 1 -- 172.21.15.6:0/4049250513 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321bd2e440 con 0x56321be3cca0 2024-09-12T03:28:52.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.881+0000 7eff8c8a3640 1 --2- 172.21.15.6:0/4049250513 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 0x56321be4de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:52.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.881+0000 7eff8d0a4640 1 --2- 172.21.15.6:0/4049250513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3cca0 0x56321be4b550 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7eff80009cf0 tx=0x7eff80011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922259ee51f56be2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:52.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.881+0000 7eff867fc640 1 -- 172.21.15.6:0/3395181910 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7eff7c0ef450 con 0x7eff88103740 2024-09-12T03:28:52.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.881+0000 7eff6f7fe640 1 -- 172.21.15.6:0/4049250513 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff80004230 con 0x56321be3cca0 2024-09-12T03:28:52.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.881+0000 7eff8d8a5640 1 --2- 172.21.15.6:0/3395181910 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7eff700a2b80 0x7eff700a5040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7eff780094d0 tx=0x7eff78009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:52.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.881+0000 7eff6f7fe640 1 -- 172.21.15.6:0/4049250513 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff800043d0 con 0x56321be3cca0 2024-09-12T03:28:52.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.881+0000 7eff6f7fe640 1 -- 172.21.15.6:0/4049250513 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff80019900 con 0x56321be3cca0 2024-09-12T03:28:52.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.882+0000 7eff916e1740 1 -- 172.21.15.6:0/4049250513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3cca0 msgr2=0x56321be4b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:52.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.882+0000 7eff916e1740 1 --2- 172.21.15.6:0/4049250513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3cca0 0x56321be4b550 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7eff80009cf0 tx=0x7eff80011810 comp rx=0 tx=0).stop 2024-09-12T03:28:52.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.882+0000 7eff916e1740 1 -- 172.21.15.6:0/4049250513 shutdown_connections 2024-09-12T03:28:52.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.882+0000 7eff916e1740 1 --2- 172.21.15.6:0/4049250513 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 0x56321be4de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.882+0000 7eff916e1740 1 --2- 172.21.15.6:0/4049250513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3cca0 0x56321be4b550 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.882+0000 7eff916e1740 1 --2- 172.21.15.6:0/4049250513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56321be3c380 0x56321be3c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.882+0000 7eff916e1740 1 -- 172.21.15.6:0/4049250513 >> 172.21.15.6:0/4049250513 conn(0x56321be38a30 msgr2=0x56321be38e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:52.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.882+0000 7eff916e1740 1 -- 172.21.15.6:0/4049250513 shutdown_connections 2024-09-12T03:28:52.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.882+0000 7eff916e1740 1 -- 172.21.15.6:0/4049250513 wait complete. 2024-09-12T03:28:52.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.883+0000 7eff916e1740 1 -- start start 2024-09-12T03:28:52.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.883+0000 7eff916e1740 1 Processor -- start 2024-09-12T03:28:52.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.883+0000 7eff916e1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56321be3c380 0x56321be4b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.883+0000 7eff916e1740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3cca0 0x56321be48620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.883+0000 7eff916e1740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 0x56321be48b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.883+0000 7eff916e1740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56321be4e660 con 0x56321be3cca0 2024-09-12T03:28:52.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.883+0000 7eff916e1740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x56321bd5d050 con 0x56321be4ba90 2024-09-12T03:28:52.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.883+0000 7eff916e1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56321be4e7e0 con 0x56321be3c380 2024-09-12T03:28:52.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.883+0000 7eff916e1740 10 client.0 _ll_register_callbacks cb 0x56321bd9d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:52.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.883+0000 7eff8c8a3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3cca0 0x56321be48620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.884+0000 7eff8c8a3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3cca0 0x56321be48620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35942/0 (socket says 172.21.15.6:35942) 2024-09-12T03:28:52.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.884+0000 7eff8c8a3640 1 -- 172.21.15.6:0/3079670720 learned_addr learned my addr 172.21.15.6:0/3079670720 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:52.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.884+0000 7eff8d0a4640 1 --2- 172.21.15.6:0/3079670720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56321be3c380 0x56321be4b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.884+0000 7eff8d8a5640 1 --2- 172.21.15.6:0/3079670720 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 0x56321be48b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.884+0000 7eff8c8a3640 1 -- 172.21.15.6:0/3079670720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56321be3c380 msgr2=0x56321be4b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:52.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.884+0000 7eff8c8a3640 1 --2- 172.21.15.6:0/3079670720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56321be3c380 0x56321be4b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.884+0000 7eff8c8a3640 1 -- 172.21.15.6:0/3079670720 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 msgr2=0x56321be48b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:52.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.884+0000 7eff8c8a3640 1 --2- 172.21.15.6:0/3079670720 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 0x56321be48b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.884+0000 7eff8c8a3640 1 -- 172.21.15.6:0/3079670720 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56321bb133c0 con 0x56321be3cca0 2024-09-12T03:28:52.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.884+0000 7eff8d0a4640 1 --2- 172.21.15.6:0/3079670720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56321be3c380 0x56321be4b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:52.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.884+0000 7eff8c8a3640 1 --2- 172.21.15.6:0/3079670720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3cca0 0x56321be48620 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7eff7c0c4420 tx=0x7eff7c0bb700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97809f969d1503d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:52.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.884+0000 7eff6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:28:52.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.884+0000 7eff6cff9640 1 -- 172.21.15.6:0/3079670720 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c049040 con 0x56321be3cca0 2024-09-12T03:28:52.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.884+0000 7eff6cff9640 1 -- 172.21.15.6:0/3079670720 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7eff7c0b9fe0 con 0x56321be3cca0 2024-09-12T03:28:52.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.884+0000 7eff916e1740 1 -- 172.21.15.6:0/3079670720 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff800043d0 con 0x56321be3cca0 2024-09-12T03:28:52.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.884+0000 7eff6cff9640 1 -- 172.21.15.6:0/3079670720 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7eff7c0d9050 con 0x56321be3cca0 2024-09-12T03:28:52.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.884+0000 7eff916e1740 10 client.16224 fetch_fsmap learned FSMap version 29 2024-09-12T03:28:52.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.884+0000 7eff916e1740 1 -- 172.21.15.6:0/3079670720 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56321be59f60 con 0x56321be3cca0 2024-09-12T03:28:52.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.885+0000 7eff6cff9640 1 -- 172.21.15.6:0/3079670720 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff800043d0 con 0x56321be3cca0 2024-09-12T03:28:52.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.885+0000 7eff6cff9640 1 -- 172.21.15.6:0/3079670720 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7eff7c0bbf40 con 0x56321be3cca0 2024-09-12T03:28:52.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.885+0000 7eff916e1740 10 client.16224 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T03:28:52.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.885+0000 7eff916e1740 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:28:52.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.885+0000 7eff916e1740 20 client.16224 populate_metadata read hostname 'smithi006' 2024-09-12T03:28:52.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.885+0000 7eff916e1740 1 --2- 172.21.15.6:0/3079670720 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56321be5b3a0 0x56321be5d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.885+0000 7eff916e1740 4 client.16224 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:52.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.885+0000 7eff916e1740 1 -- 172.21.15.6:0/3079670720 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56321bdaf850 con 0x56321be5b3a0 2024-09-12T03:28:52.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.885+0000 7eff8d0a4640 1 --2- 172.21.15.6:0/3079670720 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56321be5b3a0 0x56321be5d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.886+0000 7eff6cff9640 10 client.16224 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:52.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.886+0000 7eff8d0a4640 1 --2- 172.21.15.6:0/3079670720 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56321be5b3a0 0x56321be5d7f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10eaad547a88aaee server_cookie=6c564486f6029b65 in_seq=0 out_seq=0 2024-09-12T03:28:52.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.886+0000 7eff6cff9640 1 -- 172.21.15.6:0/3079670720 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56321bdaf850 con 0x56321be5b3a0 2024-09-12T03:28:52.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.886+0000 7eff6cff9640 10 client.16224 handle_command_reply: tid=0 2024-09-12T03:28:52.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.886+0000 7eff916e1740 1 client.16224 shutdown 2024-09-12T03:28:52.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.887+0000 7eff916e1740 1 -- 172.21.15.6:0/3079670720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3cca0 msgr2=0x56321be48620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:52.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.887+0000 7eff916e1740 1 --2- 172.21.15.6:0/3079670720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3cca0 0x56321be48620 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7eff7c0c4420 tx=0x7eff7c0bb700 comp rx=0 tx=0).stop 2024-09-12T03:28:52.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.887+0000 7eff916e1740 1 -- 172.21.15.6:0/3079670720 shutdown_connections 2024-09-12T03:28:52.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.887+0000 7eff916e1740 1 --2- 172.21.15.6:0/3079670720 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56321be5b3a0 0x56321be5d7f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.887+0000 7eff916e1740 1 --2- 172.21.15.6:0/3079670720 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 0x56321be48b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.887+0000 7eff916e1740 1 --2- 172.21.15.6:0/3079670720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3cca0 0x56321be48620 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.887+0000 7eff6cff9640 0 client.16224 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:52.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.887+0000 7eff916e1740 1 --2- 172.21.15.6:0/3079670720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56321be3c380 0x56321be4b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.887+0000 7eff916e1740 1 -- 172.21.15.6:0/3079670720 >> 172.21.15.6:0/3079670720 conn(0x56321be38a30 msgr2=0x56321be3e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:52.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.887+0000 7eff916e1740 1 -- 172.21.15.6:0/3079670720 shutdown_connections 2024-09-12T03:28:52.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.887+0000 7eff916e1740 1 -- 172.21.15.6:0/3079670720 wait complete. 2024-09-12T03:28:52.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.887+0000 7eff916e1740 20 client.16224 trim_cache size 0 max 16384 2024-09-12T03:28:52.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.905+0000 7eff916e1740 1 Processor -- start 2024-09-12T03:28:52.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.905+0000 7eff916e1740 1 -- start start 2024-09-12T03:28:52.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.905+0000 7eff916e1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56321be3c380 0x56321be57900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.905+0000 7eff916e1740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3cca0 0x56321be63100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.905+0000 7eff916e1740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 0x56321be63640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.905+0000 7eff916e1740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7eff881adcc0 con 0x56321be3cca0 2024-09-12T03:28:52.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.905+0000 7eff916e1740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7eff881ab530 con 0x56321be4ba90 2024-09-12T03:28:52.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.905+0000 7eff916e1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff881a9380 con 0x56321be3c380 2024-09-12T03:28:52.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.905+0000 7eff8c8a3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3cca0 0x56321be63100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.905+0000 7eff8c8a3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3cca0 0x56321be63100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35956/0 (socket says 172.21.15.6:35956) 2024-09-12T03:28:52.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.905+0000 7eff8c8a3640 1 -- 172.21.15.6:0/2280059524 learned_addr learned my addr 172.21.15.6:0/2280059524 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:52.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.905+0000 7eff8d0a4640 1 --2- 172.21.15.6:0/2280059524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56321be3c380 0x56321be57900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.906+0000 7eff8d8a5640 1 --2- 172.21.15.6:0/2280059524 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 0x56321be63640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.906+0000 7eff8c8a3640 1 -- 172.21.15.6:0/2280059524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56321be3c380 msgr2=0x56321be57900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:52.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.906+0000 7eff8c8a3640 1 --2- 172.21.15.6:0/2280059524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56321be3c380 0x56321be57900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.906+0000 7eff8c8a3640 1 -- 172.21.15.6:0/2280059524 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 msgr2=0x56321be63640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:52.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.906+0000 7eff8c8a3640 1 --2- 172.21.15.6:0/2280059524 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 0x56321be63640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.906+0000 7eff8c8a3640 1 -- 172.21.15.6:0/2280059524 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321be3cad0 con 0x56321be3cca0 2024-09-12T03:28:52.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.906+0000 7eff8d8a5640 1 --2- 172.21.15.6:0/2280059524 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 0x56321be63640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:52.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.906+0000 7eff8c8a3640 1 --2- 172.21.15.6:0/2280059524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3cca0 0x56321be63100 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7eff7c0ecfd0 tx=0x7eff7c03b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3163b8144e066963 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:52.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.906+0000 7eff6effd640 1 -- 172.21.15.6:0/2280059524 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c049070 con 0x56321be3cca0 2024-09-12T03:28:52.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.906+0000 7eff6effd640 1 -- 172.21.15.6:0/2280059524 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff7c09f020 con 0x56321be3cca0 2024-09-12T03:28:52.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.906+0000 7eff6effd640 1 -- 172.21.15.6:0/2280059524 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c0b9fe0 con 0x56321be3cca0 2024-09-12T03:28:52.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.907+0000 7eff916e1740 1 -- 172.21.15.6:0/2280059524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3cca0 msgr2=0x56321be63100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:52.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.907+0000 7eff916e1740 1 --2- 172.21.15.6:0/2280059524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3cca0 0x56321be63100 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7eff7c0ecfd0 tx=0x7eff7c03b5d0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.907+0000 7eff916e1740 1 -- 172.21.15.6:0/2280059524 shutdown_connections 2024-09-12T03:28:52.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.907+0000 7eff916e1740 1 --2- 172.21.15.6:0/2280059524 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 0x56321be63640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.907+0000 7eff916e1740 1 --2- 172.21.15.6:0/2280059524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3cca0 0x56321be63100 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.907+0000 7eff916e1740 1 --2- 172.21.15.6:0/2280059524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56321be3c380 0x56321be57900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.907+0000 7eff916e1740 1 -- 172.21.15.6:0/2280059524 >> 172.21.15.6:0/2280059524 conn(0x56321be38a30 msgr2=0x56321be46ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:52.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.907+0000 7eff916e1740 1 -- 172.21.15.6:0/2280059524 shutdown_connections 2024-09-12T03:28:52.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.907+0000 7eff916e1740 1 -- 172.21.15.6:0/2280059524 wait complete. 2024-09-12T03:28:52.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.907+0000 7eff916e1740 1 -- start start 2024-09-12T03:28:52.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.907+0000 7eff916e1740 1 Processor -- start 2024-09-12T03:28:52.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.908+0000 7eff916e1740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3c380 0x56321be4f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.908+0000 7eff916e1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56321be3cca0 0x56321be4fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.908+0000 7eff916e1740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 0x56321be502f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.908+0000 7eff916e1740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7eff881a99f0 con 0x56321be3c380 2024-09-12T03:28:52.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.908+0000 7eff916e1740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7eff881aae60 con 0x56321be4ba90 2024-09-12T03:28:52.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.908+0000 7eff916e1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff881aa0c0 con 0x56321be3cca0 2024-09-12T03:28:52.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.908+0000 7eff916e1740 10 client.0 _ll_register_callbacks cb 0x7eff7c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:52.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.908+0000 7eff8d0a4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3c380 0x56321be4f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.908+0000 7eff8d0a4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3c380 0x56321be4f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35970/0 (socket says 172.21.15.6:35970) 2024-09-12T03:28:52.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.908+0000 7eff8d0a4640 1 -- 172.21.15.6:0/4175171113 learned_addr learned my addr 172.21.15.6:0/4175171113 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:52.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.908+0000 7eff8c8a3640 1 --2- 172.21.15.6:0/4175171113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56321be3cca0 0x56321be4fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.908+0000 7eff8d8a5640 1 --2- 172.21.15.6:0/4175171113 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 0x56321be502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.908+0000 7eff8d0a4640 1 -- 172.21.15.6:0/4175171113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56321be3cca0 msgr2=0x56321be4fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:52.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.908+0000 7eff8d0a4640 1 --2- 172.21.15.6:0/4175171113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56321be3cca0 0x56321be4fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.908+0000 7eff8d0a4640 1 -- 172.21.15.6:0/4175171113 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 msgr2=0x56321be502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:52.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.908+0000 7eff8d0a4640 1 --2- 172.21.15.6:0/4175171113 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 0x56321be502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.908+0000 7eff8d0a4640 1 -- 172.21.15.6:0/4175171113 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56321bec5180 con 0x56321be3c380 2024-09-12T03:28:52.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.909+0000 7eff8d0a4640 1 --2- 172.21.15.6:0/4175171113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3c380 0x56321be4f870 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7eff80009cf0 tx=0x7eff80009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46ac8712b738019 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:52.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.909+0000 7eff6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:28:52.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.909+0000 7eff6cff9640 1 -- 172.21.15.6:0/4175171113 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff80005450 con 0x56321be3c380 2024-09-12T03:28:52.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.909+0000 7eff6cff9640 1 -- 172.21.15.6:0/4175171113 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7eff800055f0 con 0x56321be3c380 2024-09-12T03:28:52.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.909+0000 7eff916e1740 1 -- 172.21.15.6:0/4175171113 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff7c09f020 con 0x56321be3c380 2024-09-12T03:28:52.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.909+0000 7eff6cff9640 1 -- 172.21.15.6:0/4175171113 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7eff8003c050 con 0x56321be3c380 2024-09-12T03:28:52.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.909+0000 7eff916e1740 10 client.16230 fetch_fsmap learned FSMap version 29 2024-09-12T03:28:52.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.909+0000 7eff916e1740 1 -- 172.21.15.6:0/4175171113 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56321bec6470 con 0x56321be3c380 2024-09-12T03:28:52.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.909+0000 7eff6cff9640 1 -- 172.21.15.6:0/4175171113 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff80003ad0 con 0x56321be3c380 2024-09-12T03:28:52.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.909+0000 7eff6cff9640 1 -- 172.21.15.6:0/4175171113 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7eff80019c20 con 0x56321be3c380 2024-09-12T03:28:52.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.909+0000 7eff916e1740 10 client.16230 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T03:28:52.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.909+0000 7eff916e1740 10 client.16230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:28:52.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.909+0000 7eff916e1740 20 client.16230 populate_metadata read hostname 'smithi006' 2024-09-12T03:28:52.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.909+0000 7eff916e1740 1 --2- 172.21.15.6:0/4175171113 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56321be5b3a0 0x56321bec89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.909+0000 7eff916e1740 4 client.16230 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:28:52.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.909+0000 7eff916e1740 1 -- 172.21.15.6:0/4175171113 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7eff7c09f020 con 0x56321be5b3a0 2024-09-12T03:28:52.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.910+0000 7eff8c8a3640 1 --2- 172.21.15.6:0/4175171113 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56321be5b3a0 0x56321bec89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.910+0000 7eff6cff9640 10 client.16230 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:52.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.910+0000 7eff8c8a3640 1 --2- 172.21.15.6:0/4175171113 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56321be5b3a0 0x56321bec89b0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69e5a589a018eb1d server_cookie=8cd6810f7a0e98fd in_seq=0 out_seq=0 2024-09-12T03:28:52.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.910+0000 7eff6cff9640 1 -- 172.21.15.6:0/4175171113 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7eff7c09f020 con 0x56321be5b3a0 2024-09-12T03:28:52.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.910+0000 7eff6cff9640 10 client.16230 handle_command_reply: tid=0 2024-09-12T03:28:52.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.910+0000 7eff916e1740 1 client.16230 shutdown 2024-09-12T03:28:52.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.911+0000 7eff916e1740 1 -- 172.21.15.6:0/4175171113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3c380 msgr2=0x56321be4f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:52.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.911+0000 7eff916e1740 1 --2- 172.21.15.6:0/4175171113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3c380 0x56321be4f870 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7eff80009cf0 tx=0x7eff80009370 comp rx=0 tx=0).stop 2024-09-12T03:28:52.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.911+0000 7eff916e1740 1 -- 172.21.15.6:0/4175171113 shutdown_connections 2024-09-12T03:28:52.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.911+0000 7eff916e1740 1 --2- 172.21.15.6:0/4175171113 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x56321be5b3a0 0x56321bec89b0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.911+0000 7eff916e1740 1 --2- 172.21.15.6:0/4175171113 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56321be4ba90 0x56321be502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.911+0000 7eff916e1740 1 --2- 172.21.15.6:0/4175171113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56321be3cca0 0x56321be4fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.911+0000 7eff916e1740 1 --2- 172.21.15.6:0/4175171113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56321be3c380 0x56321be4f870 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.911+0000 7eff916e1740 1 -- 172.21.15.6:0/4175171113 >> 172.21.15.6:0/4175171113 conn(0x56321be38a30 msgr2=0x56321be4a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:52.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.911+0000 7eff6cff9640 0 client.16230 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:52.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.911+0000 7eff916e1740 1 -- 172.21.15.6:0/4175171113 shutdown_connections 2024-09-12T03:28:52.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.911+0000 7eff916e1740 1 -- 172.21.15.6:0/4175171113 wait complete. 2024-09-12T03:28:52.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.911+0000 7eff916e1740 20 client.16230 trim_cache size 0 max 16384 2024-09-12T03:28:52.921 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:28:52.921 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2024-09-12T03:28:52.921 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2024-09-12T03:28:52.921 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:28:52.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.911+0000 7eff8f308640 1 -- 172.21.15.6:0/3395181910 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7eff700a2b80 msgr2=0x7eff700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:52.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.911+0000 7eff8f308640 1 --2- 172.21.15.6:0/3395181910 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7eff700a2b80 0x7eff700a5040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7eff780094d0 tx=0x7eff78009500 comp rx=0 tx=0).stop 2024-09-12T03:28:52.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.911+0000 7eff8f308640 1 -- 172.21.15.6:0/3395181910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff88103740 msgr2=0x7eff88198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:52.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.911+0000 7eff8f308640 1 --2- 172.21.15.6:0/3395181910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff88103740 0x7eff88198b90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7eff7c005940 tx=0x7eff7c0040c0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.912+0000 7eff8f308640 1 -- 172.21.15.6:0/3395181910 shutdown_connections 2024-09-12T03:28:52.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.912+0000 7eff8f308640 1 --2- 172.21.15.6:0/3395181910 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7eff700a2b80 0x7eff700a5040 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.912+0000 7eff8f308640 1 --2- 172.21.15.6:0/3395181910 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff88108290 0x7eff8819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.912+0000 7eff8f308640 1 --2- 172.21.15.6:0/3395181910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff88103740 0x7eff88198b90 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.912+0000 7eff8f308640 1 --2- 172.21.15.6:0/3395181910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff88102d90 0x7eff88198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.912+0000 7eff8f308640 1 -- 172.21.15.6:0/3395181910 >> 172.21.15.6:0/3395181910 conn(0x7eff880fc7d0 msgr2=0x7eff88105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:52.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.912+0000 7eff8f308640 1 -- 172.21.15.6:0/3395181910 shutdown_connections 2024-09-12T03:28:52.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:52.912+0000 7eff8f308640 1 -- 172.21.15.6:0/3395181910 wait complete. 2024-09-12T03:28:53.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:52 smithi006 ceph-mon[30991]: pgmap v568: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:28:53.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:52 smithi161 ceph-mon[35485]: pgmap v568: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:28:53.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:52 smithi118 ceph-mon[35048]: pgmap v568: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:28:53.620 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:28:53.620 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:28:53.882 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:54.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.352+0000 7f74c120d640 1 -- 172.21.15.6:0/1896936360 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74bc102db0 msgr2=0x7f74bc103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:54.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.352+0000 7f74c120d640 1 --2- 172.21.15.6:0/1896936360 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74bc102db0 0x7f74bc103190 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009930 tx=0x7f74ac02a420 comp rx=0 tx=0).stop 2024-09-12T03:28:54.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.352+0000 7f74c120d640 1 -- 172.21.15.6:0/1896936360 shutdown_connections 2024-09-12T03:28:54.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.352+0000 7f74c120d640 1 --2- 172.21.15.6:0/1896936360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74bc1082e0 0x7f74bc10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:54.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.352+0000 7f74c120d640 1 --2- 172.21.15.6:0/1896936360 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74bc103760 0x7f74bc107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:54.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.352+0000 7f74c120d640 1 --2- 172.21.15.6:0/1896936360 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74bc102db0 0x7f74bc103190 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:54.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.352+0000 7f74c120d640 1 -- 172.21.15.6:0/1896936360 >> 172.21.15.6:0/1896936360 conn(0x7f74bc0fc7d0 msgr2=0x7f74bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:54.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.353+0000 7f74c120d640 1 -- 172.21.15.6:0/1896936360 shutdown_connections 2024-09-12T03:28:54.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.353+0000 7f74c120d640 1 -- 172.21.15.6:0/1896936360 wait complete. 2024-09-12T03:28:54.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.353+0000 7f74c120d640 1 Processor -- start 2024-09-12T03:28:54.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.354+0000 7f74c120d640 1 -- start start 2024-09-12T03:28:54.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.354+0000 7f74c120d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74bc102db0 0x7f74bc1a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:54.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.354+0000 7f74c120d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74bc103760 0x7f74bc1a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:54.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.354+0000 7f74c120d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74bc1082e0 0x7f74bc19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:54.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.354+0000 7f74c120d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc1120b0 con 0x7f74bc102db0 2024-09-12T03:28:54.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.354+0000 7f74c120d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc111f30 con 0x7f74bc1082e0 2024-09-12T03:28:54.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.354+0000 7f74c120d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc112230 con 0x7f74bc103760 2024-09-12T03:28:54.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.354+0000 7f74bad9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74bc102db0 0x7f74bc1a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:54.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.354+0000 7f74bad9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74bc102db0 0x7f74bc1a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35994/0 (socket says 172.21.15.6:35994) 2024-09-12T03:28:54.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.354+0000 7f74bad9d640 1 -- 172.21.15.6:0/1938746162 learned_addr learned my addr 172.21.15.6:0/1938746162 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:54.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.354+0000 7f74ba59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74bc103760 0x7f74bc1a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:54.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.354+0000 7f74bb59e640 1 --2- 172.21.15.6:0/1938746162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74bc1082e0 0x7f74bc19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:54.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.354+0000 7f74bad9d640 1 -- 172.21.15.6:0/1938746162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74bc103760 msgr2=0x7f74bc1a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:54.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.354+0000 7f74bad9d640 1 --2- 172.21.15.6:0/1938746162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74bc103760 0x7f74bc1a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:54.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.354+0000 7f74bad9d640 1 -- 172.21.15.6:0/1938746162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74bc1082e0 msgr2=0x7f74bc19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:54.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.354+0000 7f74bad9d640 1 --2- 172.21.15.6:0/1938746162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74bc1082e0 0x7f74bc19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:54.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.354+0000 7f74bad9d640 1 -- 172.21.15.6:0/1938746162 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc19ad20 con 0x7f74bc102db0 2024-09-12T03:28:54.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.355+0000 7f74bad9d640 1 --2- 172.21.15.6:0/1938746162 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74bc102db0 0x7f74bc1a0430 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f74ac004770 tx=0x7f74ac003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:54.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.355+0000 7f74b3fff640 1 -- 172.21.15.6:0/1938746162 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac03f070 con 0x7f74bc102db0 2024-09-12T03:28:54.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.355+0000 7f74c120d640 1 -- 172.21.15.6:0/1938746162 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74bc19afb0 con 0x7f74bc102db0 2024-09-12T03:28:54.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.355+0000 7f74b3fff640 1 -- 172.21.15.6:0/1938746162 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74ac031c20 con 0x7f74bc102db0 2024-09-12T03:28:54.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.355+0000 7f74b3fff640 1 -- 172.21.15.6:0/1938746162 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac035030 con 0x7f74bc102db0 2024-09-12T03:28:54.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.355+0000 7f74c120d640 1 -- 172.21.15.6:0/1938746162 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74bc1a7300 con 0x7f74bc102db0 2024-09-12T03:28:54.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.356+0000 7f74c120d640 1 -- 172.21.15.6:0/1938746162 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74bc103190 con 0x7f74bc102db0 2024-09-12T03:28:54.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.358+0000 7f74b3fff640 1 -- 172.21.15.6:0/1938746162 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f74ac042070 con 0x7f74bc102db0 2024-09-12T03:28:54.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.359+0000 7f74b3fff640 1 --2- 172.21.15.6:0/1938746162 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f74a80a2b80 0x7f74a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:54.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.359+0000 7f74ba59c640 1 --2- 172.21.15.6:0/1938746162 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f74a80a2b80 0x7f74a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:54.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.359+0000 7f74b3fff640 1 -- 172.21.15.6:0/1938746162 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f74ac0eff70 con 0x7f74bc102db0 2024-09-12T03:28:54.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.359+0000 7f74ba59c640 1 --2- 172.21.15.6:0/1938746162 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f74a80a2b80 0x7f74a80a5040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f74bc10f020 tx=0x7f74a400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:54.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.362+0000 7f74b3fff640 1 -- 172.21.15.6:0/1938746162 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f74ac0bc950 con 0x7f74bc102db0 2024-09-12T03:28:54.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.575+0000 7f74c120d640 1 -- 172.21.15.6:0/1938746162 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74bc1a7530 con 0x7f74bc102db0 2024-09-12T03:28:54.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.576+0000 7f74b3fff640 1 -- 172.21.15.6:0/1938746162 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11910 (secure 0 0 0) 0x7f74ac0c0c50 con 0x7f74bc102db0 2024-09-12T03:28:54.577 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:28:54.577 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":29,"btime":"2024-09-12T03:28:30:753620+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:28:30.102891+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24463]},"id":1}]} 2024-09-12T03:28:54.577 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 29 2024-09-12T03:28:54.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.579+0000 7f74c120d640 1 -- 172.21.15.6:0/1938746162 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f74a80a2b80 msgr2=0x7f74a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:54.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.579+0000 7f74c120d640 1 --2- 172.21.15.6:0/1938746162 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f74a80a2b80 0x7f74a80a5040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f74bc10f020 tx=0x7f74a400b390 comp rx=0 tx=0).stop 2024-09-12T03:28:54.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.579+0000 7f74c120d640 1 -- 172.21.15.6:0/1938746162 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74bc102db0 msgr2=0x7f74bc1a0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:54.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.579+0000 7f74c120d640 1 --2- 172.21.15.6:0/1938746162 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74bc102db0 0x7f74bc1a0430 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f74ac004770 tx=0x7f74ac003b90 comp rx=0 tx=0).stop 2024-09-12T03:28:54.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.579+0000 7f74c120d640 1 -- 172.21.15.6:0/1938746162 shutdown_connections 2024-09-12T03:28:54.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.579+0000 7f74c120d640 1 --2- 172.21.15.6:0/1938746162 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f74a80a2b80 0x7f74a80a5040 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:54.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.579+0000 7f74c120d640 1 --2- 172.21.15.6:0/1938746162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74bc1082e0 0x7f74bc19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:54.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.579+0000 7f74c120d640 1 --2- 172.21.15.6:0/1938746162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74bc103760 0x7f74bc1a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:54.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.579+0000 7f74c120d640 1 --2- 172.21.15.6:0/1938746162 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74bc102db0 0x7f74bc1a0430 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:54.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.579+0000 7f74c120d640 1 -- 172.21.15.6:0/1938746162 >> 172.21.15.6:0/1938746162 conn(0x7f74bc0fc7d0 msgr2=0x7f74bc10d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:54.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.579+0000 7f74c120d640 1 -- 172.21.15.6:0/1938746162 shutdown_connections 2024-09-12T03:28:54.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:54.579+0000 7f74c120d640 1 -- 172.21.15.6:0/1938746162 wait complete. 2024-09-12T03:28:55.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:54 smithi006 ceph-mon[30991]: pgmap v569: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-09-12T03:28:55.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:54 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1938746162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:55.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:54 smithi161 ceph-mon[35485]: pgmap v569: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-09-12T03:28:55.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:54 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1938746162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:55.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:54 smithi118 ceph-mon[35048]: pgmap v569: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-09-12T03:28:55.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:54 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1938746162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:55.230 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:28:55.494 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:56.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.983+0000 7f89c0a8d640 1 -- 172.21.15.6:0/1538308853 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89bc102d30 msgr2=0x7f89bc103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:56.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.983+0000 7f89c0a8d640 1 --2- 172.21.15.6:0/1538308853 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89bc102d30 0x7f89bc103110 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f89b4009cd0 tx=0x7f89b402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.983+0000 7f89c0a8d640 1 -- 172.21.15.6:0/1538308853 shutdown_connections 2024-09-12T03:28:56.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.983+0000 7f89c0a8d640 1 --2- 172.21.15.6:0/1538308853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89bc1082a0 0x7f89bc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.983+0000 7f89c0a8d640 1 --2- 172.21.15.6:0/1538308853 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89bc1036e0 0x7f89bc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.983+0000 7f89c0a8d640 1 --2- 172.21.15.6:0/1538308853 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89bc102d30 0x7f89bc103110 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.983+0000 7f89c0a8d640 1 -- 172.21.15.6:0/1538308853 >> 172.21.15.6:0/1538308853 conn(0x7f89bc0fc7d0 msgr2=0x7f89bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:56.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.983+0000 7f89c0a8d640 1 -- 172.21.15.6:0/1538308853 shutdown_connections 2024-09-12T03:28:56.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.983+0000 7f89c0a8d640 1 -- 172.21.15.6:0/1538308853 wait complete. 2024-09-12T03:28:56.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.984+0000 7f89c0a8d640 1 Processor -- start 2024-09-12T03:28:56.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.984+0000 7f89c0a8d640 1 -- start start 2024-09-12T03:28:56.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.984+0000 7f89c0a8d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89bc102d30 0x7f89bc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:56.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.984+0000 7f89c0a8d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89bc1036e0 0x7f89bc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:56.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.984+0000 7f89c0a8d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89bc1082a0 0x7f89bc101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:56.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.984+0000 7f89c0a8d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1120a0 con 0x7f89bc1036e0 2024-09-12T03:28:56.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.984+0000 7f89c0a8d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc111f20 con 0x7f89bc1082a0 2024-09-12T03:28:56.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.984+0000 7f89c0a8d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc112220 con 0x7f89bc102d30 2024-09-12T03:28:56.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.984+0000 7f89b9d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89bc1036e0 0x7f89bc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:56.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.984+0000 7f89b9d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89bc1036e0 0x7f89bc100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36010/0 (socket says 172.21.15.6:36010) 2024-09-12T03:28:56.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.985+0000 7f89b9d9b640 1 -- 172.21.15.6:0/2089297749 learned_addr learned my addr 172.21.15.6:0/2089297749 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:56.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.985+0000 7f89ba59c640 1 --2- 172.21.15.6:0/2089297749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89bc102d30 0x7f89bc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:56.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.985+0000 7f89bad9d640 1 --2- 172.21.15.6:0/2089297749 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89bc1082a0 0x7f89bc101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:56.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.985+0000 7f89b9d9b640 1 -- 172.21.15.6:0/2089297749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89bc102d30 msgr2=0x7f89bc100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:56.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.985+0000 7f89b9d9b640 1 --2- 172.21.15.6:0/2089297749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89bc102d30 0x7f89bc100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.985+0000 7f89b9d9b640 1 -- 172.21.15.6:0/2089297749 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89bc1082a0 msgr2=0x7f89bc101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:56.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.985+0000 7f89b9d9b640 1 --2- 172.21.15.6:0/2089297749 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89bc1082a0 0x7f89bc101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.985+0000 7f89b9d9b640 1 -- 172.21.15.6:0/2089297749 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89bc1a6a60 con 0x7f89bc1036e0 2024-09-12T03:28:56.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.985+0000 7f89ba59c640 1 --2- 172.21.15.6:0/2089297749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89bc102d30 0x7f89bc100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:56.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.985+0000 7f89b9d9b640 1 --2- 172.21.15.6:0/2089297749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89bc1036e0 0x7f89bc100ed0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f89a400c970 tx=0x7f89a400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:56.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.985+0000 7f89b37fe640 1 -- 172.21.15.6:0/2089297749 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a4007bb0 con 0x7f89bc1036e0 2024-09-12T03:28:56.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.985+0000 7f89b37fe640 1 -- 172.21.15.6:0/2089297749 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89a4007d50 con 0x7f89bc1036e0 2024-09-12T03:28:56.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.985+0000 7f89c0a8d640 1 -- 172.21.15.6:0/2089297749 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89bc1a6d50 con 0x7f89bc1036e0 2024-09-12T03:28:56.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.985+0000 7f89b37fe640 1 -- 172.21.15.6:0/2089297749 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a400f450 con 0x7f89bc1036e0 2024-09-12T03:28:56.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.985+0000 7f89c0a8d640 1 -- 172.21.15.6:0/2089297749 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89bc1a7230 con 0x7f89bc1036e0 2024-09-12T03:28:56.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.988+0000 7f89b37fe640 1 -- 172.21.15.6:0/2089297749 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f89a4004f60 con 0x7f89bc1036e0 2024-09-12T03:28:56.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.988+0000 7f89c2e66740 1 Processor -- start 2024-09-12T03:28:56.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.989+0000 7f89c2e66740 1 -- start start 2024-09-12T03:28:56.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.989+0000 7f89b37fe640 1 --2- 172.21.15.6:0/2089297749 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f89a80a2b80 0x7f89a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:56.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.989+0000 7f89c2e66740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb41db40 0x55e7cb41c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:56.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.989+0000 7f89c2e66740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e7cb41c5d0 0x55e7cb41ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:56.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.989+0000 7f89c2e66740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7cb427370 0x55e7cb429760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:56.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.989+0000 7f89c2e66740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e7cb3822f0 con 0x55e7cb41db40 2024-09-12T03:28:56.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.989+0000 7f89c2e66740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55e7cb329cf0 con 0x55e7cb41c5d0 2024-09-12T03:28:56.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.989+0000 7f89c2e66740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e7cb37e2b0 con 0x55e7cb427370 2024-09-12T03:28:56.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.989+0000 7f89ba59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb41db40 0x55e7cb41c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:56.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.989+0000 7f89bad9d640 1 --2- 172.21.15.6:0/2089297749 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f89a80a2b80 0x7f89a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:56.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.989+0000 7f89ba59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb41db40 0x55e7cb41c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36026/0 (socket says 172.21.15.6:36026) 2024-09-12T03:28:56.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.989+0000 7f89ba59c640 1 -- 172.21.15.6:0/2410687333 learned_addr learned my addr 172.21.15.6:0/2410687333 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:56.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.989+0000 7f89b37fe640 1 -- 172.21.15.6:0/2089297749 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f89a40ced50 con 0x7f89bc1036e0 2024-09-12T03:28:56.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.989+0000 7f89bad9d640 1 --2- 172.21.15.6:0/2410687333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7cb427370 0x55e7cb429760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:56.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.989+0000 7f89b9d9b640 1 --2- 172.21.15.6:0/2410687333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e7cb41c5d0 0x55e7cb41ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:56.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.989+0000 7f89ba59c640 1 -- 172.21.15.6:0/2410687333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7cb427370 msgr2=0x55e7cb429760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:56.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.989+0000 7f89ba59c640 1 --2- 172.21.15.6:0/2410687333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7cb427370 0x55e7cb429760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.990+0000 7f89ba59c640 1 -- 172.21.15.6:0/2410687333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e7cb41c5d0 msgr2=0x55e7cb41ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:56.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.990+0000 7f89ba59c640 1 --2- 172.21.15.6:0/2410687333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e7cb41c5d0 0x55e7cb41ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.990+0000 7f89ba59c640 1 -- 172.21.15.6:0/2410687333 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7cb30e3a0 con 0x55e7cb41db40 2024-09-12T03:28:56.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.990+0000 7f89bad9d640 1 --2- 172.21.15.6:0/2089297749 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f89a80a2b80 0x7f89a80a5040 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00b970 tx=0x7f89ac00bd70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:56.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.990+0000 7f89ba59c640 1 --2- 172.21.15.6:0/2410687333 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb41db40 0x55e7cb41c090 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f89b4009cd0 tx=0x7f89b4031970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df9d7e923b556cf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:56.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.990+0000 7f89b0ff9640 1 -- 172.21.15.6:0/2410687333 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4048070 con 0x55e7cb41db40 2024-09-12T03:28:56.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.990+0000 7f89b0ff9640 1 -- 172.21.15.6:0/2410687333 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89b4036070 con 0x55e7cb41db40 2024-09-12T03:28:56.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.990+0000 7f89b0ff9640 1 -- 172.21.15.6:0/2410687333 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b40305a0 con 0x55e7cb41db40 2024-09-12T03:28:56.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.991+0000 7f89c2e66740 1 -- 172.21.15.6:0/2410687333 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb41db40 msgr2=0x55e7cb41c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:56.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.991+0000 7f89c2e66740 1 --2- 172.21.15.6:0/2410687333 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb41db40 0x55e7cb41c090 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f89b4009cd0 tx=0x7f89b4031970 comp rx=0 tx=0).stop 2024-09-12T03:28:56.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.991+0000 7f89c2e66740 1 -- 172.21.15.6:0/2410687333 shutdown_connections 2024-09-12T03:28:56.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.991+0000 7f89c2e66740 1 --2- 172.21.15.6:0/2410687333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7cb427370 0x55e7cb429760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.991+0000 7f89c2e66740 1 --2- 172.21.15.6:0/2410687333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e7cb41c5d0 0x55e7cb41ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.991+0000 7f89c2e66740 1 --2- 172.21.15.6:0/2410687333 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb41db40 0x55e7cb41c090 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.991+0000 7f89c2e66740 1 -- 172.21.15.6:0/2410687333 >> 172.21.15.6:0/2410687333 conn(0x55e7cb3e5310 msgr2=0x55e7cb418c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:56.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.991+0000 7f89c2e66740 1 -- 172.21.15.6:0/2410687333 shutdown_connections 2024-09-12T03:28:56.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.991+0000 7f89c2e66740 1 -- 172.21.15.6:0/2410687333 wait complete. 2024-09-12T03:28:56.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.991+0000 7f89c2e66740 1 -- start start 2024-09-12T03:28:56.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.991+0000 7f89c2e66740 1 Processor -- start 2024-09-12T03:28:56.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89c2e66740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb41c5d0 0x55e7cb426d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:56.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89c2e66740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7cb41db40 0x55e7cb424000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:56.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89c2e66740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e7cb427370 0x55e7cb4245d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:56.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89c2e66740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e7cb42a010 con 0x55e7cb41c5d0 2024-09-12T03:28:56.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89c2e66740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x55e7cb33d060 con 0x55e7cb427370 2024-09-12T03:28:56.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89c2e66740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e7cb42a190 con 0x55e7cb41db40 2024-09-12T03:28:56.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89c2e66740 10 client.0 _ll_register_callbacks cb 0x55e7cb0908c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:56.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89ba59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb41c5d0 0x55e7cb426d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:56.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89ba59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb41c5d0 0x55e7cb426d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36042/0 (socket says 172.21.15.6:36042) 2024-09-12T03:28:56.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89ba59c640 1 -- 172.21.15.6:0/1350489171 learned_addr learned my addr 172.21.15.6:0/1350489171 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:56.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89b9d9b640 1 --2- 172.21.15.6:0/1350489171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7cb41db40 0x55e7cb424000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:56.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89bad9d640 1 --2- 172.21.15.6:0/1350489171 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e7cb427370 0x55e7cb4245d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:56.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89ba59c640 1 -- 172.21.15.6:0/1350489171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7cb41db40 msgr2=0x55e7cb424000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:56.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89ba59c640 1 --2- 172.21.15.6:0/1350489171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7cb41db40 0x55e7cb424000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89ba59c640 1 -- 172.21.15.6:0/1350489171 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e7cb427370 msgr2=0x55e7cb4245d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:56.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89ba59c640 1 --2- 172.21.15.6:0/1350489171 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e7cb427370 0x55e7cb4245d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89ba59c640 1 -- 172.21.15.6:0/1350489171 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7cb0f33c0 con 0x55e7cb41c5d0 2024-09-12T03:28:56.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89bad9d640 1 --2- 172.21.15.6:0/1350489171 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e7cb427370 0x55e7cb4245d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:56.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89ba59c640 1 --2- 172.21.15.6:0/1350489171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb41c5d0 0x55e7cb426d50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f89b4046040 tx=0x7f89b4031d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9b1055b22f1be8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:56.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:28:56.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89a1ffb640 1 -- 172.21.15.6:0/1350489171 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4048040 con 0x55e7cb41c5d0 2024-09-12T03:28:56.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89a1ffb640 1 -- 172.21.15.6:0/1350489171 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f89b402e040 con 0x55e7cb41c5d0 2024-09-12T03:28:56.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.992+0000 7f89c2e66740 1 -- 172.21.15.6:0/1350489171 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89b4036070 con 0x55e7cb41c5d0 2024-09-12T03:28:56.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.993+0000 7f89a1ffb640 1 -- 172.21.15.6:0/1350489171 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f89b4003970 con 0x55e7cb41c5d0 2024-09-12T03:28:56.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.993+0000 7f89c2e66740 10 client.16248 fetch_fsmap learned FSMap version 29 2024-09-12T03:28:56.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.993+0000 7f89c2e66740 1 -- 172.21.15.6:0/1350489171 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e7cb42a010 con 0x55e7cb41c5d0 2024-09-12T03:28:56.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.993+0000 7f89a1ffb640 1 -- 172.21.15.6:0/1350489171 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89b4036070 con 0x55e7cb41c5d0 2024-09-12T03:28:56.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.993+0000 7f89a1ffb640 1 -- 172.21.15.6:0/1350489171 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f89b4042030 con 0x55e7cb41c5d0 2024-09-12T03:28:56.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.993+0000 7f89c2e66740 10 client.16248 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T03:28:56.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.993+0000 7f89c2e66740 10 client.16248 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-12T03:28:56.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.993+0000 7f89c2e66740 20 client.16248 populate_metadata read hostname 'smithi006' 2024-09-12T03:28:56.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.993+0000 7f89c2e66740 1 --2- 172.21.15.6:0/1350489171 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55e7cb436c10 0x55e7cb439000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:56.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.993+0000 7f89c2e66740 4 client.16248 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:56.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.993+0000 7f89c2e66740 1 -- 172.21.15.6:0/1350489171 --> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7cb38f860 con 0x55e7cb436c10 2024-09-12T03:28:56.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.993+0000 7f89b9d9b640 1 --2- 172.21.15.6:0/1350489171 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55e7cb436c10 0x55e7cb439000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:56.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.994+0000 7f89b9d9b640 1 --2- 172.21.15.6:0/1350489171 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55e7cb436c10 0x55e7cb439000 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd5984fb8af03443 server_cookie=3f93e7e8906e08de in_seq=0 out_seq=0 2024-09-12T03:28:56.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.994+0000 7f89a1ffb640 10 client.16248 ms_handle_connect on v2:172.21.15.161:6838/3645555254 2024-09-12T03:28:56.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.994+0000 7f89a1ffb640 1 -- 172.21.15.6:0/1350489171 <== mds.2 v2:172.21.15.161:6838/3645555254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e7cb38f860 con 0x55e7cb436c10 2024-09-12T03:28:56.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.994+0000 7f89a1ffb640 10 client.16248 handle_command_reply: tid=0 2024-09-12T03:28:56.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.994+0000 7f89c2e66740 1 client.16248 shutdown 2024-09-12T03:28:56.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.995+0000 7f89c2e66740 1 -- 172.21.15.6:0/1350489171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb41c5d0 msgr2=0x55e7cb426d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:56.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.995+0000 7f89c2e66740 1 --2- 172.21.15.6:0/1350489171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb41c5d0 0x55e7cb426d50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f89b4046040 tx=0x7f89b4031d10 comp rx=0 tx=0).stop 2024-09-12T03:28:56.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.995+0000 7f89c2e66740 1 -- 172.21.15.6:0/1350489171 shutdown_connections 2024-09-12T03:28:56.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.995+0000 7f89c2e66740 1 --2- 172.21.15.6:0/1350489171 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55e7cb436c10 0x55e7cb439000 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.995+0000 7f89c2e66740 1 --2- 172.21.15.6:0/1350489171 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e7cb427370 0x55e7cb4245d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.995+0000 7f89a1ffb640 0 client.16248 ms_handle_reset on v2:172.21.15.161:6838/3645555254 2024-09-12T03:28:56.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.995+0000 7f89c2e66740 1 --2- 172.21.15.6:0/1350489171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7cb41db40 0x55e7cb424000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.995+0000 7f89c2e66740 1 --2- 172.21.15.6:0/1350489171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb41c5d0 0x55e7cb426d50 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.995+0000 7f89c2e66740 1 -- 172.21.15.6:0/1350489171 >> 172.21.15.6:0/1350489171 conn(0x55e7cb3e5310 msgr2=0x55e7cb41e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:56.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.995+0000 7f89c2e66740 1 -- 172.21.15.6:0/1350489171 shutdown_connections 2024-09-12T03:28:56.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.995+0000 7f89c2e66740 1 -- 172.21.15.6:0/1350489171 wait complete. 2024-09-12T03:28:56.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:55.995+0000 7f89c2e66740 20 client.16248 trim_cache size 0 max 16384 2024-09-12T03:28:56.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.010+0000 7f89c2e66740 1 Processor -- start 2024-09-12T03:28:56.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.010+0000 7f89c2e66740 1 -- start start 2024-09-12T03:28:56.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.010+0000 7f89c2e66740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7cb41c5d0 0x55e7cb4332b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:56.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.010+0000 7f89c2e66740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e7cb41db40 0x55e7cb4238c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:56.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.010+0000 7f89c2e66740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb427370 0x55e7cb413ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:56.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.010+0000 7f89c2e66740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1b1fe0 con 0x55e7cb427370 2024-09-12T03:28:56.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.010+0000 7f89c2e66740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1af850 con 0x55e7cb41db40 2024-09-12T03:28:56.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.010+0000 7f89c2e66740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1ad6a0 con 0x55e7cb41c5d0 2024-09-12T03:28:56.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.010+0000 7f89bad9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb427370 0x55e7cb413ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:56.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.010+0000 7f89bad9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb427370 0x55e7cb413ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36046/0 (socket says 172.21.15.6:36046) 2024-09-12T03:28:56.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.010+0000 7f89bad9d640 1 -- 172.21.15.6:0/3523433096 learned_addr learned my addr 172.21.15.6:0/3523433096 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:56.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.011+0000 7f89ba59c640 1 --2- 172.21.15.6:0/3523433096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7cb41c5d0 0x55e7cb4332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:56.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.011+0000 7f89b9d9b640 1 --2- 172.21.15.6:0/3523433096 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e7cb41db40 0x55e7cb4238c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:56.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.011+0000 7f89bad9d640 1 -- 172.21.15.6:0/3523433096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7cb41c5d0 msgr2=0x55e7cb4332b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:56.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.011+0000 7f89bad9d640 1 --2- 172.21.15.6:0/3523433096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7cb41c5d0 0x55e7cb4332b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.011+0000 7f89bad9d640 1 -- 172.21.15.6:0/3523433096 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e7cb41db40 msgr2=0x55e7cb4238c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:56.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.011+0000 7f89bad9d640 1 --2- 172.21.15.6:0/3523433096 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e7cb41db40 0x55e7cb4238c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.011+0000 7f89bad9d640 1 -- 172.21.15.6:0/3523433096 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7cb41d760 con 0x55e7cb427370 2024-09-12T03:28:56.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.011+0000 7f89bad9d640 1 --2- 172.21.15.6:0/3523433096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb427370 0x55e7cb413ab0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f89ac003fb0 tx=0x7f89ac01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6571216c36fc7e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:56.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.011+0000 7f89a3fff640 1 -- 172.21.15.6:0/3523433096 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac028610 con 0x55e7cb427370 2024-09-12T03:28:56.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.011+0000 7f89a3fff640 1 -- 172.21.15.6:0/3523433096 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89ac024070 con 0x55e7cb427370 2024-09-12T03:28:56.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.011+0000 7f89a3fff640 1 -- 172.21.15.6:0/3523433096 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac01f6f0 con 0x55e7cb427370 2024-09-12T03:28:56.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.012+0000 7f89c2e66740 1 -- 172.21.15.6:0/3523433096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb427370 msgr2=0x55e7cb413ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:56.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.012+0000 7f89c2e66740 1 --2- 172.21.15.6:0/3523433096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb427370 0x55e7cb413ab0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f89ac003fb0 tx=0x7f89ac01d000 comp rx=0 tx=0).stop 2024-09-12T03:28:56.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.012+0000 7f89c2e66740 1 -- 172.21.15.6:0/3523433096 shutdown_connections 2024-09-12T03:28:56.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.012+0000 7f89c2e66740 1 --2- 172.21.15.6:0/3523433096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb427370 0x55e7cb413ab0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.012+0000 7f89c2e66740 1 --2- 172.21.15.6:0/3523433096 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e7cb41db40 0x55e7cb4238c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.012+0000 7f89c2e66740 1 --2- 172.21.15.6:0/3523433096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7cb41c5d0 0x55e7cb4332b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.012+0000 7f89c2e66740 1 -- 172.21.15.6:0/3523433096 >> 172.21.15.6:0/3523433096 conn(0x55e7cb3e5310 msgr2=0x55e7cb4230f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:56.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.012+0000 7f89c2e66740 1 -- 172.21.15.6:0/3523433096 shutdown_connections 2024-09-12T03:28:56.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.012+0000 7f89c2e66740 1 -- 172.21.15.6:0/3523433096 wait complete. 2024-09-12T03:28:56.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.012+0000 7f89c2e66740 1 -- start start 2024-09-12T03:28:56.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.012+0000 7f89c2e66740 1 Processor -- start 2024-09-12T03:28:56.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.013+0000 7f89c2e66740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb41c5d0 0x55e7cb42ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:56.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.013+0000 7f89c2e66740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7cb41db40 0x55e7cb42b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:56.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.013+0000 7f89c2e66740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e7cb427370 0x55e7cb499630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:56.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.013+0000 7f89c2e66740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1add10 con 0x55e7cb41c5d0 2024-09-12T03:28:56.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.013+0000 7f89c2e66740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1af180 con 0x55e7cb427370 2024-09-12T03:28:56.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.013+0000 7f89c2e66740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1ae3e0 con 0x55e7cb41db40 2024-09-12T03:28:56.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.013+0000 7f89c2e66740 10 client.0 _ll_register_callbacks cb 0x7f89b4046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:56.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.013+0000 7f89ba59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb41c5d0 0x55e7cb42ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:56.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.013+0000 7f89ba59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb41c5d0 0x55e7cb42ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36060/0 (socket says 172.21.15.6:36060) 2024-09-12T03:28:56.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.013+0000 7f89ba59c640 1 -- 172.21.15.6:0/1531804710 learned_addr learned my addr 172.21.15.6:0/1531804710 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:56.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.013+0000 7f89b9d9b640 1 --2- 172.21.15.6:0/1531804710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7cb41db40 0x55e7cb42b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:56.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.013+0000 7f89bad9d640 1 --2- 172.21.15.6:0/1531804710 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e7cb427370 0x55e7cb499630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:56.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.013+0000 7f89ba59c640 1 -- 172.21.15.6:0/1531804710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7cb41db40 msgr2=0x55e7cb42b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:56.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.013+0000 7f89ba59c640 1 --2- 172.21.15.6:0/1531804710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7cb41db40 0x55e7cb42b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.013+0000 7f89ba59c640 1 -- 172.21.15.6:0/1531804710 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e7cb427370 msgr2=0x55e7cb499630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:56.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.013+0000 7f89ba59c640 1 --2- 172.21.15.6:0/1531804710 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e7cb427370 0x55e7cb499630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.013+0000 7f89ba59c640 1 -- 172.21.15.6:0/1531804710 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7cb499bf0 con 0x55e7cb41c5d0 2024-09-12T03:28:56.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.013+0000 7f89ba59c640 1 --2- 172.21.15.6:0/1531804710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb41c5d0 0x55e7cb42ab60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f89b4005e30 tx=0x7f89b4009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c8fbb910e9cb04 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:56.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.013+0000 7f89a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:28:56.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.014+0000 7f89a1ffb640 1 -- 172.21.15.6:0/1531804710 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4048040 con 0x55e7cb41c5d0 2024-09-12T03:28:56.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.014+0000 7f89a1ffb640 1 -- 172.21.15.6:0/1531804710 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f89b4053040 con 0x55e7cb41c5d0 2024-09-12T03:28:56.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.014+0000 7f89c2e66740 1 -- 172.21.15.6:0/1531804710 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89ac024070 con 0x55e7cb41c5d0 2024-09-12T03:28:56.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.014+0000 7f89a1ffb640 1 -- 172.21.15.6:0/1531804710 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f89b40042f0 con 0x55e7cb41c5d0 2024-09-12T03:28:56.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.014+0000 7f89c2e66740 10 client.16254 fetch_fsmap learned FSMap version 29 2024-09-12T03:28:56.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.014+0000 7f89c2e66740 1 -- 172.21.15.6:0/1531804710 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e7cb4a0fa0 con 0x55e7cb41c5d0 2024-09-12T03:28:56.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.014+0000 7f89a1ffb640 1 -- 172.21.15.6:0/1531804710 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89ac024070 con 0x55e7cb41c5d0 2024-09-12T03:28:56.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.014+0000 7f89a1ffb640 1 -- 172.21.15.6:0/1531804710 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f89b4042070 con 0x55e7cb41c5d0 2024-09-12T03:28:56.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.014+0000 7f89c2e66740 10 client.16254 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T03:28:56.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.014+0000 7f89c2e66740 10 client.16254 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-12T03:28:56.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.014+0000 7f89c2e66740 20 client.16254 populate_metadata read hostname 'smithi006' 2024-09-12T03:28:56.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.014+0000 7f89c2e66740 1 --2- 172.21.15.6:0/1531804710 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55e7cb4a16c0 0x55e7cb4a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:56.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.014+0000 7f89c2e66740 4 client.16254 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:28:56.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.014+0000 7f89c2e66740 1 -- 172.21.15.6:0/1531804710 --> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e7cb389d70 con 0x55e7cb4a16c0 2024-09-12T03:28:56.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.015+0000 7f89b9d9b640 1 --2- 172.21.15.6:0/1531804710 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55e7cb4a16c0 0x55e7cb4a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:56.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.015+0000 7f89b9d9b640 1 --2- 172.21.15.6:0/1531804710 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55e7cb4a16c0 0x55e7cb4a3ae0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7e09ec4982c9b346 server_cookie=bc80f6947b7812f9 in_seq=0 out_seq=0 2024-09-12T03:28:56.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.015+0000 7f89a1ffb640 10 client.16254 ms_handle_connect on v2:172.21.15.161:6838/3645555254 2024-09-12T03:28:56.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.015+0000 7f89a1ffb640 1 -- 172.21.15.6:0/1531804710 <== mds.2 v2:172.21.15.161:6838/3645555254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e7cb389d70 con 0x55e7cb4a16c0 2024-09-12T03:28:56.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.015+0000 7f89a1ffb640 10 client.16254 handle_command_reply: tid=0 2024-09-12T03:28:56.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.015+0000 7f89c2e66740 1 client.16254 shutdown 2024-09-12T03:28:56.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.015+0000 7f89c2e66740 1 -- 172.21.15.6:0/1531804710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb41c5d0 msgr2=0x55e7cb42ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:56.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.015+0000 7f89c2e66740 1 --2- 172.21.15.6:0/1531804710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb41c5d0 0x55e7cb42ab60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f89b4005e30 tx=0x7f89b4009540 comp rx=0 tx=0).stop 2024-09-12T03:28:56.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.015+0000 7f89c2e66740 1 -- 172.21.15.6:0/1531804710 shutdown_connections 2024-09-12T03:28:56.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.015+0000 7f89c2e66740 1 --2- 172.21.15.6:0/1531804710 >> [v2:172.21.15.161:6838/3645555254,v1:172.21.15.161:6839/3645555254] conn(0x55e7cb4a16c0 0x55e7cb4a3ae0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.015+0000 7f89c2e66740 1 --2- 172.21.15.6:0/1531804710 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e7cb427370 0x55e7cb499630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.015+0000 7f89c2e66740 1 --2- 172.21.15.6:0/1531804710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7cb41db40 0x55e7cb42b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.015+0000 7f89a1ffb640 0 client.16254 ms_handle_reset on v2:172.21.15.161:6838/3645555254 2024-09-12T03:28:56.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.015+0000 7f89c2e66740 1 --2- 172.21.15.6:0/1531804710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e7cb41c5d0 0x55e7cb42ab60 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.015+0000 7f89c2e66740 1 -- 172.21.15.6:0/1531804710 >> 172.21.15.6:0/1531804710 conn(0x55e7cb3e5310 msgr2=0x55e7cb4174e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:56.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.015+0000 7f89c2e66740 1 -- 172.21.15.6:0/1531804710 shutdown_connections 2024-09-12T03:28:56.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.015+0000 7f89c2e66740 1 -- 172.21.15.6:0/1531804710 wait complete. 2024-09-12T03:28:56.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.016+0000 7f89c2e66740 20 client.16254 trim_cache size 0 max 16384 2024-09-12T03:28:56.024 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-09-12T03:28:56.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.016+0000 7f89c0a8d640 1 -- 172.21.15.6:0/2089297749 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f89a80a2b80 msgr2=0x7f89a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:56.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.016+0000 7f89c0a8d640 1 --2- 172.21.15.6:0/2089297749 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f89a80a2b80 0x7f89a80a5040 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00b970 tx=0x7f89ac00bd70 comp rx=0 tx=0).stop 2024-09-12T03:28:56.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.016+0000 7f89c0a8d640 1 -- 172.21.15.6:0/2089297749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89bc1036e0 msgr2=0x7f89bc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:56.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.016+0000 7f89c0a8d640 1 --2- 172.21.15.6:0/2089297749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89bc1036e0 0x7f89bc100ed0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f89a400c970 tx=0x7f89a400ce30 comp rx=0 tx=0).stop 2024-09-12T03:28:56.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.016+0000 7f89c0a8d640 1 -- 172.21.15.6:0/2089297749 shutdown_connections 2024-09-12T03:28:56.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.016+0000 7f89c0a8d640 1 --2- 172.21.15.6:0/2089297749 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f89a80a2b80 0x7f89a80a5040 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.016+0000 7f89c0a8d640 1 --2- 172.21.15.6:0/2089297749 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89bc1082a0 0x7f89bc101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.016+0000 7f89c0a8d640 1 --2- 172.21.15.6:0/2089297749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89bc1036e0 0x7f89bc100ed0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.016+0000 7f89c0a8d640 1 --2- 172.21.15.6:0/2089297749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89bc102d30 0x7f89bc100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:56.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.016+0000 7f89c0a8d640 1 -- 172.21.15.6:0/2089297749 >> 172.21.15.6:0/2089297749 conn(0x7f89bc0fc7d0 msgr2=0x7f89bc105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:56.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.016+0000 7f89c0a8d640 1 -- 172.21.15.6:0/2089297749 shutdown_connections 2024-09-12T03:28:56.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:56.016+0000 7f89c0a8d640 1 -- 172.21.15.6:0/2089297749 wait complete. 2024-09-12T03:28:56.810 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:28:57.076 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:57.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:56 smithi006 ceph-mon[30991]: pgmap v570: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:28:57.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:56 smithi161 ceph-mon[35485]: pgmap v570: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:28:57.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:56 smithi118 ceph-mon[35048]: pgmap v570: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:28:57.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.623+0000 7f56ff37b640 1 -- 172.21.15.6:0/3665352268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56f81036e0 msgr2=0x7f56f8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:57.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.623+0000 7f56ff37b640 1 --2- 172.21.15.6:0/3665352268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56f81036e0 0x7f56f8107b90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009930 tx=0x7f56e802a420 comp rx=0 tx=0).stop 2024-09-12T03:28:57.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.623+0000 7f56ff37b640 1 -- 172.21.15.6:0/3665352268 shutdown_connections 2024-09-12T03:28:57.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.623+0000 7f56ff37b640 1 --2- 172.21.15.6:0/3665352268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56f81082a0 0x7f56f810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.623+0000 7f56ff37b640 1 --2- 172.21.15.6:0/3665352268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56f81036e0 0x7f56f8107b90 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.623+0000 7f56ff37b640 1 --2- 172.21.15.6:0/3665352268 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f56f8102d30 0x7f56f8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.623+0000 7f56ff37b640 1 -- 172.21.15.6:0/3665352268 >> 172.21.15.6:0/3665352268 conn(0x7f56f80fc7d0 msgr2=0x7f56f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:57.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.623+0000 7f56ff37b640 1 -- 172.21.15.6:0/3665352268 shutdown_connections 2024-09-12T03:28:57.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.623+0000 7f56ff37b640 1 -- 172.21.15.6:0/3665352268 wait complete. 2024-09-12T03:28:57.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.624+0000 7f56ff37b640 1 Processor -- start 2024-09-12T03:28:57.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.624+0000 7f56ff37b640 1 -- start start 2024-09-12T03:28:57.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.624+0000 7f56ff37b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56f8102d30 0x7f56f8198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.624+0000 7f56ff37b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f56f81036e0 0x7f56f8198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.624+0000 7f56ff37b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56f81082a0 0x7f56f819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.624+0000 7f56ff37b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f56f81120a0 con 0x7f56f8102d30 2024-09-12T03:28:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.624+0000 7f56ff37b640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f56f8111f20 con 0x7f56f81036e0 2024-09-12T03:28:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.624+0000 7f56ff37b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f56f8112220 con 0x7f56f81082a0 2024-09-12T03:28:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.624+0000 7f56fd117640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56f8102d30 0x7f56f8198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.624+0000 7f56fd117640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56f8102d30 0x7f56f8198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36074/0 (socket says 172.21.15.6:36074) 2024-09-12T03:28:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.624+0000 7f56fd117640 1 -- 172.21.15.6:0/806580256 learned_addr learned my addr 172.21.15.6:0/806580256 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.624+0000 7f56fc916640 1 --2- 172.21.15.6:0/806580256 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f56f81036e0 0x7f56f8198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.624+0000 7f56fd918640 1 --2- 172.21.15.6:0/806580256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56f81082a0 0x7f56f819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.625+0000 7f56fd117640 1 -- 172.21.15.6:0/806580256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56f81082a0 msgr2=0x7f56f819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.625+0000 7f56fd117640 1 --2- 172.21.15.6:0/806580256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56f81082a0 0x7f56f819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.625+0000 7f56fd117640 1 -- 172.21.15.6:0/806580256 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f56f81036e0 msgr2=0x7f56f8198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:57.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.625+0000 7f56fd117640 1 --2- 172.21.15.6:0/806580256 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f56f81036e0 0x7f56f8198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.625+0000 7f56fd117640 1 -- 172.21.15.6:0/806580256 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f819d5f0 con 0x7f56f8102d30 2024-09-12T03:28:57.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.625+0000 7f56fd117640 1 --2- 172.21.15.6:0/806580256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56f8102d30 0x7f56f8198640 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f56f400d950 tx=0x7f56f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:57.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.625+0000 7f56ee7fc640 1 -- 172.21.15.6:0/806580256 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f4017070 con 0x7f56f8102d30 2024-09-12T03:28:57.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.625+0000 7f56ee7fc640 1 -- 172.21.15.6:0/806580256 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f4009c30 con 0x7f56f8102d30 2024-09-12T03:28:57.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.625+0000 7f56ff37b640 1 -- 172.21.15.6:0/806580256 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f819d8e0 con 0x7f56f8102d30 2024-09-12T03:28:57.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.625+0000 7f56ee7fc640 1 -- 172.21.15.6:0/806580256 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f400f6b0 con 0x7f56f8102d30 2024-09-12T03:28:57.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.625+0000 7f56ff37b640 1 -- 172.21.15.6:0/806580256 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f81a51c0 con 0x7f56f8102d30 2024-09-12T03:28:57.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.627+0000 7f56ee7fc640 1 -- 172.21.15.6:0/806580256 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f56f40109d0 con 0x7f56f8102d30 2024-09-12T03:28:57.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.628+0000 7f5701754740 1 Processor -- start 2024-09-12T03:28:57.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.628+0000 7f5701754740 1 -- start start 2024-09-12T03:28:57.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.628+0000 7f5701754740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce1b40 0x5597bcce0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:57.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.628+0000 7f5701754740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5597bcce05d0 0x5597bcce0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:57.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.628+0000 7f5701754740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5597bccef780 0x5597bccf1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.628+0000 7f56ee7fc640 1 --2- 172.21.15.6:0/806580256 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f56e40a2b80 0x7f56e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.628+0000 7f5701754740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5597bcc462f0 con 0x5597bcce1b40 2024-09-12T03:28:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.628+0000 7f5701754740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5597bcbedcf0 con 0x5597bcce05d0 2024-09-12T03:28:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.628+0000 7f5701754740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5597bcc422b0 con 0x5597bccef780 2024-09-12T03:28:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.629+0000 7f56fd918640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce1b40 0x5597bcce0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.629+0000 7f56ee7fc640 1 -- 172.21.15.6:0/806580256 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f56f40cebf0 con 0x7f56f8102d30 2024-09-12T03:28:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.629+0000 7f56fd918640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce1b40 0x5597bcce0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36078/0 (socket says 172.21.15.6:36078) 2024-09-12T03:28:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.629+0000 7f56fd918640 1 -- 172.21.15.6:0/3052627123 learned_addr learned my addr 172.21.15.6:0/3052627123 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.629+0000 7f56fc916640 1 --2- 172.21.15.6:0/3052627123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5597bccef780 0x5597bccf1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.629+0000 7f56fd117640 1 --2- 172.21.15.6:0/3052627123 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5597bcce05d0 0x5597bcce0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.629+0000 7f56fd918640 1 --2- 172.21.15.6:0/806580256 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f56e40a2b80 0x7f56e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:57.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.629+0000 7f56fc916640 1 -- 172.21.15.6:0/3052627123 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5597bcce05d0 msgr2=0x5597bcce0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:57.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.629+0000 7f56fc916640 1 --2- 172.21.15.6:0/3052627123 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5597bcce05d0 0x5597bcce0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.629+0000 7f56fc916640 1 -- 172.21.15.6:0/3052627123 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce1b40 msgr2=0x5597bcce0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:57.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.629+0000 7f56fc916640 1 --2- 172.21.15.6:0/3052627123 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce1b40 0x5597bcce0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.629+0000 7f56fc916640 1 -- 172.21.15.6:0/3052627123 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597bcbd23a0 con 0x5597bccef780 2024-09-12T03:28:57.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.629+0000 7f56fc916640 1 --2- 172.21.15.6:0/3052627123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5597bccef780 0x5597bccf1b70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f56e802a940 tx=0x7f56e8005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d99952325b8b3f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:57.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.629+0000 7f56fd918640 1 --2- 172.21.15.6:0/806580256 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f56e40a2b80 0x7f56e40a5040 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009fd0 tx=0x7f56f0013040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:57.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.629+0000 7f56e37fe640 1 -- 172.21.15.6:0/3052627123 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8040070 con 0x5597bccef780 2024-09-12T03:28:57.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.629+0000 7f56e37fe640 1 -- 172.21.15.6:0/3052627123 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56e8004030 con 0x5597bccef780 2024-09-12T03:28:57.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.630+0000 7f56e37fe640 1 -- 172.21.15.6:0/3052627123 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8035040 con 0x5597bccef780 2024-09-12T03:28:57.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.630+0000 7f5701754740 1 -- 172.21.15.6:0/3052627123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5597bccef780 msgr2=0x5597bccf1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:57.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.630+0000 7f5701754740 1 --2- 172.21.15.6:0/3052627123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5597bccef780 0x5597bccf1b70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f56e802a940 tx=0x7f56e8005e20 comp rx=0 tx=0).stop 2024-09-12T03:28:57.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.630+0000 7f5701754740 1 -- 172.21.15.6:0/3052627123 shutdown_connections 2024-09-12T03:28:57.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.630+0000 7f5701754740 1 --2- 172.21.15.6:0/3052627123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5597bccef780 0x5597bccf1b70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.630+0000 7f5701754740 1 --2- 172.21.15.6:0/3052627123 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5597bcce05d0 0x5597bcce0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.630+0000 7f5701754740 1 --2- 172.21.15.6:0/3052627123 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce1b40 0x5597bcce0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.630+0000 7f5701754740 1 -- 172.21.15.6:0/3052627123 >> 172.21.15.6:0/3052627123 conn(0x5597bcca9310 msgr2=0x5597bccdcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:57.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.630+0000 7f5701754740 1 -- 172.21.15.6:0/3052627123 shutdown_connections 2024-09-12T03:28:57.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.630+0000 7f5701754740 1 -- 172.21.15.6:0/3052627123 wait complete. 2024-09-12T03:28:57.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.631+0000 7f5701754740 1 -- start start 2024-09-12T03:28:57.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.631+0000 7f5701754740 1 Processor -- start 2024-09-12T03:28:57.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.631+0000 7f5701754740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce05d0 0x5597bccef140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:57.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.631+0000 7f5701754740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5597bcce1b40 0x5597bccec3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:57.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.631+0000 7f5701754740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5597bccef780 0x5597bccec9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:57.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.631+0000 7f5701754740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5597bccf2420 con 0x5597bcce05d0 2024-09-12T03:28:57.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.631+0000 7f5701754740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x5597bcc01060 con 0x5597bcce1b40 2024-09-12T03:28:57.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.631+0000 7f5701754740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5597bccf25a0 con 0x5597bccef780 2024-09-12T03:28:57.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.631+0000 7f5701754740 10 client.0 _ll_register_callbacks cb 0x5597bc9548c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:57.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.631+0000 7f56fd117640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce05d0 0x5597bccef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:57.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.631+0000 7f56fd117640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce05d0 0x5597bccef140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36092/0 (socket says 172.21.15.6:36092) 2024-09-12T03:28:57.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.631+0000 7f56fd117640 1 -- 172.21.15.6:0/3229930907 learned_addr learned my addr 172.21.15.6:0/3229930907 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:57.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.631+0000 7f56fd918640 1 --2- 172.21.15.6:0/3229930907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5597bccef780 0x5597bccec9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:57.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.631+0000 7f56fc916640 1 --2- 172.21.15.6:0/3229930907 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5597bcce1b40 0x5597bccec3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:57.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.632+0000 7f56fd117640 1 -- 172.21.15.6:0/3229930907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5597bccef780 msgr2=0x5597bccec9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:57.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.632+0000 7f56fd117640 1 --2- 172.21.15.6:0/3229930907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5597bccef780 0x5597bccec9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.632+0000 7f56fd117640 1 -- 172.21.15.6:0/3229930907 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5597bcce1b40 msgr2=0x5597bccec3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:57.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.632+0000 7f56fd117640 1 --2- 172.21.15.6:0/3229930907 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5597bcce1b40 0x5597bccec3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.632+0000 7f56fd117640 1 -- 172.21.15.6:0/3229930907 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597bc9b73c0 con 0x5597bcce05d0 2024-09-12T03:28:57.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.632+0000 7f56fd117640 1 --2- 172.21.15.6:0/3229930907 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce05d0 0x5597bccef140 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f56f4010fd0 tx=0x7f56f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2207d259d6e74093 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:57.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.632+0000 7f56e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:28:57.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.632+0000 7f56e0ff9640 1 -- 172.21.15.6:0/3229930907 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f4020040 con 0x5597bcce05d0 2024-09-12T03:28:57.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.632+0000 7f56e0ff9640 1 -- 172.21.15.6:0/3229930907 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f56f40ab2f0 con 0x5597bcce05d0 2024-09-12T03:28:57.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.632+0000 7f5701754740 1 -- 172.21.15.6:0/3229930907 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56e8004030 con 0x5597bcce05d0 2024-09-12T03:28:57.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.632+0000 7f56e0ff9640 1 -- 172.21.15.6:0/3229930907 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f56f409f830 con 0x5597bcce05d0 2024-09-12T03:28:57.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.632+0000 7f5701754740 10 client.16266 fetch_fsmap learned FSMap version 29 2024-09-12T03:28:57.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.632+0000 7f5701754740 1 -- 172.21.15.6:0/3229930907 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5597bccfdee0 con 0x5597bcce05d0 2024-09-12T03:28:57.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.632+0000 7f56e0ff9640 1 -- 172.21.15.6:0/3229930907 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56e8004030 con 0x5597bcce05d0 2024-09-12T03:28:57.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.633+0000 7f56e0ff9640 1 -- 172.21.15.6:0/3229930907 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f56f409a980 con 0x5597bcce05d0 2024-09-12T03:28:57.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.633+0000 7f5701754740 10 client.16266 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T03:28:57.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.633+0000 7f5701754740 10 client.16266 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-12T03:28:57.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.633+0000 7f5701754740 20 client.16266 populate_metadata read hostname 'smithi006' 2024-09-12T03:28:57.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.633+0000 7f5701754740 1 --2- 172.21.15.6:0/3229930907 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x5597bccff2f0 0x5597bcd01760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:57.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.633+0000 7f5701754740 4 client.16266 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:57.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.633+0000 7f5701754740 1 -- 172.21.15.6:0/3229930907 --> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597bcc53860 con 0x5597bccff2f0 2024-09-12T03:28:57.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.633+0000 7f56fc916640 1 --2- 172.21.15.6:0/3229930907 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x5597bccff2f0 0x5597bcd01760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:57.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.633+0000 7f56e0ff9640 10 client.16266 ms_handle_connect on v2:172.21.15.118:6836/3879191727 2024-09-12T03:28:57.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.633+0000 7f56fc916640 1 --2- 172.21.15.6:0/3229930907 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x5597bccff2f0 0x5597bcd01760 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e6bc5a8d14df4e4 server_cookie=abd9cf812227bbc1 in_seq=0 out_seq=0 2024-09-12T03:28:57.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.634+0000 7f56e0ff9640 1 -- 172.21.15.6:0/3229930907 <== mds.1 v2:172.21.15.118:6836/3879191727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5597bcc53860 con 0x5597bccff2f0 2024-09-12T03:28:57.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.634+0000 7f56e0ff9640 10 client.16266 handle_command_reply: tid=0 2024-09-12T03:28:57.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.634+0000 7f5701754740 1 client.16266 shutdown 2024-09-12T03:28:57.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.634+0000 7f5701754740 1 -- 172.21.15.6:0/3229930907 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce05d0 msgr2=0x5597bccef140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:57.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.634+0000 7f5701754740 1 --2- 172.21.15.6:0/3229930907 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce05d0 0x5597bccef140 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f56f4010fd0 tx=0x7f56f4005880 comp rx=0 tx=0).stop 2024-09-12T03:28:57.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.635+0000 7f5701754740 1 -- 172.21.15.6:0/3229930907 shutdown_connections 2024-09-12T03:28:57.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.635+0000 7f5701754740 1 --2- 172.21.15.6:0/3229930907 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x5597bccff2f0 0x5597bcd01760 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.635+0000 7f5701754740 1 --2- 172.21.15.6:0/3229930907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5597bccef780 0x5597bccec9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.635+0000 7f5701754740 1 --2- 172.21.15.6:0/3229930907 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5597bcce1b40 0x5597bccec3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.635+0000 7f5701754740 1 --2- 172.21.15.6:0/3229930907 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce05d0 0x5597bccef140 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.635+0000 7f5701754740 1 -- 172.21.15.6:0/3229930907 >> 172.21.15.6:0/3229930907 conn(0x5597bcca9310 msgr2=0x5597bcce2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:57.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.635+0000 7f56e0ff9640 0 client.16266 ms_handle_reset on v2:172.21.15.118:6836/3879191727 2024-09-12T03:28:57.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.635+0000 7f5701754740 1 -- 172.21.15.6:0/3229930907 shutdown_connections 2024-09-12T03:28:57.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.635+0000 7f5701754740 1 -- 172.21.15.6:0/3229930907 wait complete. 2024-09-12T03:28:57.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.635+0000 7f5701754740 20 client.16266 trim_cache size 0 max 16384 2024-09-12T03:28:57.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.648+0000 7f5701754740 1 Processor -- start 2024-09-12T03:28:57.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.648+0000 7f5701754740 1 -- start start 2024-09-12T03:28:57.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.648+0000 7f5701754740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce05d0 0x5597bccdce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:57.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.648+0000 7f5701754740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5597bcce1b40 0x5597bccfb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:57.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.648+0000 7f5701754740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5597bccef780 0x5597bcd07080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:57.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.648+0000 7f5701754740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f56f81adb40 con 0x5597bcce05d0 2024-09-12T03:28:57.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.648+0000 7f5701754740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f56f81ab3b0 con 0x5597bcce1b40 2024-09-12T03:28:57.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.648+0000 7f5701754740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f56f81a9200 con 0x5597bccef780 2024-09-12T03:28:57.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.649+0000 7f56fd117640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce05d0 0x5597bccdce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:57.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.649+0000 7f56fc916640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5597bcce1b40 0x5597bccfb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:57.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.649+0000 7f56fd117640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce05d0 0x5597bccdce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36102/0 (socket says 172.21.15.6:36102) 2024-09-12T03:28:57.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.649+0000 7f56fd117640 1 -- 172.21.15.6:0/1302638518 learned_addr learned my addr 172.21.15.6:0/1302638518 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:57.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.649+0000 7f56fd918640 1 --2- 172.21.15.6:0/1302638518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5597bccef780 0x5597bcd07080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:57.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.649+0000 7f56fd117640 1 -- 172.21.15.6:0/1302638518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5597bccef780 msgr2=0x5597bcd07080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:57.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.649+0000 7f56fd117640 1 --2- 172.21.15.6:0/1302638518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5597bccef780 0x5597bcd07080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.649+0000 7f56fd117640 1 -- 172.21.15.6:0/1302638518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5597bcce1b40 msgr2=0x5597bccfb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:57.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.649+0000 7f56fd117640 1 --2- 172.21.15.6:0/1302638518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5597bcce1b40 0x5597bccfb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.649+0000 7f56fd117640 1 -- 172.21.15.6:0/1302638518 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597bccf1ee0 con 0x5597bcce05d0 2024-09-12T03:28:57.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.649+0000 7f56fc916640 1 --2- 172.21.15.6:0/1302638518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5597bcce1b40 0x5597bccfb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:57.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.649+0000 7f56fd117640 1 --2- 172.21.15.6:0/1302638518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce05d0 0x5597bccdce10 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f56f4097210 tx=0x7f56f4099a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33368a44b685fec7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:57.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.649+0000 7f56e2ffd640 1 -- 172.21.15.6:0/1302638518 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f40b3030 con 0x5597bcce05d0 2024-09-12T03:28:57.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.649+0000 7f56e2ffd640 1 -- 172.21.15.6:0/1302638518 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f40a40f0 con 0x5597bcce05d0 2024-09-12T03:28:57.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.649+0000 7f56e2ffd640 1 -- 172.21.15.6:0/1302638518 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f4020070 con 0x5597bcce05d0 2024-09-12T03:28:57.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.650+0000 7f5701754740 1 -- 172.21.15.6:0/1302638518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce05d0 msgr2=0x5597bccdce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:57.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.650+0000 7f5701754740 1 --2- 172.21.15.6:0/1302638518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce05d0 0x5597bccdce10 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f56f4097210 tx=0x7f56f4099a40 comp rx=0 tx=0).stop 2024-09-12T03:28:57.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.650+0000 7f5701754740 1 -- 172.21.15.6:0/1302638518 shutdown_connections 2024-09-12T03:28:57.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.650+0000 7f5701754740 1 --2- 172.21.15.6:0/1302638518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5597bccef780 0x5597bcd07080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.650+0000 7f5701754740 1 --2- 172.21.15.6:0/1302638518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5597bcce1b40 0x5597bccfb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.650+0000 7f5701754740 1 --2- 172.21.15.6:0/1302638518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce05d0 0x5597bccdce10 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.650+0000 7f5701754740 1 -- 172.21.15.6:0/1302638518 >> 172.21.15.6:0/1302638518 conn(0x5597bcca9310 msgr2=0x5597bccdc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:57.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.650+0000 7f5701754740 1 -- 172.21.15.6:0/1302638518 shutdown_connections 2024-09-12T03:28:57.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.650+0000 7f5701754740 1 -- 172.21.15.6:0/1302638518 wait complete. 2024-09-12T03:28:57.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.650+0000 7f5701754740 1 -- start start 2024-09-12T03:28:57.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.651+0000 7f5701754740 1 Processor -- start 2024-09-12T03:28:57.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.651+0000 7f5701754740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce05d0 0x5597bccf7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:57.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.651+0000 7f5701754740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5597bcce1b40 0x5597bccf7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:57.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.651+0000 7f5701754740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5597bccef780 0x5597bcce8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:57.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.651+0000 7f5701754740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f56f81a9870 con 0x5597bcce05d0 2024-09-12T03:28:57.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.651+0000 7f5701754740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f56f81aace0 con 0x5597bccef780 2024-09-12T03:28:57.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.651+0000 7f5701754740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f56f81a9f40 con 0x5597bcce1b40 2024-09-12T03:28:57.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.651+0000 7f5701754740 10 client.0 _ll_register_callbacks cb 0x7f56f409b300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:57.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.651+0000 7f56fd117640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce05d0 0x5597bccf7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:57.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.651+0000 7f56fd117640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce05d0 0x5597bccf7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36114/0 (socket says 172.21.15.6:36114) 2024-09-12T03:28:57.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.651+0000 7f56fd117640 1 -- 172.21.15.6:0/545181457 learned_addr learned my addr 172.21.15.6:0/545181457 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:57.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.651+0000 7f56fc916640 1 --2- 172.21.15.6:0/545181457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5597bcce1b40 0x5597bccf7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:57.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.651+0000 7f56fd918640 1 --2- 172.21.15.6:0/545181457 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5597bccef780 0x5597bcce8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:57.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.651+0000 7f56fd117640 1 -- 172.21.15.6:0/545181457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5597bcce1b40 msgr2=0x5597bccf7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:57.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.651+0000 7f56fd117640 1 --2- 172.21.15.6:0/545181457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5597bcce1b40 0x5597bccf7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.651+0000 7f56fd117640 1 -- 172.21.15.6:0/545181457 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5597bccef780 msgr2=0x5597bcce8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:57.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.651+0000 7f56fd117640 1 --2- 172.21.15.6:0/545181457 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5597bccef780 0x5597bcce8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.651+0000 7f56fd117640 1 -- 172.21.15.6:0/545181457 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597bcce9280 con 0x5597bcce05d0 2024-09-12T03:28:57.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.652+0000 7f56fd117640 1 --2- 172.21.15.6:0/545181457 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce05d0 0x5597bccf7830 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f56f4097990 tx=0x7f56f4005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5364eff8c23519b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:57.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.652+0000 7f56e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:28:57.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.652+0000 7f56e0ff9640 1 -- 172.21.15.6:0/545181457 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f40b8060 con 0x5597bcce05d0 2024-09-12T03:28:57.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.652+0000 7f56e0ff9640 1 -- 172.21.15.6:0/545181457 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f56f4097280 con 0x5597bcce05d0 2024-09-12T03:28:57.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.652+0000 7f5701754740 1 -- 172.21.15.6:0/545181457 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56f40a40f0 con 0x5597bcce05d0 2024-09-12T03:28:57.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.652+0000 7f56e0ff9640 1 -- 172.21.15.6:0/545181457 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f56f40b3070 con 0x5597bcce05d0 2024-09-12T03:28:57.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.652+0000 7f5701754740 10 client.16272 fetch_fsmap learned FSMap version 29 2024-09-12T03:28:57.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.652+0000 7f5701754740 1 -- 172.21.15.6:0/545181457 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5597bcd69950 con 0x5597bcce05d0 2024-09-12T03:28:57.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.652+0000 7f56e0ff9640 1 -- 172.21.15.6:0/545181457 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f40a40f0 con 0x5597bcce05d0 2024-09-12T03:28:57.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.652+0000 7f56e0ff9640 1 -- 172.21.15.6:0/545181457 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f56f40ac070 con 0x5597bcce05d0 2024-09-12T03:28:57.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.652+0000 7f5701754740 10 client.16272 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T03:28:57.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.652+0000 7f5701754740 10 client.16272 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-12T03:28:57.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.652+0000 7f5701754740 20 client.16272 populate_metadata read hostname 'smithi006' 2024-09-12T03:28:57.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.652+0000 7f5701754740 1 --2- 172.21.15.6:0/545181457 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x5597bcd69d80 0x5597bcd6c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:57.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.652+0000 7f5701754740 4 client.16272 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:28:57.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.652+0000 7f5701754740 1 -- 172.21.15.6:0/545181457 --> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5597bcc4dd70 con 0x5597bcd69d80 2024-09-12T03:28:57.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.653+0000 7f56fc916640 1 --2- 172.21.15.6:0/545181457 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x5597bcd69d80 0x5597bcd6c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:57.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.653+0000 7f56fc916640 1 --2- 172.21.15.6:0/545181457 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x5597bcd69d80 0x5597bcd6c1a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e31fff2f86d8cd57 server_cookie=3cfb38d6aeb53500 in_seq=0 out_seq=0 2024-09-12T03:28:57.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.653+0000 7f56e0ff9640 10 client.16272 ms_handle_connect on v2:172.21.15.118:6836/3879191727 2024-09-12T03:28:57.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.653+0000 7f56e0ff9640 1 -- 172.21.15.6:0/545181457 <== mds.1 v2:172.21.15.118:6836/3879191727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5597bcc4dd70 con 0x5597bcd69d80 2024-09-12T03:28:57.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.653+0000 7f56e0ff9640 10 client.16272 handle_command_reply: tid=0 2024-09-12T03:28:57.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.653+0000 7f5701754740 1 client.16272 shutdown 2024-09-12T03:28:57.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.653+0000 7f5701754740 1 -- 172.21.15.6:0/545181457 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce05d0 msgr2=0x5597bccf7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:57.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.653+0000 7f5701754740 1 --2- 172.21.15.6:0/545181457 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce05d0 0x5597bccf7830 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f56f4097990 tx=0x7f56f4005bb0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.653+0000 7f5701754740 1 -- 172.21.15.6:0/545181457 shutdown_connections 2024-09-12T03:28:57.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.653+0000 7f5701754740 1 --2- 172.21.15.6:0/545181457 >> [v2:172.21.15.118:6836/3879191727,v1:172.21.15.118:6837/3879191727] conn(0x5597bcd69d80 0x5597bcd6c1a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.653+0000 7f5701754740 1 --2- 172.21.15.6:0/545181457 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5597bccef780 0x5597bcce8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.653+0000 7f5701754740 1 --2- 172.21.15.6:0/545181457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5597bcce1b40 0x5597bccf7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.653+0000 7f5701754740 1 --2- 172.21.15.6:0/545181457 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5597bcce05d0 0x5597bccf7830 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.653+0000 7f56e0ff9640 0 client.16272 ms_handle_reset on v2:172.21.15.118:6836/3879191727 2024-09-12T03:28:57.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.653+0000 7f5701754740 1 -- 172.21.15.6:0/545181457 >> 172.21.15.6:0/545181457 conn(0x5597bcca9310 msgr2=0x5597bcce2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:57.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.654+0000 7f5701754740 1 -- 172.21.15.6:0/545181457 shutdown_connections 2024-09-12T03:28:57.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.654+0000 7f5701754740 1 -- 172.21.15.6:0/545181457 wait complete. 2024-09-12T03:28:57.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.654+0000 7f5701754740 20 client.16272 trim_cache size 0 max 16384 2024-09-12T03:28:57.661 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-09-12T03:28:57.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.654+0000 7f56ff37b640 1 -- 172.21.15.6:0/806580256 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f56e40a2b80 msgr2=0x7f56e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:57.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.654+0000 7f56ff37b640 1 --2- 172.21.15.6:0/806580256 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f56e40a2b80 0x7f56e40a5040 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009fd0 tx=0x7f56f0013040 comp rx=0 tx=0).stop 2024-09-12T03:28:57.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.654+0000 7f56ff37b640 1 -- 172.21.15.6:0/806580256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56f8102d30 msgr2=0x7f56f8198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:57.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.654+0000 7f56ff37b640 1 --2- 172.21.15.6:0/806580256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56f8102d30 0x7f56f8198640 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f56f400d950 tx=0x7f56f400de10 comp rx=0 tx=0).stop 2024-09-12T03:28:57.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.654+0000 7f56ff37b640 1 -- 172.21.15.6:0/806580256 shutdown_connections 2024-09-12T03:28:57.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.654+0000 7f56ff37b640 1 --2- 172.21.15.6:0/806580256 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f56e40a2b80 0x7f56e40a5040 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.654+0000 7f56ff37b640 1 --2- 172.21.15.6:0/806580256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56f81082a0 0x7f56f819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.654+0000 7f56ff37b640 1 --2- 172.21.15.6:0/806580256 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f56f81036e0 0x7f56f8198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.654+0000 7f56ff37b640 1 --2- 172.21.15.6:0/806580256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56f8102d30 0x7f56f8198640 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:57.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.654+0000 7f56ff37b640 1 -- 172.21.15.6:0/806580256 >> 172.21.15.6:0/806580256 conn(0x7f56f80fc7d0 msgr2=0x7f56f8106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:57.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.654+0000 7f56ff37b640 1 -- 172.21.15.6:0/806580256 shutdown_connections 2024-09-12T03:28:57.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:57.655+0000 7f56ff37b640 1 -- 172.21.15.6:0/806580256 wait complete. 2024-09-12T03:28:58.417 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:28:58.676 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:59.076 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:28:58 smithi006 ceph-mon[30991]: pgmap v571: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:28:59.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:28:58 smithi161 ceph-mon[35485]: pgmap v571: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:28:59.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.160+0000 7f00f8cbf640 1 -- 172.21.15.6:0/3995383935 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f41036c0 msgr2=0x7f00f410bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:59.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.160+0000 7f00f8cbf640 1 --2- 172.21.15.6:0/3995383935 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f41036c0 0x7f00f410bbb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009900 tx=0x7f00e802a480 comp rx=0 tx=0).stop 2024-09-12T03:28:59.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.161+0000 7f00f8cbf640 1 -- 172.21.15.6:0/3995383935 shutdown_connections 2024-09-12T03:28:59.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.161+0000 7f00f8cbf640 1 --2- 172.21.15.6:0/3995383935 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00f410c180 0x7f00f410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.161+0000 7f00f8cbf640 1 --2- 172.21.15.6:0/3995383935 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f41036c0 0x7f00f410bbb0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.161+0000 7f00f8cbf640 1 --2- 172.21.15.6:0/3995383935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00f4102da0 0x7f00f4103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.161+0000 7f00f8cbf640 1 -- 172.21.15.6:0/3995383935 >> 172.21.15.6:0/3995383935 conn(0x7f00f40fc7a0 msgr2=0x7f00f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:59.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.161+0000 7f00f8cbf640 1 -- 172.21.15.6:0/3995383935 shutdown_connections 2024-09-12T03:28:59.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.161+0000 7f00f8cbf640 1 -- 172.21.15.6:0/3995383935 wait complete. 2024-09-12T03:28:59.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.162+0000 7f00f8cbf640 1 Processor -- start 2024-09-12T03:28:59.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.162+0000 7f00f8cbf640 1 -- start start 2024-09-12T03:28:59.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.162+0000 7f00f8cbf640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00f4102da0 0x7f00f4198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.162+0000 7f00f8cbf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00f41036c0 0x7f00f4198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.162+0000 7f00f8cbf640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f410c180 0x7f00f419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.162+0000 7f00f8cbf640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f00f4111f60 con 0x7f00f410c180 2024-09-12T03:28:59.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.162+0000 7f00f8cbf640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f00f4111de0 con 0x7f00f4102da0 2024-09-12T03:28:59.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.162+0000 7f00f8cbf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f00f41120e0 con 0x7f00f41036c0 2024-09-12T03:28:59.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.162+0000 7f00f2d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f410c180 0x7f00f419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.162+0000 7f00f2d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f410c180 0x7f00f419cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56178/0 (socket says 172.21.15.6:56178) 2024-09-12T03:28:59.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.162+0000 7f00f2d9d640 1 -- 172.21.15.6:0/2645095825 learned_addr learned my addr 172.21.15.6:0/2645095825 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:59.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.162+0000 7f00f1d9b640 1 --2- 172.21.15.6:0/2645095825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00f41036c0 0x7f00f4198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.162+0000 7f00f259c640 1 --2- 172.21.15.6:0/2645095825 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00f4102da0 0x7f00f4198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.162+0000 7f00f2d9d640 1 -- 172.21.15.6:0/2645095825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00f41036c0 msgr2=0x7f00f4198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:59.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.162+0000 7f00f2d9d640 1 --2- 172.21.15.6:0/2645095825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00f41036c0 0x7f00f4198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.162+0000 7f00f2d9d640 1 -- 172.21.15.6:0/2645095825 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00f4102da0 msgr2=0x7f00f4198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:59.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.162+0000 7f00f2d9d640 1 --2- 172.21.15.6:0/2645095825 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00f4102da0 0x7f00f4198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.163+0000 7f00f2d9d640 1 -- 172.21.15.6:0/2645095825 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f419d3f0 con 0x7f00f410c180 2024-09-12T03:28:59.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.163+0000 7f00f2d9d640 1 --2- 172.21.15.6:0/2645095825 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f410c180 0x7f00f419cd10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00eaa0 tx=0x7f00ec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:59.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.163+0000 7f00e37fe640 1 -- 172.21.15.6:0/2645095825 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec00cdb0 con 0x7f00f410c180 2024-09-12T03:28:59.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.163+0000 7f00e37fe640 1 -- 172.21.15.6:0/2645095825 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00ec005020 con 0x7f00f410c180 2024-09-12T03:28:59.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.163+0000 7f00f8cbf640 1 -- 172.21.15.6:0/2645095825 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f419d6e0 con 0x7f00f410c180 2024-09-12T03:28:59.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.163+0000 7f00e37fe640 1 -- 172.21.15.6:0/2645095825 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec0116b0 con 0x7f00f410c180 2024-09-12T03:28:59.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.163+0000 7f00f8cbf640 1 -- 172.21.15.6:0/2645095825 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f41a4fc0 con 0x7f00f410c180 2024-09-12T03:28:59.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.165+0000 7f00e37fe640 1 -- 172.21.15.6:0/2645095825 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f00ec005560 con 0x7f00f410c180 2024-09-12T03:28:59.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.165+0000 7f00fb098740 1 Processor -- start 2024-09-12T03:28:59.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.166+0000 7f00fb098740 1 -- start start 2024-09-12T03:28:59.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.166+0000 7f00fb098740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c0836eb40 0x557c0836d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.166+0000 7f00fb098740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0836d5d0 0x557c0836da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.166+0000 7f00fb098740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557c08378370 0x557c0837a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.166+0000 7f00fb098740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x557c082d32f0 con 0x557c0836eb40 2024-09-12T03:28:59.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.166+0000 7f00fb098740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x557c0827acf0 con 0x557c08378370 2024-09-12T03:28:59.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.166+0000 7f00fb098740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557c082cf2b0 con 0x557c0836d5d0 2024-09-12T03:28:59.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.166+0000 7f00f1d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c0836eb40 0x557c0836d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.166+0000 7f00f1d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c0836eb40 0x557c0836d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56184/0 (socket says 172.21.15.6:56184) 2024-09-12T03:28:59.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.166+0000 7f00f1d9b640 1 -- 172.21.15.6:0/905741465 learned_addr learned my addr 172.21.15.6:0/905741465 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:59.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.166+0000 7f00f259c640 1 --2- 172.21.15.6:0/905741465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0836d5d0 0x557c0836da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.167+0000 7f00f1d9b640 1 --2- 172.21.15.6:0/905741465 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557c08378370 0x557c0837a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.167+0000 7f00e37fe640 1 --2- 172.21.15.6:0/2645095825 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f00dc0a2c00 0x7f00dc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.167+0000 7f00f259c640 1 -- 172.21.15.6:0/905741465 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557c08378370 msgr2=0x557c0837a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:59.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.167+0000 7f00f259c640 1 --2- 172.21.15.6:0/905741465 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557c08378370 0x557c0837a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.167+0000 7f00f259c640 1 -- 172.21.15.6:0/905741465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c0836eb40 msgr2=0x557c0836d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:59.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.167+0000 7f00f259c640 1 --2- 172.21.15.6:0/905741465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c0836eb40 0x557c0836d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.167+0000 7f00f259c640 1 -- 172.21.15.6:0/905741465 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c0825f3a0 con 0x557c0836d5d0 2024-09-12T03:28:59.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.167+0000 7f00f1d9b640 1 --2- 172.21.15.6:0/905741465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c0836eb40 0x557c0836d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:59.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.167+0000 7f00f2d9d640 1 --2- 172.21.15.6:0/2645095825 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f00dc0a2c00 0x7f00dc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.167+0000 7f00e37fe640 1 -- 172.21.15.6:0/2645095825 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f00ec0cf1a0 con 0x7f00f410c180 2024-09-12T03:28:59.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.167+0000 7f00f259c640 1 --2- 172.21.15.6:0/905741465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0836d5d0 0x557c0836da10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f00e400bbe0 tx=0x7f00e400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adc3280d17eafa2f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:59.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.167+0000 7f00e0ff9640 1 -- 172.21.15.6:0/905741465 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e4015070 con 0x557c0836d5d0 2024-09-12T03:28:59.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.167+0000 7f00e0ff9640 1 -- 172.21.15.6:0/905741465 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e40035f0 con 0x557c0836d5d0 2024-09-12T03:28:59.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.167+0000 7f00e0ff9640 1 -- 172.21.15.6:0/905741465 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e40116a0 con 0x557c0836d5d0 2024-09-12T03:28:59.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.168+0000 7f00fb098740 1 -- 172.21.15.6:0/905741465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0836d5d0 msgr2=0x557c0836da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:59.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.168+0000 7f00f2d9d640 1 --2- 172.21.15.6:0/2645095825 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f00dc0a2c00 0x7f00dc0a50c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0cd040 tx=0x7f00ec0666b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:59.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.168+0000 7f00fb098740 1 --2- 172.21.15.6:0/905741465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0836d5d0 0x557c0836da10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f00e400bbe0 tx=0x7f00e400f730 comp rx=0 tx=0).stop 2024-09-12T03:28:59.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.168+0000 7f00fb098740 1 -- 172.21.15.6:0/905741465 shutdown_connections 2024-09-12T03:28:59.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.168+0000 7f00fb098740 1 --2- 172.21.15.6:0/905741465 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557c08378370 0x557c0837a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.168+0000 7f00fb098740 1 --2- 172.21.15.6:0/905741465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0836d5d0 0x557c0836da10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.168+0000 7f00fb098740 1 --2- 172.21.15.6:0/905741465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c0836eb40 0x557c0836d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.168+0000 7f00fb098740 1 -- 172.21.15.6:0/905741465 >> 172.21.15.6:0/905741465 conn(0x557c08336310 msgr2=0x557c08369c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:59.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.168+0000 7f00fb098740 1 -- 172.21.15.6:0/905741465 shutdown_connections 2024-09-12T03:28:59.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.168+0000 7f00fb098740 1 -- 172.21.15.6:0/905741465 wait complete. 2024-09-12T03:28:59.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.168+0000 7f00fb098740 1 -- start start 2024-09-12T03:28:59.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.169+0000 7f00fb098740 1 Processor -- start 2024-09-12T03:28:59.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.169+0000 7f00fb098740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557c0836d5d0 0x557c08377d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.169+0000 7f00fb098740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c0836eb40 0x557c08375040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.169+0000 7f00fb098740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c08378370 0x557c08375650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.169+0000 7f00fb098740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x557c0837b070 con 0x557c0836eb40 2024-09-12T03:28:59.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.169+0000 7f00fb098740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x557c0828e060 con 0x557c0836d5d0 2024-09-12T03:28:59.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.169+0000 7f00fb098740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557c0837b1f0 con 0x557c08378370 2024-09-12T03:28:59.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.169+0000 7f00f259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c0836eb40 0x557c08375040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.169+0000 7f00fb098740 10 client.0 _ll_register_callbacks cb 0x557c07fe18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:59.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.169+0000 7f00f259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c0836eb40 0x557c08375040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56196/0 (socket says 172.21.15.6:56196) 2024-09-12T03:28:59.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.169+0000 7f00f259c640 1 -- 172.21.15.6:0/4146089112 learned_addr learned my addr 172.21.15.6:0/4146089112 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:59.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.169+0000 7f00f1d9b640 1 --2- 172.21.15.6:0/4146089112 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557c0836d5d0 0x557c08377d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.169+0000 7f00f1d9b640 1 --2- 172.21.15.6:0/4146089112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c08378370 0x557c08375650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.169+0000 7f00f259c640 1 -- 172.21.15.6:0/4146089112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c08378370 msgr2=0x557c08375650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:59.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.169+0000 7f00f259c640 1 --2- 172.21.15.6:0/4146089112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c08378370 0x557c08375650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.169+0000 7f00f259c640 1 -- 172.21.15.6:0/4146089112 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557c0836d5d0 msgr2=0x557c08377d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:59.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.169+0000 7f00f259c640 1 --2- 172.21.15.6:0/4146089112 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557c0836d5d0 0x557c08377d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.169+0000 7f00f259c640 1 -- 172.21.15.6:0/4146089112 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c080443c0 con 0x557c0836eb40 2024-09-12T03:28:59.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.169+0000 7f00f1d9b640 1 --2- 172.21.15.6:0/4146089112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c08378370 0x557c08375650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:59.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.170+0000 7f00f259c640 1 --2- 172.21.15.6:0/4146089112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c0836eb40 0x557c08375040 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f00e400fc30 tx=0x7f00e400b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2307c6309aaac2c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:59.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.170+0000 7f00d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:28:59.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.170+0000 7f00d9ffb640 1 -- 172.21.15.6:0/4146089112 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e4015070 con 0x557c0836eb40 2024-09-12T03:28:59.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.170+0000 7f00d9ffb640 1 -- 172.21.15.6:0/4146089112 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f00e40039b0 con 0x557c0836eb40 2024-09-12T03:28:59.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.170+0000 7f00fb098740 1 -- 172.21.15.6:0/4146089112 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00e40035f0 con 0x557c0836eb40 2024-09-12T03:28:59.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.170+0000 7f00d9ffb640 1 -- 172.21.15.6:0/4146089112 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f00e402b070 con 0x557c0836eb40 2024-09-12T03:28:59.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.170+0000 7f00fb098740 10 client.16284 fetch_fsmap learned FSMap version 29 2024-09-12T03:28:59.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.170+0000 7f00fb098740 1 -- 172.21.15.6:0/4146089112 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557c08386b40 con 0x557c0836eb40 2024-09-12T03:28:59.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.170+0000 7f00d9ffb640 1 -- 172.21.15.6:0/4146089112 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e40035f0 con 0x557c0836eb40 2024-09-12T03:28:59.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.170+0000 7f00d9ffb640 1 -- 172.21.15.6:0/4146089112 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f00e40046c0 con 0x557c0836eb40 2024-09-12T03:28:59.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.170+0000 7f00fb098740 10 client.16284 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T03:28:59.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.170+0000 7f00fb098740 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:28:59.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.170+0000 7f00fb098740 20 client.16284 populate_metadata read hostname 'smithi006' 2024-09-12T03:28:59.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.170+0000 7f00fb098740 1 --2- 172.21.15.6:0/4146089112 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557c08387f40 0x557c0838a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.170+0000 7f00fb098740 4 client.16284 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:59.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.170+0000 7f00fb098740 1 -- 172.21.15.6:0/4146089112 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c082e0860 con 0x557c08387f40 2024-09-12T03:28:59.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.171+0000 7f00f1d9b640 1 --2- 172.21.15.6:0/4146089112 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557c08387f40 0x557c0838a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.171+0000 7f00d9ffb640 10 client.16284 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:59.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.171+0000 7f00f1d9b640 1 --2- 172.21.15.6:0/4146089112 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557c08387f40 0x557c0838a330 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d49a6c76fa27a9f7 server_cookie=4f8c26c3365a558f in_seq=0 out_seq=0 2024-09-12T03:28:59.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.172+0000 7f00d9ffb640 1 -- 172.21.15.6:0/4146089112 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c082e0860 con 0x557c08387f40 2024-09-12T03:28:59.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.172+0000 7f00d9ffb640 10 client.16284 handle_command_reply: tid=0 2024-09-12T03:28:59.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.172+0000 7f00fb098740 1 client.16284 shutdown 2024-09-12T03:28:59.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.172+0000 7f00fb098740 1 -- 172.21.15.6:0/4146089112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c0836eb40 msgr2=0x557c08375040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:59.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.172+0000 7f00fb098740 1 --2- 172.21.15.6:0/4146089112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c0836eb40 0x557c08375040 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f00e400fc30 tx=0x7f00e400b920 comp rx=0 tx=0).stop 2024-09-12T03:28:59.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.172+0000 7f00fb098740 1 -- 172.21.15.6:0/4146089112 shutdown_connections 2024-09-12T03:28:59.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.172+0000 7f00fb098740 1 --2- 172.21.15.6:0/4146089112 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557c08387f40 0x557c0838a330 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.172+0000 7f00fb098740 1 --2- 172.21.15.6:0/4146089112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c08378370 0x557c08375650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.172+0000 7f00fb098740 1 --2- 172.21.15.6:0/4146089112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c0836eb40 0x557c08375040 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.172+0000 7f00fb098740 1 --2- 172.21.15.6:0/4146089112 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557c0836d5d0 0x557c08377d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.172+0000 7f00fb098740 1 -- 172.21.15.6:0/4146089112 >> 172.21.15.6:0/4146089112 conn(0x557c08336310 msgr2=0x557c0836f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:59.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.172+0000 7f00d9ffb640 0 client.16284 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:59.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.172+0000 7f00fb098740 1 -- 172.21.15.6:0/4146089112 shutdown_connections 2024-09-12T03:28:59.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.172+0000 7f00fb098740 1 -- 172.21.15.6:0/4146089112 wait complete. 2024-09-12T03:28:59.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.172+0000 7f00fb098740 20 client.16284 trim_cache size 0 max 16384 2024-09-12T03:28:59.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.186+0000 7f00fb098740 1 Processor -- start 2024-09-12T03:28:59.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.186+0000 7f00fb098740 1 -- start start 2024-09-12T03:28:59.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.186+0000 7f00fb098740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0836d5d0 0x557c08384490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.186+0000 7f00fb098740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c0836eb40 0x557c0836cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.186+0000 7f00fb098740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557c08378370 0x557c08379760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.186+0000 7f00fb098740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f00f41afe80 con 0x557c0836eb40 2024-09-12T03:28:59.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.186+0000 7f00fb098740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f00f41ad6f0 con 0x557c08378370 2024-09-12T03:28:59.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.186+0000 7f00fb098740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f00f41ab540 con 0x557c0836d5d0 2024-09-12T03:28:59.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.186+0000 7f00f259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c0836eb40 0x557c0836cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.186+0000 7f00f259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c0836eb40 0x557c0836cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56208/0 (socket says 172.21.15.6:56208) 2024-09-12T03:28:59.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.186+0000 7f00f259c640 1 -- 172.21.15.6:0/2169861857 learned_addr learned my addr 172.21.15.6:0/2169861857 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:59.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.186+0000 7f00f1d9b640 1 --2- 172.21.15.6:0/2169861857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0836d5d0 0x557c08384490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.186+0000 7f00f1d9b640 1 --2- 172.21.15.6:0/2169861857 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557c08378370 0x557c08379760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.186+0000 7f00f259c640 1 -- 172.21.15.6:0/2169861857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0836d5d0 msgr2=0x557c08384490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:59.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.186+0000 7f00f259c640 1 --2- 172.21.15.6:0/2169861857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0836d5d0 0x557c08384490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.186+0000 7f00f259c640 1 -- 172.21.15.6:0/2169861857 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557c08378370 msgr2=0x557c08379760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:59.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.186+0000 7f00f259c640 1 --2- 172.21.15.6:0/2169861857 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557c08378370 0x557c08379760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.187+0000 7f00f259c640 1 -- 172.21.15.6:0/2169861857 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c0836a960 con 0x557c0836eb40 2024-09-12T03:28:59.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.187+0000 7f00f259c640 1 --2- 172.21.15.6:0/2169861857 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c0836eb40 0x557c0836cde0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f00e4004c80 tx=0x7f00e400b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ad91438181ddd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:59.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.187+0000 7f00dbfff640 1 -- 172.21.15.6:0/2169861857 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e4015040 con 0x557c0836eb40 2024-09-12T03:28:59.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.187+0000 7f00dbfff640 1 -- 172.21.15.6:0/2169861857 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e4035070 con 0x557c0836eb40 2024-09-12T03:28:59.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.187+0000 7f00dbfff640 1 -- 172.21.15.6:0/2169861857 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e40054b0 con 0x557c0836eb40 2024-09-12T03:28:59.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.187+0000 7f00fb098740 1 -- 172.21.15.6:0/2169861857 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c0836eb40 msgr2=0x557c0836cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:59.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.187+0000 7f00fb098740 1 --2- 172.21.15.6:0/2169861857 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c0836eb40 0x557c0836cde0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f00e4004c80 tx=0x7f00e400b3a0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.188+0000 7f00fb098740 1 -- 172.21.15.6:0/2169861857 shutdown_connections 2024-09-12T03:28:59.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.188+0000 7f00fb098740 1 --2- 172.21.15.6:0/2169861857 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557c08378370 0x557c08379760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.188+0000 7f00fb098740 1 --2- 172.21.15.6:0/2169861857 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c0836eb40 0x557c0836cde0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.188+0000 7f00fb098740 1 --2- 172.21.15.6:0/2169861857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0836d5d0 0x557c08384490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.188+0000 7f00fb098740 1 -- 172.21.15.6:0/2169861857 >> 172.21.15.6:0/2169861857 conn(0x557c08336310 msgr2=0x557c08373910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:59.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.188+0000 7f00fb098740 1 -- 172.21.15.6:0/2169861857 shutdown_connections 2024-09-12T03:28:59.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.188+0000 7f00fb098740 1 -- 172.21.15.6:0/2169861857 wait complete. 2024-09-12T03:28:59.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.188+0000 7f00fb098740 1 -- start start 2024-09-12T03:28:59.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.188+0000 7f00fb098740 1 Processor -- start 2024-09-12T03:28:59.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.188+0000 7f00fb098740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557c0836d5d0 0x557c0838da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.188+0000 7f00fb098740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0836eb40 0x557c08388de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.188+0000 7f00fb098740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c08378370 0x557c08389320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.188+0000 7f00fb098740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f00f41abbb0 con 0x557c08378370 2024-09-12T03:28:59.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.188+0000 7f00fb098740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f00f41ad020 con 0x557c0836d5d0 2024-09-12T03:28:59.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.188+0000 7f00fb098740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f00f41ac280 con 0x557c0836eb40 2024-09-12T03:28:59.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.188+0000 7f00fb098740 10 client.0 _ll_register_callbacks cb 0x7f00e8005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:59.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.188+0000 7f00f1d9b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557c0836d5d0 0x557c0838da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.188+0000 7f00f259c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0836eb40 0x557c08388de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.188+0000 7f00f1d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c08378370 0x557c08389320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.189+0000 7f00f1d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c08378370 0x557c08389320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56224/0 (socket says 172.21.15.6:56224) 2024-09-12T03:28:59.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.189+0000 7f00f1d9b640 1 -- 172.21.15.6:0/4292226265 learned_addr learned my addr 172.21.15.6:0/4292226265 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:28:59.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.189+0000 7f00f259c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0836eb40 0x557c08388de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:43148/0 (socket says 172.21.15.6:43148) 2024-09-12T03:28:59.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.189+0000 7f00f1d9b640 1 -- 172.21.15.6:0/4292226265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0836eb40 msgr2=0x557c08388de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:59.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.189+0000 7f00f1d9b640 1 --2- 172.21.15.6:0/4292226265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0836eb40 0x557c08388de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.189+0000 7f00f1d9b640 1 -- 172.21.15.6:0/4292226265 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557c0836d5d0 msgr2=0x557c0838da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:59.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.189+0000 7f00f1d9b640 1 --2- 172.21.15.6:0/4292226265 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557c0836d5d0 0x557c0838da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.189+0000 7f00f1d9b640 1 -- 172.21.15.6:0/4292226265 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c08369510 con 0x557c08378370 2024-09-12T03:28:59.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.189+0000 7f00f259c640 1 --2- 172.21.15.6:0/4292226265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0836eb40 0x557c08388de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:59.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.189+0000 7f00f1d9b640 1 --2- 172.21.15.6:0/4292226265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c08378370 0x557c08389320 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f00e8031bf0 tx=0x7f00e80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46cb7c5c97769774 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:59.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.189+0000 7f00d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-09-12T03:28:59.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.189+0000 7f00d9ffb640 1 -- 172.21.15.6:0/4292226265 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e8041070 con 0x557c08378370 2024-09-12T03:28:59.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.189+0000 7f00d9ffb640 1 -- 172.21.15.6:0/4292226265 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f00e804ce80 con 0x557c08378370 2024-09-12T03:28:59.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.189+0000 7f00fb098740 1 -- 172.21.15.6:0/4292226265 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00e4035070 con 0x557c08378370 2024-09-12T03:28:59.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.189+0000 7f00d9ffb640 1 -- 172.21.15.6:0/4292226265 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f00e8051070 con 0x557c08378370 2024-09-12T03:28:59.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.189+0000 7f00fb098740 10 client.16290 fetch_fsmap learned FSMap version 29 2024-09-12T03:28:59.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.189+0000 7f00fb098740 1 -- 172.21.15.6:0/4292226265 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557c083f26c0 con 0x557c08378370 2024-09-12T03:28:59.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.190+0000 7f00d9ffb640 1 -- 172.21.15.6:0/4292226265 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e4035070 con 0x557c08378370 2024-09-12T03:28:59.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.190+0000 7f00d9ffb640 1 -- 172.21.15.6:0/4292226265 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f00e804c880 con 0x557c08378370 2024-09-12T03:28:59.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.190+0000 7f00fb098740 10 client.16290 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T03:28:59.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.190+0000 7f00fb098740 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:28:59.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.190+0000 7f00fb098740 20 client.16290 populate_metadata read hostname 'smithi006' 2024-09-12T03:28:59.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.190+0000 7f00fb098740 1 --2- 172.21.15.6:0/4292226265 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557c083f2e10 0x557c083f5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.190+0000 7f00fb098740 4 client.16290 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:28:59.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.190+0000 7f00fb098740 1 -- 172.21.15.6:0/4292226265 --> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c082dad70 con 0x557c083f2e10 2024-09-12T03:28:59.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.190+0000 7f00f259c640 1 --2- 172.21.15.6:0/4292226265 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557c083f2e10 0x557c083f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.191+0000 7f00f259c640 1 --2- 172.21.15.6:0/4292226265 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557c083f2e10 0x557c083f5230 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7781591d3bff9ddb server_cookie=eac77bd4720b4be3 in_seq=0 out_seq=0 2024-09-12T03:28:59.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.191+0000 7f00d9ffb640 10 client.16290 ms_handle_connect on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:59.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.191+0000 7f00d9ffb640 1 -- 172.21.15.6:0/4292226265 <== mds.0 v2:172.21.15.118:6838/3035878957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c082dad70 con 0x557c083f2e10 2024-09-12T03:28:59.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.191+0000 7f00d9ffb640 10 client.16290 handle_command_reply: tid=0 2024-09-12T03:28:59.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.191+0000 7f00fb098740 1 client.16290 shutdown 2024-09-12T03:28:59.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.191+0000 7f00fb098740 1 -- 172.21.15.6:0/4292226265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c08378370 msgr2=0x557c08389320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:59.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.191+0000 7f00fb098740 1 --2- 172.21.15.6:0/4292226265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c08378370 0x557c08389320 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f00e8031bf0 tx=0x7f00e80058d0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.191+0000 7f00fb098740 1 -- 172.21.15.6:0/4292226265 shutdown_connections 2024-09-12T03:28:59.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.191+0000 7f00fb098740 1 --2- 172.21.15.6:0/4292226265 >> [v2:172.21.15.118:6838/3035878957,v1:172.21.15.118:6839/3035878957] conn(0x557c083f2e10 0x557c083f5230 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.191+0000 7f00fb098740 1 --2- 172.21.15.6:0/4292226265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557c08378370 0x557c08389320 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.191+0000 7f00fb098740 1 --2- 172.21.15.6:0/4292226265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0836eb40 0x557c08388de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.191+0000 7f00fb098740 1 --2- 172.21.15.6:0/4292226265 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557c0836d5d0 0x557c0838da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.191+0000 7f00fb098740 1 -- 172.21.15.6:0/4292226265 >> 172.21.15.6:0/4292226265 conn(0x557c08336310 msgr2=0x557c083770c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:59.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.191+0000 7f00d9ffb640 0 client.16290 ms_handle_reset on v2:172.21.15.118:6838/3035878957 2024-09-12T03:28:59.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.191+0000 7f00fb098740 1 -- 172.21.15.6:0/4292226265 shutdown_connections 2024-09-12T03:28:59.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.191+0000 7f00fb098740 1 -- 172.21.15.6:0/4292226265 wait complete. 2024-09-12T03:28:59.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.191+0000 7f00fb098740 20 client.16290 trim_cache size 0 max 16384 2024-09-12T03:28:59.198 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-09-12T03:28:59.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.192+0000 7f00f8cbf640 1 -- 172.21.15.6:0/2645095825 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f00dc0a2c00 msgr2=0x7f00dc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:59.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.192+0000 7f00f8cbf640 1 --2- 172.21.15.6:0/2645095825 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f00dc0a2c00 0x7f00dc0a50c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0cd040 tx=0x7f00ec0666b0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.192+0000 7f00f8cbf640 1 -- 172.21.15.6:0/2645095825 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f410c180 msgr2=0x7f00f419cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:59.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.192+0000 7f00f8cbf640 1 --2- 172.21.15.6:0/2645095825 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f410c180 0x7f00f419cd10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00eaa0 tx=0x7f00ec00ef60 comp rx=0 tx=0).stop 2024-09-12T03:28:59.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.192+0000 7f00f8cbf640 1 -- 172.21.15.6:0/2645095825 shutdown_connections 2024-09-12T03:28:59.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.192+0000 7f00f8cbf640 1 --2- 172.21.15.6:0/2645095825 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f00dc0a2c00 0x7f00dc0a50c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.192+0000 7f00f8cbf640 1 --2- 172.21.15.6:0/2645095825 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f410c180 0x7f00f419cd10 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.192+0000 7f00f8cbf640 1 --2- 172.21.15.6:0/2645095825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00f41036c0 0x7f00f4198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.192+0000 7f00f8cbf640 1 --2- 172.21.15.6:0/2645095825 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00f4102da0 0x7f00f4198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.192+0000 7f00f8cbf640 1 -- 172.21.15.6:0/2645095825 >> 172.21.15.6:0/2645095825 conn(0x7f00f40fc7a0 msgr2=0x7f00f40fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:59.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.192+0000 7f00f8cbf640 1 -- 172.21.15.6:0/2645095825 shutdown_connections 2024-09-12T03:28:59.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:28:59.192+0000 7f00f8cbf640 1 -- 172.21.15.6:0/2645095825 wait complete. 2024-09-12T03:28:59.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:28:58 smithi118 ceph-mon[35048]: pgmap v571: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:29:00.008 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-12T03:29:00.009 INFO:tasks.fwd_scrub:done joining 2024-09-12T03:29:00.009 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-12T03:29:00.019 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:29:00.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.125+0000 7fb1e1391640 1 -- 172.21.15.6:0/324908725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1dc1025c0 msgr2=0x7fb1dc108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.125+0000 7fb1e1391640 1 --2- 172.21.15.6:0/324908725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1dc1025c0 0x7fb1dc108e50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009ca0 tx=0x7fb1d402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.125+0000 7fb1e1391640 1 -- 172.21.15.6:0/324908725 shutdown_connections 2024-09-12T03:29:00.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.125+0000 7fb1e1391640 1 --2- 172.21.15.6:0/324908725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1dc1025c0 0x7fb1dc108e50 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.125+0000 7fb1e1391640 1 --2- 172.21.15.6:0/324908725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1dc101c00 0x7fb1dc102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.125+0000 7fb1e1391640 1 --2- 172.21.15.6:0/324908725 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1dc100a10 0x7fb1dc100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.125+0000 7fb1e1391640 1 -- 172.21.15.6:0/324908725 >> 172.21.15.6:0/324908725 conn(0x7fb1dc0fc170 msgr2=0x7fb1dc0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:00.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.125+0000 7fb1e1391640 1 -- 172.21.15.6:0/324908725 shutdown_connections 2024-09-12T03:29:00.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.125+0000 7fb1e1391640 1 -- 172.21.15.6:0/324908725 wait complete. 2024-09-12T03:29:00.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.126+0000 7fb1e1391640 1 Processor -- start 2024-09-12T03:29:00.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.126+0000 7fb1e1391640 1 -- start start 2024-09-12T03:29:00.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.126+0000 7fb1e1391640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1dc100a10 0x7fb1dc1049a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:00.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.126+0000 7fb1e1391640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1dc101c00 0x7fb1dc102fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:00.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.126+0000 7fb1e1391640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1dc1025c0 0x7fb1dc103520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:00.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.126+0000 7fb1e1391640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc10c690 con 0x7fb1dc101c00 2024-09-12T03:29:00.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.126+0000 7fb1e1391640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc10c510 con 0x7fb1dc1025c0 2024-09-12T03:29:00.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.126+0000 7fb1e1391640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc10c810 con 0x7fb1dc100a10 2024-09-12T03:29:00.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.126+0000 7fb1d3fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1dc101c00 0x7fb1dc102fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:00.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.126+0000 7fb1d3fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1dc101c00 0x7fb1dc102fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56244/0 (socket says 172.21.15.6:56244) 2024-09-12T03:29:00.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.126+0000 7fb1d3fff640 1 -- 172.21.15.6:0/2323571143 learned_addr learned my addr 172.21.15.6:0/2323571143 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:29:00.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.126+0000 7fb1daffd640 1 --2- 172.21.15.6:0/2323571143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1dc100a10 0x7fb1dc1049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:00.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.126+0000 7fb1db7fe640 1 --2- 172.21.15.6:0/2323571143 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1dc1025c0 0x7fb1dc103520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:00.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.126+0000 7fb1d3fff640 1 -- 172.21.15.6:0/2323571143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1dc100a10 msgr2=0x7fb1dc1049a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.126+0000 7fb1d3fff640 1 --2- 172.21.15.6:0/2323571143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1dc100a10 0x7fb1dc1049a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.126+0000 7fb1d3fff640 1 -- 172.21.15.6:0/2323571143 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1dc1025c0 msgr2=0x7fb1dc103520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.126+0000 7fb1d3fff640 1 --2- 172.21.15.6:0/2323571143 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1dc1025c0 0x7fb1dc103520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.126+0000 7fb1d3fff640 1 -- 172.21.15.6:0/2323571143 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1dc103db0 con 0x7fb1dc101c00 2024-09-12T03:29:00.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.127+0000 7fb1d3fff640 1 --2- 172.21.15.6:0/2323571143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1dc101c00 0x7fb1dc102fe0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400d770 tx=0x7fb1c400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:00.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.127+0000 7fb1d8ff9640 1 -- 172.21.15.6:0/2323571143 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4017070 con 0x7fb1dc101c00 2024-09-12T03:29:00.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.127+0000 7fb1d8ff9640 1 -- 172.21.15.6:0/2323571143 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1c4005500 con 0x7fb1dc101c00 2024-09-12T03:29:00.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.127+0000 7fb1d8ff9640 1 -- 172.21.15.6:0/2323571143 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4004e00 con 0x7fb1dc101c00 2024-09-12T03:29:00.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.127+0000 7fb1e1391640 1 -- 172.21.15.6:0/2323571143 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1dc1a1f30 con 0x7fb1dc101c00 2024-09-12T03:29:00.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.127+0000 7fb1e1391640 1 -- 172.21.15.6:0/2323571143 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1dc1a2390 con 0x7fb1dc101c00 2024-09-12T03:29:00.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.128+0000 7fb1e1391640 1 -- 172.21.15.6:0/2323571143 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1dc1a2680 con 0x7fb1dc101c00 2024-09-12T03:29:00.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.129+0000 7fb1d8ff9640 1 -- 172.21.15.6:0/2323571143 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb1c4005000 con 0x7fb1dc101c00 2024-09-12T03:29:00.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.130+0000 7fb1d8ff9640 1 --2- 172.21.15.6:0/2323571143 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb1c80a2c10 0x7fb1c80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:00.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.130+0000 7fb1daffd640 1 --2- 172.21.15.6:0/2323571143 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb1c80a2c10 0x7fb1c80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:00.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.130+0000 7fb1d8ff9640 1 -- 172.21.15.6:0/2323571143 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fb1c40cea10 con 0x7fb1dc101c00 2024-09-12T03:29:00.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.130+0000 7fb1daffd640 1 --2- 172.21.15.6:0/2323571143 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb1c80a2c10 0x7fb1c80a50d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc005fd0 tx=0x7fb1cc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:00.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.134+0000 7fb1d8ff9640 1 -- 172.21.15.6:0/2323571143 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb1c409b9a0 con 0x7fb1dc101c00 2024-09-12T03:29:00.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.311+0000 7fb1e1391640 1 -- 172.21.15.6:0/2323571143 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1dc1a2f60 con 0x7fb1dc101c00 2024-09-12T03:29:00.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.312+0000 7fb1d8ff9640 1 -- 172.21.15.6:0/2323571143 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22607 (secure 0 0 0) 0x7fb1c409b2a0 con 0x7fb1dc101c00 2024-09-12T03:29:00.313 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:29:00.313 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":82,"fsid":"89d99938-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:11:29.530545+0000","modified":"2024-09-12T03:19:20.366710+0000","last_up_change":"2024-09-12T03:18:21.292572+0000","last_in_change":"2024-09-12T03:17:58.013800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:14:37.662010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:06.143398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:08.305279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f55d68b-95a0-4ee2-a46f-bc422b38471e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6803","nonce":726729286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6805","nonce":726729286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6809","nonce":726729286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6807","nonce":726729286}]},"public_addr":"172.21.15.6:6803/726729286","cluster_addr":"172.21.15.6:6805/726729286","heartbeat_back_addr":"172.21.15.6:6809/726729286","heartbeat_front_addr":"172.21.15.6:6807/726729286","state":["exists","up"]},{"osd":1,"uuid":"83afdb01-c731-4de1-96c6-3f798549b1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6801","nonce":3377555415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6803","nonce":3377555415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6807","nonce":3377555415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6805","nonce":3377555415}]},"public_addr":"172.21.15.118:6801/3377555415","cluster_addr":"172.21.15.118:6803/3377555415","heartbeat_back_addr":"172.21.15.118:6807/3377555415","heartbeat_front_addr":"172.21.15.118:6805/3377555415","state":["exists","up"]},{"osd":2,"uuid":"aed718b4-b849-403a-b81c-3a140b5c380c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6801","nonce":392541009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6803","nonce":392541009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6807","nonce":392541009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6805","nonce":392541009}]},"public_addr":"172.21.15.161:6801/392541009","cluster_addr":"172.21.15.161:6803/392541009","heartbeat_back_addr":"172.21.15.161:6807/392541009","heartbeat_front_addr":"172.21.15.161:6805/392541009","state":["exists","up"]},{"osd":3,"uuid":"25ced4c1-bf31-402d-8397-d42fd57f615f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6811","nonce":1892039151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6813","nonce":1892039151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6817","nonce":1892039151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6815","nonce":1892039151}]},"public_addr":"172.21.15.6:6811/1892039151","cluster_addr":"172.21.15.6:6813/1892039151","heartbeat_back_addr":"172.21.15.6:6817/1892039151","heartbeat_front_addr":"172.21.15.6:6815/1892039151","state":["exists","up"]},{"osd":4,"uuid":"1d884604-920b-4c35-b41e-d629ad9f9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6809","nonce":150177024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6811","nonce":150177024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6815","nonce":150177024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6813","nonce":150177024}]},"public_addr":"172.21.15.118:6809/150177024","cluster_addr":"172.21.15.118:6811/150177024","heartbeat_back_addr":"172.21.15.118:6815/150177024","heartbeat_front_addr":"172.21.15.118:6813/150177024","state":["exists","up"]},{"osd":5,"uuid":"1d40c0ef-99f6-426f-90c7-27579edc66af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6809","nonce":3758167854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6811","nonce":3758167854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6815","nonce":3758167854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6813","nonce":3758167854}]},"public_addr":"172.21.15.161:6809/3758167854","cluster_addr":"172.21.15.161:6811/3758167854","heartbeat_back_addr":"172.21.15.161:6815/3758167854","heartbeat_front_addr":"172.21.15.161:6813/3758167854","state":["exists","up"]},{"osd":6,"uuid":"9d9f9510-b107-40c2-9a98-9012715944b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6819","nonce":2129224874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6821","nonce":2129224874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6825","nonce":2129224874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6823","nonce":2129224874}]},"public_addr":"172.21.15.6:6819/2129224874","cluster_addr":"172.21.15.6:6821/2129224874","heartbeat_back_addr":"172.21.15.6:6825/2129224874","heartbeat_front_addr":"172.21.15.6:6823/2129224874","state":["exists","up"]},{"osd":7,"uuid":"84635ae1-3240-4954-af14-76bb3aa162e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6817","nonce":269980353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6819","nonce":269980353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6823","nonce":269980353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6821","nonce":269980353}]},"public_addr":"172.21.15.118:6817/269980353","cluster_addr":"172.21.15.118:6819/269980353","heartbeat_back_addr":"172.21.15.118:6823/269980353","heartbeat_front_addr":"172.21.15.118:6821/269980353","state":["exists","up"]},{"osd":8,"uuid":"6fd58e55-1492-4595-8647-823ea3e3c5d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6817","nonce":1783416134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6819","nonce":1783416134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6823","nonce":1783416134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6821","nonce":1783416134}]},"public_addr":"172.21.15.161:6817/1783416134","cluster_addr":"172.21.15.161:6819/1783416134","heartbeat_back_addr":"172.21.15.161:6823/1783416134","heartbeat_front_addr":"172.21.15.161:6821/1783416134","state":["exists","up"]},{"osd":9,"uuid":"90508a6d-a49c-43c6-a600-5bbccd8702ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6827","nonce":2987332365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6829","nonce":2987332365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6833","nonce":2987332365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6831","nonce":2987332365}]},"public_addr":"172.21.15.6:6827/2987332365","cluster_addr":"172.21.15.6:6829/2987332365","heartbeat_back_addr":"172.21.15.6:6833/2987332365","heartbeat_front_addr":"172.21.15.6:6831/2987332365","state":["exists","up"]},{"osd":10,"uuid":"911e86d1-669a-4935-82c0-7da51d141df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6825","nonce":2167146341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6827","nonce":2167146341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6831","nonce":2167146341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6829","nonce":2167146341}]},"public_addr":"172.21.15.118:6825/2167146341","cluster_addr":"172.21.15.118:6827/2167146341","heartbeat_back_addr":"172.21.15.118:6831/2167146341","heartbeat_front_addr":"172.21.15.118:6829/2167146341","state":["exists","up"]},{"osd":11,"uuid":"5a15c6a7-ff43-473c-bb48-0c129c316df5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6825","nonce":2889795796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6827","nonce":2889795796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6831","nonce":2889795796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6829","nonce":2889795796}]},"public_addr":"172.21.15.161:6825/2889795796","cluster_addr":"172.21.15.161:6827/2889795796","heartbeat_back_addr":"172.21.15.161:6831/2889795796","heartbeat_front_addr":"172.21.15.161:6829/2889795796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:42.912205+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:08.410669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:34.299134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:59.093181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:24.116635+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:49.566433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:15.719320+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:40.541017+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:05.178178+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:31.163877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:55.405110+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:19.417156+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6800/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/2620491916":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/4037656392":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/1377116573":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/3062800125":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:6800/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/4145338968":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/825701128":"2024-09-13T03:11:55.244257+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:29:00.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.314+0000 7fb1e1391640 1 -- 172.21.15.6:0/2323571143 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb1c80a2c10 msgr2=0x7fb1c80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.314+0000 7fb1e1391640 1 --2- 172.21.15.6:0/2323571143 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb1c80a2c10 0x7fb1c80a50d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc005fd0 tx=0x7fb1cc005850 comp rx=0 tx=0).stop 2024-09-12T03:29:00.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.314+0000 7fb1e1391640 1 -- 172.21.15.6:0/2323571143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1dc101c00 msgr2=0x7fb1dc102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.314+0000 7fb1e1391640 1 --2- 172.21.15.6:0/2323571143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1dc101c00 0x7fb1dc102fe0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400d770 tx=0x7fb1c400dc30 comp rx=0 tx=0).stop 2024-09-12T03:29:00.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.314+0000 7fb1e1391640 1 -- 172.21.15.6:0/2323571143 shutdown_connections 2024-09-12T03:29:00.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.314+0000 7fb1e1391640 1 --2- 172.21.15.6:0/2323571143 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7fb1c80a2c10 0x7fb1c80a50d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.314+0000 7fb1e1391640 1 --2- 172.21.15.6:0/2323571143 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1dc1025c0 0x7fb1dc103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.314+0000 7fb1e1391640 1 --2- 172.21.15.6:0/2323571143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1dc101c00 0x7fb1dc102fe0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.314+0000 7fb1e1391640 1 --2- 172.21.15.6:0/2323571143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1dc100a10 0x7fb1dc1049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.314+0000 7fb1e1391640 1 -- 172.21.15.6:0/2323571143 >> 172.21.15.6:0/2323571143 conn(0x7fb1dc0fc170 msgr2=0x7fb1dc0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:00.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.314+0000 7fb1e1391640 1 -- 172.21.15.6:0/2323571143 shutdown_connections 2024-09-12T03:29:00.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.314+0000 7fb1e1391640 1 -- 172.21.15.6:0/2323571143 wait complete. 2024-09-12T03:29:00.326 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T03:14:37.662010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T03:19:06.143398+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T03:19:08.305279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:29:00.326 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:29:00.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.433+0000 7f49c8be8640 1 -- 172.21.15.6:0/4146917172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49c4100a00 msgr2=0x7f49c4100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.433+0000 7f49c8be8640 1 --2- 172.21.15.6:0/4146917172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49c4100a00 0x7f49c4100e00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009900 tx=0x7f49b402a410 comp rx=0 tx=0).stop 2024-09-12T03:29:00.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.433+0000 7f49c8be8640 1 -- 172.21.15.6:0/4146917172 shutdown_connections 2024-09-12T03:29:00.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.433+0000 7f49c8be8640 1 --2- 172.21.15.6:0/4146917172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c41025b0 0x7f49c4108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.433+0000 7f49c8be8640 1 --2- 172.21.15.6:0/4146917172 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49c4101bf0 0x7f49c4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.433+0000 7f49c8be8640 1 --2- 172.21.15.6:0/4146917172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49c4100a00 0x7f49c4100e00 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.433+0000 7f49c8be8640 1 -- 172.21.15.6:0/4146917172 >> 172.21.15.6:0/4146917172 conn(0x7f49c40fc140 msgr2=0x7f49c40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:00.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.433+0000 7f49c8be8640 1 -- 172.21.15.6:0/4146917172 shutdown_connections 2024-09-12T03:29:00.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.433+0000 7f49c8be8640 1 -- 172.21.15.6:0/4146917172 wait complete. 2024-09-12T03:29:00.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.434+0000 7f49c8be8640 1 Processor -- start 2024-09-12T03:29:00.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.434+0000 7f49c8be8640 1 -- start start 2024-09-12T03:29:00.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.434+0000 7f49c8be8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49c4100a00 0x7f49c4069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:00.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.434+0000 7f49c8be8640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49c4101bf0 0x7f49c419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:00.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.434+0000 7f49c8be8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c41025b0 0x7f49c419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:00.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.434+0000 7f49c8be8640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f49c410c6b0 con 0x7f49c4100a00 2024-09-12T03:29:00.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.434+0000 7f49c8be8640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f49c410c530 con 0x7f49c4101bf0 2024-09-12T03:29:00.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.434+0000 7f49c8be8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f49c410c830 con 0x7f49c41025b0 2024-09-12T03:29:00.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.434+0000 7f49c259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49c4100a00 0x7f49c4069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:00.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.434+0000 7f49c259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49c4100a00 0x7f49c4069ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56258/0 (socket says 172.21.15.6:56258) 2024-09-12T03:29:00.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.434+0000 7f49c259c640 1 -- 172.21.15.6:0/3812837615 learned_addr learned my addr 172.21.15.6:0/3812837615 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:29:00.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.435+0000 7f49c1d9b640 1 --2- 172.21.15.6:0/3812837615 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49c4101bf0 0x7f49c419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:00.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.435+0000 7f49c2d9d640 1 --2- 172.21.15.6:0/3812837615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c41025b0 0x7f49c419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:00.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.435+0000 7f49c259c640 1 -- 172.21.15.6:0/3812837615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c41025b0 msgr2=0x7f49c419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.435+0000 7f49c259c640 1 --2- 172.21.15.6:0/3812837615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c41025b0 0x7f49c419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.435+0000 7f49c259c640 1 -- 172.21.15.6:0/3812837615 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49c4101bf0 msgr2=0x7f49c419cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.435+0000 7f49c259c640 1 --2- 172.21.15.6:0/3812837615 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49c4101bf0 0x7f49c419cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.435+0000 7f49c259c640 1 -- 172.21.15.6:0/3812837615 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c419fa40 con 0x7f49c4100a00 2024-09-12T03:29:00.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.435+0000 7f49c259c640 1 --2- 172.21.15.6:0/3812837615 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49c4100a00 0x7f49c4069ef0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f49b4031440 tx=0x7f49b4032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:00.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.435+0000 7f49b37fe640 1 -- 172.21.15.6:0/3812837615 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b403f030 con 0x7f49c4100a00 2024-09-12T03:29:00.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.435+0000 7f49b37fe640 1 -- 172.21.15.6:0/3812837615 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49b4003ce0 con 0x7f49c4100a00 2024-09-12T03:29:00.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.435+0000 7f49c8be8640 1 -- 172.21.15.6:0/3812837615 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c419fc70 con 0x7f49c4100a00 2024-09-12T03:29:00.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.435+0000 7f49b37fe640 1 -- 172.21.15.6:0/3812837615 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b4035070 con 0x7f49c4100a00 2024-09-12T03:29:00.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.435+0000 7f49c8be8640 1 -- 172.21.15.6:0/3812837615 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c41a01b0 con 0x7f49c4100a00 2024-09-12T03:29:00.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.436+0000 7f49c8be8640 1 -- 172.21.15.6:0/3812837615 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c4107df0 con 0x7f49c4100a00 2024-09-12T03:29:00.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.438+0000 7f49b37fe640 1 -- 172.21.15.6:0/3812837615 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f49b4042030 con 0x7f49c4100a00 2024-09-12T03:29:00.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.439+0000 7f49b37fe640 1 --2- 172.21.15.6:0/3812837615 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f49ac0a2c10 0x7f49ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:00.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.439+0000 7f49c1d9b640 1 --2- 172.21.15.6:0/3812837615 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f49ac0a2c10 0x7f49ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:00.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.439+0000 7f49b37fe640 1 -- 172.21.15.6:0/3812837615 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f49b40efaa0 con 0x7f49c4100a00 2024-09-12T03:29:00.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.439+0000 7f49c1d9b640 1 --2- 172.21.15.6:0/3812837615 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f49ac0a2c10 0x7f49ac0a50d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f49b8005e10 tx=0x7f49b8005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:00.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.442+0000 7f49b37fe640 1 -- 172.21.15.6:0/3812837615 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f49b40bca30 con 0x7f49c4100a00 2024-09-12T03:29:00.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.623+0000 7f49c8be8640 1 -- 172.21.15.6:0/3812837615 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f49c402da10 con 0x7f49c4100a00 2024-09-12T03:29:00.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.623+0000 7f49b37fe640 1 -- 172.21.15.6:0/3812837615 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f49b40bc330 con 0x7f49c4100a00 2024-09-12T03:29:00.623 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-09-12T03:29:00.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.625+0000 7f49c8be8640 1 -- 172.21.15.6:0/3812837615 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f49ac0a2c10 msgr2=0x7f49ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.625+0000 7f49c8be8640 1 --2- 172.21.15.6:0/3812837615 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f49ac0a2c10 0x7f49ac0a50d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f49b8005e10 tx=0x7f49b8005da0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.625+0000 7f49c8be8640 1 -- 172.21.15.6:0/3812837615 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49c4100a00 msgr2=0x7f49c4069ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.625+0000 7f49c8be8640 1 --2- 172.21.15.6:0/3812837615 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49c4100a00 0x7f49c4069ef0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f49b4031440 tx=0x7f49b4032e20 comp rx=0 tx=0).stop 2024-09-12T03:29:00.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.625+0000 7f49c8be8640 1 -- 172.21.15.6:0/3812837615 shutdown_connections 2024-09-12T03:29:00.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.625+0000 7f49c8be8640 1 --2- 172.21.15.6:0/3812837615 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f49ac0a2c10 0x7f49ac0a50d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.625+0000 7f49c8be8640 1 --2- 172.21.15.6:0/3812837615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c41025b0 0x7f49c419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.625+0000 7f49c8be8640 1 --2- 172.21.15.6:0/3812837615 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49c4101bf0 0x7f49c419cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.625+0000 7f49c8be8640 1 --2- 172.21.15.6:0/3812837615 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49c4100a00 0x7f49c4069ef0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.625+0000 7f49c8be8640 1 -- 172.21.15.6:0/3812837615 >> 172.21.15.6:0/3812837615 conn(0x7f49c40fc140 msgr2=0x7f49c4104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:00.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.625+0000 7f49c8be8640 1 -- 172.21.15.6:0/3812837615 shutdown_connections 2024-09-12T03:29:00.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.625+0000 7f49c8be8640 1 -- 172.21.15.6:0/3812837615 wait complete. 2024-09-12T03:29:00.636 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T03:29:00.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.742+0000 7f70fd4c9640 1 -- 172.21.15.6:0/2542970519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70f8100a00 msgr2=0x7f70f8100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.742+0000 7f70fd4c9640 1 --2- 172.21.15.6:0/2542970519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70f8100a00 0x7f70f8100e00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009900 tx=0x7f70e802a410 comp rx=0 tx=0).stop 2024-09-12T03:29:00.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.742+0000 7f70fd4c9640 1 -- 172.21.15.6:0/2542970519 shutdown_connections 2024-09-12T03:29:00.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.742+0000 7f70fd4c9640 1 --2- 172.21.15.6:0/2542970519 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f70f81025b0 0x7f70f8108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.742+0000 7f70fd4c9640 1 --2- 172.21.15.6:0/2542970519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f8101bf0 0x7f70f8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.742+0000 7f70fd4c9640 1 --2- 172.21.15.6:0/2542970519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70f8100a00 0x7f70f8100e00 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.742+0000 7f70fd4c9640 1 -- 172.21.15.6:0/2542970519 >> 172.21.15.6:0/2542970519 conn(0x7f70f80fc140 msgr2=0x7f70f80fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:00.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.742+0000 7f70fd4c9640 1 -- 172.21.15.6:0/2542970519 shutdown_connections 2024-09-12T03:29:00.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.742+0000 7f70fd4c9640 1 -- 172.21.15.6:0/2542970519 wait complete. 2024-09-12T03:29:00.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.743+0000 7f70fd4c9640 1 Processor -- start 2024-09-12T03:29:00.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.743+0000 7f70fd4c9640 1 -- start start 2024-09-12T03:29:00.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.743+0000 7f70fd4c9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f8100a00 0x7f70f8197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:00.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.743+0000 7f70fd4c9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70f8101bf0 0x7f70f8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:00.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.743+0000 7f70fd4c9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f70f81025b0 0x7f70f819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:00.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.743+0000 7f70fd4c9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f70f810d7d0 con 0x7f70f8101bf0 2024-09-12T03:29:00.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.743+0000 7f70fd4c9640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f70f810d650 con 0x7f70f81025b0 2024-09-12T03:29:00.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.743+0000 7f70fd4c9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f70f810d950 con 0x7f70f8100a00 2024-09-12T03:29:00.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.743+0000 7f70f67fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70f8101bf0 0x7f70f8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:00.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.743+0000 7f70f6ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f8100a00 0x7f70f8197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:00.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.743+0000 7f70f67fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70f8101bf0 0x7f70f8198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56272/0 (socket says 172.21.15.6:56272) 2024-09-12T03:29:00.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.743+0000 7f70f6ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f8100a00 0x7f70f8197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:43208/0 (socket says 172.21.15.6:43208) 2024-09-12T03:29:00.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.743+0000 7f70f67fc640 1 -- 172.21.15.6:0/685143016 learned_addr learned my addr 172.21.15.6:0/685143016 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:29:00.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.743+0000 7f70f77fe640 1 --2- 172.21.15.6:0/685143016 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f70f81025b0 0x7f70f819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:00.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.743+0000 7f70f67fc640 1 -- 172.21.15.6:0/685143016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f8100a00 msgr2=0x7f70f8197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.743+0000 7f70f67fc640 1 --2- 172.21.15.6:0/685143016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f8100a00 0x7f70f8197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.743+0000 7f70f67fc640 1 -- 172.21.15.6:0/685143016 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f70f81025b0 msgr2=0x7f70f819f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.743+0000 7f70f67fc640 1 --2- 172.21.15.6:0/685143016 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f70f81025b0 0x7f70f819f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.744+0000 7f70f67fc640 1 -- 172.21.15.6:0/685143016 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f819fa60 con 0x7f70f8101bf0 2024-09-12T03:29:00.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.744+0000 7f70f77fe640 1 --2- 172.21.15.6:0/685143016 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f70f81025b0 0x7f70f819f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:29:00.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.744+0000 7f70f67fc640 1 --2- 172.21.15.6:0/685143016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70f8101bf0 0x7f70f8198380 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00e9f0 tx=0x7f70ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:00.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.744+0000 7f70dffff640 1 -- 172.21.15.6:0/685143016 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec00cd80 con 0x7f70f8101bf0 2024-09-12T03:29:00.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.744+0000 7f70dffff640 1 -- 172.21.15.6:0/685143016 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70ec005500 con 0x7f70f8101bf0 2024-09-12T03:29:00.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.744+0000 7f70fd4c9640 1 -- 172.21.15.6:0/685143016 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f819fd50 con 0x7f70f8101bf0 2024-09-12T03:29:00.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.744+0000 7f70dffff640 1 -- 172.21.15.6:0/685143016 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec010690 con 0x7f70f8101bf0 2024-09-12T03:29:00.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.744+0000 7f70fd4c9640 1 -- 172.21.15.6:0/685143016 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f81a01f0 con 0x7f70f8101bf0 2024-09-12T03:29:00.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.745+0000 7f70fd4c9640 1 -- 172.21.15.6:0/685143016 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70f8107e40 con 0x7f70f8101bf0 2024-09-12T03:29:00.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.746+0000 7f70dffff640 1 -- 172.21.15.6:0/685143016 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f70ec0046c0 con 0x7f70f8101bf0 2024-09-12T03:29:00.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.747+0000 7f70dffff640 1 --2- 172.21.15.6:0/685143016 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f70e00a2b90 0x7f70e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:00.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.747+0000 7f70f6ffd640 1 --2- 172.21.15.6:0/685143016 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f70e00a2b90 0x7f70e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:00.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.747+0000 7f70dffff640 1 -- 172.21.15.6:0/685143016 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f70ec0cf670 con 0x7f70f8101bf0 2024-09-12T03:29:00.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.747+0000 7f70f6ffd640 1 --2- 172.21.15.6:0/685143016 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f70e00a2b90 0x7f70e00a5050 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f70e8005ab0 tx=0x7f70e8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:00.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.750+0000 7f70dffff640 1 -- 172.21.15.6:0/685143016 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70ec0541b0 con 0x7f70f8101bf0 2024-09-12T03:29:00.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.926+0000 7f70fd4c9640 1 -- 172.21.15.6:0/685143016 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f70f802da10 con 0x7f70f8101bf0 2024-09-12T03:29:00.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.927+0000 7f70dffff640 1 -- 172.21.15.6:0/685143016 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7f70ec09c600 con 0x7f70f8101bf0 2024-09-12T03:29:00.927 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2024-09-12T03:29:00.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.929+0000 7f70fd4c9640 1 -- 172.21.15.6:0/685143016 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f70e00a2b90 msgr2=0x7f70e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.929+0000 7f70fd4c9640 1 --2- 172.21.15.6:0/685143016 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f70e00a2b90 0x7f70e00a5050 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f70e8005ab0 tx=0x7f70e8000bc0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.929+0000 7f70fd4c9640 1 -- 172.21.15.6:0/685143016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70f8101bf0 msgr2=0x7f70f8198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.929+0000 7f70fd4c9640 1 --2- 172.21.15.6:0/685143016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70f8101bf0 0x7f70f8198380 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00e9f0 tx=0x7f70ec00edf0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.929+0000 7f70fd4c9640 1 -- 172.21.15.6:0/685143016 shutdown_connections 2024-09-12T03:29:00.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.929+0000 7f70fd4c9640 1 --2- 172.21.15.6:0/685143016 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f70e00a2b90 0x7f70e00a5050 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.929+0000 7f70fd4c9640 1 --2- 172.21.15.6:0/685143016 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f70f81025b0 0x7f70f819f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.929+0000 7f70fd4c9640 1 --2- 172.21.15.6:0/685143016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70f8101bf0 0x7f70f8198380 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.929+0000 7f70fd4c9640 1 --2- 172.21.15.6:0/685143016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f8100a00 0x7f70f8197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.929+0000 7f70fd4c9640 1 -- 172.21.15.6:0/685143016 >> 172.21.15.6:0/685143016 conn(0x7f70f80fc140 msgr2=0x7f70f8105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:00.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.929+0000 7f70fd4c9640 1 -- 172.21.15.6:0/685143016 shutdown_connections 2024-09-12T03:29:00.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:00.929+0000 7f70fd4c9640 1 -- 172.21.15.6:0/685143016 wait complete. 2024-09-12T03:29:00.939 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:00 smithi006 ceph-mon[30991]: pgmap v572: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:29:00.939 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:00 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2323571143' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:29:00.939 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:00 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3812837615' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:29:00.941 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T03:29:01.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.047+0000 7f8109f86640 1 -- 172.21.15.6:0/489304694 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81041025b0 msgr2=0x7f8104108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:01.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.047+0000 7f8109f86640 1 --2- 172.21.15.6:0/489304694 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81041025b0 0x7f8104108e40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f80fc009ca0 tx=0x7f80fc02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:29:01.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.047+0000 7f8109f86640 1 -- 172.21.15.6:0/489304694 shutdown_connections 2024-09-12T03:29:01.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.047+0000 7f8109f86640 1 --2- 172.21.15.6:0/489304694 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81041025b0 0x7f8104108e40 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:01.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.047+0000 7f8109f86640 1 --2- 172.21.15.6:0/489304694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8104101bf0 0x7f8104102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:01.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.047+0000 7f8109f86640 1 --2- 172.21.15.6:0/489304694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8104100a00 0x7f8104100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:01.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.047+0000 7f8109f86640 1 -- 172.21.15.6:0/489304694 >> 172.21.15.6:0/489304694 conn(0x7f81040fc140 msgr2=0x7f81040fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:01.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.047+0000 7f8109f86640 1 -- 172.21.15.6:0/489304694 shutdown_connections 2024-09-12T03:29:01.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.047+0000 7f8109f86640 1 -- 172.21.15.6:0/489304694 wait complete. 2024-09-12T03:29:01.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.048+0000 7f8109f86640 1 Processor -- start 2024-09-12T03:29:01.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.048+0000 7f8109f86640 1 -- start start 2024-09-12T03:29:01.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.048+0000 7f8109f86640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8104100a00 0x7f8104069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:01.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.048+0000 7f8109f86640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8104101bf0 0x7f810419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:01.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.048+0000 7f8109f86640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81041025b0 0x7f810419f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:01.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.048+0000 7f8109f86640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f810410c660 con 0x7f8104100a00 2024-09-12T03:29:01.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.048+0000 7f8109f86640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f810410c4e0 con 0x7f8104101bf0 2024-09-12T03:29:01.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.048+0000 7f8109f86640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f810410c7e0 con 0x7f81041025b0 2024-09-12T03:29:01.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.048+0000 7f81037fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8104100a00 0x7f8104069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:01.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.048+0000 7f81037fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8104100a00 0x7f8104069ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56300/0 (socket says 172.21.15.6:56300) 2024-09-12T03:29:01.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.048+0000 7f81037fe640 1 -- 172.21.15.6:0/3114284010 learned_addr learned my addr 172.21.15.6:0/3114284010 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:29:01.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.048+0000 7f8103fff640 1 --2- 172.21.15.6:0/3114284010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81041025b0 0x7f810419f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:01.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.048+0000 7f8102ffd640 1 --2- 172.21.15.6:0/3114284010 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8104101bf0 0x7f810419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:01.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.049+0000 7f81037fe640 1 -- 172.21.15.6:0/3114284010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81041025b0 msgr2=0x7f810419f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:01.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.049+0000 7f81037fe640 1 --2- 172.21.15.6:0/3114284010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81041025b0 0x7f810419f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:01.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.049+0000 7f81037fe640 1 -- 172.21.15.6:0/3114284010 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8104101bf0 msgr2=0x7f810419cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:01.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.049+0000 7f81037fe640 1 --2- 172.21.15.6:0/3114284010 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8104101bf0 0x7f810419cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:01.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.049+0000 7f81037fe640 1 -- 172.21.15.6:0/3114284010 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810419fa90 con 0x7f8104100a00 2024-09-12T03:29:01.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.049+0000 7f81037fe640 1 --2- 172.21.15.6:0/3114284010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8104100a00 0x7f8104069ef0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f80f400e9c0 tx=0x7f80f400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:01.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.049+0000 7f8100ff9640 1 -- 172.21.15.6:0/3114284010 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f400cd80 con 0x7f8104100a00 2024-09-12T03:29:01.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.049+0000 7f8109f86640 1 -- 172.21.15.6:0/3114284010 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810419fd20 con 0x7f8104100a00 2024-09-12T03:29:01.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.049+0000 7f8100ff9640 1 -- 172.21.15.6:0/3114284010 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f40035f0 con 0x7f8104100a00 2024-09-12T03:29:01.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.049+0000 7f8100ff9640 1 -- 172.21.15.6:0/3114284010 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4010690 con 0x7f8104100a00 2024-09-12T03:29:01.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.049+0000 7f8109f86640 1 -- 172.21.15.6:0/3114284010 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81041a01c0 con 0x7f8104100a00 2024-09-12T03:29:01.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.050+0000 7f8109f86640 1 -- 172.21.15.6:0/3114284010 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8104107df0 con 0x7f8104100a00 2024-09-12T03:29:01.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.051+0000 7f8100ff9640 1 -- 172.21.15.6:0/3114284010 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f80f40037a0 con 0x7f8104100a00 2024-09-12T03:29:01.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.052+0000 7f8100ff9640 1 --2- 172.21.15.6:0/3114284010 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f80ec0a2b90 0x7f80ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:01.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.052+0000 7f8102ffd640 1 --2- 172.21.15.6:0/3114284010 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f80ec0a2b90 0x7f80ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:01.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.052+0000 7f8100ff9640 1 -- 172.21.15.6:0/3114284010 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f80f40cf450 con 0x7f8104100a00 2024-09-12T03:29:01.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.053+0000 7f8102ffd640 1 --2- 172.21.15.6:0/3114284010 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f80ec0a2b90 0x7f80ec0a5050 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8104069c60 tx=0x7f80f800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:01.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.056+0000 7f8100ff9640 1 -- 172.21.15.6:0/3114284010 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f80f409c380 con 0x7f8104100a00 2024-09-12T03:29:01.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:00 smithi161 ceph-mon[35485]: pgmap v572: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:29:01.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:00 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2323571143' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:29:01.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:00 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3812837615' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:29:01.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:00 smithi118 ceph-mon[35048]: pgmap v572: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:29:01.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:00 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2323571143' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:29:01.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:00 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3812837615' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:29:01.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.231+0000 7f8109f86640 1 -- 172.21.15.6:0/3114284010 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8104062f30 con 0x7f8104100a00 2024-09-12T03:29:01.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.231+0000 7f8100ff9640 1 -- 172.21.15.6:0/3114284010 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7f80f409bc80 con 0x7f8104100a00 2024-09-12T03:29:01.232 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2024-09-12T03:29:01.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.233+0000 7f8109f86640 1 -- 172.21.15.6:0/3114284010 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f80ec0a2b90 msgr2=0x7f80ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:01.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.233+0000 7f8109f86640 1 --2- 172.21.15.6:0/3114284010 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f80ec0a2b90 0x7f80ec0a5050 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8104069c60 tx=0x7f80f800b390 comp rx=0 tx=0).stop 2024-09-12T03:29:01.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.233+0000 7f8109f86640 1 -- 172.21.15.6:0/3114284010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8104100a00 msgr2=0x7f8104069ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:01.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.233+0000 7f8109f86640 1 --2- 172.21.15.6:0/3114284010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8104100a00 0x7f8104069ef0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f80f400e9c0 tx=0x7f80f400ee80 comp rx=0 tx=0).stop 2024-09-12T03:29:01.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.233+0000 7f8109f86640 1 -- 172.21.15.6:0/3114284010 shutdown_connections 2024-09-12T03:29:01.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.233+0000 7f8109f86640 1 --2- 172.21.15.6:0/3114284010 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f80ec0a2b90 0x7f80ec0a5050 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:01.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.233+0000 7f8109f86640 1 --2- 172.21.15.6:0/3114284010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81041025b0 0x7f810419f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:01.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.233+0000 7f8109f86640 1 --2- 172.21.15.6:0/3114284010 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8104101bf0 0x7f810419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:01.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.233+0000 7f8109f86640 1 --2- 172.21.15.6:0/3114284010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8104100a00 0x7f8104069ef0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:01.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.233+0000 7f8109f86640 1 -- 172.21.15.6:0/3114284010 >> 172.21.15.6:0/3114284010 conn(0x7f81040fc140 msgr2=0x7f8104104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:01.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.234+0000 7f8109f86640 1 -- 172.21.15.6:0/3114284010 shutdown_connections 2024-09-12T03:29:01.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.234+0000 7f8109f86640 1 -- 172.21.15.6:0/3114284010 wait complete. 2024-09-12T03:29:01.245 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-12T03:29:01.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.348+0000 7f5bcbb6d640 1 -- 172.21.15.6:0/1987340271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bc41078b0 msgr2=0x7f5bc4109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:01.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.348+0000 7f5bcbb6d640 1 --2- 172.21.15.6:0/1987340271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bc41078b0 0x7f5bc4109ca0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009ca0 tx=0x7f5bbc02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:29:01.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.349+0000 7f5bcbb6d640 1 -- 172.21.15.6:0/1987340271 shutdown_connections 2024-09-12T03:29:01.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.349+0000 7f5bcbb6d640 1 --2- 172.21.15.6:0/1987340271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bc41078b0 0x7f5bc4109ca0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.349+0000 7f5bcbb6d640 1 --2- 172.21.15.6:0/1987340271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bc4104f80 0x7f5bc4107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.349+0000 7f5bcbb6d640 1 --2- 172.21.15.6:0/1987340271 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bc4102650 0x7f5bc4104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.349+0000 7f5bcbb6d640 1 -- 172.21.15.6:0/1987340271 >> 172.21.15.6:0/1987340271 conn(0x7f5bc40fc170 msgr2=0x7f5bc40fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.349+0000 7f5bcbb6d640 1 -- 172.21.15.6:0/1987340271 shutdown_connections 2024-09-12T03:29:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.349+0000 7f5bcbb6d640 1 -- 172.21.15.6:0/1987340271 wait complete. 2024-09-12T03:29:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.349+0000 7f5bcbb6d640 1 Processor -- start 2024-09-12T03:29:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.349+0000 7f5bcbb6d640 1 -- start start 2024-09-12T03:29:01.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.349+0000 7f5bcbb6d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bc4102650 0x7f5bc4101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:01.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bcbb6d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bc4104f80 0x7f5bc4100600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bcbb6d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bc41078b0 0x7f5bc4100b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bcbb6d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc410d7a0 con 0x7f5bc4102650 2024-09-12T03:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bcbb6d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc410d620 con 0x7f5bc41078b0 2024-09-12T03:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bcbb6d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc410d920 con 0x7f5bc4104f80 2024-09-12T03:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bc9909640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bc4102650 0x7f5bc4101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bc9909640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bc4102650 0x7f5bc4101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56332/0 (socket says 172.21.15.6:56332) 2024-09-12T03:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bc9909640 1 -- 172.21.15.6:0/2724961351 learned_addr learned my addr 172.21.15.6:0/2724961351 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bc9108640 1 --2- 172.21.15.6:0/2724961351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bc4104f80 0x7f5bc4100600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bca10a640 1 --2- 172.21.15.6:0/2724961351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bc41078b0 0x7f5bc4100b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bc9909640 1 -- 172.21.15.6:0/2724961351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bc4104f80 msgr2=0x7f5bc4100600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bc9909640 1 --2- 172.21.15.6:0/2724961351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bc4104f80 0x7f5bc4100600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bc9909640 1 -- 172.21.15.6:0/2724961351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bc41078b0 msgr2=0x7f5bc4100b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bc9909640 1 --2- 172.21.15.6:0/2724961351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bc41078b0 0x7f5bc4100b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bc9909640 1 -- 172.21.15.6:0/2724961351 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc41013d0 con 0x7f5bc4102650 2024-09-12T03:29:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bc9108640 1 --2- 172.21.15.6:0/2724961351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bc4104f80 0x7f5bc4100600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bc9909640 1 --2- 172.21.15.6:0/2724961351 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bc4102650 0x7f5bc4101fc0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000c920 tx=0x7f5bc000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bbaffd640 1 -- 172.21.15.6:0/2724961351 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0007c70 con 0x7f5bc4102650 2024-09-12T03:29:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bbaffd640 1 -- 172.21.15.6:0/2724961351 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bc0007e10 con 0x7f5bc4102650 2024-09-12T03:29:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bbaffd640 1 -- 172.21.15.6:0/2724961351 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0018890 con 0x7f5bc4102650 2024-09-12T03:29:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bcbb6d640 1 -- 172.21.15.6:0/2724961351 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc41a1ec0 con 0x7f5bc4102650 2024-09-12T03:29:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.350+0000 7f5bcbb6d640 1 -- 172.21.15.6:0/2724961351 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc41a22d0 con 0x7f5bc4102650 2024-09-12T03:29:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.352+0000 7f5bcbb6d640 1 -- 172.21.15.6:0/2724961351 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bc40fdc60 con 0x7f5bc4102650 2024-09-12T03:29:01.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.353+0000 7f5bbaffd640 1 -- 172.21.15.6:0/2724961351 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5bc0018a50 con 0x7f5bc4102650 2024-09-12T03:29:01.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.353+0000 7f5bbaffd640 1 --2- 172.21.15.6:0/2724961351 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5bb00a2b90 0x7f5bb00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:01.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.354+0000 7f5bc9108640 1 --2- 172.21.15.6:0/2724961351 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5bb00a2b90 0x7f5bb00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:01.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.354+0000 7f5bbaffd640 1 -- 172.21.15.6:0/2724961351 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f5bc00cf4e0 con 0x7f5bc4102650 2024-09-12T03:29:01.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.354+0000 7f5bc9108640 1 --2- 172.21.15.6:0/2724961351 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5bb00a2b90 0x7f5bb00a5050 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5bb40058c0 tx=0x7f5bb4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:01.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.357+0000 7f5bbaffd640 1 -- 172.21.15.6:0/2724961351 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5bc0014030 con 0x7f5bc4102650 2024-09-12T03:29:01.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.584+0000 7f5bcbb6d640 1 -- 172.21.15.6:0/2724961351 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f5bc402da70 con 0x7f5bc4102650 2024-09-12T03:29:01.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.592+0000 7f5bbaffd640 1 -- 172.21.15.6:0/2724961351 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740353 (secure 0 0 0) 0x7f5bc000de20 con 0x7f5bc4102650 2024-09-12T03:29:01.602 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:29:01.602 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:29:01.602 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 20, 2024-09-12T03:29:01.602 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.602 INFO:teuthology.orchestra.run.smithi006.stdout: "active_gid": 14150, 2024-09-12T03:29:01.602 INFO:teuthology.orchestra.run.smithi006.stdout: "active_name": "x", 2024-09-12T03:29:01.602 INFO:teuthology.orchestra.run.smithi006.stdout: "active_addrs": { 2024-09-12T03:29:01.602 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2024-09-12T03:29:01.602 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.603 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2024-09-12T03:29:01.603 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:6800", 2024-09-12T03:29:01.603 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 5477767 2024-09-12T03:29:01.603 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.603 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.603 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2024-09-12T03:29:01.603 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:6801", 2024-09-12T03:29:01.603 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 5477767 2024-09-12T03:29:01.603 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.603 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-09-12T03:29:01.603 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.603 INFO:teuthology.orchestra.run.smithi006.stdout: "active_addr": "172.21.15.6:6801/5477767", 2024-09-12T03:29:01.603 INFO:teuthology.orchestra.run.smithi006.stdout: "active_change": "2024-09-12T03:12:19.155375+0000", 2024-09-12T03:29:01.604 INFO:teuthology.orchestra.run.smithi006.stdout: "active_mgr_features": 4540701547738038271, 2024-09-12T03:29:01.604 INFO:teuthology.orchestra.run.smithi006.stdout: "available": true, 2024-09-12T03:29:01.604 INFO:teuthology.orchestra.run.smithi006.stdout: "standbys": [ 2024-09-12T03:29:01.604 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.604 INFO:teuthology.orchestra.run.smithi006.stdout: "gid": 24115, 2024-09-12T03:29:01.604 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "z", 2024-09-12T03:29:01.604 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_features": 4540701547738038271, 2024-09-12T03:29:01.604 INFO:teuthology.orchestra.run.smithi006.stdout: "available_modules": [ 2024-09-12T03:29:01.604 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.604 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "alerts", 2024-09-12T03:29:01.604 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.604 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.605 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.605 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-09-12T03:29:01.605 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-09-12T03:29:01.605 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:01.605 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.605 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.605 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-09-12T03:29:01.605 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.605 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.605 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.605 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:29:01.605 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.605 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.606 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.606 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.606 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.606 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.606 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.606 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.606 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.606 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.606 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.606 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.606 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.606 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.606 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.606 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.607 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.607 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.607 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.607 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.607 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.607 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.607 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.607 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.607 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.607 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.607 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.607 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.608 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.608 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.608 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.608 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.608 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.608 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.608 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.608 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.608 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.609 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.609 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.609 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.609 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.609 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.609 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.609 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.609 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.609 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.609 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.609 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.609 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.609 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.610 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.610 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.610 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.610 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.610 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.610 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.610 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.610 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.610 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.610 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.610 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.610 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.610 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.611 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.611 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.611 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.611 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.611 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.611 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.611 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.611 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.611 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.611 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.611 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.611 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_destination": { 2024-09-12T03:29:01.611 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_destination", 2024-09-12T03:29:01.611 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.612 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.612 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.612 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.612 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.612 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.612 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.612 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:29:01.612 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.612 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.612 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.612 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.612 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_from_name": { 2024-09-12T03:29:01.612 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_from_name", 2024-09-12T03:29:01.613 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.613 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.613 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.613 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "Ceph", 2024-09-12T03:29:01.613 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.613 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.613 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.613 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email From: name", 2024-09-12T03:29:01.613 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.613 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.613 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.613 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.613 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_host": { 2024-09-12T03:29:01.614 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_host", 2024-09-12T03:29:01.614 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.614 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.614 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.614 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.614 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.614 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.614 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.614 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP server", 2024-09-12T03:29:01.614 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.614 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.614 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.614 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.614 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_password": { 2024-09-12T03:29:01.615 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_password", 2024-09-12T03:29:01.615 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.615 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.615 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.615 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.615 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.615 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.615 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.615 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Password to authenticate with", 2024-09-12T03:29:01.615 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.615 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.615 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.615 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.616 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_port": { 2024-09-12T03:29:01.616 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_port", 2024-09-12T03:29:01.616 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.616 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.616 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.616 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "465", 2024-09-12T03:29:01.616 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.616 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.616 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.616 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP port", 2024-09-12T03:29:01.616 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.616 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.616 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.617 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.617 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_sender": { 2024-09-12T03:29:01.617 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_sender", 2024-09-12T03:29:01.617 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.617 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.617 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.617 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.617 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.617 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.617 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.617 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:29:01.617 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.617 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.617 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.618 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.618 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_ssl": { 2024-09-12T03:29:01.618 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_ssl", 2024-09-12T03:29:01.618 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.618 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.618 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.618 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.618 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.618 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.618 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.618 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:29:01.619 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.619 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.619 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.619 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.619 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_user": { 2024-09-12T03:29:01.619 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_user", 2024-09-12T03:29:01.619 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.619 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.619 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.619 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.619 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.619 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.619 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.620 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "User to authenticate as", 2024-09-12T03:29:01.620 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.620 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.620 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.620 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.620 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.620 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.620 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.620 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "balancer", 2024-09-12T03:29:01.620 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.620 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.620 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.621 INFO:teuthology.orchestra.run.smithi006.stdout: "active": { 2024-09-12T03:29:01.621 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "active", 2024-09-12T03:29:01.621 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.621 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.621 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.621 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.621 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.621 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.621 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.621 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:29:01.621 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.621 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.622 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.622 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.622 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_time": { 2024-09-12T03:29:01.622 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_time", 2024-09-12T03:29:01.622 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.622 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.622 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.622 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0000", 2024-09-12T03:29:01.622 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.622 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.622 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.622 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:29:01.622 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:29:01.622 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.623 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.623 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.623 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_weekday": { 2024-09-12T03:29:01.623 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_weekday", 2024-09-12T03:29:01.623 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-09-12T03:29:01.623 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.623 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.623 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-09-12T03:29:01.623 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2024-09-12T03:29:01.623 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2024-09-12T03:29:01.623 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.623 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:29:01.623 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:29:01.624 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.624 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.624 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.624 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_max_iterations": { 2024-09-12T03:29:01.624 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:29:01.624 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-09-12T03:29:01.624 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.624 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.624 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "25", 2024-09-12T03:29:01.624 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-09-12T03:29:01.624 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "250", 2024-09-12T03:29:01.624 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.624 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:29:01.625 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.625 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.625 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.625 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.625 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_metrics": { 2024-09-12T03:29:01.625 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_metrics", 2024-09-12T03:29:01.625 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.625 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.625 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.625 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:29:01.625 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.625 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.625 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.625 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:29:01.626 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:29:01.626 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.626 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.626 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.626 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_step": { 2024-09-12T03:29:01.626 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_step", 2024-09-12T03:29:01.626 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-09-12T03:29:01.626 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.626 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.626 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.5", 2024-09-12T03:29:01.626 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0.001", 2024-09-12T03:29:01.626 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "0.999", 2024-09-12T03:29:01.626 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.627 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:29:01.627 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:29:01.627 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.627 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.627 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.627 INFO:teuthology.orchestra.run.smithi006.stdout: "end_time": { 2024-09-12T03:29:01.627 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_time", 2024-09-12T03:29:01.627 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.627 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.627 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.627 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2359", 2024-09-12T03:29:01.627 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.627 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.627 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.628 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:29:01.628 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:29:01.628 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.628 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.628 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.628 INFO:teuthology.orchestra.run.smithi006.stdout: "end_weekday": { 2024-09-12T03:29:01.628 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_weekday", 2024-09-12T03:29:01.628 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-09-12T03:29:01.628 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.628 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.628 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-09-12T03:29:01.628 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2024-09-12T03:29:01.628 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2024-09-12T03:29:01.628 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.629 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:29:01.629 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:29:01.629 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.629 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.629 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.629 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.629 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.629 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.629 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.629 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.629 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.629 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.629 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.630 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.630 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.630 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.630 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.630 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.630 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.630 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.630 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.630 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.630 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.630 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.630 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.630 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.631 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.631 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.631 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.631 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.631 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.631 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.631 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.631 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.631 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.631 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.631 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.631 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.632 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.632 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.632 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.632 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.632 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.632 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.632 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.632 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.632 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.632 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.632 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.632 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.632 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.632 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.633 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.633 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.633 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.633 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.633 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.633 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.633 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.633 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.633 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.633 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.633 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.633 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.633 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.634 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.634 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.634 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.634 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.634 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.634 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.634 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.634 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.634 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.634 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.634 INFO:teuthology.orchestra.run.smithi006.stdout: "min_score": { 2024-09-12T03:29:01.634 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_score", 2024-09-12T03:29:01.634 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-09-12T03:29:01.635 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.635 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.635 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-09-12T03:29:01.635 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.635 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.635 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.635 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:29:01.635 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.635 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.635 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.635 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.635 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2024-09-12T03:29:01.635 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2024-09-12T03:29:01.635 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.636 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.636 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.636 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "upmap", 2024-09-12T03:29:01.636 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.636 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.636 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.636 INFO:teuthology.orchestra.run.smithi006.stdout: "crush-compat", 2024-09-12T03:29:01.636 INFO:teuthology.orchestra.run.smithi006.stdout: "none", 2024-09-12T03:29:01.636 INFO:teuthology.orchestra.run.smithi006.stdout: "read", 2024-09-12T03:29:01.636 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap", 2024-09-12T03:29:01.636 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap-read" 2024-09-12T03:29:01.636 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.636 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Balancer mode", 2024-09-12T03:29:01.637 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.637 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.637 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.637 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.637 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_ids": { 2024-09-12T03:29:01.637 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_ids", 2024-09-12T03:29:01.637 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.637 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.637 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.637 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.637 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.637 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.637 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.638 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:29:01.638 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.638 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.638 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.638 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.638 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-09-12T03:29:01.638 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-09-12T03:29:01.638 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:01.638 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.638 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.638 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-09-12T03:29:01.638 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.638 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.638 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.639 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:29:01.639 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.639 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.639 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.639 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.639 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_deviation": { 2024-09-12T03:29:01.639 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_deviation", 2024-09-12T03:29:01.639 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.639 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.639 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.639 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-09-12T03:29:01.639 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-09-12T03:29:01.639 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.640 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.640 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:29:01.640 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:29:01.640 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.640 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.640 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.640 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_optimizations": { 2024-09-12T03:29:01.640 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:29:01.640 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-09-12T03:29:01.640 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.640 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.640 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:01.640 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.641 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.641 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.641 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:29:01.641 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.641 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.641 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.641 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.641 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.641 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.641 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.641 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cephadm", 2024-09-12T03:29:01.641 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.641 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.642 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.642 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_down_multiplier": { 2024-09-12T03:29:01.642 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_down_multiplier", 2024-09-12T03:29:01.642 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-09-12T03:29:01.642 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.642 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.642 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2024-09-12T03:29:01.642 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.642 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.642 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.642 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:29:01.642 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.642 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.643 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.643 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.643 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_refresh_rate": { 2024-09-12T03:29:01.643 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_refresh_rate", 2024-09-12T03:29:01.643 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:01.643 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.643 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.643 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "20", 2024-09-12T03:29:01.643 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.643 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.643 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.643 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:29:01.643 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.644 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.644 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.644 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.644 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_starting_port": { 2024-09-12T03:29:01.644 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_starting_port", 2024-09-12T03:29:01.644 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.644 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.644 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.644 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4721", 2024-09-12T03:29:01.644 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.644 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.644 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.645 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:29:01.645 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.645 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.645 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.645 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.645 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_ptrace": { 2024-09-12T03:29:01.645 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_ptrace", 2024-09-12T03:29:01.645 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.645 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.645 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.645 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.645 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.646 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.646 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.646 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:29:01.646 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:29:01.646 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.646 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.646 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.646 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_interval": { 2024-09-12T03:29:01.646 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_interval", 2024-09-12T03:29:01.646 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:01.646 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.646 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.646 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-09-12T03:29:01.647 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.647 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.647 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.647 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:29:01.647 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.647 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.647 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.647 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.647 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:29:01.647 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:29:01.647 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-09-12T03:29:01.647 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.648 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.648 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.7", 2024-09-12T03:29:01.648 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.648 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.648 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.648 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:29:01.648 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.648 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.648 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.648 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.648 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm_log_destination": { 2024-09-12T03:29:01.648 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cephadm_log_destination", 2024-09-12T03:29:01.649 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.649 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.649 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.649 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.649 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.649 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.649 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.649 INFO:teuthology.orchestra.run.smithi006.stdout: "file", 2024-09-12T03:29:01.649 INFO:teuthology.orchestra.run.smithi006.stdout: "file,syslog", 2024-09-12T03:29:01.649 INFO:teuthology.orchestra.run.smithi006.stdout: "syslog" 2024-09-12T03:29:01.649 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.649 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:29:01.649 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.650 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.650 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.650 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.650 INFO:teuthology.orchestra.run.smithi006.stdout: "cgroups_split": { 2024-09-12T03:29:01.650 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cgroups_split", 2024-09-12T03:29:01.650 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.650 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.650 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.650 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.650 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.650 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.650 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.650 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:29:01.650 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.651 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.651 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.651 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.651 INFO:teuthology.orchestra.run.smithi006.stdout: "config_checks_enabled": { 2024-09-12T03:29:01.651 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_checks_enabled", 2024-09-12T03:29:01.651 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.651 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.651 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.651 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.651 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.651 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.651 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.651 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:29:01.652 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.652 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.652 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.652 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.652 INFO:teuthology.orchestra.run.smithi006.stdout: "config_dashboard": { 2024-09-12T03:29:01.652 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_dashboard", 2024-09-12T03:29:01.652 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.652 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.652 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.652 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.652 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.652 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.653 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.653 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:29:01.653 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.653 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.653 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.653 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.653 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_alertmanager": { 2024-09-12T03:29:01.653 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_alertmanager", 2024-09-12T03:29:01.653 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.653 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.653 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.653 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:29:01.653 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.654 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.654 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.654 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-09-12T03:29:01.654 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.654 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.654 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.654 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.654 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_base": { 2024-09-12T03:29:01.654 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_base", 2024-09-12T03:29:01.654 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.654 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.654 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.654 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:29:01.654 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.655 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.655 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.655 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:29:01.655 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.655 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.655 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.655 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.655 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_elasticsearch": { 2024-09-12T03:29:01.655 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:29:01.655 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.655 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.655 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.655 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:29:01.656 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.656 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.656 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.656 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "elasticsearch container image", 2024-09-12T03:29:01.656 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.656 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.656 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.656 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.656 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_grafana": { 2024-09-12T03:29:01.656 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_grafana", 2024-09-12T03:29:01.656 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.656 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.657 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.657 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:29:01.657 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.657 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.657 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.657 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-09-12T03:29:01.657 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.657 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.657 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.657 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.657 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_haproxy": { 2024-09-12T03:29:01.657 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_haproxy", 2024-09-12T03:29:01.657 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.657 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.658 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.658 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:29:01.658 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.658 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.658 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.658 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "HAproxy container image", 2024-09-12T03:29:01.658 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.658 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.658 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.658 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.658 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_agent": { 2024-09-12T03:29:01.659 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:29:01.659 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.659 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.659 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.659 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:29:01.659 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.659 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.659 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.659 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:29:01.659 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.659 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.659 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.659 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.660 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_collector": { 2024-09-12T03:29:01.660 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:29:01.660 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.660 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.660 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.660 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:29:01.660 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.660 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.660 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.660 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:29:01.660 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.660 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.660 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.661 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.661 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_query": { 2024-09-12T03:29:01.661 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:29:01.661 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.661 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.661 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.661 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:29:01.661 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.661 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.661 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.661 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger query container image", 2024-09-12T03:29:01.661 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.661 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.662 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.662 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.662 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_keepalived": { 2024-09-12T03:29:01.662 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_keepalived", 2024-09-12T03:29:01.662 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.662 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.662 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.662 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:29:01.662 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.662 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.662 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.662 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Keepalived container image", 2024-09-12T03:29:01.662 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.662 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.663 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.663 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.663 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_loki": { 2024-09-12T03:29:01.663 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_loki", 2024-09-12T03:29:01.663 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.663 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.663 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.663 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:29:01.663 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.663 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.663 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.663 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Loki container image", 2024-09-12T03:29:01.663 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.663 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.664 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.664 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.664 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_node_exporter": { 2024-09-12T03:29:01.664 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_node_exporter", 2024-09-12T03:29:01.664 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.664 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.664 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.664 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:29:01.664 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.664 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.664 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.664 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-09-12T03:29:01.664 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.665 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.665 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.665 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.665 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_nvmeof": { 2024-09-12T03:29:01.665 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_nvmeof", 2024-09-12T03:29:01.665 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.665 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.665 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.665 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:29:01.665 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.665 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.665 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.665 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Nvme-of container image", 2024-09-12T03:29:01.666 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.666 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.666 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.666 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.666 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_prometheus": { 2024-09-12T03:29:01.666 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_prometheus", 2024-09-12T03:29:01.666 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.666 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.666 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.666 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:29:01.666 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.666 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.666 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.666 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-09-12T03:29:01.666 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.667 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.667 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.667 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.667 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_promtail": { 2024-09-12T03:29:01.667 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_promtail", 2024-09-12T03:29:01.667 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.667 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.667 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.667 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:29:01.667 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.667 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.667 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.667 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Promtail container image", 2024-09-12T03:29:01.668 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.668 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.668 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.668 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.668 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_samba": { 2024-09-12T03:29:01.668 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_samba", 2024-09-12T03:29:01.668 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.668 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.668 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.668 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:29:01.668 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.668 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.668 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.668 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:29:01.669 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.669 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.669 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.669 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.669 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_snmp_gateway": { 2024-09-12T03:29:01.669 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:29:01.669 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.669 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.669 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.669 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:29:01.669 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.669 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.669 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.670 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:29:01.670 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.670 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.670 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.670 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.670 INFO:teuthology.orchestra.run.smithi006.stdout: "container_init": { 2024-09-12T03:29:01.670 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_init", 2024-09-12T03:29:01.670 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.670 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.670 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.670 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.670 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.671 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.671 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.671 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:29:01.671 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.671 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.671 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.671 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.671 INFO:teuthology.orchestra.run.smithi006.stdout: "daemon_cache_timeout": { 2024-09-12T03:29:01.671 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:29:01.671 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:01.671 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.671 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.672 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-09-12T03:29:01.672 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.672 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.672 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.672 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:29:01.672 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.672 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.672 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.672 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.672 INFO:teuthology.orchestra.run.smithi006.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:29:01.672 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:29:01.672 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.672 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.673 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.673 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "900", 2024-09-12T03:29:01.673 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.673 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.673 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.673 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:29:01.673 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.673 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.673 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.673 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.673 INFO:teuthology.orchestra.run.smithi006.stdout: "default_registry": { 2024-09-12T03:29:01.673 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_registry", 2024-09-12T03:29:01.673 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.674 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.674 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.674 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io", 2024-09-12T03:29:01.674 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.674 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.674 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.674 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:29:01.674 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.674 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.674 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.674 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.674 INFO:teuthology.orchestra.run.smithi006.stdout: "device_cache_timeout": { 2024-09-12T03:29:01.674 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_cache_timeout", 2024-09-12T03:29:01.675 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:01.675 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.675 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.675 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1800", 2024-09-12T03:29:01.675 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.675 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.675 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.675 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:29:01.675 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.675 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.675 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.675 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.675 INFO:teuthology.orchestra.run.smithi006.stdout: "device_enhanced_scan": { 2024-09-12T03:29:01.676 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_enhanced_scan", 2024-09-12T03:29:01.676 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.676 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.676 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.676 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.676 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.676 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.676 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.676 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:29:01.676 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.676 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.676 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.676 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.677 INFO:teuthology.orchestra.run.smithi006.stdout: "facts_cache_timeout": { 2024-09-12T03:29:01.677 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "facts_cache_timeout", 2024-09-12T03:29:01.677 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:01.677 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.677 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.677 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-09-12T03:29:01.677 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.677 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.677 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.677 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:29:01.677 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.677 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.677 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.677 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.678 INFO:teuthology.orchestra.run.smithi006.stdout: "grafana_dashboards_path": { 2024-09-12T03:29:01.678 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:29:01.678 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.678 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.678 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.678 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:29:01.678 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.678 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.678 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.678 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:29:01.678 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.678 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.678 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.679 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.679 INFO:teuthology.orchestra.run.smithi006.stdout: "host_check_interval": { 2024-09-12T03:29:01.679 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "host_check_interval", 2024-09-12T03:29:01.679 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:01.679 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.679 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.679 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-09-12T03:29:01.679 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.679 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.679 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.679 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:29:01.679 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.679 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.679 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.680 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.680 INFO:teuthology.orchestra.run.smithi006.stdout: "hw_monitoring": { 2024-09-12T03:29:01.680 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "hw_monitoring", 2024-09-12T03:29:01.680 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.680 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.680 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.680 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.680 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.680 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.680 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.680 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:29:01.680 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.680 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.681 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.681 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.681 INFO:teuthology.orchestra.run.smithi006.stdout: "inventory_list_all": { 2024-09-12T03:29:01.681 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "inventory_list_all", 2024-09-12T03:29:01.681 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.681 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.681 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.681 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.681 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.681 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.681 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.681 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:29:01.681 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.681 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.682 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.682 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.682 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.682 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.682 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.682 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.682 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.682 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.682 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.682 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.682 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.682 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.682 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.682 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.683 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.683 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.683 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.683 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.683 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.683 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.683 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.683 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.683 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.683 INFO:teuthology.orchestra.run.smithi006.stdout: "log_refresh_metadata": { 2024-09-12T03:29:01.683 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_refresh_metadata", 2024-09-12T03:29:01.683 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.683 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.684 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.684 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.684 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.684 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.684 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.684 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:29:01.684 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.684 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.684 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.684 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.684 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.684 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.685 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.685 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.685 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.685 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.685 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.685 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.685 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.685 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:29:01.685 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.685 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.688 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.688 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.688 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.689 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.689 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.689 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.689 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.689 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.689 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.689 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.689 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.689 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.689 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.689 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.689 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.689 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.689 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.690 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.690 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.690 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.690 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.690 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.690 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.690 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.690 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.690 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.690 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.690 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.690 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.690 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.691 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.691 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.691 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.691 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.691 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.691 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.691 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.691 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:29:01.691 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:29:01.691 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.691 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.691 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.691 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.691 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.692 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.692 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.692 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:29:01.692 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.692 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.692 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.692 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.692 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:29:01.692 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:29:01.692 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.692 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.692 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.692 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "*", 2024-09-12T03:29:01.693 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.693 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.693 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.693 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:29:01.693 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.693 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.693 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.693 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.693 INFO:teuthology.orchestra.run.smithi006.stdout: "max_count_per_host": { 2024-09-12T03:29:01.693 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_count_per_host", 2024-09-12T03:29:01.693 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.693 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.693 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.694 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:01.694 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.694 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.694 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.694 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:29:01.694 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.694 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.694 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.694 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.694 INFO:teuthology.orchestra.run.smithi006.stdout: "max_osd_draining_count": { 2024-09-12T03:29:01.694 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_osd_draining_count", 2024-09-12T03:29:01.694 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.694 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.695 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.695 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:01.695 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.695 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.695 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.695 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:29:01.695 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.695 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.695 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.695 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.695 INFO:teuthology.orchestra.run.smithi006.stdout: "migration_current": { 2024-09-12T03:29:01.695 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "migration_current", 2024-09-12T03:29:01.695 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.695 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.696 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.696 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.696 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.696 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.696 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.696 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "internal - do not modify", 2024-09-12T03:29:01.696 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.696 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.696 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.696 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.696 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2024-09-12T03:29:01.696 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2024-09-12T03:29:01.696 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.696 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.697 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.697 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "root", 2024-09-12T03:29:01.697 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.697 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.697 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.697 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm-package", 2024-09-12T03:29:01.697 INFO:teuthology.orchestra.run.smithi006.stdout: "root" 2024-09-12T03:29:01.697 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.697 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:29:01.697 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.697 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.697 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.697 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.697 INFO:teuthology.orchestra.run.smithi006.stdout: "oob_default_addr": { 2024-09-12T03:29:01.698 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "oob_default_addr", 2024-09-12T03:29:01.698 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.698 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.698 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.698 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "169.254.1.1", 2024-09-12T03:29:01.698 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.698 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.698 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.698 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:29:01.698 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.698 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.698 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.698 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.699 INFO:teuthology.orchestra.run.smithi006.stdout: "prometheus_alerts_path": { 2024-09-12T03:29:01.699 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:29:01.699 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.699 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.699 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.699 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:29:01.699 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.699 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.699 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.699 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:29:01.699 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.699 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.699 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.699 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.700 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_insecure": { 2024-09-12T03:29:01.700 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_insecure", 2024-09-12T03:29:01.700 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.700 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.700 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.700 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.700 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.700 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.700 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.700 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:29:01.700 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.700 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.701 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.701 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.701 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_password": { 2024-09-12T03:29:01.701 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_password", 2024-09-12T03:29:01.701 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.701 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.701 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.701 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.701 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.701 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.701 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.701 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:29:01.701 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.702 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.702 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.702 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.702 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_url": { 2024-09-12T03:29:01.702 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_url", 2024-09-12T03:29:01.702 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.702 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.702 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.702 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.702 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.702 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.702 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.702 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:29:01.702 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_username": { 2024-09-12T03:29:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_username", 2024-09-12T03:29:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:29:01.704 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.704 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.704 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.704 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.704 INFO:teuthology.orchestra.run.smithi006.stdout: "secure_monitoring_stack": { 2024-09-12T03:29:01.704 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:29:01.704 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.704 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.704 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.704 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.704 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.704 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.704 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.705 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:29:01.705 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.705 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.705 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.705 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.705 INFO:teuthology.orchestra.run.smithi006.stdout: "service_discovery_port": { 2024-09-12T03:29:01.705 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "service_discovery_port", 2024-09-12T03:29:01.705 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.705 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.705 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.705 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8765", 2024-09-12T03:29:01.705 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.705 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.705 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.706 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:29:01.706 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.706 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.706 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.706 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.706 INFO:teuthology.orchestra.run.smithi006.stdout: "ssh_config_file": { 2024-09-12T03:29:01.706 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssh_config_file", 2024-09-12T03:29:01.706 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.706 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.706 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.706 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.706 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.706 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.707 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.707 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:29:01.707 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.707 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.707 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.707 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.707 INFO:teuthology.orchestra.run.smithi006.stdout: "use_agent": { 2024-09-12T03:29:01.707 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_agent", 2024-09-12T03:29:01.707 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.707 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.707 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.707 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.707 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.707 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.708 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.708 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:29:01.708 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.708 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.708 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.708 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.708 INFO:teuthology.orchestra.run.smithi006.stdout: "use_repo_digest": { 2024-09-12T03:29:01.708 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_repo_digest", 2024-09-12T03:29:01.708 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.708 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.708 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.708 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.708 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.709 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.709 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.709 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:29:01.709 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.709 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.709 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.709 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.709 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_failed_host_check": { 2024-09-12T03:29:01.709 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:29:01.709 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.709 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.709 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.709 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.710 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.710 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.710 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.710 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:29:01.710 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.710 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.710 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.710 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.710 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_daemons": { 2024-09-12T03:29:01.710 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:29:01.710 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.710 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.710 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.710 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.711 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.711 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.711 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.711 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:29:01.711 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.711 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.711 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.711 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.711 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_hosts": { 2024-09-12T03:29:01.711 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:29:01.711 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.711 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.711 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.711 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.712 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.712 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.712 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.712 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:29:01.712 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.712 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.712 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.712 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.712 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.712 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.712 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.712 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crash", 2024-09-12T03:29:01.712 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.713 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.713 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.713 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.713 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.713 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.713 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.713 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.713 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.713 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.713 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.713 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.713 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.714 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.714 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.714 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.714 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.714 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.714 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.714 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.714 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.714 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.714 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.714 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.714 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.714 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.714 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.715 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.715 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.715 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.715 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.715 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.715 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.715 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.715 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.715 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.715 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.715 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.715 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.715 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.716 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.716 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.716 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.716 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.716 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.716 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.716 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.716 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.716 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.716 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.716 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.716 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.716 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.717 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.717 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.717 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.717 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.717 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.717 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.717 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.717 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.717 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.717 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.717 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.717 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.717 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.718 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.718 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.718 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.718 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.718 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.718 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.718 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.718 INFO:teuthology.orchestra.run.smithi006.stdout: "retain_interval": { 2024-09-12T03:29:01.718 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retain_interval", 2024-09-12T03:29:01.718 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:01.718 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.718 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.718 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "31536000", 2024-09-12T03:29:01.719 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.719 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.719 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.719 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:29:01.719 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.719 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.719 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.719 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.719 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_recent_interval": { 2024-09-12T03:29:01.719 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_recent_interval", 2024-09-12T03:29:01.719 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:01.719 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.719 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.719 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1209600", 2024-09-12T03:29:01.720 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.720 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.720 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.720 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:29:01.720 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.720 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.720 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.720 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.720 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.720 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.720 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.720 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dashboard", 2024-09-12T03:29:01.720 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.721 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.721 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.721 INFO:teuthology.orchestra.run.smithi006.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:29:01.721 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:29:01.721 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.721 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.721 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.721 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:01.721 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.721 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.721 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.721 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.721 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.722 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.722 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.722 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.722 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:29:01.722 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:29:01.722 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.722 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.722 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.722 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.722 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.722 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.722 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.722 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.722 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.723 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.723 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.723 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.723 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:29:01.723 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:29:01.723 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.723 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.723 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.723 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.723 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.723 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.723 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.723 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.724 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.724 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.724 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.724 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.724 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:29:01.724 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:29:01.724 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.724 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.724 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.724 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.724 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.724 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.724 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.725 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.725 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.725 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.725 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.725 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.725 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:29:01.725 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:29:01.725 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.725 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.725 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.725 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.725 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.725 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.725 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.726 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.726 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.726 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.726 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.726 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.726 INFO:teuthology.orchestra.run.smithi006.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:29:01.726 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:29:01.726 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.726 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.726 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.726 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.726 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.726 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.727 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.727 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.727 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.727 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.727 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.727 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.727 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:29:01.727 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:29:01.727 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.727 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.727 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.727 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.728 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.728 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.728 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.728 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.728 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.728 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.728 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.728 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.728 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:29:01.728 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:29:01.728 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.728 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.728 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.729 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.729 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.729 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.729 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.729 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.729 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.729 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.729 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.729 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.729 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:29:01.729 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:29:01.729 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.729 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.730 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.730 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.730 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.730 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.730 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.730 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.730 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.730 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.730 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.730 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.730 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:29:01.730 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:29:01.730 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.731 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.731 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.731 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.731 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.731 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.731 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.731 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.731 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.731 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.731 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.731 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.731 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:29:01.731 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:29:01.731 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.732 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.732 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.732 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.732 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.732 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.732 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.732 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.732 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.732 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.732 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.732 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.732 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:29:01.732 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:29:01.733 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.733 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.733 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.733 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.733 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.733 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.733 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.733 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.733 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.733 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.733 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.733 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.734 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:29:01.734 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:29:01.734 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.734 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.734 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.734 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.734 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.734 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.734 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.734 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.734 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.734 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.734 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.734 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.735 INFO:teuthology.orchestra.run.smithi006.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:29:01.735 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:29:01.735 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.735 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.735 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.735 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.735 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.735 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.735 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.735 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.735 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.735 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.735 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.736 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.736 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:29:01.736 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:29:01.736 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.736 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.736 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.736 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-09-12T03:29:01.736 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.736 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.736 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.736 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.736 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.736 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.736 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.737 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.737 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:29:01.737 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:29:01.737 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.737 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.737 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.737 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.737 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.737 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.737 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.737 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.737 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.737 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.737 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.738 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.738 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_URL": { 2024-09-12T03:29:01.738 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:29:01.738 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.738 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.738 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.738 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.738 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.738 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.738 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.738 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.738 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.738 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.739 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.739 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.739 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:29:01.739 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:29:01.739 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.739 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.739 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.739 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-09-12T03:29:01.739 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.739 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.739 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.739 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.739 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.739 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.740 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.740 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.740 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:29:01.740 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:29:01.740 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.740 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.740 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.740 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.740 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.740 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.740 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.740 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.740 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.740 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.741 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.741 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.741 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:29:01.741 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:29:01.741 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.741 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.741 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.741 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.741 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.741 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.741 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.741 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.742 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.742 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.742 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.742 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.742 INFO:teuthology.orchestra.run.smithi006.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:29:01.742 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:29:01.742 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.742 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.742 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.742 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.742 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.742 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.742 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.743 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.743 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.743 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.743 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.743 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.743 INFO:teuthology.orchestra.run.smithi006.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:29:01.743 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:29:01.743 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.743 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.743 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.743 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.743 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.743 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.743 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.744 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.744 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.744 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.744 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.744 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.744 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:29:01.744 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:29:01.744 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.744 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.744 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.744 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.744 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.744 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.745 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.745 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.745 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.745 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.745 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.745 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.745 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:29:01.745 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:29:01.745 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.745 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.745 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.745 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.745 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.746 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.746 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.746 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.746 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.746 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.746 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.746 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.746 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:29:01.746 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:29:01.746 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.746 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.746 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.746 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.747 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.747 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.747 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.747 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.747 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.747 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.747 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.747 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.747 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:29:01.747 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:29:01.747 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.747 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.747 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.747 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.748 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.748 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.748 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.748 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.748 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.748 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.748 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.748 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.748 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:29:01.748 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:29:01.748 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.748 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.748 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.749 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.749 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.749 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.749 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.749 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.749 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.749 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.749 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.749 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.749 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:29:01.749 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:29:01.749 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.749 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.749 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.750 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.750 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.750 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.750 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.750 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.750 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.750 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.750 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.750 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.750 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:29:01.750 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:29:01.750 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.750 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.750 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.751 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.751 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.751 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.751 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.751 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.751 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.751 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.751 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.751 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.751 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:29:01.751 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:29:01.751 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.751 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.751 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.752 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.752 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.752 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.752 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.752 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.752 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.752 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.752 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.752 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.752 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:29:01.752 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:29:01.752 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.752 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.752 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.753 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.753 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.753 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.753 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.753 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.753 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.753 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.753 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.753 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.753 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:29:01.753 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:29:01.753 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.753 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.754 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.754 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.754 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.754 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.754 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.754 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.754 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.754 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.754 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.754 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.754 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:29:01.754 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:29:01.754 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.755 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.755 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.755 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:29:01.755 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.755 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.755 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.755 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.755 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.755 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.755 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.755 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.755 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:29:01.755 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:29:01.756 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.756 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.756 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.756 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:01.756 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.756 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.756 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.756 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.756 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.756 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.756 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.756 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.756 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:29:01.756 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:29:01.757 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.757 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.757 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.757 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8", 2024-09-12T03:29:01.757 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.757 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.757 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.757 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.757 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.757 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.757 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.757 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.757 INFO:teuthology.orchestra.run.smithi006.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:29:01.758 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:29:01.758 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.758 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.758 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.758 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "45", 2024-09-12T03:29:01.758 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.758 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.758 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.758 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.758 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.758 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.758 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.758 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.759 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:29:01.759 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:29:01.759 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.759 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.759 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.759 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.759 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.759 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.759 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.759 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.759 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.759 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.759 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.759 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.760 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:29:01.760 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:29:01.760 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.760 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.760 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.760 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-09-12T03:29:01.760 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.760 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.760 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.760 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.760 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.760 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.760 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.761 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.761 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:29:01.761 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:29:01.761 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.761 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.761 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.761 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.761 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.761 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.761 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.761 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.761 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.761 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.762 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.762 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.762 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:29:01.762 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:29:01.762 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.762 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.762 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.762 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.762 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.762 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.762 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.762 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.762 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.762 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.763 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.763 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.763 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:29:01.763 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:29:01.763 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.763 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.763 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.763 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-09-12T03:29:01.763 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.763 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.763 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.763 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.763 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.763 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.764 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.764 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.764 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:29:01.764 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:29:01.764 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.764 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.764 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.764 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:01.764 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.764 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.764 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.764 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.764 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.765 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.765 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.765 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.765 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:29:01.765 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:29:01.765 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.765 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.765 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.765 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-09-12T03:29:01.765 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.765 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.765 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.765 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.765 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.766 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.766 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.766 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.766 INFO:teuthology.orchestra.run.smithi006.stdout: "cross_origin_url": { 2024-09-12T03:29:01.766 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cross_origin_url", 2024-09-12T03:29:01.766 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.766 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.766 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.766 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.766 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.766 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.766 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.766 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.767 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.767 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.767 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.767 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.767 INFO:teuthology.orchestra.run.smithi006.stdout: "crt_file": { 2024-09-12T03:29:01.767 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crt_file", 2024-09-12T03:29:01.767 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.767 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.767 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.767 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.767 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.767 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.767 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.767 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.768 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.768 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.768 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.768 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.768 INFO:teuthology.orchestra.run.smithi006.stdout: "debug": { 2024-09-12T03:29:01.768 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "debug", 2024-09-12T03:29:01.768 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.768 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.768 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.768 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.768 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.768 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.769 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.769 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:29:01.769 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.769 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.769 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.769 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.769 INFO:teuthology.orchestra.run.smithi006.stdout: "jwt_token_ttl": { 2024-09-12T03:29:01.769 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "jwt_token_ttl", 2024-09-12T03:29:01.769 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.769 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.769 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.769 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "28800", 2024-09-12T03:29:01.769 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.770 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.770 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.770 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.770 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.770 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.770 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.770 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.770 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2024-09-12T03:29:01.770 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2024-09-12T03:29:01.770 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.770 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.770 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.770 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.771 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.771 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.771 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.771 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.771 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.771 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.771 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.771 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.771 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.771 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.771 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.771 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.771 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.771 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.772 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.772 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.772 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.772 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.772 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.772 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.772 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.772 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.772 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.772 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.772 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.772 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.772 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.773 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.773 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.773 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.773 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.773 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.773 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.773 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.773 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.773 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.773 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.773 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.773 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.773 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.773 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.774 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.774 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.774 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.774 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.774 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.774 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.774 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.774 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.774 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.774 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.774 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.774 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.774 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.775 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.775 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.775 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.775 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.775 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.775 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.775 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.775 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.775 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.775 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.775 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.775 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.775 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.776 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.776 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.776 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.776 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.776 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.776 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.776 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.776 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.776 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.776 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.776 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.776 INFO:teuthology.orchestra.run.smithi006.stdout: "motd": { 2024-09-12T03:29:01.776 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "motd", 2024-09-12T03:29:01.777 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.777 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.777 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.777 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.777 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.777 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.777 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.777 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "The message of the day", 2024-09-12T03:29:01.777 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.777 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.777 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.777 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.777 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:29:01.777 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:29:01.778 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.778 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.778 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.778 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.778 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.778 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.778 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.778 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.778 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.778 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.778 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.778 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.778 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-09-12T03:29:01.779 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-09-12T03:29:01.779 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.779 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.779 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.779 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2024-09-12T03:29:01.779 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.779 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.779 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.779 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.779 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.779 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.779 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.779 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.779 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-09-12T03:29:01.780 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-09-12T03:29:01.780 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.780 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.780 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.780 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8080", 2024-09-12T03:29:01.780 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.780 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.780 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.780 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.780 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.780 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.780 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.780 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.780 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2024-09-12T03:29:01.781 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2024-09-12T03:29:01.781 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.781 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.781 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.781 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.781 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.781 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.781 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.781 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.781 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.781 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.781 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.781 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.782 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl_server_port": { 2024-09-12T03:29:01.782 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl_server_port", 2024-09-12T03:29:01.782 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.782 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.782 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.782 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8443", 2024-09-12T03:29:01.782 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.782 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.782 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.782 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.782 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.782 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.782 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.783 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.783 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2024-09-12T03:29:01.783 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2024-09-12T03:29:01.783 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.783 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.783 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.783 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "redirect", 2024-09-12T03:29:01.783 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.783 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.783 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.783 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.783 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect" 2024-09-12T03:29:01.783 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.784 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.784 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.784 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.784 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.784 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.784 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2024-09-12T03:29:01.784 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2024-09-12T03:29:01.784 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.784 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.784 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.784 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2024-09-12T03:29:01.784 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2024-09-12T03:29:01.784 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2024-09-12T03:29:01.785 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.785 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.785 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.785 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.785 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.785 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.785 INFO:teuthology.orchestra.run.smithi006.stdout: "url_prefix": { 2024-09-12T03:29:01.785 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url_prefix", 2024-09-12T03:29:01.785 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.785 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.785 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.785 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.785 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.786 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.786 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.786 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.786 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.786 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.786 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.786 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.786 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.786 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.786 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.786 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "devicehealth", 2024-09-12T03:29:01.786 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.786 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.786 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.787 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_monitoring": { 2024-09-12T03:29:01.787 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_monitoring", 2024-09-12T03:29:01.787 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.787 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.787 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.787 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.787 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.787 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.787 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.787 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "monitor device health metrics", 2024-09-12T03:29:01.787 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.787 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.787 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.789 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.789 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.789 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.789 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.789 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.789 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.789 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.789 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.789 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.789 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.789 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.789 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.789 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.792 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.792 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.792 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.792 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.792 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.792 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.792 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.792 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.792 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.792 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.792 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.792 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.792 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.792 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.793 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.793 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.793 INFO:teuthology.orchestra.run.smithi006.stdout: "mark_out_threshold": { 2024-09-12T03:29:01.793 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mark_out_threshold", 2024-09-12T03:29:01.793 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:01.793 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.793 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.793 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2419200", 2024-09-12T03:29:01.793 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.793 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.793 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.793 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:29:01.793 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.794 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.794 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.794 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.794 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_name": { 2024-09-12T03:29:01.794 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_name", 2024-09-12T03:29:01.794 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.794 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.794 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.794 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "device_health_metrics", 2024-09-12T03:29:01.794 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.794 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.794 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.794 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:29:01.794 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.795 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.795 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.795 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.795 INFO:teuthology.orchestra.run.smithi006.stdout: "retention_period": { 2024-09-12T03:29:01.795 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retention_period", 2024-09-12T03:29:01.795 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:01.795 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.795 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.795 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15552000", 2024-09-12T03:29:01.795 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.795 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.795 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.795 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:29:01.795 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.796 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.796 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.796 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.796 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_frequency": { 2024-09-12T03:29:01.796 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_frequency", 2024-09-12T03:29:01.796 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:01.796 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.796 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.796 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2024-09-12T03:29:01.796 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.796 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.796 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.797 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:29:01.797 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.797 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.797 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.797 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.797 INFO:teuthology.orchestra.run.smithi006.stdout: "self_heal": { 2024-09-12T03:29:01.797 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "self_heal", 2024-09-12T03:29:01.797 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.797 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.797 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.797 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.797 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.797 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.798 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.798 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:29:01.798 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.798 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.798 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.798 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.798 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-09-12T03:29:01.798 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-09-12T03:29:01.798 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:01.798 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.798 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.798 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-09-12T03:29:01.798 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.798 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.799 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.799 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:29:01.799 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.799 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.799 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.799 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.799 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_threshold": { 2024-09-12T03:29:01.799 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_threshold", 2024-09-12T03:29:01.799 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:01.799 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.799 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.799 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7257600", 2024-09-12T03:29:01.799 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.799 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.800 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.800 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:29:01.800 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.800 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.800 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.800 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.800 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.800 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.800 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.800 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "diskprediction_local", 2024-09-12T03:29:01.800 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.800 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.800 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.801 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.801 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.801 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.801 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.801 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.801 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.801 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.801 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.801 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.801 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.801 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.801 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.801 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.802 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.802 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.802 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.802 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.802 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.802 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.802 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.802 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.802 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.802 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.802 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.802 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.802 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.803 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.803 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.803 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.803 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.803 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.803 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.803 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.803 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.803 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.803 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.803 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.803 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.803 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.803 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.804 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.804 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.804 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.804 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.804 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.804 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.804 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.804 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.804 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.804 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.804 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.804 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.805 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.805 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.805 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.805 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.805 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.805 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.805 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.805 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.805 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.805 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.805 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.805 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.805 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.805 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.806 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.806 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.806 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.806 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.806 INFO:teuthology.orchestra.run.smithi006.stdout: "predict_interval": { 2024-09-12T03:29:01.806 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predict_interval", 2024-09-12T03:29:01.806 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.806 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.806 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.806 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2024-09-12T03:29:01.806 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.806 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.806 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.807 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.807 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.807 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.807 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.807 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.807 INFO:teuthology.orchestra.run.smithi006.stdout: "predictor_model": { 2024-09-12T03:29:01.807 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predictor_model", 2024-09-12T03:29:01.807 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.807 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.807 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.807 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "prophetstor", 2024-09-12T03:29:01.807 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.807 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.807 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.808 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.808 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.808 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.808 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.808 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.808 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-09-12T03:29:01.808 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-09-12T03:29:01.808 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.808 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.808 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.808 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-09-12T03:29:01.808 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.808 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.809 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.809 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.809 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.809 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.809 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.809 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.809 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.809 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.809 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.809 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "influx", 2024-09-12T03:29:01.809 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": false, 2024-09-12T03:29:01.809 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:29:01.809 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.809 INFO:teuthology.orchestra.run.smithi006.stdout: "batch_size": { 2024-09-12T03:29:01.810 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "batch_size", 2024-09-12T03:29:01.810 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.810 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.810 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.810 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5000", 2024-09-12T03:29:01.810 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.810 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.810 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.810 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:29:01.810 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.810 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.810 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.810 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.811 INFO:teuthology.orchestra.run.smithi006.stdout: "database": { 2024-09-12T03:29:01.811 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "database", 2024-09-12T03:29:01.811 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.811 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.811 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.811 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "ceph", 2024-09-12T03:29:01.811 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.811 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.811 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.811 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:29:01.811 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.811 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.811 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.811 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.812 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": { 2024-09-12T03:29:01.812 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "hostname", 2024-09-12T03:29:01.812 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.812 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.812 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.812 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.812 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.812 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.812 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.812 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:29:01.812 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.812 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.812 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.813 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.813 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-09-12T03:29:01.813 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-09-12T03:29:01.813 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:01.813 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.813 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.813 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "30", 2024-09-12T03:29:01.813 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "5", 2024-09-12T03:29:01.813 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.813 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.813 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:29:01.813 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.813 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.813 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.814 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.814 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.814 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.814 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.814 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.814 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.814 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.814 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.814 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.814 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.814 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.814 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.814 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.815 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.815 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.815 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.815 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.815 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.815 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.815 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.815 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.815 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.815 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.815 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.815 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.815 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.815 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.816 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.816 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.816 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.816 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.816 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.816 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.816 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.816 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.816 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.816 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.816 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.816 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.816 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.817 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.817 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.817 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.817 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.817 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.817 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.817 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.817 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.817 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.817 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.817 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.817 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.817 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.817 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.818 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.818 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.818 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.818 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.818 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.818 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.818 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.818 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.818 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.818 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.818 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.818 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.818 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.818 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.819 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.819 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.819 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.819 INFO:teuthology.orchestra.run.smithi006.stdout: "password": { 2024-09-12T03:29:01.819 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "password", 2024-09-12T03:29:01.819 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.819 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.819 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.819 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.819 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.819 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.819 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.819 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:29:01.819 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.820 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.820 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.820 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.820 INFO:teuthology.orchestra.run.smithi006.stdout: "port": { 2024-09-12T03:29:01.820 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "port", 2024-09-12T03:29:01.820 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.820 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.820 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.820 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8086", 2024-09-12T03:29:01.820 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.820 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.820 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.820 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server port", 2024-09-12T03:29:01.821 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.821 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.821 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.821 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.821 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2024-09-12T03:29:01.821 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2024-09-12T03:29:01.821 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.821 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.821 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.821 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "false", 2024-09-12T03:29:01.821 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.821 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.821 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.821 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:29:01.822 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.822 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.822 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.822 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.822 INFO:teuthology.orchestra.run.smithi006.stdout: "threads": { 2024-09-12T03:29:01.822 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threads", 2024-09-12T03:29:01.822 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.822 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.822 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.822 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-09-12T03:29:01.822 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-09-12T03:29:01.822 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "32", 2024-09-12T03:29:01.822 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.822 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:29:01.823 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.823 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.823 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.823 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.823 INFO:teuthology.orchestra.run.smithi006.stdout: "username": { 2024-09-12T03:29:01.823 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "username", 2024-09-12T03:29:01.823 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.823 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.823 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.823 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.823 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.823 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.823 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.824 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:29:01.824 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.824 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.824 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.824 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.824 INFO:teuthology.orchestra.run.smithi006.stdout: "verify_ssl": { 2024-09-12T03:29:01.824 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "verify_ssl", 2024-09-12T03:29:01.824 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.824 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.824 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.824 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "true", 2024-09-12T03:29:01.824 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.825 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.825 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.825 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:29:01.825 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.825 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.825 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.825 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.825 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.825 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.825 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.825 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "insights", 2024-09-12T03:29:01.825 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.825 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.826 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.826 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.826 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.826 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.826 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.826 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.826 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.826 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.826 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.826 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.826 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.826 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.826 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.826 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.827 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.827 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.827 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.827 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.827 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.827 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.827 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.827 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.827 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.827 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.827 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.827 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.827 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.828 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.828 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.828 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.828 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.828 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.828 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.828 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.828 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.828 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.828 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.828 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.828 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.828 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "iostat", 2024-09-12T03:29:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.832 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.832 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.832 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.832 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.832 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.832 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.832 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.832 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.832 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.832 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.832 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.832 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.832 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.833 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.833 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.833 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.833 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.833 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.833 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.833 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.833 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.833 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.833 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.833 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.833 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.833 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.834 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.834 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.834 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.834 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.834 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.834 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.834 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.834 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.834 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.834 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.834 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.834 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.834 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.834 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.835 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.835 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.835 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.835 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.835 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.835 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.835 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.835 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.835 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.835 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.835 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.835 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.835 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.835 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.836 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.836 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.836 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.836 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.836 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.836 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.836 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.836 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.836 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.836 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.836 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.836 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.836 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.837 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.837 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.837 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.837 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "k8sevents", 2024-09-12T03:29:01.837 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.837 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.837 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.837 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_event_retention_days": { 2024-09-12T03:29:01.837 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:29:01.837 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.837 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.837 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.837 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7", 2024-09-12T03:29:01.837 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.838 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.838 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.838 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:29:01.838 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.838 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.838 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.838 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.838 INFO:teuthology.orchestra.run.smithi006.stdout: "config_check_secs": { 2024-09-12T03:29:01.838 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_check_secs", 2024-09-12T03:29:01.838 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.838 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.838 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.838 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:01.838 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "10", 2024-09-12T03:29:01.839 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.839 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.839 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:29:01.839 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.839 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.839 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.839 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.839 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.839 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.839 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.839 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.840 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.840 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.840 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.840 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.840 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.840 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.840 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.840 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.840 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.840 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.840 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.840 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.840 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.841 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.841 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.841 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.841 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.841 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.841 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.841 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.841 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.841 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.841 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.841 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.841 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.841 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.841 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.842 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.842 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.842 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.842 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.842 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.842 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.842 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.842 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.842 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.842 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.842 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.842 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.842 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.843 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.843 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.843 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.843 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.843 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.843 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.843 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.843 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.843 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.843 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.843 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.843 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.843 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.843 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.844 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.844 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.844 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.844 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.844 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.844 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.844 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.844 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.844 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.844 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.844 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.844 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.845 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.845 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.845 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.845 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "localpool", 2024-09-12T03:29:01.845 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.845 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.845 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.845 INFO:teuthology.orchestra.run.smithi006.stdout: "failure_domain": { 2024-09-12T03:29:01.845 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "failure_domain", 2024-09-12T03:29:01.845 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.845 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.845 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.845 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "host", 2024-09-12T03:29:01.845 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.846 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.846 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.846 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:29:01.846 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:29:01.846 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.846 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.846 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.846 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.846 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.846 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.846 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.846 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.846 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.846 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.847 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.847 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.847 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.847 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.847 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.847 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.847 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.847 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.847 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.847 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.847 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.847 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.847 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.848 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.848 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.848 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.848 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.848 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.848 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.848 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.848 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.848 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.848 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.848 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.848 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.848 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.848 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.849 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.849 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.849 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.849 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.849 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.849 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.849 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.849 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.849 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.849 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.849 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.849 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.849 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.850 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.850 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.850 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.850 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.850 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.850 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.850 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.850 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.850 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.850 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.850 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.850 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.850 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.850 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.851 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.851 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.851 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.851 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.851 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.851 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.851 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.851 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.851 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.851 INFO:teuthology.orchestra.run.smithi006.stdout: "min_size": { 2024-09-12T03:29:01.851 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_size", 2024-09-12T03:29:01.851 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.851 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.852 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.852 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.852 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.852 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.852 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.852 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:29:01.852 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:29:01.852 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.852 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.852 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.852 INFO:teuthology.orchestra.run.smithi006.stdout: "num_rep": { 2024-09-12T03:29:01.852 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "num_rep", 2024-09-12T03:29:01.852 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.853 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.853 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.853 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3", 2024-09-12T03:29:01.853 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.853 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.853 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.853 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:29:01.853 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.853 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.853 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.853 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.853 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_num": { 2024-09-12T03:29:01.853 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_num", 2024-09-12T03:29:01.854 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.854 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.854 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.854 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "128", 2024-09-12T03:29:01.854 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.854 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.854 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.854 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:29:01.854 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.854 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.854 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.854 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.854 INFO:teuthology.orchestra.run.smithi006.stdout: "prefix": { 2024-09-12T03:29:01.854 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prefix", 2024-09-12T03:29:01.855 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.855 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.855 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.855 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.855 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.855 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.855 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.855 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:29:01.855 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.855 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.855 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.855 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.855 INFO:teuthology.orchestra.run.smithi006.stdout: "subtree": { 2024-09-12T03:29:01.856 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "subtree", 2024-09-12T03:29:01.856 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.856 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.856 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.856 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "rack", 2024-09-12T03:29:01.856 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.856 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.856 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.856 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:29:01.856 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:29:01.856 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.856 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.856 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.856 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.857 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.857 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.857 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mds_autoscaler", 2024-09-12T03:29:01.857 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.857 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.857 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.857 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.857 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.857 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.857 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.857 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.857 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.857 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.858 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.858 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.858 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.858 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.858 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.858 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.858 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.858 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.858 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.858 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.858 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.858 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.858 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.859 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.859 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.859 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.859 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.859 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.859 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.859 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.859 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.859 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.859 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.859 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.859 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.859 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.859 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.860 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.860 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.860 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.860 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.860 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.860 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.860 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.860 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.860 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.860 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.860 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.860 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.860 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.860 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.861 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.861 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.861 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.861 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.861 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.861 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.861 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.861 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.861 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.861 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.861 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.861 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.861 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.862 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.862 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.862 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.862 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.862 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.862 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.862 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.862 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.862 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.862 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.862 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.862 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.862 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirroring", 2024-09-12T03:29:01.862 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.863 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.863 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.863 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.863 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.863 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.863 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.863 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.863 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.863 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.863 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.863 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.863 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.864 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.864 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.864 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.864 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.864 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.864 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.864 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.864 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.864 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.864 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.864 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.864 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.864 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.865 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.865 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.865 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.865 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.865 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.865 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.865 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.865 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.865 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.865 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.865 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.865 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.865 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.865 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.866 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.866 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.866 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.866 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.866 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.866 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.866 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.866 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.866 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.866 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.866 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.866 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.866 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.867 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.867 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.867 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.867 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.867 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.867 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.867 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.867 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.867 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.867 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.867 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.867 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.867 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.868 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.868 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.868 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.868 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.868 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.868 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.868 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.868 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.868 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.868 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.868 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "nfs", 2024-09-12T03:29:01.868 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.868 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.869 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.869 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.869 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.869 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.869 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.869 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.869 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.869 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.869 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.869 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.869 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.869 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.869 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.870 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.870 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.870 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.870 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.870 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.870 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.870 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.870 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.870 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.870 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.870 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.870 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.870 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.870 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.871 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.871 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.871 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.871 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.871 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.871 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.871 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.871 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.871 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.871 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.871 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.871 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.871 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.872 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.872 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.872 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.872 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.872 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.872 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.872 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.872 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.872 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.872 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.872 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.872 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.872 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.873 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.873 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.873 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.873 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.873 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.873 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.873 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.873 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.873 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.873 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.873 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.873 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.873 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.874 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.874 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.874 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.874 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.874 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.874 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.874 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.874 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.874 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2024-09-12T03:29:01.874 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.874 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.874 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.874 INFO:teuthology.orchestra.run.smithi006.stdout: "fail_fs": { 2024-09-12T03:29:01.874 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "fail_fs", 2024-09-12T03:29:01.875 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.875 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.875 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.875 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.875 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.875 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.875 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.875 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:29:01.875 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.875 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.875 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.875 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.875 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.875 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.876 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.876 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.876 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.876 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.876 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.876 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.876 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.876 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.876 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.876 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.876 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.876 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.876 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.877 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.877 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.877 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.877 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.877 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.877 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.877 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.877 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.877 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.877 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.877 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.877 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.877 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.877 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.878 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.878 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.878 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.878 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.878 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.878 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.878 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.878 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.878 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.878 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.878 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.878 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.878 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.879 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.879 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.879 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.879 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.879 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.879 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.879 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.879 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.879 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.879 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.879 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.879 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.879 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.879 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.880 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.880 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.880 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.880 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.880 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.880 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.880 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.880 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.880 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.880 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.880 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.880 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.880 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.880 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.881 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator": { 2024-09-12T03:29:01.881 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2024-09-12T03:29:01.881 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.881 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.881 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.881 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.881 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.881 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.881 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.881 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm", 2024-09-12T03:29:01.881 INFO:teuthology.orchestra.run.smithi006.stdout: "rook", 2024-09-12T03:29:01.881 INFO:teuthology.orchestra.run.smithi006.stdout: "test_orchestrator" 2024-09-12T03:29:01.881 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.882 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Orchestrator backend", 2024-09-12T03:29:01.882 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.882 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.882 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.882 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.882 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.882 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.882 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.882 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_perf_query", 2024-09-12T03:29:01.882 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.882 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.882 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.883 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.883 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.883 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.883 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.883 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.883 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.883 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.883 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.883 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.883 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.883 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.883 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.883 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.883 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.884 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.884 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.884 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.884 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.884 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.884 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.884 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.884 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.884 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.884 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.884 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.884 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.884 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.885 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.885 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.885 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.885 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.885 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.885 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.885 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.885 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.885 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.885 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.885 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.885 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.885 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.886 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.886 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.886 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.886 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.886 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.886 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.886 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.886 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.886 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.886 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.886 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.886 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.886 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.886 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.887 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.887 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.887 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.887 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.887 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.887 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.887 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.887 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.887 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.887 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.887 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.887 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.887 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.887 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.888 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.888 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.888 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.888 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.888 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.888 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_support", 2024-09-12T03:29:01.888 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.888 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.888 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.888 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.888 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.888 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.888 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.889 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.889 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.889 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.889 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.889 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.889 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.889 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.889 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.889 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.889 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.889 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.889 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.889 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.890 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.890 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.890 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.890 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.890 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.890 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.890 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.890 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.890 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.890 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.890 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.890 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.890 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.890 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.891 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.891 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.891 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.891 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.891 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.891 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.891 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.891 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.891 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.891 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.891 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.891 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.891 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.892 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.892 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.892 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.892 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.892 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.892 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.892 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.892 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.892 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.892 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.892 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.892 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.892 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.893 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.893 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.893 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.893 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.893 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.893 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.893 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.893 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.893 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.893 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.893 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.893 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.893 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.894 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.894 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.894 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.896 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_autoscaler", 2024-09-12T03:29:01.897 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.897 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.897 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.897 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.897 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.897 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.897 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.897 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.897 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.897 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.897 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.898 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.898 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.898 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.898 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.898 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.898 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.898 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.898 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.898 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.898 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.898 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.898 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.898 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.899 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.899 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.899 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.899 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.899 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.899 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.899 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.899 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.899 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.899 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.899 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.899 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.899 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.900 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.900 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.900 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.900 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.900 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.900 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.900 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.900 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.900 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.900 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.900 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.901 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.901 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.901 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.901 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.901 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.901 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.901 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.901 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.901 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.901 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.901 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.901 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.901 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.901 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.902 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.902 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.902 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.902 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.902 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.902 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.902 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.902 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.902 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.902 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.902 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.902 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-09-12T03:29:01.902 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-09-12T03:29:01.903 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:01.903 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.903 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.903 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-09-12T03:29:01.903 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.903 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.903 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.903 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.903 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.903 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.903 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.903 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.903 INFO:teuthology.orchestra.run.smithi006.stdout: "threshold": { 2024-09-12T03:29:01.904 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threshold", 2024-09-12T03:29:01.904 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-09-12T03:29:01.904 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.904 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.904 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2024-09-12T03:29:01.904 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1.0", 2024-09-12T03:29:01.904 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.904 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.904 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "scaling threshold", 2024-09-12T03:29:01.904 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:29:01.904 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.904 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.905 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.905 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.905 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.905 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.905 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "progress", 2024-09-12T03:29:01.905 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.905 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.905 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.905 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_pg_recovery_event": { 2024-09-12T03:29:01.905 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:29:01.905 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.905 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.905 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.906 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.906 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.906 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.906 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.906 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:29:01.906 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.906 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.906 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.906 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.906 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2024-09-12T03:29:01.906 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2024-09-12T03:29:01.906 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.906 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.907 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.907 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.907 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.907 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.907 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.907 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.907 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.907 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.907 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.907 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.907 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.907 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.907 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.907 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.908 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.908 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.908 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.908 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.908 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.908 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.908 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.908 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.908 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.908 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.908 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.908 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.908 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.909 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.909 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.909 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.909 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.909 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.909 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.909 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.909 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.909 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.909 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.909 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.909 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.909 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.910 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.910 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.910 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.910 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.910 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.910 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.910 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.910 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.910 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.910 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.910 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.910 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.910 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.910 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.911 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.911 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.911 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.911 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.911 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.911 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.911 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.911 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.911 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.911 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.911 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.911 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.911 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.912 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.912 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.912 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.912 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.912 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.912 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.912 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.912 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.912 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.912 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.912 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.912 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.912 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.913 INFO:teuthology.orchestra.run.smithi006.stdout: "max_completed_events": { 2024-09-12T03:29:01.913 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_completed_events", 2024-09-12T03:29:01.913 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.913 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.913 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.913 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "50", 2024-09-12T03:29:01.913 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.913 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.913 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.913 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:29:01.913 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.913 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.913 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.913 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.914 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-09-12T03:29:01.914 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-09-12T03:29:01.914 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:01.914 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.914 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.914 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-09-12T03:29:01.914 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.914 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.914 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.914 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:29:01.914 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.915 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.915 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.915 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.915 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.915 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.915 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.915 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus", 2024-09-12T03:29:01.915 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.915 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.915 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.915 INFO:teuthology.orchestra.run.smithi006.stdout: "cache": { 2024-09-12T03:29:01.915 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cache", 2024-09-12T03:29:01.916 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.916 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.916 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.916 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.916 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.916 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.916 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.916 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.916 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.916 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.916 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.916 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.916 INFO:teuthology.orchestra.run.smithi006.stdout: "exclude_perf_counters": { 2024-09-12T03:29:01.917 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "exclude_perf_counters", 2024-09-12T03:29:01.917 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.917 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.917 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.917 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.917 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.917 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.917 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.917 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:29:01.917 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:29:01.917 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.917 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.917 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.918 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.918 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.918 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.918 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.918 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.918 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.918 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.918 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.918 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.918 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.918 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.918 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.918 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.919 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.919 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.919 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.919 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.919 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.919 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.919 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.919 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.919 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.919 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.919 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.919 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.920 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.920 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.920 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.920 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.920 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.920 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.920 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.920 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.920 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.920 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.920 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.920 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.920 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.921 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.921 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.921 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.921 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.921 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.921 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.921 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.921 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.921 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.921 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.921 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.921 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.922 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.922 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.922 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.922 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.922 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.922 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.922 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.922 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.922 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.922 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.922 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.922 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.922 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.922 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.923 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.923 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.923 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.923 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.923 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.923 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.923 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools": { 2024-09-12T03:29:01.923 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools", 2024-09-12T03:29:01.923 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.923 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.923 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.923 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.923 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.924 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.924 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.924 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.924 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.924 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.924 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.924 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.924 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:29:01.924 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:29:01.924 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.924 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.924 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.924 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "300", 2024-09-12T03:29:01.925 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.925 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.925 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.925 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.925 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.925 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.925 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.925 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.925 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_interval": { 2024-09-12T03:29:01.925 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_interval", 2024-09-12T03:29:01.925 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-09-12T03:29:01.925 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.925 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.926 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15.0", 2024-09-12T03:29:01.926 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.926 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.926 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.926 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.926 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.926 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.926 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.926 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.926 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-09-12T03:29:01.926 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-09-12T03:29:01.926 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.926 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.926 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.927 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2024-09-12T03:29:01.927 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.927 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.927 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.927 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:29:01.927 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.927 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.927 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.927 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.927 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-09-12T03:29:01.927 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-09-12T03:29:01.927 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.927 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.927 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.928 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "9283", 2024-09-12T03:29:01.928 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.928 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.928 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.928 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:29:01.928 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.928 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.928 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.928 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.928 INFO:teuthology.orchestra.run.smithi006.stdout: "stale_cache_strategy": { 2024-09-12T03:29:01.928 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stale_cache_strategy", 2024-09-12T03:29:01.928 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.929 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.929 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.929 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "log", 2024-09-12T03:29:01.929 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.929 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.929 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.929 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.929 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.929 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.929 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.929 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.929 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2024-09-12T03:29:01.929 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2024-09-12T03:29:01.930 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.930 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.930 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.930 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "default", 2024-09-12T03:29:01.930 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.930 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.930 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.930 INFO:teuthology.orchestra.run.smithi006.stdout: "default", 2024-09-12T03:29:01.930 INFO:teuthology.orchestra.run.smithi006.stdout: "error" 2024-09-12T03:29:01.930 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.930 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.930 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.930 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.931 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.931 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.931 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2024-09-12T03:29:01.931 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2024-09-12T03:29:01.931 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.931 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.931 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.931 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2024-09-12T03:29:01.931 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2024-09-12T03:29:01.931 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2024-09-12T03:29:01.931 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.931 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.931 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.931 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.932 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.932 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.932 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.932 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.932 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.932 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_support", 2024-09-12T03:29:01.932 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.932 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.932 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.932 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.932 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.932 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.932 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.933 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.933 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.933 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.933 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.933 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.933 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.933 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.933 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.933 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.933 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.933 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.933 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.933 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.934 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.934 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.934 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.934 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.934 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.934 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.934 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.934 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.934 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.934 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.934 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.934 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.934 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.934 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.935 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.935 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.935 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.935 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.935 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.935 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.935 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.935 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.935 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.935 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.935 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.935 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.936 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.936 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.936 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.936 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.936 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.936 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.936 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.936 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.936 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.936 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.936 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.936 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.936 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.936 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.937 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.937 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.937 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.937 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.937 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.937 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.937 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.937 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.937 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.937 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.937 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.937 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.937 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_snap_create": { 2024-09-12T03:29:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:29:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.939 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.939 INFO:teuthology.orchestra.run.smithi006.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:29:01.939 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:29:01.939 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.939 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.939 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.939 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.939 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.939 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.939 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.939 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.939 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.939 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.939 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.940 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.940 INFO:teuthology.orchestra.run.smithi006.stdout: "trash_purge_schedule": { 2024-09-12T03:29:01.940 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "trash_purge_schedule", 2024-09-12T03:29:01.940 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.940 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.940 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.940 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.940 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.940 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.940 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.940 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.940 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.940 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "restful", 2024-09-12T03:29:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_auth": { 2024-09-12T03:29:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_auth", 2024-09-12T03:29:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.942 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.942 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.942 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.942 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.942 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.942 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.942 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.942 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.942 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.942 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.942 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2024-09-12T03:29:01.942 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2024-09-12T03:29:01.942 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.942 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.943 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.943 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.943 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.943 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.943 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.943 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.943 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.943 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.943 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.944 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.944 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.944 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.944 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.944 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.944 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.944 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.944 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.944 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.944 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.944 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.944 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.945 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.945 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.945 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.945 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.945 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.945 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.945 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.945 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.945 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.945 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.945 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.945 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.945 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.946 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.946 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.946 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.946 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.946 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.946 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.946 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.946 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.946 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.946 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.946 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.946 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.946 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.947 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.947 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.947 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.947 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.947 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.947 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.947 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.947 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.947 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.947 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.947 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.947 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.947 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.947 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.948 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.948 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.948 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.948 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.948 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.948 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.948 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.948 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.948 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.948 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.948 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.948 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.948 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.949 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.949 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.949 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.949 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.949 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.949 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.949 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-09-12T03:29:01.949 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-09-12T03:29:01.949 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.949 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.949 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.949 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.949 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.950 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.950 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.950 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.950 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.950 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.950 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.950 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.950 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-09-12T03:29:01.950 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-09-12T03:29:01.950 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.950 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.950 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.950 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.951 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.951 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.951 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.951 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.951 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.951 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.951 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.951 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.951 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.951 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.951 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.951 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rgw", 2024-09-12T03:29:01.951 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.951 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.952 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.952 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.952 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.952 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.952 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.952 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.952 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.952 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.952 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.952 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.952 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.952 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.952 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.953 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.953 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.953 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.953 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.953 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.953 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.953 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.953 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.953 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.953 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.953 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.953 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.953 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.953 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.954 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.954 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.954 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.954 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.954 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.954 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.954 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.954 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.954 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.954 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.954 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.954 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.954 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.955 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.955 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.955 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.955 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.955 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.955 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.955 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.955 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.955 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.955 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.955 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.955 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.955 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.955 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.956 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.956 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.956 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.956 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.956 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.956 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.956 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.956 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.956 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.956 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.956 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.956 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.956 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.957 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.957 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.957 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.957 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.957 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.957 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.957 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.957 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rook", 2024-09-12T03:29:01.957 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.957 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.957 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.957 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.957 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.958 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.958 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.958 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.958 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.958 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.958 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.958 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.958 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.958 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.958 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.958 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.958 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.959 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.959 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.959 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.959 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.959 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.959 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.959 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.959 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.959 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.959 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.959 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.959 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.959 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.960 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.960 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.960 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.960 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.960 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.960 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.960 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.960 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.960 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.960 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.960 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.960 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.960 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.961 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.961 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.961 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.961 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.961 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.961 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.961 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.961 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.961 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.961 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.961 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.961 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.961 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.962 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.962 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.962 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.962 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.962 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.962 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.962 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.962 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.962 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.962 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.962 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.962 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.962 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.962 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.963 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.963 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.963 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.963 INFO:teuthology.orchestra.run.smithi006.stdout: "storage_class": { 2024-09-12T03:29:01.963 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "storage_class", 2024-09-12T03:29:01.963 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.963 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.963 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.963 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "local", 2024-09-12T03:29:01.963 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.963 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.963 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.963 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:29:01.964 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.964 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.964 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.964 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.964 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.964 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.964 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.964 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "selftest", 2024-09-12T03:29:01.964 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.964 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.964 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.964 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.964 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.965 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.965 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.965 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.965 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.965 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.965 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.965 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.965 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.965 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.965 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.965 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.965 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.965 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.966 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.966 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.966 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.966 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.966 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.966 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.966 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.966 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.966 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.966 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.966 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.966 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.966 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.966 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.967 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.967 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.967 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.967 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.967 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.967 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.967 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.967 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.967 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.967 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.967 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.967 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.967 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.968 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.968 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.968 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.968 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.968 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.968 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.968 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.968 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.968 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.968 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.968 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.968 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.970 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.970 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.970 INFO:teuthology.orchestra.run.smithi006.stdout: "roption1": { 2024-09-12T03:29:01.970 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption1", 2024-09-12T03:29:01.970 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.970 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.970 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.970 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.970 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.970 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.970 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.970 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.970 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.970 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.971 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.971 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.971 INFO:teuthology.orchestra.run.smithi006.stdout: "roption2": { 2024-09-12T03:29:01.971 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption2", 2024-09-12T03:29:01.971 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.971 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.971 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.971 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "xyz", 2024-09-12T03:29:01.971 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.971 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.971 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.971 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.971 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.971 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.972 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.972 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.972 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption1": { 2024-09-12T03:29:01.972 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption1", 2024-09-12T03:29:01.972 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.972 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.972 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.972 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.972 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.972 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.972 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.972 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.972 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.973 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.973 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.973 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.973 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption2": { 2024-09-12T03:29:01.973 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption2", 2024-09-12T03:29:01.973 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.973 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.973 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.973 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.973 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.973 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.973 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.974 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.974 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.974 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.974 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.974 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.974 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption3": { 2024-09-12T03:29:01.974 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption3", 2024-09-12T03:29:01.974 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-09-12T03:29:01.974 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.974 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.974 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.974 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.974 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.975 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.975 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.975 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.975 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.975 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.975 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.975 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption4": { 2024-09-12T03:29:01.975 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption4", 2024-09-12T03:29:01.975 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.975 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.975 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.975 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.975 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.976 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.976 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.976 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.976 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.976 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.976 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.976 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.976 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption5": { 2024-09-12T03:29:01.976 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption5", 2024-09-12T03:29:01.976 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.976 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.976 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.976 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.977 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.977 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.977 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.977 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.977 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.977 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.977 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.977 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.977 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption6": { 2024-09-12T03:29:01.977 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption6", 2024-09-12T03:29:01.977 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.977 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.977 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.978 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:01.978 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.978 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.978 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.978 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.978 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.978 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.978 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.978 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.978 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption7": { 2024-09-12T03:29:01.978 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption7", 2024-09-12T03:29:01.978 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:01.978 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.978 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.979 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.979 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-09-12T03:29:01.979 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "42", 2024-09-12T03:29:01.979 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.979 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.979 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.979 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.979 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.979 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.979 INFO:teuthology.orchestra.run.smithi006.stdout: "testkey": { 2024-09-12T03:29:01.979 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testkey", 2024-09-12T03:29:01.979 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.979 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.980 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.980 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.980 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.980 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.980 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.980 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.980 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.980 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.980 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.980 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.980 INFO:teuthology.orchestra.run.smithi006.stdout: "testlkey": { 2024-09-12T03:29:01.980 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testlkey", 2024-09-12T03:29:01.980 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.981 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.981 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.981 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.981 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.981 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.981 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.981 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.981 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.981 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.981 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.981 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.981 INFO:teuthology.orchestra.run.smithi006.stdout: "testnewline": { 2024-09-12T03:29:01.981 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testnewline", 2024-09-12T03:29:01.981 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.982 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.982 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:01.982 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.982 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.982 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.982 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.982 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.982 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.982 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.982 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.982 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.982 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.982 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.982 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.983 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snap_schedule", 2024-09-12T03:29:01.983 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.983 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.983 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.983 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_m_granularity": { 2024-09-12T03:29:01.983 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_m_granularity", 2024-09-12T03:29:01.983 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.983 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.983 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.983 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.983 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.983 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.983 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.984 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:29:01.984 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.984 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.984 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.984 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.984 INFO:teuthology.orchestra.run.smithi006.stdout: "dump_on_update": { 2024-09-12T03:29:01.984 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dump_on_update", 2024-09-12T03:29:01.984 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.984 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.984 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.984 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.984 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.984 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.985 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.985 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:29:01.985 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.985 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.985 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.985 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.985 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.985 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.985 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.985 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.985 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.985 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.985 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.985 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.986 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.986 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.986 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.986 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.986 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.986 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.986 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.986 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.986 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.986 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.986 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.986 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.986 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.987 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.987 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.987 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.987 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.987 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.987 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.987 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.987 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.987 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.987 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.987 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.987 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.987 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.987 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.988 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.988 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.988 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.988 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.988 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.988 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.988 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.988 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.988 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.988 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.988 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.989 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.989 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.989 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.989 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.989 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.989 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.989 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.989 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.989 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.989 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.989 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.989 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.989 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.990 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.990 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.990 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.990 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.990 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.990 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.990 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.990 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.990 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.990 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.990 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.990 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.990 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.990 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.991 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stats", 2024-09-12T03:29:01.991 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.991 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.991 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.991 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.991 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.991 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.991 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.991 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.991 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.991 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.991 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.991 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.992 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.992 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.992 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.992 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.992 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.992 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.992 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.992 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.992 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.992 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.992 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.992 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.992 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.993 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.993 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.993 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.993 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.993 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.993 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.993 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.993 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.993 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.993 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.993 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.993 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.993 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.993 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.994 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.994 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.994 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.994 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.994 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:01.994 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.994 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.994 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.994 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.994 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.994 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.994 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.994 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.995 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.995 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.995 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.995 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.995 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.995 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.995 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.995 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:01.995 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:01.995 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.995 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.995 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.995 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.996 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.996 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.996 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.996 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.996 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.996 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.996 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.996 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.996 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:01.996 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.996 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:01.996 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "status", 2024-09-12T03:29:01.996 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:01.996 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:01.997 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:01.997 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:01.997 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:01.997 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.997 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.997 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.997 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:01.997 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.997 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.997 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:01.997 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:01.997 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:01.997 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:01.998 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:01.998 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:01.998 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:01.998 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:01.998 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.998 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.998 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.998 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.998 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.998 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:01.998 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:01.998 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:01.998 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:01.998 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:01.999 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:01.999 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:01.999 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:01.999 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:01.999 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:01.999 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:01.999 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:01.999 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:01.999 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:01.999 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:01.999 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:01.999 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:01.999 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.000 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.000 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.000 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.000 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.000 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.000 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.000 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.000 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.000 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.000 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.000 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.000 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.000 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.000 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.001 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.001 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.001 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.001 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.001 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.001 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.001 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.001 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.001 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.001 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.001 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.001 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.001 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.001 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.002 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.002 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.002 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.002 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.002 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.002 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.002 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telegraf", 2024-09-12T03:29:02.002 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.002 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.002 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.002 INFO:teuthology.orchestra.run.smithi006.stdout: "address": { 2024-09-12T03:29:02.002 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "address", 2024-09-12T03:29:02.002 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.003 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.003 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.003 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:29:02.003 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.003 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.003 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.003 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.003 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.003 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.003 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.003 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.004 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-09-12T03:29:02.004 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-09-12T03:29:02.004 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.004 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.004 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.004 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15", 2024-09-12T03:29:02.004 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.004 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.004 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.004 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.004 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.004 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.004 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.006 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.006 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.006 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.006 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.006 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.006 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.006 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.006 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.006 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.006 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.006 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.006 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.006 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.007 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.007 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.007 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.007 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.007 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.007 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.007 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.007 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.007 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.007 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.007 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.007 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.007 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.008 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.008 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.008 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.008 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.008 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.008 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.008 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.008 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.008 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.008 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.008 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.008 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.008 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.009 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.009 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.009 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.009 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.009 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.009 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.009 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.009 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.009 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.009 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.009 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.009 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.009 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.009 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.010 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.010 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.010 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.010 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.010 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.010 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.010 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.010 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telemetry", 2024-09-12T03:29:02.010 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.010 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.010 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.010 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_basic": { 2024-09-12T03:29:02.010 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_basic", 2024-09-12T03:29:02.011 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.011 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.011 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.011 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.011 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.011 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.011 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.011 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:29:02.011 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.011 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.011 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.011 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.011 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_crash": { 2024-09-12T03:29:02.011 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_crash", 2024-09-12T03:29:02.012 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.012 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.012 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.012 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.012 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.012 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.012 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.012 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:29:02.012 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.012 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.012 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.012 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.013 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_device": { 2024-09-12T03:29:02.013 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_device", 2024-09-12T03:29:02.013 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.013 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.013 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.013 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.013 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.013 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.013 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.013 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:29:02.013 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.013 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.013 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.014 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.014 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_ident": { 2024-09-12T03:29:02.014 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_ident", 2024-09-12T03:29:02.014 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.014 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.014 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.014 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.014 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.014 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.014 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.014 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:29:02.014 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.014 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.014 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.015 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.015 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_perf": { 2024-09-12T03:29:02.015 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_perf", 2024-09-12T03:29:02.015 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.015 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.015 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.015 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.015 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.015 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.015 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.015 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:29:02.015 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.015 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.015 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.016 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.016 INFO:teuthology.orchestra.run.smithi006.stdout: "contact": { 2024-09-12T03:29:02.016 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "contact", 2024-09-12T03:29:02.016 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.016 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.016 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.016 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.016 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.016 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.016 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.016 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.016 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.016 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.017 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.017 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.017 INFO:teuthology.orchestra.run.smithi006.stdout: "description": { 2024-09-12T03:29:02.017 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "description", 2024-09-12T03:29:02.017 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.017 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.017 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.017 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.017 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.017 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.017 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.017 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.018 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.018 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.018 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.018 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.018 INFO:teuthology.orchestra.run.smithi006.stdout: "device_url": { 2024-09-12T03:29:02.018 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_url", 2024-09-12T03:29:02.018 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.018 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.018 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.018 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:29:02.018 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.018 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.018 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.018 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.019 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.019 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.019 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.019 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.019 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2024-09-12T03:29:02.019 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2024-09-12T03:29:02.019 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.019 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.019 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.019 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.019 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.019 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.019 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.020 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.020 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.020 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.020 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.020 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.020 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-09-12T03:29:02.020 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-09-12T03:29:02.020 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.020 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.020 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.020 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "24", 2024-09-12T03:29:02.020 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "8", 2024-09-12T03:29:02.020 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.021 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.021 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.021 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.021 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.021 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.021 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.021 INFO:teuthology.orchestra.run.smithi006.stdout: "last_opt_revision": { 2024-09-12T03:29:02.021 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "last_opt_revision", 2024-09-12T03:29:02.021 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.021 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.021 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.021 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1", 2024-09-12T03:29:02.021 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.022 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.022 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.022 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.022 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.022 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.022 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.022 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.022 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard": { 2024-09-12T03:29:02.022 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard", 2024-09-12T03:29:02.022 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.022 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.022 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.022 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.022 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.023 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.023 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.023 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.023 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.023 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.023 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.023 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.023 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard_description": { 2024-09-12T03:29:02.023 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard_description", 2024-09-12T03:29:02.023 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.023 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.023 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.023 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.024 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.024 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.024 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.024 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.024 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.024 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.024 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.024 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.024 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.024 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.024 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.024 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.024 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.025 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.025 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.025 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.025 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.025 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.025 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.025 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.025 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.025 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.025 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.025 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.025 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.025 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.025 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.026 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.026 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.026 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.026 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.026 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.026 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.026 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.026 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.026 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.026 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.026 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.026 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.026 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.026 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.027 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.027 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.027 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.027 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.027 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.027 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.027 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.027 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.027 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.027 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.027 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.027 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.027 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.028 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.028 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.028 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.028 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.028 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.028 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.028 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.028 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.028 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.028 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.028 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.028 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.028 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.029 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.029 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.029 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.029 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.029 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.029 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.029 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.029 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.029 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.029 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.029 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.029 INFO:teuthology.orchestra.run.smithi006.stdout: "organization": { 2024-09-12T03:29:02.029 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "organization", 2024-09-12T03:29:02.029 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.030 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.030 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.030 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.030 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.030 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.030 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.030 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.030 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.030 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.030 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.030 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.030 INFO:teuthology.orchestra.run.smithi006.stdout: "proxy": { 2024-09-12T03:29:02.030 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "proxy", 2024-09-12T03:29:02.030 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.031 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.031 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.031 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.031 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.031 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.031 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.031 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.031 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.031 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.031 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.031 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.031 INFO:teuthology.orchestra.run.smithi006.stdout: "url": { 2024-09-12T03:29:02.032 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url", 2024-09-12T03:29:02.032 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.032 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.032 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.032 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:29:02.032 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.032 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.032 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.032 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.032 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.032 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.032 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.032 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.033 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.033 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.033 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.033 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "test_orchestrator", 2024-09-12T03:29:02.033 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.033 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.033 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.033 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.033 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.033 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.033 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.033 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.033 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.034 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.034 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.034 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.034 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.034 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.034 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.034 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.034 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.034 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.034 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.034 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.034 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.034 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.035 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.035 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.035 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.035 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.035 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.035 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.035 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.035 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.035 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.035 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.035 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.035 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.035 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.035 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.036 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.036 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.036 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.036 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.036 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.036 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.036 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.036 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.036 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.036 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.036 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.036 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.037 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.037 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.037 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.037 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.037 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.037 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.037 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.037 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.037 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.037 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.037 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.037 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.037 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.037 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.038 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.038 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.038 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.038 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.038 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.038 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.038 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.038 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.038 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.038 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.038 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.038 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.038 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.039 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.039 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "volumes", 2024-09-12T03:29:02.039 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.039 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.039 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.039 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.039 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.039 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.039 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.039 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.039 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.039 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.039 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.039 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.040 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.040 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.040 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.040 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.040 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.040 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.040 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.040 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.040 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.040 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.040 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.040 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.040 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.041 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.041 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.041 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.041 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.041 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.041 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.041 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.041 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.041 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.041 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.041 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.041 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.041 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.042 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.042 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.042 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.042 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.042 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.042 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.042 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.042 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.042 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.042 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.042 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.042 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.042 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.042 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.043 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.043 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.043 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.043 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.043 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.043 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.043 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.043 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.043 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.043 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.043 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.043 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.043 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.044 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.044 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.044 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.044 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.044 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.044 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.044 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.044 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.044 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_clones": { 2024-09-12T03:29:02.044 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_clones", 2024-09-12T03:29:02.044 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.044 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.044 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.045 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4", 2024-09-12T03:29:02.045 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.045 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.045 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.045 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:29:02.045 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.045 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.045 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.045 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.045 INFO:teuthology.orchestra.run.smithi006.stdout: "periodic_async_work": { 2024-09-12T03:29:02.045 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "periodic_async_work", 2024-09-12T03:29:02.045 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.045 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.045 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.046 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.046 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.046 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.046 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.046 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Periodically check for async work", 2024-09-12T03:29:02.046 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.046 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.046 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.046 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.046 INFO:teuthology.orchestra.run.smithi006.stdout: "snapshot_clone_delay": { 2024-09-12T03:29:02.046 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:29:02.046 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.047 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.047 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.047 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-09-12T03:29:02.047 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.047 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.047 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.047 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:29:02.047 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.047 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.047 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.047 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.049 INFO:teuthology.orchestra.run.smithi006.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:29:02.049 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:29:02.050 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.050 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.050 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.050 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.050 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.050 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.050 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.050 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:29:02.050 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.050 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.050 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.050 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.050 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.051 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.051 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.051 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix", 2024-09-12T03:29:02.051 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.051 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.051 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.051 INFO:teuthology.orchestra.run.smithi006.stdout: "discovery_interval": { 2024-09-12T03:29:02.051 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "discovery_interval", 2024-09-12T03:29:02.051 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-09-12T03:29:02.051 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.051 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.051 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "100", 2024-09-12T03:29:02.051 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.052 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.052 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.052 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.052 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.052 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.052 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.052 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.052 INFO:teuthology.orchestra.run.smithi006.stdout: "identifier": { 2024-09-12T03:29:02.052 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "identifier", 2024-09-12T03:29:02.052 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.052 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.052 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.052 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.053 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.053 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.053 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.053 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.053 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.053 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.053 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.053 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.053 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-09-12T03:29:02.053 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-09-12T03:29:02.053 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.053 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.053 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.053 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-09-12T03:29:02.054 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.054 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.054 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.054 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.054 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.054 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.054 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.054 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.054 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.054 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.054 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.054 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.054 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.055 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.055 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.055 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.055 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.055 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.055 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.055 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.055 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.055 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.055 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.055 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.055 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.055 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.056 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.056 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.056 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.056 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.056 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.056 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.056 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.056 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.056 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.056 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.056 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.056 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.056 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.057 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.057 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.057 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.057 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.057 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.057 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.057 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.057 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.057 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.057 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.057 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.057 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.057 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.057 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.058 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.058 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.058 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.058 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.058 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.058 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.058 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.058 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.058 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.058 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.058 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.058 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.058 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.059 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.059 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.059 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.059 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.059 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.059 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.059 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.059 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.059 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.059 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.059 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.059 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.059 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_host": { 2024-09-12T03:29:02.059 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_host", 2024-09-12T03:29:02.060 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.060 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.060 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.060 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.060 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.060 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.060 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.060 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.060 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.061 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.061 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.061 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.061 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_port": { 2024-09-12T03:29:02.061 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_port", 2024-09-12T03:29:02.061 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.061 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.061 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.061 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10051", 2024-09-12T03:29:02.061 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.061 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.061 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.061 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.062 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.062 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.062 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.062 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.062 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_sender": { 2024-09-12T03:29:02.062 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_sender", 2024-09-12T03:29:02.062 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.062 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.062 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.062 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:29:02.063 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.063 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.063 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.063 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.063 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.063 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.063 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.063 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.063 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.063 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.063 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-09-12T03:29:02.064 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.064 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.064 INFO:teuthology.orchestra.run.smithi006.stdout: "gid": 24119, 2024-09-12T03:29:02.064 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "y", 2024-09-12T03:29:02.064 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_features": 4540701547738038271, 2024-09-12T03:29:02.064 INFO:teuthology.orchestra.run.smithi006.stdout: "available_modules": [ 2024-09-12T03:29:02.064 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.064 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "alerts", 2024-09-12T03:29:02.064 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.064 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.065 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.065 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-09-12T03:29:02.065 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-09-12T03:29:02.065 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.065 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.065 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.065 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-09-12T03:29:02.065 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.065 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.065 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.065 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:29:02.066 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.066 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.066 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.066 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.066 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.066 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.066 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.066 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.066 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.066 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.066 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.066 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.066 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.066 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.067 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.067 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.067 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.067 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.067 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.067 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.067 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.067 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.067 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.067 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.067 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.067 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.067 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.068 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.068 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.068 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.068 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.068 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.068 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.068 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.068 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.068 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.068 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.068 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.068 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.069 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.069 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.069 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.069 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.069 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.069 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.069 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.069 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.069 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.069 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.070 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.070 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.070 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.070 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.070 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.070 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.070 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.070 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.070 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.070 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.070 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.071 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.071 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.071 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.071 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.071 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.071 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.071 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.071 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.071 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.071 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.071 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.072 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.072 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.072 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.072 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_destination": { 2024-09-12T03:29:02.072 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_destination", 2024-09-12T03:29:02.072 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.072 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.072 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.072 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.073 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.073 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.073 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.073 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:29:02.073 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.073 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.073 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.073 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.073 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_from_name": { 2024-09-12T03:29:02.073 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_from_name", 2024-09-12T03:29:02.074 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.074 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.074 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.074 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "Ceph", 2024-09-12T03:29:02.074 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.074 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.074 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.074 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email From: name", 2024-09-12T03:29:02.074 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.074 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.075 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.075 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.075 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_host": { 2024-09-12T03:29:02.075 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_host", 2024-09-12T03:29:02.075 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.075 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.075 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.075 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.075 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.075 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.075 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.075 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP server", 2024-09-12T03:29:02.076 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.076 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.076 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.076 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.076 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_password": { 2024-09-12T03:29:02.076 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_password", 2024-09-12T03:29:02.076 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.076 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.076 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.076 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.076 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.076 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.076 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.077 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Password to authenticate with", 2024-09-12T03:29:02.077 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.077 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.077 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.077 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.077 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_port": { 2024-09-12T03:29:02.077 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_port", 2024-09-12T03:29:02.077 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.077 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.077 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.077 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "465", 2024-09-12T03:29:02.077 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.077 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.077 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.078 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP port", 2024-09-12T03:29:02.078 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.078 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.078 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.078 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.078 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_sender": { 2024-09-12T03:29:02.078 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_sender", 2024-09-12T03:29:02.078 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.078 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.078 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.078 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.078 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.078 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.079 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.079 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:29:02.079 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.079 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.079 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.079 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.079 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_ssl": { 2024-09-12T03:29:02.079 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_ssl", 2024-09-12T03:29:02.079 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.079 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.079 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.079 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.079 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.080 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.080 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.080 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:29:02.080 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.080 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.080 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.080 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.080 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_user": { 2024-09-12T03:29:02.080 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_user", 2024-09-12T03:29:02.080 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.080 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.080 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.081 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.081 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.081 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.081 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.081 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "User to authenticate as", 2024-09-12T03:29:02.081 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.081 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.081 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.081 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.081 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.081 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.081 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.081 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "balancer", 2024-09-12T03:29:02.082 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.082 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.082 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.082 INFO:teuthology.orchestra.run.smithi006.stdout: "active": { 2024-09-12T03:29:02.082 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "active", 2024-09-12T03:29:02.082 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.082 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.082 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.082 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.082 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.082 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.082 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.082 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:29:02.083 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.083 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.083 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.083 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.083 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_time": { 2024-09-12T03:29:02.083 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_time", 2024-09-12T03:29:02.083 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.083 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.083 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.083 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0000", 2024-09-12T03:29:02.083 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.083 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.083 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.084 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:29:02.084 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:29:02.084 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.084 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.084 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.084 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_weekday": { 2024-09-12T03:29:02.084 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_weekday", 2024-09-12T03:29:02.084 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-09-12T03:29:02.084 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.084 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.084 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-09-12T03:29:02.084 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2024-09-12T03:29:02.084 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2024-09-12T03:29:02.085 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.085 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:29:02.085 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:29:02.085 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.085 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.085 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.085 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_max_iterations": { 2024-09-12T03:29:02.085 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:29:02.085 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-09-12T03:29:02.085 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.085 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.085 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "25", 2024-09-12T03:29:02.085 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-09-12T03:29:02.086 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "250", 2024-09-12T03:29:02.086 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.086 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:29:02.086 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.086 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.086 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.086 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.086 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_metrics": { 2024-09-12T03:29:02.086 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_metrics", 2024-09-12T03:29:02.086 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.086 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.086 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.086 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:29:02.086 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.087 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.087 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.087 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:29:02.087 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:29:02.087 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.087 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.087 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.087 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_step": { 2024-09-12T03:29:02.087 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_step", 2024-09-12T03:29:02.087 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-09-12T03:29:02.087 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.087 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.087 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.5", 2024-09-12T03:29:02.088 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0.001", 2024-09-12T03:29:02.088 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "0.999", 2024-09-12T03:29:02.088 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.088 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:29:02.088 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:29:02.088 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.088 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.088 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.088 INFO:teuthology.orchestra.run.smithi006.stdout: "end_time": { 2024-09-12T03:29:02.088 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_time", 2024-09-12T03:29:02.088 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.088 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.088 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.089 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2359", 2024-09-12T03:29:02.089 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.089 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.089 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.089 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:29:02.089 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:29:02.089 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.089 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.089 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.089 INFO:teuthology.orchestra.run.smithi006.stdout: "end_weekday": { 2024-09-12T03:29:02.089 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_weekday", 2024-09-12T03:29:02.089 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-09-12T03:29:02.089 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.090 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.090 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-09-12T03:29:02.090 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2024-09-12T03:29:02.090 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2024-09-12T03:29:02.090 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.090 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:29:02.090 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:29:02.090 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.090 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.090 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.090 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.090 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.090 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.091 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.091 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.091 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.091 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.091 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.091 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.091 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.091 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.091 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.091 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.091 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.091 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.091 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.091 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.092 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.092 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.092 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.092 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.092 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.092 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.092 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.092 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.092 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.092 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.092 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.092 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.092 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.093 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.093 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.093 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.093 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.093 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.093 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.093 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.093 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.093 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.093 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.093 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.093 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.094 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.094 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.094 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.094 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.094 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.094 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.094 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.094 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.094 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.094 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.094 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.094 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.094 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.095 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.095 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.095 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.095 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.095 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.095 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.095 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.095 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.095 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.095 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.095 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.095 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.095 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.096 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.096 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.096 INFO:teuthology.orchestra.run.smithi006.stdout: "min_score": { 2024-09-12T03:29:02.096 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_score", 2024-09-12T03:29:02.096 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-09-12T03:29:02.096 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.096 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.096 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-09-12T03:29:02.096 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.097 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.097 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.097 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:29:02.097 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.097 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.097 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.097 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.097 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2024-09-12T03:29:02.097 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2024-09-12T03:29:02.097 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.097 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.097 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.097 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "upmap", 2024-09-12T03:29:02.098 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.098 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.098 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.098 INFO:teuthology.orchestra.run.smithi006.stdout: "crush-compat", 2024-09-12T03:29:02.098 INFO:teuthology.orchestra.run.smithi006.stdout: "none", 2024-09-12T03:29:02.098 INFO:teuthology.orchestra.run.smithi006.stdout: "read", 2024-09-12T03:29:02.098 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap", 2024-09-12T03:29:02.098 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap-read" 2024-09-12T03:29:02.098 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.099 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Balancer mode", 2024-09-12T03:29:02.099 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.099 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.099 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.099 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.099 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_ids": { 2024-09-12T03:29:02.099 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_ids", 2024-09-12T03:29:02.099 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.099 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.099 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.099 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.099 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.100 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.100 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.100 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:29:02.100 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.100 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.100 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.100 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.100 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-09-12T03:29:02.100 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-09-12T03:29:02.100 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.100 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.101 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.101 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-09-12T03:29:02.101 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.101 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.101 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.101 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:29:02.101 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.101 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.101 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.101 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.101 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_deviation": { 2024-09-12T03:29:02.102 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_deviation", 2024-09-12T03:29:02.102 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.102 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.102 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.102 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-09-12T03:29:02.102 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-09-12T03:29:02.102 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.102 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.102 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:29:02.102 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:29:02.102 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.102 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.102 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.103 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_optimizations": { 2024-09-12T03:29:02.103 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:29:02.103 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-09-12T03:29:02.103 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.103 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.103 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:02.103 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.103 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.103 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.103 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:29:02.103 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.103 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.103 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.104 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.104 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.104 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.104 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.104 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cephadm", 2024-09-12T03:29:02.104 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.104 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.104 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.104 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_down_multiplier": { 2024-09-12T03:29:02.104 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_down_multiplier", 2024-09-12T03:29:02.104 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-09-12T03:29:02.104 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.104 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.105 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2024-09-12T03:29:02.105 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.105 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.105 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.105 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:29:02.105 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.105 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.105 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.105 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.105 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_refresh_rate": { 2024-09-12T03:29:02.105 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_refresh_rate", 2024-09-12T03:29:02.105 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.105 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.106 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.106 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "20", 2024-09-12T03:29:02.106 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.106 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.106 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.106 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:29:02.106 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.106 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.106 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.106 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.106 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_starting_port": { 2024-09-12T03:29:02.106 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_starting_port", 2024-09-12T03:29:02.106 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.106 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.107 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.107 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4721", 2024-09-12T03:29:02.107 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.107 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.107 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.107 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:29:02.107 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.107 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.107 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.107 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.107 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_ptrace": { 2024-09-12T03:29:02.107 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_ptrace", 2024-09-12T03:29:02.107 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.108 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.108 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.108 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.108 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.108 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.108 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.108 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:29:02.108 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:29:02.108 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.108 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.108 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.108 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_interval": { 2024-09-12T03:29:02.109 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_interval", 2024-09-12T03:29:02.109 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.109 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.109 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.109 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-09-12T03:29:02.109 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.109 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.109 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.109 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:29:02.109 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.109 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.109 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.109 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.110 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:29:02.110 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:29:02.110 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-09-12T03:29:02.110 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.110 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.110 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.7", 2024-09-12T03:29:02.110 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.110 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.110 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.110 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:29:02.110 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.110 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.110 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.110 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.111 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm_log_destination": { 2024-09-12T03:29:02.111 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cephadm_log_destination", 2024-09-12T03:29:02.111 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.111 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.111 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.111 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.111 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.111 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.111 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.111 INFO:teuthology.orchestra.run.smithi006.stdout: "file", 2024-09-12T03:29:02.111 INFO:teuthology.orchestra.run.smithi006.stdout: "file,syslog", 2024-09-12T03:29:02.111 INFO:teuthology.orchestra.run.smithi006.stdout: "syslog" 2024-09-12T03:29:02.111 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.112 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:29:02.112 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.112 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.112 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.112 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.112 INFO:teuthology.orchestra.run.smithi006.stdout: "cgroups_split": { 2024-09-12T03:29:02.112 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cgroups_split", 2024-09-12T03:29:02.112 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.112 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.112 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.112 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.112 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.112 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.113 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.113 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:29:02.113 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.113 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.113 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.113 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.113 INFO:teuthology.orchestra.run.smithi006.stdout: "config_checks_enabled": { 2024-09-12T03:29:02.113 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_checks_enabled", 2024-09-12T03:29:02.113 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.113 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.113 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.113 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.113 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.114 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.114 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.114 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:29:02.114 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.114 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.114 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.114 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.114 INFO:teuthology.orchestra.run.smithi006.stdout: "config_dashboard": { 2024-09-12T03:29:02.114 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_dashboard", 2024-09-12T03:29:02.114 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.114 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.114 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.114 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.114 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.115 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.115 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.115 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:29:02.115 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.115 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.115 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.115 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.115 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_alertmanager": { 2024-09-12T03:29:02.115 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_alertmanager", 2024-09-12T03:29:02.115 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.115 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.115 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.115 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:29:02.116 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.116 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.116 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.116 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-09-12T03:29:02.116 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.116 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.116 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.116 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.116 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_base": { 2024-09-12T03:29:02.116 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_base", 2024-09-12T03:29:02.116 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.116 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.116 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.117 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:29:02.117 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.117 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.117 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.117 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:29:02.117 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.117 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.117 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.117 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.117 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_elasticsearch": { 2024-09-12T03:29:02.117 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:29:02.117 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.117 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.117 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.118 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:29:02.118 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.118 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.118 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.118 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "elasticsearch container image", 2024-09-12T03:29:02.118 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.118 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.118 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.118 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.118 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_grafana": { 2024-09-12T03:29:02.118 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_grafana", 2024-09-12T03:29:02.118 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.118 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.118 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.119 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:29:02.119 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.119 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.119 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.119 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-09-12T03:29:02.119 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.119 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.119 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.119 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.119 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_haproxy": { 2024-09-12T03:29:02.119 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_haproxy", 2024-09-12T03:29:02.119 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.119 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.119 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.120 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:29:02.120 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.120 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.120 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.120 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "HAproxy container image", 2024-09-12T03:29:02.120 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.120 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.120 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.120 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.120 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_agent": { 2024-09-12T03:29:02.120 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:29:02.120 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.121 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.121 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.121 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:29:02.121 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.121 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.121 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.121 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:29:02.121 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.121 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.121 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.121 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.121 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_collector": { 2024-09-12T03:29:02.121 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:29:02.122 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.122 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.122 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.122 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:29:02.122 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.122 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.122 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.122 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:29:02.122 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.122 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.122 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.122 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.122 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_query": { 2024-09-12T03:29:02.122 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:29:02.123 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.123 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.123 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.123 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:29:02.123 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.123 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.123 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.123 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger query container image", 2024-09-12T03:29:02.123 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.123 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.123 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.123 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.123 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_keepalived": { 2024-09-12T03:29:02.124 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_keepalived", 2024-09-12T03:29:02.124 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.124 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.124 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.124 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:29:02.124 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.124 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.124 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.124 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Keepalived container image", 2024-09-12T03:29:02.124 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.124 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.124 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.124 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.125 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_loki": { 2024-09-12T03:29:02.125 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_loki", 2024-09-12T03:29:02.125 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.125 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.125 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.125 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:29:02.125 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.125 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.125 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.125 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Loki container image", 2024-09-12T03:29:02.125 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.125 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.125 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.126 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.126 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_node_exporter": { 2024-09-12T03:29:02.126 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_node_exporter", 2024-09-12T03:29:02.126 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.126 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.126 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.126 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:29:02.126 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.126 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.126 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.126 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-09-12T03:29:02.126 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.126 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.126 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.127 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.127 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_nvmeof": { 2024-09-12T03:29:02.127 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_nvmeof", 2024-09-12T03:29:02.127 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.127 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.127 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.127 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:29:02.127 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.127 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.127 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.127 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Nvme-of container image", 2024-09-12T03:29:02.127 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.127 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.128 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.128 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.128 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_prometheus": { 2024-09-12T03:29:02.128 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_prometheus", 2024-09-12T03:29:02.128 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.128 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.128 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.128 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:29:02.128 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.128 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.128 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.128 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-09-12T03:29:02.128 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.128 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.129 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.129 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.129 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_promtail": { 2024-09-12T03:29:02.129 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_promtail", 2024-09-12T03:29:02.129 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.129 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.129 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.129 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:29:02.129 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.129 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.129 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.129 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Promtail container image", 2024-09-12T03:29:02.129 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.130 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.130 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.130 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.130 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_samba": { 2024-09-12T03:29:02.130 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_samba", 2024-09-12T03:29:02.130 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.130 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.130 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.130 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:29:02.130 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.130 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.130 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.130 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:29:02.130 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.131 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.131 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.131 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.131 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_snmp_gateway": { 2024-09-12T03:29:02.131 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:29:02.131 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.131 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.131 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.131 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:29:02.131 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.131 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.131 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.131 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:29:02.131 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.132 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.132 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.132 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.132 INFO:teuthology.orchestra.run.smithi006.stdout: "container_init": { 2024-09-12T03:29:02.132 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_init", 2024-09-12T03:29:02.132 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.132 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.132 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.132 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.132 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.132 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.132 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.132 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:29:02.133 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.133 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.133 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.133 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.133 INFO:teuthology.orchestra.run.smithi006.stdout: "daemon_cache_timeout": { 2024-09-12T03:29:02.133 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:29:02.133 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.133 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.133 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.133 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-09-12T03:29:02.133 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.133 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.134 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.134 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:29:02.134 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.134 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.134 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.134 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.134 INFO:teuthology.orchestra.run.smithi006.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:29:02.134 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:29:02.134 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.134 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.134 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.134 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "900", 2024-09-12T03:29:02.134 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.135 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.135 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.135 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:29:02.135 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.135 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.135 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.135 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.135 INFO:teuthology.orchestra.run.smithi006.stdout: "default_registry": { 2024-09-12T03:29:02.135 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_registry", 2024-09-12T03:29:02.135 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.135 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.135 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.135 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io", 2024-09-12T03:29:02.136 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.136 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.136 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.136 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:29:02.136 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.136 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.136 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.136 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.136 INFO:teuthology.orchestra.run.smithi006.stdout: "device_cache_timeout": { 2024-09-12T03:29:02.136 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_cache_timeout", 2024-09-12T03:29:02.136 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.136 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.136 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.137 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1800", 2024-09-12T03:29:02.137 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.137 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.137 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.137 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:29:02.137 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.137 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.137 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.137 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.137 INFO:teuthology.orchestra.run.smithi006.stdout: "device_enhanced_scan": { 2024-09-12T03:29:02.137 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_enhanced_scan", 2024-09-12T03:29:02.137 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.137 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.138 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.138 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.138 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.138 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.138 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.138 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:29:02.138 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.138 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.138 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.138 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.138 INFO:teuthology.orchestra.run.smithi006.stdout: "facts_cache_timeout": { 2024-09-12T03:29:02.138 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "facts_cache_timeout", 2024-09-12T03:29:02.138 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.138 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.139 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.139 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-09-12T03:29:02.139 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.139 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.139 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.139 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:29:02.139 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.139 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.139 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.139 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.139 INFO:teuthology.orchestra.run.smithi006.stdout: "grafana_dashboards_path": { 2024-09-12T03:29:02.139 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:29:02.139 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.140 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.140 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.140 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:29:02.140 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.140 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.140 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.140 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:29:02.140 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.140 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.140 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.140 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.140 INFO:teuthology.orchestra.run.smithi006.stdout: "host_check_interval": { 2024-09-12T03:29:02.140 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "host_check_interval", 2024-09-12T03:29:02.141 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.141 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.141 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.141 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-09-12T03:29:02.141 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.141 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.141 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.141 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:29:02.141 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.141 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.141 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.141 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.141 INFO:teuthology.orchestra.run.smithi006.stdout: "hw_monitoring": { 2024-09-12T03:29:02.142 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "hw_monitoring", 2024-09-12T03:29:02.142 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.142 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.142 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.142 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.142 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.142 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.142 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.142 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:29:02.142 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.142 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.142 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.142 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.142 INFO:teuthology.orchestra.run.smithi006.stdout: "inventory_list_all": { 2024-09-12T03:29:02.143 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "inventory_list_all", 2024-09-12T03:29:02.143 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.143 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.143 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.143 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.143 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.143 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.143 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.143 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:29:02.143 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.143 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.143 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.143 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.144 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.144 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.144 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.144 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.144 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.144 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.144 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.144 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.144 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.144 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.144 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.144 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.144 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.145 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.145 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.145 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.145 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.145 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.145 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.145 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.145 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.145 INFO:teuthology.orchestra.run.smithi006.stdout: "log_refresh_metadata": { 2024-09-12T03:29:02.145 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_refresh_metadata", 2024-09-12T03:29:02.145 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.145 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.145 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.145 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.146 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.146 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.146 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.146 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:29:02.146 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.146 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.146 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.146 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.146 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.146 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.146 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.146 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.147 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.147 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.147 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.147 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.147 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.147 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:29:02.147 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.147 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.147 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.147 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.147 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.147 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.147 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.147 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.148 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.148 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.148 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.148 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.148 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.148 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.148 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.148 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.148 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.148 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.148 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.148 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.148 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.149 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.149 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.149 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.149 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.149 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.149 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.149 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.149 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.149 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.149 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.149 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.149 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.149 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.150 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.150 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.150 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.150 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.150 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.150 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:29:02.150 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:29:02.150 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.150 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.150 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.150 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.150 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.150 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.150 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.151 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:29:02.151 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.151 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.151 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.151 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.151 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:29:02.151 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:29:02.151 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.151 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.151 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.151 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "*", 2024-09-12T03:29:02.151 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.151 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.152 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.152 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:29:02.152 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.152 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.152 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.152 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.152 INFO:teuthology.orchestra.run.smithi006.stdout: "max_count_per_host": { 2024-09-12T03:29:02.152 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_count_per_host", 2024-09-12T03:29:02.152 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.152 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.152 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.152 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:02.152 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.153 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.153 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.153 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:29:02.153 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.153 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.153 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.153 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.153 INFO:teuthology.orchestra.run.smithi006.stdout: "max_osd_draining_count": { 2024-09-12T03:29:02.153 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_osd_draining_count", 2024-09-12T03:29:02.153 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.153 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.153 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.153 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:02.154 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.154 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.154 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.154 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:29:02.154 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.154 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.154 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.154 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.154 INFO:teuthology.orchestra.run.smithi006.stdout: "migration_current": { 2024-09-12T03:29:02.154 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "migration_current", 2024-09-12T03:29:02.154 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.154 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.154 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.154 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.155 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.155 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.155 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.155 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "internal - do not modify", 2024-09-12T03:29:02.155 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.155 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.155 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.155 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.155 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2024-09-12T03:29:02.155 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2024-09-12T03:29:02.155 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.155 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.155 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.155 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "root", 2024-09-12T03:29:02.156 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.156 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.156 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.156 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm-package", 2024-09-12T03:29:02.156 INFO:teuthology.orchestra.run.smithi006.stdout: "root" 2024-09-12T03:29:02.156 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.156 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:29:02.156 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.156 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.156 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.156 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.156 INFO:teuthology.orchestra.run.smithi006.stdout: "oob_default_addr": { 2024-09-12T03:29:02.156 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "oob_default_addr", 2024-09-12T03:29:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "169.254.1.1", 2024-09-12T03:29:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:29:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: "prometheus_alerts_path": { 2024-09-12T03:29:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:29:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:29:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:29:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_insecure": { 2024-09-12T03:29:02.159 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_insecure", 2024-09-12T03:29:02.159 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.159 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.159 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.159 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.159 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.159 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.159 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.159 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:29:02.159 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.159 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.159 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.159 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.160 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_password": { 2024-09-12T03:29:02.160 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_password", 2024-09-12T03:29:02.160 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.160 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.160 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.160 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.160 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.160 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.160 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.160 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:29:02.160 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.160 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.160 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.161 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.161 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_url": { 2024-09-12T03:29:02.161 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_url", 2024-09-12T03:29:02.161 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.161 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.161 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.161 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.161 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.161 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.161 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.161 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:29:02.161 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.161 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.162 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.162 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.162 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_username": { 2024-09-12T03:29:02.162 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_username", 2024-09-12T03:29:02.162 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.162 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.162 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.162 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.162 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.162 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.162 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.162 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:29:02.162 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.163 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.163 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.163 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.163 INFO:teuthology.orchestra.run.smithi006.stdout: "secure_monitoring_stack": { 2024-09-12T03:29:02.163 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:29:02.163 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.163 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.163 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.163 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.163 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.163 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.163 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.163 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:29:02.163 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.164 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.164 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.164 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.164 INFO:teuthology.orchestra.run.smithi006.stdout: "service_discovery_port": { 2024-09-12T03:29:02.164 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "service_discovery_port", 2024-09-12T03:29:02.164 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.164 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.164 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.164 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8765", 2024-09-12T03:29:02.164 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.164 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.164 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.164 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:29:02.165 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.165 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.165 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.165 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.165 INFO:teuthology.orchestra.run.smithi006.stdout: "ssh_config_file": { 2024-09-12T03:29:02.165 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssh_config_file", 2024-09-12T03:29:02.165 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.165 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.165 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.165 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.165 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.165 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.165 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.166 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:29:02.166 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.166 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.166 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.166 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.166 INFO:teuthology.orchestra.run.smithi006.stdout: "use_agent": { 2024-09-12T03:29:02.166 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_agent", 2024-09-12T03:29:02.166 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.166 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.166 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.166 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.166 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.166 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.167 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.167 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:29:02.167 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.167 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.167 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.167 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.167 INFO:teuthology.orchestra.run.smithi006.stdout: "use_repo_digest": { 2024-09-12T03:29:02.167 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_repo_digest", 2024-09-12T03:29:02.167 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.167 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.167 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.167 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.167 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.167 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.168 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.168 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:29:02.168 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.168 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.168 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.168 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.168 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_failed_host_check": { 2024-09-12T03:29:02.168 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:29:02.168 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.168 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.168 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.168 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.168 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.169 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.169 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.169 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:29:02.169 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.169 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.169 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.169 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.169 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_daemons": { 2024-09-12T03:29:02.169 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:29:02.169 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.169 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.169 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.169 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.170 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.170 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.170 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.170 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:29:02.170 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.170 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.170 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.170 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.170 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_hosts": { 2024-09-12T03:29:02.170 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:29:02.170 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.170 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.170 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.170 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.171 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.171 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.171 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.171 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:29:02.171 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.171 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.171 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.171 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.171 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.171 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.171 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.171 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crash", 2024-09-12T03:29:02.171 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.171 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.172 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.172 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.172 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.172 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.172 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.172 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.172 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.172 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.172 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.172 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.172 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.172 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.173 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.173 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.173 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.173 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.173 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.173 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.173 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.173 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.173 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.173 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.173 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.173 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.173 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.174 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.174 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.174 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.174 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.174 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.174 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.174 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.174 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.174 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.174 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.174 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.174 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.174 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.175 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.175 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.175 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.175 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.175 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.175 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.175 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.175 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.175 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.175 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.175 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.175 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.175 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.175 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.176 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.176 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.176 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.176 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.176 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.176 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.176 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.176 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.176 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.176 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.176 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.176 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.177 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.177 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.177 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.177 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.177 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.177 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.177 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.177 INFO:teuthology.orchestra.run.smithi006.stdout: "retain_interval": { 2024-09-12T03:29:02.177 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retain_interval", 2024-09-12T03:29:02.177 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.177 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.177 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.177 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "31536000", 2024-09-12T03:29:02.178 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.178 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.178 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.178 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:29:02.178 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.178 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.178 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.178 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.178 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_recent_interval": { 2024-09-12T03:29:02.178 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_recent_interval", 2024-09-12T03:29:02.178 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.178 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.178 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.179 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1209600", 2024-09-12T03:29:02.179 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.179 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.179 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.179 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:29:02.179 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.179 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.179 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.179 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.179 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.179 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.179 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.179 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dashboard", 2024-09-12T03:29:02.179 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.180 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.180 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.180 INFO:teuthology.orchestra.run.smithi006.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:29:02.180 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:29:02.180 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.180 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.180 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.180 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:02.180 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.180 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.180 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.180 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.180 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.180 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.181 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.181 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.181 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:29:02.181 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:29:02.181 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.181 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.181 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.181 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.181 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.181 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.181 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.181 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.181 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.182 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.182 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.182 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.182 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:29:02.182 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:29:02.182 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.182 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.182 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.182 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.182 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.182 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.182 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.182 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.182 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.183 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.183 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.183 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.183 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:29:02.183 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:29:02.183 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.183 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.183 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.183 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.183 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.183 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.183 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.183 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.183 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.184 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.184 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.184 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.184 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:29:02.184 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:29:02.184 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.184 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.184 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.184 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.184 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.184 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.184 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.184 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.185 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.185 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.185 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.185 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.185 INFO:teuthology.orchestra.run.smithi006.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:29:02.185 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:29:02.185 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.185 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.185 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.185 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.185 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.185 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.185 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.185 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.186 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.186 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.186 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.186 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.186 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:29:02.186 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:29:02.186 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.186 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.186 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.186 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.186 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.186 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.186 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.187 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.187 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.187 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.187 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.187 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.187 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:29:02.187 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:29:02.187 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.187 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.187 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.187 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.187 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.187 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.188 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.188 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.188 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.188 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.188 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.188 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.188 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:29:02.188 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:29:02.188 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.188 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.188 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.188 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.188 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.189 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.189 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.189 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.189 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.189 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.189 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.189 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.189 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:29:02.189 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:29:02.189 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.189 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.189 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.189 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.190 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.190 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.190 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.190 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.190 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.190 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.190 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.190 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.190 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:29:02.190 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:29:02.190 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.190 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.190 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.190 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.191 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.191 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.191 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.191 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.191 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.191 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.191 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.191 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.191 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:29:02.191 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:29:02.191 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.191 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.191 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.192 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.192 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.192 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.192 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.192 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.192 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.192 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.192 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.192 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.192 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:29:02.192 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:29:02.192 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.192 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.192 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.193 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.193 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.193 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.193 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.193 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.193 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.193 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.193 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.193 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.193 INFO:teuthology.orchestra.run.smithi006.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:29:02.193 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:29:02.193 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.193 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.194 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.194 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.194 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.194 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.194 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.194 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.194 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.194 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.194 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.194 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.194 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:29:02.194 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:29:02.194 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.194 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.195 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.195 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-09-12T03:29:02.195 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.195 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.195 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.195 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.195 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.195 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.195 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.195 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.195 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:29:02.195 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:29:02.195 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.195 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.196 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.196 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.196 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.196 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.196 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.196 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.196 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.196 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.196 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.196 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.196 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_URL": { 2024-09-12T03:29:02.196 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:29:02.196 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.197 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.197 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.197 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.197 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.197 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.197 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.197 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.197 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.197 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.197 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.197 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.197 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:29:02.197 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:29:02.198 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.198 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.198 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.198 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-09-12T03:29:02.198 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.198 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.198 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.198 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.198 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.198 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.198 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.198 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.198 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:29:02.199 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:29:02.199 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.199 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.199 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.199 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.199 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.199 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.199 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.199 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.199 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.199 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.199 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.199 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.199 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:29:02.200 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:29:02.200 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.200 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.200 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.200 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.200 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.200 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.200 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.200 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.200 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.200 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.200 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.201 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.201 INFO:teuthology.orchestra.run.smithi006.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:29:02.201 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:29:02.201 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.201 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.201 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.201 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.201 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.201 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.201 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.201 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.201 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.201 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.202 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.202 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.202 INFO:teuthology.orchestra.run.smithi006.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:29:02.202 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:29:02.202 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.202 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.202 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.202 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.202 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.202 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.202 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.202 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.202 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.203 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.203 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.203 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.203 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:29:02.203 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:29:02.203 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.203 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.203 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.203 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.203 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.203 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.203 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.203 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.203 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.204 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.204 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.204 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.204 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:29:02.204 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:29:02.204 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.204 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.204 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.204 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.204 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.204 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.204 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.204 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.205 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.205 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.205 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.205 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.205 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:29:02.205 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:29:02.205 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.205 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.205 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.205 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.205 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.205 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.205 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.206 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.206 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.206 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.206 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.206 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.206 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:29:02.206 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:29:02.206 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.206 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.206 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.206 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.206 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.206 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.206 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.207 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.207 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.207 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.207 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.207 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.207 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:29:02.207 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:29:02.207 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.207 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.207 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.207 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.207 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.207 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.208 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.208 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.208 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.208 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.208 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.208 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.208 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:29:02.208 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:29:02.208 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.208 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.208 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.208 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.208 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.209 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.209 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.209 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.209 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.209 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.209 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.209 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.209 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:29:02.209 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:29:02.209 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.209 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.209 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.209 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.209 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.210 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.210 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.210 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.210 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.210 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.210 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.210 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.210 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:29:02.210 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:29:02.210 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.210 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.210 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.210 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.211 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.211 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.211 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.211 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.211 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.211 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.211 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.211 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.211 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:29:02.211 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:29:02.211 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.211 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.211 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.211 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.212 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.212 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.212 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.212 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.212 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.212 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.212 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.212 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.212 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:29:02.212 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:29:02.212 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.212 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.212 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.213 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.213 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.213 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.213 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.213 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.213 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.213 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.213 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.213 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.213 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:29:02.213 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:29:02.213 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.213 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.213 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.214 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:29:02.214 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.214 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.214 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.214 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.214 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.214 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.214 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.214 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.214 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:29:02.214 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:29:02.214 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.215 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.215 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.215 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:02.215 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.215 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.215 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.215 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.215 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.215 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.215 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.215 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.215 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:29:02.215 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:29:02.216 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.216 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.216 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.216 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8", 2024-09-12T03:29:02.216 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.216 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.216 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.216 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.216 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.216 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.216 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.216 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.216 INFO:teuthology.orchestra.run.smithi006.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:29:02.217 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:29:02.217 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.217 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.217 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.217 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "45", 2024-09-12T03:29:02.217 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.217 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.217 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.217 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.217 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.217 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.217 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.217 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.218 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:29:02.218 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:29:02.218 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.218 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.218 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.218 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.218 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.218 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.218 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.218 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.218 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.218 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.218 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.218 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.219 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:29:02.219 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:29:02.219 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.219 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.219 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.219 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-09-12T03:29:02.219 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.219 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.219 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.219 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.219 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.219 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.219 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.220 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.220 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:29:02.220 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:29:02.220 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.220 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.220 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.220 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.220 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.220 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.220 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.220 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.220 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.220 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.221 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.221 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.221 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:29:02.221 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:29:02.221 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.221 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.221 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.221 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.221 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.221 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.221 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.221 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.221 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.221 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.222 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.222 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.222 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:29:02.222 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:29:02.222 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.222 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.222 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.222 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-09-12T03:29:02.222 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.222 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.222 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.222 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.222 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.223 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.223 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.223 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.223 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:29:02.223 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:29:02.223 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.223 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.223 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.223 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:02.223 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.223 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.223 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.223 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.223 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.224 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.224 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.224 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.224 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:29:02.224 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:29:02.224 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.224 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.224 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.224 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-09-12T03:29:02.224 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.224 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.224 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.224 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.225 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.225 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.225 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.225 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.225 INFO:teuthology.orchestra.run.smithi006.stdout: "cross_origin_url": { 2024-09-12T03:29:02.225 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cross_origin_url", 2024-09-12T03:29:02.225 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.225 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.225 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.225 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.225 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.225 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.225 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.225 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.226 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.226 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.226 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.226 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.226 INFO:teuthology.orchestra.run.smithi006.stdout: "crt_file": { 2024-09-12T03:29:02.226 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crt_file", 2024-09-12T03:29:02.226 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.226 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.226 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.226 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.226 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.226 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.226 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.226 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.227 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.227 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.227 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.227 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.227 INFO:teuthology.orchestra.run.smithi006.stdout: "debug": { 2024-09-12T03:29:02.227 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "debug", 2024-09-12T03:29:02.227 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.227 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.227 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.227 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.227 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.227 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.227 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.228 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:29:02.228 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.228 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.228 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.228 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.228 INFO:teuthology.orchestra.run.smithi006.stdout: "jwt_token_ttl": { 2024-09-12T03:29:02.228 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "jwt_token_ttl", 2024-09-12T03:29:02.228 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.228 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.228 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.228 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "28800", 2024-09-12T03:29:02.228 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.229 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.229 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.229 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.229 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.229 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.229 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.229 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.229 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2024-09-12T03:29:02.229 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2024-09-12T03:29:02.229 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.229 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.229 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.229 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.230 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.230 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.230 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.230 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.230 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.230 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.230 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.230 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.230 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.230 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.230 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.230 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.230 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.231 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.231 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.231 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.231 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.231 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.231 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.231 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.231 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.231 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.231 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.231 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.231 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.231 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.231 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.232 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.232 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.232 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.232 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.232 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.232 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.232 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.232 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.232 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.232 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.232 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.232 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.232 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.233 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.233 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.233 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.233 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.233 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.233 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.233 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.233 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.233 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.233 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.233 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.233 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.233 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.234 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.234 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.234 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.234 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.234 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.234 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.234 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.234 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.234 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.234 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.234 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.234 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.234 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.235 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.235 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.235 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.235 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.235 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.235 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.235 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.235 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.235 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.235 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.235 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.235 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.235 INFO:teuthology.orchestra.run.smithi006.stdout: "motd": { 2024-09-12T03:29:02.235 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "motd", 2024-09-12T03:29:02.236 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.236 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.236 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.236 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.236 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.236 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.236 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.236 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "The message of the day", 2024-09-12T03:29:02.236 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.236 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.236 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.236 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.236 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:29:02.237 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:29:02.237 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.237 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.237 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.237 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.237 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.237 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.237 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.237 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.237 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.237 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.237 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.237 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.238 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-09-12T03:29:02.238 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-09-12T03:29:02.238 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.238 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.238 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.238 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2024-09-12T03:29:02.238 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.238 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.238 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.238 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.238 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.238 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.238 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.239 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.239 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-09-12T03:29:02.239 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-09-12T03:29:02.239 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.239 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.239 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.239 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8080", 2024-09-12T03:29:02.239 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.239 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.239 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.239 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.239 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.239 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.239 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.240 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.240 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2024-09-12T03:29:02.240 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2024-09-12T03:29:02.240 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.240 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.240 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.240 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.240 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.240 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.240 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.240 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.240 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.240 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.241 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.241 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.241 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl_server_port": { 2024-09-12T03:29:02.241 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl_server_port", 2024-09-12T03:29:02.241 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.241 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.241 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.241 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8443", 2024-09-12T03:29:02.241 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.241 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.241 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.241 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.241 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.242 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.242 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.242 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.242 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2024-09-12T03:29:02.242 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2024-09-12T03:29:02.242 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.242 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.242 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.242 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "redirect", 2024-09-12T03:29:02.242 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.242 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.242 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.242 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.243 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect" 2024-09-12T03:29:02.243 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.243 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.243 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.243 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.243 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.243 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.243 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2024-09-12T03:29:02.243 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2024-09-12T03:29:02.243 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.243 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.243 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.243 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2024-09-12T03:29:02.244 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2024-09-12T03:29:02.244 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2024-09-12T03:29:02.244 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.244 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.244 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.244 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.244 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.244 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.244 INFO:teuthology.orchestra.run.smithi006.stdout: "url_prefix": { 2024-09-12T03:29:02.244 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url_prefix", 2024-09-12T03:29:02.244 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.244 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.244 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.245 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.245 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.245 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.245 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.245 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.245 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.245 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.245 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.245 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.245 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.245 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.245 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.245 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "devicehealth", 2024-09-12T03:29:02.245 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.246 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.246 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.246 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_monitoring": { 2024-09-12T03:29:02.246 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_monitoring", 2024-09-12T03:29:02.246 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.246 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.246 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.246 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.246 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.246 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.246 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.246 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "monitor device health metrics", 2024-09-12T03:29:02.246 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.247 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.247 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.247 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.247 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.247 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.247 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.247 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.247 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.247 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.247 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.247 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.247 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.247 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.248 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.248 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.248 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.248 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.248 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.248 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.248 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.248 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.248 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.248 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.248 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.248 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.249 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.249 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.249 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.249 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.249 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.249 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.249 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.249 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.249 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.249 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.249 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.249 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.249 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.250 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.250 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.250 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.250 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.250 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.250 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.250 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.250 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.250 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.250 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.250 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.250 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.250 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.251 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.251 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.251 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.251 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.251 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.251 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.251 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.251 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.251 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.251 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.251 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.251 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.251 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.251 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.252 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.252 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.252 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.252 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.252 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.252 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.252 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.252 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.252 INFO:teuthology.orchestra.run.smithi006.stdout: "mark_out_threshold": { 2024-09-12T03:29:02.252 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mark_out_threshold", 2024-09-12T03:29:02.252 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.252 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.252 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.253 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2419200", 2024-09-12T03:29:02.253 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.253 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.253 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.253 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:29:02.253 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.253 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.253 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.253 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.253 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_name": { 2024-09-12T03:29:02.253 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_name", 2024-09-12T03:29:02.253 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.253 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.253 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.254 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "device_health_metrics", 2024-09-12T03:29:02.254 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.254 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.254 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.254 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:29:02.254 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.254 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.254 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.254 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.254 INFO:teuthology.orchestra.run.smithi006.stdout: "retention_period": { 2024-09-12T03:29:02.254 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retention_period", 2024-09-12T03:29:02.254 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.254 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.255 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.255 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15552000", 2024-09-12T03:29:02.255 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.255 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.255 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.255 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:29:02.255 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.255 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.255 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.255 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.255 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_frequency": { 2024-09-12T03:29:02.255 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_frequency", 2024-09-12T03:29:02.255 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.255 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.256 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.256 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2024-09-12T03:29:02.256 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.256 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.256 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.256 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:29:02.256 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.256 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.256 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.256 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.256 INFO:teuthology.orchestra.run.smithi006.stdout: "self_heal": { 2024-09-12T03:29:02.256 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "self_heal", 2024-09-12T03:29:02.256 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.257 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.257 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.257 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.257 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.257 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.257 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.257 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:29:02.257 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.257 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.257 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.257 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.257 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-09-12T03:29:02.258 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-09-12T03:29:02.258 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.258 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.258 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.258 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-09-12T03:29:02.258 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.258 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.258 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.258 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:29:02.258 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.258 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.258 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.258 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.258 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_threshold": { 2024-09-12T03:29:02.259 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_threshold", 2024-09-12T03:29:02.259 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.259 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.259 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.259 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7257600", 2024-09-12T03:29:02.259 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.259 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.259 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.259 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:29:02.259 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.259 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.259 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.259 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.260 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.260 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.260 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.260 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "diskprediction_local", 2024-09-12T03:29:02.260 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.260 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.260 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.260 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.260 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.260 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.260 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.260 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.260 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.261 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.261 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.261 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.261 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.261 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.261 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.261 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.261 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.261 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.261 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.261 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.261 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.262 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.262 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.262 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.262 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.262 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.262 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.262 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.262 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.262 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.262 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.262 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.262 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.262 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.262 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.263 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.263 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.263 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.263 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.263 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.263 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.263 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.263 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.263 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.263 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.263 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.263 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.263 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.264 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.264 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.264 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.264 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.264 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.264 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.264 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.264 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.264 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.264 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.264 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.264 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.264 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.265 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.265 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.265 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.265 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.265 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.265 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.265 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.265 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.265 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.265 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.265 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.265 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.265 INFO:teuthology.orchestra.run.smithi006.stdout: "predict_interval": { 2024-09-12T03:29:02.266 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predict_interval", 2024-09-12T03:29:02.266 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.266 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.266 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.266 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2024-09-12T03:29:02.266 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.266 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.266 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.266 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.266 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.266 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.266 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.266 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.267 INFO:teuthology.orchestra.run.smithi006.stdout: "predictor_model": { 2024-09-12T03:29:02.267 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predictor_model", 2024-09-12T03:29:02.267 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.267 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.267 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.267 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "prophetstor", 2024-09-12T03:29:02.267 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.267 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.267 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.267 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.267 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.267 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.267 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.267 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.268 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-09-12T03:29:02.268 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-09-12T03:29:02.268 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.268 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.268 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.268 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-09-12T03:29:02.268 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.268 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.268 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.268 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.268 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.268 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.268 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.269 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.269 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.269 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.269 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.269 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "influx", 2024-09-12T03:29:02.269 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": false, 2024-09-12T03:29:02.269 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:29:02.269 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.269 INFO:teuthology.orchestra.run.smithi006.stdout: "batch_size": { 2024-09-12T03:29:02.269 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "batch_size", 2024-09-12T03:29:02.269 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.269 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.269 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.269 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5000", 2024-09-12T03:29:02.270 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.270 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.270 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.270 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:29:02.270 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.270 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.270 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.270 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.270 INFO:teuthology.orchestra.run.smithi006.stdout: "database": { 2024-09-12T03:29:02.270 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "database", 2024-09-12T03:29:02.270 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.270 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.270 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.271 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "ceph", 2024-09-12T03:29:02.271 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.271 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.271 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.271 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:29:02.271 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.271 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.271 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.271 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.271 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": { 2024-09-12T03:29:02.271 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "hostname", 2024-09-12T03:29:02.271 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.272 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.272 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.272 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.272 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.272 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.272 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.272 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:29:02.272 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.272 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.272 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.272 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.272 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-09-12T03:29:02.272 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-09-12T03:29:02.273 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.273 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.273 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.273 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "30", 2024-09-12T03:29:02.273 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "5", 2024-09-12T03:29:02.273 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.273 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.273 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:29:02.273 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.273 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.273 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.273 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.273 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.273 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.274 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.274 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.274 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.274 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.274 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.274 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.274 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.274 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.274 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.274 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.274 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.274 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.274 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.275 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.275 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.275 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.275 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.275 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.275 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.275 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.275 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.275 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.275 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.275 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.275 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.275 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.276 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.276 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.276 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.276 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.276 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.276 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.276 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.276 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.276 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.276 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.276 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.276 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.276 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.277 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.277 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.277 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.277 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.277 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.277 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.277 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.277 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.277 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.277 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.277 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.277 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.277 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.277 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.278 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.278 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.278 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.278 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.278 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.278 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.278 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.278 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.278 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.278 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.278 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.278 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.278 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.279 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.279 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.279 INFO:teuthology.orchestra.run.smithi006.stdout: "password": { 2024-09-12T03:29:02.279 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "password", 2024-09-12T03:29:02.279 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.279 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.279 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.279 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.279 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.279 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.279 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.279 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:29:02.279 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.279 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.280 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.280 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.280 INFO:teuthology.orchestra.run.smithi006.stdout: "port": { 2024-09-12T03:29:02.280 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "port", 2024-09-12T03:29:02.280 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.280 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.280 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.280 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8086", 2024-09-12T03:29:02.280 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.280 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.280 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.280 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server port", 2024-09-12T03:29:02.280 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.281 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.281 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.281 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.281 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2024-09-12T03:29:02.281 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2024-09-12T03:29:02.281 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.281 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.281 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.281 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "false", 2024-09-12T03:29:02.281 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.281 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.281 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.281 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:29:02.282 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.282 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.282 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.282 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.282 INFO:teuthology.orchestra.run.smithi006.stdout: "threads": { 2024-09-12T03:29:02.282 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threads", 2024-09-12T03:29:02.282 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.282 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.282 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.282 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-09-12T03:29:02.282 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-09-12T03:29:02.282 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "32", 2024-09-12T03:29:02.282 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.283 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:29:02.283 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.283 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.283 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.283 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.283 INFO:teuthology.orchestra.run.smithi006.stdout: "username": { 2024-09-12T03:29:02.283 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "username", 2024-09-12T03:29:02.283 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.283 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.283 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.283 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.283 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.283 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.283 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.284 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:29:02.284 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.284 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.284 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.284 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.284 INFO:teuthology.orchestra.run.smithi006.stdout: "verify_ssl": { 2024-09-12T03:29:02.284 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "verify_ssl", 2024-09-12T03:29:02.284 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.284 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.284 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.284 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "true", 2024-09-12T03:29:02.284 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.284 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.285 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.285 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:29:02.285 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.285 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.285 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.285 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.285 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.285 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.285 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.285 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "insights", 2024-09-12T03:29:02.285 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.285 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.286 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.286 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.286 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.286 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.286 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.286 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.286 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.286 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.286 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.286 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.286 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.286 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.286 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.287 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.287 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.287 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.287 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.287 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.287 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.287 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.287 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.287 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.287 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.287 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.287 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.287 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.288 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.288 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.288 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.288 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.288 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.288 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.288 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.288 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.288 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.288 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.288 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.288 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.288 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.289 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.289 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.289 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.289 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.289 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.289 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.289 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.289 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.289 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.289 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.289 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.289 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.289 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.290 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.290 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.290 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.290 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.290 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.290 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.290 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.290 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.290 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.290 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.290 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.290 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.290 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.291 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.291 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.291 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.291 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.291 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.291 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.291 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.291 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.291 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.291 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "iostat", 2024-09-12T03:29:02.291 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.291 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.291 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.292 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.292 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.292 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.292 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.292 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.292 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.292 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.292 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.292 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.292 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.292 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.292 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.292 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.293 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.293 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.293 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.293 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.293 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.293 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.293 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.293 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.293 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.293 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.293 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.293 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.293 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.294 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.294 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.294 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.294 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.294 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.294 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.294 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.294 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.294 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.294 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.294 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.294 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.294 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.295 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.295 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.295 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.295 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.295 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.295 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.295 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.295 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.295 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.295 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.295 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.295 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.295 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.295 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.296 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.296 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.296 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.296 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.296 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.296 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.296 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.296 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.296 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.296 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.296 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.296 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.296 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.297 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.297 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.297 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.297 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.297 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.297 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.297 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.297 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "k8sevents", 2024-09-12T03:29:02.297 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.297 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.297 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.297 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_event_retention_days": { 2024-09-12T03:29:02.297 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:29:02.298 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.298 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.298 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.298 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7", 2024-09-12T03:29:02.298 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.298 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.298 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.298 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:29:02.298 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.298 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.298 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.298 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.298 INFO:teuthology.orchestra.run.smithi006.stdout: "config_check_secs": { 2024-09-12T03:29:02.298 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_check_secs", 2024-09-12T03:29:02.299 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.299 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.299 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.299 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:02.299 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "10", 2024-09-12T03:29:02.299 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.299 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.299 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:29:02.299 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.299 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.299 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.299 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.299 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.299 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.300 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.300 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.300 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.300 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.300 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.300 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.300 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.300 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.300 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.300 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.300 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.300 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.301 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.301 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.301 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.301 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.301 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.301 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.301 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.301 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.301 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.301 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.301 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.301 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.301 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.302 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.302 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.302 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.302 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.302 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.304 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.304 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.304 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.304 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.304 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.304 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.304 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.305 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.305 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.305 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.305 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.305 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.305 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.305 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.305 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.305 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.305 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.305 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.305 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.305 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.305 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.306 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.306 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.306 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.306 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.306 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.306 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.306 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.306 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.306 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.306 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.306 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.306 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.306 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.307 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.307 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.307 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.307 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.307 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.307 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.307 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.307 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "localpool", 2024-09-12T03:29:02.307 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.307 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.307 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.307 INFO:teuthology.orchestra.run.smithi006.stdout: "failure_domain": { 2024-09-12T03:29:02.307 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "failure_domain", 2024-09-12T03:29:02.308 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.308 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.308 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.308 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "host", 2024-09-12T03:29:02.308 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.308 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.308 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.308 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:29:02.308 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:29:02.308 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.308 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.308 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.308 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.309 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.309 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.309 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.309 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.309 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.309 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.309 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.309 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.309 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.309 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.309 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.309 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.309 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.310 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.310 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.310 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.310 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.310 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.310 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.310 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.310 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.310 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.310 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.310 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.310 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.310 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.310 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.311 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.311 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.311 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.311 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.311 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.311 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.311 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.311 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.311 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.311 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.311 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.311 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.311 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.312 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.312 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.312 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.312 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.312 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.312 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.312 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.312 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.312 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.312 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.312 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.312 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.312 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.312 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.313 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.313 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.313 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.313 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.313 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.313 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.313 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.313 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.313 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.313 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.313 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.313 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.313 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.314 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.314 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.314 INFO:teuthology.orchestra.run.smithi006.stdout: "min_size": { 2024-09-12T03:29:02.314 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_size", 2024-09-12T03:29:02.314 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.314 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.314 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.314 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.314 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.314 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.314 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.314 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:29:02.314 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:29:02.314 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.315 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.315 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.315 INFO:teuthology.orchestra.run.smithi006.stdout: "num_rep": { 2024-09-12T03:29:02.315 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "num_rep", 2024-09-12T03:29:02.315 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.315 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.315 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.315 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3", 2024-09-12T03:29:02.315 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.315 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.315 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.315 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:29:02.315 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.316 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.316 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.316 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.316 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_num": { 2024-09-12T03:29:02.316 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_num", 2024-09-12T03:29:02.316 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.316 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.316 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.316 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "128", 2024-09-12T03:29:02.317 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.317 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.317 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.317 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:29:02.317 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.317 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.317 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.317 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.317 INFO:teuthology.orchestra.run.smithi006.stdout: "prefix": { 2024-09-12T03:29:02.317 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prefix", 2024-09-12T03:29:02.317 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.317 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.317 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.318 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.318 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.318 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.318 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.318 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:29:02.318 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.318 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.318 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.318 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.318 INFO:teuthology.orchestra.run.smithi006.stdout: "subtree": { 2024-09-12T03:29:02.318 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "subtree", 2024-09-12T03:29:02.318 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.318 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.319 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.319 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "rack", 2024-09-12T03:29:02.319 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.319 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.319 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.319 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:29:02.319 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:29:02.319 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.319 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.319 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.319 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.319 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.319 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.320 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mds_autoscaler", 2024-09-12T03:29:02.320 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.320 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.320 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.320 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.320 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.320 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.320 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.320 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.320 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.320 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.320 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.320 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.321 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.321 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.321 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.321 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.321 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.321 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.321 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.321 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.321 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.321 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.321 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.321 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.321 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.322 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.322 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.322 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.322 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.322 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.322 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.322 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.322 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.322 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.322 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.322 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.322 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.322 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.322 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.323 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.323 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.323 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.323 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.323 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.323 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.323 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.323 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.323 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.323 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.323 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.323 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.323 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.324 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.324 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.324 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.324 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.324 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.324 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.324 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.324 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.324 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.324 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.324 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.324 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.324 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.325 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.325 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.325 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.325 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.325 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.325 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.325 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.325 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.325 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.325 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.325 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.325 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirroring", 2024-09-12T03:29:02.325 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.326 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.326 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.326 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.326 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.326 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.326 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.326 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.326 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.326 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.326 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.326 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.326 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.326 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.327 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.327 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.327 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.327 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.327 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.327 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.327 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.327 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.327 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.327 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.327 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.327 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.327 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.328 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.328 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.328 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.328 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.328 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.328 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.328 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.328 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.328 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.328 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.328 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.328 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.328 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.329 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.329 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.329 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.329 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.329 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.329 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.329 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.329 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.329 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.329 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.329 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.329 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.329 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.329 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.330 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.330 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.330 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.330 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.330 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.330 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.330 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.330 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.330 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.330 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.330 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.330 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.330 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.330 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.331 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.331 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.331 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.331 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.331 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.331 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.331 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.331 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.331 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "nfs", 2024-09-12T03:29:02.331 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.331 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.331 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.332 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.332 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.332 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.332 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.332 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.332 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.332 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.332 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.332 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.332 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.332 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.332 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.332 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.333 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.333 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.333 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.333 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.333 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.333 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.333 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.333 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.333 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.333 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.333 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.333 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.333 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.334 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.334 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.334 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.334 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.334 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.334 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.334 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.334 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.334 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.334 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.334 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.334 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.334 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.335 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.335 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.335 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.335 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.335 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.335 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.335 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.335 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.335 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.335 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.335 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.335 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.335 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.336 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.336 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.336 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.336 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.336 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.336 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.336 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.336 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.336 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.336 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.336 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.336 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.336 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.337 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.337 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.337 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.337 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.337 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.337 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.337 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.337 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.337 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2024-09-12T03:29:02.337 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.337 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.337 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.337 INFO:teuthology.orchestra.run.smithi006.stdout: "fail_fs": { 2024-09-12T03:29:02.338 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "fail_fs", 2024-09-12T03:29:02.338 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.338 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.338 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.338 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.338 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.338 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.338 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.338 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:29:02.338 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.338 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.338 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.338 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.338 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.339 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.339 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.339 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.339 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.339 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.339 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.339 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.339 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.339 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.339 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.339 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.339 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.339 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.340 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.340 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.340 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.340 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.340 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.340 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.340 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.340 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.340 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.340 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.340 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.340 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.341 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.341 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.341 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.341 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.341 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.341 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.341 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.341 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.341 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.341 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.341 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.341 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.341 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.342 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.342 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.342 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.342 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.342 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.342 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.342 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.342 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.342 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.342 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.342 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.342 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.342 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.343 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.343 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.343 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.343 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.343 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.343 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.343 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.343 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.343 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.343 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.343 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.343 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.344 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.344 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.344 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.344 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.344 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.344 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.344 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator": { 2024-09-12T03:29:02.344 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2024-09-12T03:29:02.344 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.344 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.344 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.344 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.345 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.345 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.345 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.345 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm", 2024-09-12T03:29:02.345 INFO:teuthology.orchestra.run.smithi006.stdout: "rook", 2024-09-12T03:29:02.345 INFO:teuthology.orchestra.run.smithi006.stdout: "test_orchestrator" 2024-09-12T03:29:02.345 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.345 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Orchestrator backend", 2024-09-12T03:29:02.345 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.345 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.345 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.346 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.346 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.346 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.346 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.346 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_perf_query", 2024-09-12T03:29:02.346 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.346 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.346 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.346 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.346 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.346 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.346 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.347 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.347 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.347 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.347 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.347 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.347 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.347 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.347 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.347 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.347 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.348 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.348 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.348 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.348 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.348 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.348 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.348 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.348 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.348 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.348 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.348 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.349 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.349 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.349 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.349 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.349 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.349 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.349 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.349 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.349 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.349 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.349 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.350 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.350 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.350 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.350 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.350 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.350 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.350 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.350 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.350 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.350 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.350 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.350 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.350 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.350 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.351 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.351 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.351 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.351 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.351 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.351 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.351 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.351 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.351 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.351 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.351 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.351 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.351 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.352 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.352 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.352 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.352 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.352 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.352 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.352 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.352 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.352 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.352 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.352 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_support", 2024-09-12T03:29:02.352 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.352 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.353 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.353 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.353 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.353 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.353 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.353 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.353 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.353 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.353 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.353 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.353 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.353 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.353 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.354 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.354 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.354 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.354 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.354 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.354 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.354 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.354 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.354 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.354 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.354 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.354 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.354 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.354 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.355 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.355 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.355 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.355 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.355 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.355 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.355 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.355 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.355 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.355 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.355 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.355 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.355 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.356 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.356 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.356 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.356 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.356 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.356 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.356 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.356 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.356 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.356 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.356 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.356 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.356 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.356 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.357 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.357 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.357 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.357 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.357 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.357 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.357 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.357 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.357 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.357 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.357 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.357 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.357 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.357 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.358 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.358 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.358 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.358 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.358 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.358 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.358 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_autoscaler", 2024-09-12T03:29:02.358 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.358 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.358 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.358 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.358 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.358 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.359 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.359 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.359 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.359 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.359 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.359 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.359 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.359 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.359 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.359 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.359 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.359 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.359 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.359 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.360 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.360 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.360 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.360 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.360 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.360 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.360 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.360 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.360 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.360 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.360 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.360 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.360 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.360 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.361 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.361 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.361 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.361 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.361 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.361 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.361 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.361 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.361 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.361 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.361 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.361 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.361 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.362 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.362 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.362 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.362 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.362 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.362 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.362 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.362 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.362 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.362 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.362 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.362 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.363 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.363 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.363 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.363 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.363 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.363 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.363 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.363 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.363 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.363 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.363 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.363 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.363 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.364 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.364 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-09-12T03:29:02.364 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-09-12T03:29:02.364 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.364 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.364 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.364 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-09-12T03:29:02.364 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.364 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.364 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.364 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.364 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.364 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.365 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.365 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.365 INFO:teuthology.orchestra.run.smithi006.stdout: "threshold": { 2024-09-12T03:29:02.365 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threshold", 2024-09-12T03:29:02.365 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-09-12T03:29:02.365 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.365 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.365 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2024-09-12T03:29:02.365 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1.0", 2024-09-12T03:29:02.365 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.365 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.365 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "scaling threshold", 2024-09-12T03:29:02.365 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:29:02.366 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.366 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.366 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.366 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.366 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.366 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.366 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "progress", 2024-09-12T03:29:02.366 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.366 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.366 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.366 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_pg_recovery_event": { 2024-09-12T03:29:02.366 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:29:02.366 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.366 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.367 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.367 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.367 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.367 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.367 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.367 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:29:02.367 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.367 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.367 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.367 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.367 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2024-09-12T03:29:02.367 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2024-09-12T03:29:02.367 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.368 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.368 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.368 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.368 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.368 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.368 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.368 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.368 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.368 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.368 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.368 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.368 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.368 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.369 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.369 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.369 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.369 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.369 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.369 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.369 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.369 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.369 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.369 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.369 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.369 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.369 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.370 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.370 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.370 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.370 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.370 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.370 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.370 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.370 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.370 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.370 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.370 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.370 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.370 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.370 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.371 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.371 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.371 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.371 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.371 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.371 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.371 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.371 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.371 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.371 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.371 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.371 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.371 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.371 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.372 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.372 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.372 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.372 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.372 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.372 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.372 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.372 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.372 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.372 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.372 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.372 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.372 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.373 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.373 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.373 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.373 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.373 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.373 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.373 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.373 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.373 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.373 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.373 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.373 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.373 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.373 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.374 INFO:teuthology.orchestra.run.smithi006.stdout: "max_completed_events": { 2024-09-12T03:29:02.374 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_completed_events", 2024-09-12T03:29:02.374 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.374 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.374 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.374 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "50", 2024-09-12T03:29:02.374 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.374 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.374 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.374 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:29:02.374 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.374 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.375 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.375 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.375 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-09-12T03:29:02.375 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-09-12T03:29:02.375 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.375 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.375 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.375 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-09-12T03:29:02.375 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.375 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.375 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.375 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:29:02.375 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus", 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi006.stdout: "cache": { 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cache", 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi006.stdout: "exclude_perf_counters": { 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "exclude_perf_counters", 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools": { 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools", 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "300", 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.386 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.386 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.386 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.386 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.386 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.386 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.386 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.386 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_interval": { 2024-09-12T03:29:02.386 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_interval", 2024-09-12T03:29:02.386 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-09-12T03:29:02.386 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.386 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.386 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15.0", 2024-09-12T03:29:02.387 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.387 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.387 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.387 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.387 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.387 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.387 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.387 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.387 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-09-12T03:29:02.387 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-09-12T03:29:02.387 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.387 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.387 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.387 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2024-09-12T03:29:02.388 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.388 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.388 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.388 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:29:02.388 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.388 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.388 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.388 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.388 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-09-12T03:29:02.388 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-09-12T03:29:02.388 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.388 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.388 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.389 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "9283", 2024-09-12T03:29:02.389 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.389 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.389 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.389 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:29:02.389 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.389 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.389 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.389 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.389 INFO:teuthology.orchestra.run.smithi006.stdout: "stale_cache_strategy": { 2024-09-12T03:29:02.389 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stale_cache_strategy", 2024-09-12T03:29:02.389 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.389 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.389 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "log", 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "default", 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi006.stdout: "default", 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi006.stdout: "error" 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_support", 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.395 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.395 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.395 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.395 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.395 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.395 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.395 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.395 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.395 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.395 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.395 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.395 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.395 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.396 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.396 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.396 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.396 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.396 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.396 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.396 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.396 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.396 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.396 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.396 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.396 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.396 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.398 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.398 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.398 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.398 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.398 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.398 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.398 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.398 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.398 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.398 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.398 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.398 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.398 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.398 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_snap_create": { 2024-09-12T03:29:02.399 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:29:02.399 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.399 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.399 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.399 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:02.399 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.399 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.399 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.399 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.399 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.399 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.399 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.399 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.400 INFO:teuthology.orchestra.run.smithi006.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:29:02.400 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:29:02.400 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.400 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.400 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.400 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.400 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.400 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.400 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.400 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.400 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.400 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.400 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.401 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.401 INFO:teuthology.orchestra.run.smithi006.stdout: "trash_purge_schedule": { 2024-09-12T03:29:02.401 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "trash_purge_schedule", 2024-09-12T03:29:02.401 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.401 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.401 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.401 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.401 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.401 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.401 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.401 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.401 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.401 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.401 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.402 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.402 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.402 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.402 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.402 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "restful", 2024-09-12T03:29:02.402 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.402 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.402 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.402 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_auth": { 2024-09-12T03:29:02.402 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_auth", 2024-09-12T03:29:02.402 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.402 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.402 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.403 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.403 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.403 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.403 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.403 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.403 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.403 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.403 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.403 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.403 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2024-09-12T03:29:02.403 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2024-09-12T03:29:02.403 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.403 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.403 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.404 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.404 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.404 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.404 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.404 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.404 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.404 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.404 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.404 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.404 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.404 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.404 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.404 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.405 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.405 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.405 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.405 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.405 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.405 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.405 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.405 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.405 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.405 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.405 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.405 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.405 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.406 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.406 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.406 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.406 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.406 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.406 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.406 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.406 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.406 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.406 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.406 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.406 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.406 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.407 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.407 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.407 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.407 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.407 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.407 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.407 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.407 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.407 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.407 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.407 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.407 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.407 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.408 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.408 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.408 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.408 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.408 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.408 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.408 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.408 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.408 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.408 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.408 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.408 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.408 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.409 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.409 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.409 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.409 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.409 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.409 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.409 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.409 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.409 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.409 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.409 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.409 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.409 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.409 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.410 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-09-12T03:29:02.410 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-09-12T03:29:02.410 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.410 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.410 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.410 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.410 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.410 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.410 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.410 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.410 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.410 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.410 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.411 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.411 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-09-12T03:29:02.411 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-09-12T03:29:02.411 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.411 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.411 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.411 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.411 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.411 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.411 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.411 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.411 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.411 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.412 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.412 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.412 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.412 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.412 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.412 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rgw", 2024-09-12T03:29:02.412 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.412 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.412 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.412 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.412 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.412 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.412 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.413 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.413 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.413 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.413 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.413 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.413 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.413 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.413 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.413 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.413 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.413 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.413 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.413 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.414 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.414 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.414 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.414 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.414 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.414 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.414 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.414 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.414 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.414 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.414 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.414 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.414 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.415 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.415 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.415 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.415 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.415 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.415 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.415 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.415 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.415 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.415 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.415 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.415 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.415 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.415 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.416 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.416 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.416 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.416 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.416 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.416 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.416 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.416 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.416 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.416 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.416 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.416 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.416 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.417 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.417 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.417 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.417 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.417 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.417 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.417 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.417 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.417 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.417 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.417 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.417 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.418 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.418 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.418 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.418 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.418 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rook", 2024-09-12T03:29:02.418 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.418 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.418 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.418 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.418 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.418 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.419 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.419 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.419 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.419 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.419 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.419 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.419 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.419 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.419 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.419 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.419 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.419 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.420 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.420 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.420 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.420 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.420 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.420 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.420 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.420 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.420 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.420 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.420 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.421 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.421 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.421 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.421 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.421 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.421 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.421 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.421 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.421 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.421 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.421 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.421 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.422 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.422 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.422 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.422 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.422 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.422 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.422 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.422 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.422 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.422 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.422 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.423 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.423 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.423 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.423 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.423 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.423 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.423 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.423 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.423 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.423 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.423 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.424 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.424 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.424 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.424 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.424 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.424 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.424 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.424 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.424 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.424 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.424 INFO:teuthology.orchestra.run.smithi006.stdout: "storage_class": { 2024-09-12T03:29:02.425 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "storage_class", 2024-09-12T03:29:02.425 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.425 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.425 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.425 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "local", 2024-09-12T03:29:02.425 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.425 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.425 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.425 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:29:02.425 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.425 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.425 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.426 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.426 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.426 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.426 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.426 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "selftest", 2024-09-12T03:29:02.426 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.426 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.426 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.426 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.426 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.426 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.426 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.427 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.427 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.427 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.427 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.427 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.427 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.427 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.427 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.427 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.427 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.427 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.428 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.428 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.428 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.428 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.428 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.428 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.428 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.428 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.428 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.428 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.428 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.428 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.429 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.429 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.429 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.429 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.429 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.429 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.429 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.429 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.429 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.429 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.429 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.429 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.429 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.429 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.430 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.430 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.430 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.430 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.430 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.430 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.430 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.430 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.430 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.430 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.430 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.430 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.430 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.431 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.431 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.431 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.431 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.431 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.431 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.431 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.431 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.431 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.431 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.431 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.431 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.431 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.431 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.432 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.432 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.432 INFO:teuthology.orchestra.run.smithi006.stdout: "roption1": { 2024-09-12T03:29:02.432 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption1", 2024-09-12T03:29:02.432 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.432 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.432 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.432 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.432 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.432 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.432 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.432 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.432 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.433 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.433 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.433 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.433 INFO:teuthology.orchestra.run.smithi006.stdout: "roption2": { 2024-09-12T03:29:02.433 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption2", 2024-09-12T03:29:02.433 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.433 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.433 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.433 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "xyz", 2024-09-12T03:29:02.433 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.433 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.433 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.436 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.436 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.436 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.437 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.437 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.437 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption1": { 2024-09-12T03:29:02.437 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption1", 2024-09-12T03:29:02.437 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.437 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.437 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.437 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.437 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.437 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.437 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.437 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.437 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.438 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.438 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.438 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.438 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption2": { 2024-09-12T03:29:02.438 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption2", 2024-09-12T03:29:02.438 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.438 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.438 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.438 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.438 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.438 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.438 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.438 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.439 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.439 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.439 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.439 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.439 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption3": { 2024-09-12T03:29:02.439 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption3", 2024-09-12T03:29:02.439 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-09-12T03:29:02.439 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.439 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.439 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.439 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.439 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.439 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.440 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.440 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.440 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.440 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.440 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.440 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption4": { 2024-09-12T03:29:02.440 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption4", 2024-09-12T03:29:02.440 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.440 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.440 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.440 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.440 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.440 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.441 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.441 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.441 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.441 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.441 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.441 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.441 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption5": { 2024-09-12T03:29:02.441 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption5", 2024-09-12T03:29:02.441 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.441 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.441 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.441 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.441 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.442 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.442 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.442 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.442 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.442 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.442 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.442 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.442 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption6": { 2024-09-12T03:29:02.442 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption6", 2024-09-12T03:29:02.442 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.442 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.442 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.442 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.443 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.443 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.443 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.443 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.443 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.443 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.443 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.443 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.443 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption7": { 2024-09-12T03:29:02.443 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption7", 2024-09-12T03:29:02.443 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.443 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.443 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.444 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.444 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-09-12T03:29:02.444 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "42", 2024-09-12T03:29:02.444 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.444 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.444 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.444 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.444 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.444 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.444 INFO:teuthology.orchestra.run.smithi006.stdout: "testkey": { 2024-09-12T03:29:02.444 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testkey", 2024-09-12T03:29:02.444 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.444 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.445 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.445 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.445 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.445 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.445 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.445 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.445 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.445 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.445 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.445 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.445 INFO:teuthology.orchestra.run.smithi006.stdout: "testlkey": { 2024-09-12T03:29:02.445 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testlkey", 2024-09-12T03:29:02.445 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.445 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.446 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.446 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.446 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.446 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.446 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.446 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.446 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.446 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.446 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.446 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.446 INFO:teuthology.orchestra.run.smithi006.stdout: "testnewline": { 2024-09-12T03:29:02.447 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testnewline", 2024-09-12T03:29:02.447 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.447 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.447 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.447 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.447 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.447 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.447 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.447 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.447 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.447 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.447 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.448 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.448 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.448 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.448 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.448 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snap_schedule", 2024-09-12T03:29:02.448 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.448 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.448 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.448 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_m_granularity": { 2024-09-12T03:29:02.448 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_m_granularity", 2024-09-12T03:29:02.448 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.448 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.448 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.449 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.449 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.449 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.449 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.449 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:29:02.449 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.449 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.449 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.449 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.449 INFO:teuthology.orchestra.run.smithi006.stdout: "dump_on_update": { 2024-09-12T03:29:02.449 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dump_on_update", 2024-09-12T03:29:02.449 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.449 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.450 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.450 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.450 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.450 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.450 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.450 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:29:02.450 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.450 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.450 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.450 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.450 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.450 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.450 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.450 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.451 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.451 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.451 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.451 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.451 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.451 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.451 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.451 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.451 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.451 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.451 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.451 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.451 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.451 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.452 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.452 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.452 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.452 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.452 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.452 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.452 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.452 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.452 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.452 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.452 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.452 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.452 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.453 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.453 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.453 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.453 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.453 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.453 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.453 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.453 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.453 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.453 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.453 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.453 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.453 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.453 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.454 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.454 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.454 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.454 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.454 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.454 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.454 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.454 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.454 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.454 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.454 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.454 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.455 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.455 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.455 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.455 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.455 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.455 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.455 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.455 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.455 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.455 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.455 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.455 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.456 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.456 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.456 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.456 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.456 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stats", 2024-09-12T03:29:02.456 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.456 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.456 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.456 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.456 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.456 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.456 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.456 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.457 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.457 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.457 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.457 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.457 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.457 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.457 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.457 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.457 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.457 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.457 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.457 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.457 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.458 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.458 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.458 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.458 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.458 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.458 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.458 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.458 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.458 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.458 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.458 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.458 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.458 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.458 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.459 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.459 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.459 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.459 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.459 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.459 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.459 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.459 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.459 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.459 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.459 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.459 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.459 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.460 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.460 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.460 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.460 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.460 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.460 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.460 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.460 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.460 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.460 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.460 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.460 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.460 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.461 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.461 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.461 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.461 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.461 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.461 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.461 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.461 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.461 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.461 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.461 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.461 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.461 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.461 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.462 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.462 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "status", 2024-09-12T03:29:02.462 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.462 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.462 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.462 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.462 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.462 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.462 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.462 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.462 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.462 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.462 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.463 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.463 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.463 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.463 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.463 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.463 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.463 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.463 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.463 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.463 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.463 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.463 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.463 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.463 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.464 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.464 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.464 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.464 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.464 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.464 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.464 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.464 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.464 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.464 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.464 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.464 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.464 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.465 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.465 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.465 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.465 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.465 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.465 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.465 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.465 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.465 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.465 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.465 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.465 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.465 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.466 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.466 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.466 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.466 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.466 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.466 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.466 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.466 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.466 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.466 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.466 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.466 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.466 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.466 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.467 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.467 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.467 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.467 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.467 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.467 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.467 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.467 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.467 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.467 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.467 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.467 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telegraf", 2024-09-12T03:29:02.467 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.467 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.468 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.468 INFO:teuthology.orchestra.run.smithi006.stdout: "address": { 2024-09-12T03:29:02.468 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "address", 2024-09-12T03:29:02.468 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.468 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.468 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.468 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:29:02.468 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.468 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.468 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.468 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.468 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.468 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.469 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.469 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.469 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-09-12T03:29:02.469 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-09-12T03:29:02.469 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.469 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.469 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.469 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15", 2024-09-12T03:29:02.469 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.469 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.469 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.469 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.470 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.470 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.470 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.470 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.470 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.470 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.470 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.470 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.470 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.470 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.470 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.470 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.470 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.470 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.471 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.471 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.471 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.471 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.471 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.471 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.471 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.471 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.471 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.471 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.471 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.471 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.471 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.472 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.472 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.472 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.472 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.472 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.472 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.472 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.472 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.472 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.472 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.472 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.472 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.472 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.473 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.473 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.473 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.473 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.473 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.473 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.473 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.473 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.473 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.473 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.473 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.473 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.473 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.474 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.474 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.474 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.474 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.474 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.474 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.474 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.474 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.474 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.474 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.474 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.474 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.474 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.474 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.475 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.475 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.475 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.475 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.475 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.475 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.475 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.475 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.475 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.475 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.475 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telemetry", 2024-09-12T03:29:02.475 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.475 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.476 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.476 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_basic": { 2024-09-12T03:29:02.476 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_basic", 2024-09-12T03:29:02.476 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.476 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.476 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.476 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.476 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.476 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.476 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.476 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:29:02.476 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.476 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.477 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.477 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.477 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_crash": { 2024-09-12T03:29:02.477 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_crash", 2024-09-12T03:29:02.477 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.477 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.477 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.477 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.477 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.477 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.477 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.477 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:29:02.477 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.478 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.478 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.478 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.478 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_device": { 2024-09-12T03:29:02.478 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_device", 2024-09-12T03:29:02.478 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.478 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.478 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.478 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.478 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.478 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.478 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.478 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:29:02.478 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.479 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.479 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.479 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.479 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_ident": { 2024-09-12T03:29:02.479 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_ident", 2024-09-12T03:29:02.479 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.479 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.479 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.479 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.479 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.479 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.479 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.479 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:29:02.480 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.480 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.480 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.480 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.480 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_perf": { 2024-09-12T03:29:02.480 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_perf", 2024-09-12T03:29:02.480 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.480 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.480 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.480 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.480 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.480 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.481 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.481 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:29:02.481 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.481 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.481 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.481 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.481 INFO:teuthology.orchestra.run.smithi006.stdout: "contact": { 2024-09-12T03:29:02.481 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "contact", 2024-09-12T03:29:02.481 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.481 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.481 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.481 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.481 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.482 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.482 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.482 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.482 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.482 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.482 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.482 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.482 INFO:teuthology.orchestra.run.smithi006.stdout: "description": { 2024-09-12T03:29:02.482 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "description", 2024-09-12T03:29:02.482 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.482 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.482 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.482 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.483 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.483 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.483 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.483 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.483 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.483 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.483 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.483 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.483 INFO:teuthology.orchestra.run.smithi006.stdout: "device_url": { 2024-09-12T03:29:02.483 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_url", 2024-09-12T03:29:02.483 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.483 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.484 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.484 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:29:02.484 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.484 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.484 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.484 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.484 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.484 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.484 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.484 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.484 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2024-09-12T03:29:02.484 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2024-09-12T03:29:02.484 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.485 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.485 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.485 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.485 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.485 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.485 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.485 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.485 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.485 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.485 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.485 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.485 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-09-12T03:29:02.485 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-09-12T03:29:02.486 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.486 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.486 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.486 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "24", 2024-09-12T03:29:02.486 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "8", 2024-09-12T03:29:02.486 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.486 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.486 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.486 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.486 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.486 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.486 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.486 INFO:teuthology.orchestra.run.smithi006.stdout: "last_opt_revision": { 2024-09-12T03:29:02.486 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "last_opt_revision", 2024-09-12T03:29:02.487 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.487 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.487 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.487 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1", 2024-09-12T03:29:02.487 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.487 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.487 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.487 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.487 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.487 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.487 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.487 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.487 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard": { 2024-09-12T03:29:02.488 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard", 2024-09-12T03:29:02.488 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.488 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.488 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.488 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.488 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.488 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.488 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.488 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.488 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.488 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.488 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.488 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.489 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard_description": { 2024-09-12T03:29:02.489 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard_description", 2024-09-12T03:29:02.489 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.489 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.489 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.489 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.489 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.489 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.489 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.489 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.489 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.489 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.489 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.489 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.490 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.490 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.490 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.490 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.490 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.490 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.490 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.490 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.490 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.490 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.490 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.490 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.490 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.491 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.491 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.491 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.491 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.491 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.491 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.491 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.491 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.491 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.491 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.491 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.491 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.491 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.492 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.492 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.492 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.492 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.492 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.492 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.492 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.492 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.492 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.492 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.492 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.492 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.492 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.493 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.493 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.493 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.493 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.493 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.493 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.493 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.493 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.493 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.493 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.493 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.493 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.493 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.494 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.494 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.494 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.494 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.494 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.494 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.494 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.494 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.494 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.494 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.494 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.494 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.494 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.495 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.495 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.495 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.495 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.495 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.495 INFO:teuthology.orchestra.run.smithi006.stdout: "organization": { 2024-09-12T03:29:02.495 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "organization", 2024-09-12T03:29:02.495 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.495 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.495 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.495 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.495 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.495 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.496 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.496 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.496 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.496 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.496 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.496 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.496 INFO:teuthology.orchestra.run.smithi006.stdout: "proxy": { 2024-09-12T03:29:02.496 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "proxy", 2024-09-12T03:29:02.496 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.496 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.496 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.496 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.496 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.496 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.497 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.497 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.497 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.497 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.497 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.497 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.497 INFO:teuthology.orchestra.run.smithi006.stdout: "url": { 2024-09-12T03:29:02.497 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url", 2024-09-12T03:29:02.497 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.497 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.497 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.497 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:29:02.497 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.498 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.498 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.498 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.498 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.498 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.498 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.498 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.498 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.498 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.498 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.498 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "test_orchestrator", 2024-09-12T03:29:02.498 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.498 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.499 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.499 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.499 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.499 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.499 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.499 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.499 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.499 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.499 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.499 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.499 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.499 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.499 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.500 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.500 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.500 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.500 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.500 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.500 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.500 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.500 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.500 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.500 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.500 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.500 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.500 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.501 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.501 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.501 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.501 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.501 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.501 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.501 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.501 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.501 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.501 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.501 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.501 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.501 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.501 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.502 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.502 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.502 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.502 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.502 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.502 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.502 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.502 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.502 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.502 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.502 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.502 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.502 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.503 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.503 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.503 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.503 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.503 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.503 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.503 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.503 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.503 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.503 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.503 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.503 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.503 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.503 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.504 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.504 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.504 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.504 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.504 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.504 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.504 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.504 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "volumes", 2024-09-12T03:29:02.504 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.504 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.504 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.504 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.505 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.505 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.505 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.505 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.505 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.505 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.505 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.505 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.505 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.505 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.505 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.505 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.505 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.505 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.506 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.506 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.506 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.506 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.506 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.506 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.506 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.506 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.506 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.506 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.506 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.506 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.506 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.507 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.507 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.507 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.507 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.507 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.507 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.507 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.507 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.507 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.507 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.507 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.507 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.507 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.507 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.508 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.508 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.508 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.508 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.508 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.508 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.508 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.508 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.508 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.508 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.508 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.508 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.508 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.509 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.509 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.509 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.509 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.509 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.509 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.509 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.509 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.509 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.509 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.509 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.509 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.509 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.509 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.510 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.510 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_clones": { 2024-09-12T03:29:02.510 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_clones", 2024-09-12T03:29:02.510 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.510 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.510 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.510 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4", 2024-09-12T03:29:02.510 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.510 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.510 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.510 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:29:02.510 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.510 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.511 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.511 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.511 INFO:teuthology.orchestra.run.smithi006.stdout: "periodic_async_work": { 2024-09-12T03:29:02.511 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "periodic_async_work", 2024-09-12T03:29:02.511 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.511 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.511 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.511 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.511 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.511 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.511 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.511 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Periodically check for async work", 2024-09-12T03:29:02.511 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.511 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.512 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.512 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.512 INFO:teuthology.orchestra.run.smithi006.stdout: "snapshot_clone_delay": { 2024-09-12T03:29:02.512 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:29:02.512 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.512 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.512 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.512 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-09-12T03:29:02.512 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.512 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.512 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.513 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:29:02.513 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.513 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.513 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.513 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.513 INFO:teuthology.orchestra.run.smithi006.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:29:02.513 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:29:02.513 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.513 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.513 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.513 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.513 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.513 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.514 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.514 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:29:02.514 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.514 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.514 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.514 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.514 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.514 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.514 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.514 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix", 2024-09-12T03:29:02.514 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.514 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.514 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.515 INFO:teuthology.orchestra.run.smithi006.stdout: "discovery_interval": { 2024-09-12T03:29:02.515 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "discovery_interval", 2024-09-12T03:29:02.515 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-09-12T03:29:02.515 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.515 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.515 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "100", 2024-09-12T03:29:02.515 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.515 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.515 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.515 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.515 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.515 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.515 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.515 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.516 INFO:teuthology.orchestra.run.smithi006.stdout: "identifier": { 2024-09-12T03:29:02.516 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "identifier", 2024-09-12T03:29:02.516 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.516 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.516 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.516 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.516 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.516 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.516 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.516 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.516 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.516 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.516 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.517 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.517 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-09-12T03:29:02.517 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-09-12T03:29:02.517 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.517 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.517 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.517 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-09-12T03:29:02.517 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.517 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.517 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.517 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.517 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.517 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.518 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.518 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.518 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.518 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.518 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.518 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.518 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.518 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.518 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.518 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.518 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.518 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.518 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.519 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.519 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.519 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.519 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.519 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.519 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.519 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.519 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.519 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.519 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.519 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.519 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.519 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.519 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.520 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.520 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.520 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.520 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.520 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.520 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.520 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.520 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.520 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.520 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.520 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.520 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.521 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.521 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.521 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.521 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.521 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.521 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.521 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.521 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.521 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.521 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.521 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.521 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.521 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.522 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.522 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.522 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.522 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.522 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.522 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.522 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.522 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.522 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.522 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.522 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.522 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.522 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.522 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.523 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.523 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.523 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.523 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.523 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.523 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.523 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_host": { 2024-09-12T03:29:02.523 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_host", 2024-09-12T03:29:02.523 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.523 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.523 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.523 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.523 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.523 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.524 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.524 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.524 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.524 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.524 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.524 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.524 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_port": { 2024-09-12T03:29:02.524 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_port", 2024-09-12T03:29:02.524 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.524 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.524 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.524 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10051", 2024-09-12T03:29:02.524 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.525 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.525 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.525 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.525 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.525 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.525 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.525 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.525 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_sender": { 2024-09-12T03:29:02.525 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_sender", 2024-09-12T03:29:02.525 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.525 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.525 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.525 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:29:02.526 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.526 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.526 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.526 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.526 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.526 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.526 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.526 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.526 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.526 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.526 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-09-12T03:29:02.526 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.526 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.526 INFO:teuthology.orchestra.run.smithi006.stdout: "modules": [ 2024-09-12T03:29:02.527 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm", 2024-09-12T03:29:02.527 INFO:teuthology.orchestra.run.smithi006.stdout: "dashboard", 2024-09-12T03:29:02.527 INFO:teuthology.orchestra.run.smithi006.stdout: "iostat", 2024-09-12T03:29:02.527 INFO:teuthology.orchestra.run.smithi006.stdout: "nfs", 2024-09-12T03:29:02.527 INFO:teuthology.orchestra.run.smithi006.stdout: "restful" 2024-09-12T03:29:02.527 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.527 INFO:teuthology.orchestra.run.smithi006.stdout: "available_modules": [ 2024-09-12T03:29:02.527 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.527 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "alerts", 2024-09-12T03:29:02.527 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.527 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.527 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.527 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-09-12T03:29:02.527 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-09-12T03:29:02.528 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.528 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.528 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.528 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-09-12T03:29:02.528 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.528 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.528 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.528 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:29:02.528 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.528 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.528 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.529 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.529 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.529 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.529 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.529 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.529 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.529 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.529 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.529 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.529 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.529 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.529 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.529 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.530 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.530 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.530 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.530 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.530 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.530 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.530 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.530 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.530 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.530 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.530 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.530 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.530 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.531 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.531 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.531 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.531 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.531 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.531 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.531 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.531 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.531 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.531 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.531 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.531 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.531 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.532 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.532 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.532 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.532 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.532 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.532 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.532 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.532 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.532 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.532 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.532 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.532 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.532 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.532 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.533 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.533 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.533 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.533 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.533 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.533 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.533 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.533 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.533 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.533 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.533 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.533 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.533 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.534 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.534 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.534 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.534 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.534 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.534 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_destination": { 2024-09-12T03:29:02.534 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_destination", 2024-09-12T03:29:02.534 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.534 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.534 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.534 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.534 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.535 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.535 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.535 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:29:02.535 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.535 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.535 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.535 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.535 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_from_name": { 2024-09-12T03:29:02.535 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_from_name", 2024-09-12T03:29:02.535 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.535 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.535 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.535 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "Ceph", 2024-09-12T03:29:02.536 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.536 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.536 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.536 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email From: name", 2024-09-12T03:29:02.536 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.536 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.536 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.536 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.536 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_host": { 2024-09-12T03:29:02.536 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_host", 2024-09-12T03:29:02.536 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.536 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.536 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.537 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.537 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.537 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.537 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.537 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP server", 2024-09-12T03:29:02.537 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.537 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.537 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.537 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.537 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_password": { 2024-09-12T03:29:02.537 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_password", 2024-09-12T03:29:02.537 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.537 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.538 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.538 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.538 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.538 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.538 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.538 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Password to authenticate with", 2024-09-12T03:29:02.538 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.538 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.538 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.538 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.538 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_port": { 2024-09-12T03:29:02.538 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_port", 2024-09-12T03:29:02.538 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.538 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.539 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.539 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "465", 2024-09-12T03:29:02.539 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.539 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.539 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.539 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP port", 2024-09-12T03:29:02.539 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.539 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.539 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.539 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.539 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_sender": { 2024-09-12T03:29:02.539 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_sender", 2024-09-12T03:29:02.539 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.539 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.540 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.540 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.540 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.540 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.540 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.540 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:29:02.540 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.540 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.540 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.540 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.540 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_ssl": { 2024-09-12T03:29:02.540 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_ssl", 2024-09-12T03:29:02.540 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.541 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.541 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.541 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.541 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.541 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.541 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.541 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:29:02.541 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.541 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.541 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.541 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.541 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_user": { 2024-09-12T03:29:02.541 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_user", 2024-09-12T03:29:02.542 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.542 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.542 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.542 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.542 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.542 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.542 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.542 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "User to authenticate as", 2024-09-12T03:29:02.542 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.542 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.542 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.542 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.542 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.542 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.543 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.543 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "balancer", 2024-09-12T03:29:02.543 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.543 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.543 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.543 INFO:teuthology.orchestra.run.smithi006.stdout: "active": { 2024-09-12T03:29:02.543 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "active", 2024-09-12T03:29:02.543 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.543 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.543 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.543 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.543 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.543 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.543 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.544 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:29:02.544 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.544 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.544 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.544 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.544 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_time": { 2024-09-12T03:29:02.544 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_time", 2024-09-12T03:29:02.544 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.544 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.544 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.544 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0000", 2024-09-12T03:29:02.544 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.544 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.545 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.545 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:29:02.545 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:29:02.545 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.545 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.545 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.545 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_weekday": { 2024-09-12T03:29:02.545 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_weekday", 2024-09-12T03:29:02.545 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-09-12T03:29:02.545 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.545 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.545 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-09-12T03:29:02.545 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2024-09-12T03:29:02.546 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2024-09-12T03:29:02.546 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.546 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:29:02.546 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:29:02.546 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.546 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.546 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.546 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_max_iterations": { 2024-09-12T03:29:02.546 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:29:02.546 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-09-12T03:29:02.546 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.546 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.546 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "25", 2024-09-12T03:29:02.547 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-09-12T03:29:02.547 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "250", 2024-09-12T03:29:02.547 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.547 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:29:02.547 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.547 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.547 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.547 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.547 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_metrics": { 2024-09-12T03:29:02.547 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_metrics", 2024-09-12T03:29:02.547 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.547 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.547 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.548 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:29:02.548 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.548 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.548 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.548 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:29:02.548 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:29:02.548 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.548 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.548 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.548 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_step": { 2024-09-12T03:29:02.548 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_step", 2024-09-12T03:29:02.548 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-09-12T03:29:02.548 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.549 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.549 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.5", 2024-09-12T03:29:02.549 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0.001", 2024-09-12T03:29:02.549 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "0.999", 2024-09-12T03:29:02.549 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.549 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:29:02.549 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:29:02.549 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.549 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.549 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.549 INFO:teuthology.orchestra.run.smithi006.stdout: "end_time": { 2024-09-12T03:29:02.549 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_time", 2024-09-12T03:29:02.549 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.550 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.550 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.550 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2359", 2024-09-12T03:29:02.550 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.550 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.550 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.550 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:29:02.550 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:29:02.550 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.550 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.550 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.550 INFO:teuthology.orchestra.run.smithi006.stdout: "end_weekday": { 2024-09-12T03:29:02.550 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_weekday", 2024-09-12T03:29:02.550 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-09-12T03:29:02.551 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.551 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.551 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-09-12T03:29:02.551 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2024-09-12T03:29:02.551 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2024-09-12T03:29:02.551 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.551 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:29:02.551 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:29:02.551 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.551 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.551 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.551 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.551 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.552 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.552 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.552 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.552 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.552 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.552 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.552 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.552 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.552 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.552 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.552 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.552 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.552 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.553 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.553 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.553 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.553 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.553 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.553 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.553 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.553 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.553 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.553 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.553 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.553 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.553 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.554 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.554 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.554 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.554 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.554 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.554 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.554 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.554 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.554 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.554 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.554 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.554 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.554 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.554 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.555 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.555 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.555 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.555 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.555 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.555 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.555 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.555 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.555 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.555 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.555 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.555 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.555 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.555 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.556 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.556 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.556 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.556 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.556 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.556 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.556 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.556 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.556 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.556 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.556 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.556 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.556 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.557 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.557 INFO:teuthology.orchestra.run.smithi006.stdout: "min_score": { 2024-09-12T03:29:02.557 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_score", 2024-09-12T03:29:02.557 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-09-12T03:29:02.557 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.557 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.557 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-09-12T03:29:02.557 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.557 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.557 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.557 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:29:02.557 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.557 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.558 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.558 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.558 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2024-09-12T03:29:02.558 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2024-09-12T03:29:02.558 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.558 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.558 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.558 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "upmap", 2024-09-12T03:29:02.558 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.558 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.558 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.558 INFO:teuthology.orchestra.run.smithi006.stdout: "crush-compat", 2024-09-12T03:29:02.558 INFO:teuthology.orchestra.run.smithi006.stdout: "none", 2024-09-12T03:29:02.558 INFO:teuthology.orchestra.run.smithi006.stdout: "read", 2024-09-12T03:29:02.559 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap", 2024-09-12T03:29:02.559 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap-read" 2024-09-12T03:29:02.559 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.559 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Balancer mode", 2024-09-12T03:29:02.559 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.559 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.559 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.559 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.559 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_ids": { 2024-09-12T03:29:02.559 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_ids", 2024-09-12T03:29:02.559 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.559 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.559 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.560 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.560 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.560 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.560 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.560 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:29:02.560 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.560 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.560 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.560 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.560 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-09-12T03:29:02.560 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-09-12T03:29:02.560 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.561 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.561 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.561 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-09-12T03:29:02.561 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.561 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.561 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.561 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:29:02.561 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.561 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.561 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.561 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.561 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_deviation": { 2024-09-12T03:29:02.561 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_deviation", 2024-09-12T03:29:02.562 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.562 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.562 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.562 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-09-12T03:29:02.562 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-09-12T03:29:02.562 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.562 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.562 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:29:02.562 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:29:02.562 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.562 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.562 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.562 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_optimizations": { 2024-09-12T03:29:02.563 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:29:02.563 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-09-12T03:29:02.563 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.563 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.563 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:02.563 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.563 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.563 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.563 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:29:02.563 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.563 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.563 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.563 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.564 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.564 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.564 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.564 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cephadm", 2024-09-12T03:29:02.564 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.564 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.564 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.564 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_down_multiplier": { 2024-09-12T03:29:02.564 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_down_multiplier", 2024-09-12T03:29:02.564 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-09-12T03:29:02.564 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.564 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.564 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2024-09-12T03:29:02.565 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.565 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.565 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.565 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:29:02.565 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.565 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.565 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.565 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.565 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_refresh_rate": { 2024-09-12T03:29:02.565 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_refresh_rate", 2024-09-12T03:29:02.565 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.565 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.565 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.566 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "20", 2024-09-12T03:29:02.566 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.566 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.566 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.566 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:29:02.566 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.566 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.566 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.566 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.566 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_starting_port": { 2024-09-12T03:29:02.566 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_starting_port", 2024-09-12T03:29:02.566 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.566 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.567 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.567 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4721", 2024-09-12T03:29:02.567 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.567 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.567 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.567 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:29:02.567 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.567 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.567 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.567 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.567 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_ptrace": { 2024-09-12T03:29:02.567 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_ptrace", 2024-09-12T03:29:02.567 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.568 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.568 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.568 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.568 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.568 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.568 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.568 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:29:02.568 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:29:02.568 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.568 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.568 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.568 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_interval": { 2024-09-12T03:29:02.568 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_interval", 2024-09-12T03:29:02.569 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.569 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.569 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.569 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-09-12T03:29:02.569 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.569 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.569 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.569 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:29:02.569 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.569 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.569 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.569 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.569 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:29:02.570 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:29:02.570 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-09-12T03:29:02.570 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.570 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.570 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.7", 2024-09-12T03:29:02.570 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.570 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.570 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.570 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:29:02.570 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.570 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.570 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.570 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.570 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm_log_destination": { 2024-09-12T03:29:02.571 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cephadm_log_destination", 2024-09-12T03:29:02.571 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.571 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.571 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.571 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.571 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.571 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.571 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.571 INFO:teuthology.orchestra.run.smithi006.stdout: "file", 2024-09-12T03:29:02.571 INFO:teuthology.orchestra.run.smithi006.stdout: "file,syslog", 2024-09-12T03:29:02.571 INFO:teuthology.orchestra.run.smithi006.stdout: "syslog" 2024-09-12T03:29:02.571 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.571 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:29:02.572 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.572 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.572 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.572 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.572 INFO:teuthology.orchestra.run.smithi006.stdout: "cgroups_split": { 2024-09-12T03:29:02.572 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cgroups_split", 2024-09-12T03:29:02.572 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.572 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.572 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.572 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.572 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.572 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.572 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.573 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:29:02.573 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.573 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.573 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.573 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.573 INFO:teuthology.orchestra.run.smithi006.stdout: "config_checks_enabled": { 2024-09-12T03:29:02.573 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_checks_enabled", 2024-09-12T03:29:02.573 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.573 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.573 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.573 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.573 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.573 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.573 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.574 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:29:02.574 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.574 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.574 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.574 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.574 INFO:teuthology.orchestra.run.smithi006.stdout: "config_dashboard": { 2024-09-12T03:29:02.574 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_dashboard", 2024-09-12T03:29:02.574 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.574 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.574 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.574 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.574 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.574 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.575 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.575 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:29:02.575 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.575 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.575 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.575 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.575 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_alertmanager": { 2024-09-12T03:29:02.575 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_alertmanager", 2024-09-12T03:29:02.575 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.575 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.575 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.575 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:29:02.575 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.575 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.576 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.576 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-09-12T03:29:02.576 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.576 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.576 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.576 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.576 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_base": { 2024-09-12T03:29:02.576 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_base", 2024-09-12T03:29:02.576 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.576 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.576 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.576 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:29:02.576 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.577 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.577 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.577 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:29:02.577 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.577 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.577 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.577 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.577 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_elasticsearch": { 2024-09-12T03:29:02.577 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:29:02.577 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.577 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.577 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.577 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:29:02.578 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.578 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.578 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.578 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "elasticsearch container image", 2024-09-12T03:29:02.578 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.578 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.578 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.578 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.578 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_grafana": { 2024-09-12T03:29:02.578 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_grafana", 2024-09-12T03:29:02.578 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.578 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.578 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.579 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:29:02.579 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.579 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.579 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.579 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-09-12T03:29:02.579 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.579 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.579 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.579 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.579 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_haproxy": { 2024-09-12T03:29:02.579 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_haproxy", 2024-09-12T03:29:02.579 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.579 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.580 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.580 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:29:02.580 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.580 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.580 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.580 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "HAproxy container image", 2024-09-12T03:29:02.580 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.580 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.580 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.580 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.580 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_agent": { 2024-09-12T03:29:02.580 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:29:02.580 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.581 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.581 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.581 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:29:02.581 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.581 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.581 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.581 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:29:02.581 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.581 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.581 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.581 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.581 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_collector": { 2024-09-12T03:29:02.581 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:29:02.582 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.582 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.582 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.582 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:29:02.582 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.582 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.582 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.582 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:29:02.582 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.582 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.582 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.582 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.582 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_query": { 2024-09-12T03:29:02.583 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:29:02.583 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.583 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.583 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.583 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:29:02.583 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.583 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.583 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.583 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger query container image", 2024-09-12T03:29:02.583 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.583 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.583 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.583 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.584 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_keepalived": { 2024-09-12T03:29:02.584 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_keepalived", 2024-09-12T03:29:02.584 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.584 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.584 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.584 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:29:02.584 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.584 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.584 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.584 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Keepalived container image", 2024-09-12T03:29:02.584 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.584 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.584 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.585 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.585 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_loki": { 2024-09-12T03:29:02.585 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_loki", 2024-09-12T03:29:02.585 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.585 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.585 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.585 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:29:02.585 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.585 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.585 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.585 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Loki container image", 2024-09-12T03:29:02.585 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.585 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.586 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.586 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.586 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_node_exporter": { 2024-09-12T03:29:02.586 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_node_exporter", 2024-09-12T03:29:02.586 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.586 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.586 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.586 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:29:02.586 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.586 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.586 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.586 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-09-12T03:29:02.586 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.587 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.587 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.587 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.587 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_nvmeof": { 2024-09-12T03:29:02.587 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_nvmeof", 2024-09-12T03:29:02.587 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.587 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.587 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.587 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:29:02.587 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.587 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.587 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.587 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Nvme-of container image", 2024-09-12T03:29:02.587 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.588 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.588 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.588 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.588 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_prometheus": { 2024-09-12T03:29:02.588 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_prometheus", 2024-09-12T03:29:02.588 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.588 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.588 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.588 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:29:02.588 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.588 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.588 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.588 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-09-12T03:29:02.588 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.589 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.589 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.589 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.589 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_promtail": { 2024-09-12T03:29:02.589 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_promtail", 2024-09-12T03:29:02.589 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.589 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.589 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.589 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:29:02.589 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.589 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.589 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.589 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Promtail container image", 2024-09-12T03:29:02.590 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.590 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.590 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.590 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.590 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_samba": { 2024-09-12T03:29:02.590 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_samba", 2024-09-12T03:29:02.590 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.590 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.590 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.590 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:29:02.590 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.590 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.590 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.590 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:29:02.591 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.591 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.591 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.591 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.591 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_snmp_gateway": { 2024-09-12T03:29:02.591 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:29:02.591 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.591 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.591 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.591 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:29:02.591 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.591 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.591 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.591 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:29:02.592 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.592 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.592 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.592 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.592 INFO:teuthology.orchestra.run.smithi006.stdout: "container_init": { 2024-09-12T03:29:02.592 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_init", 2024-09-12T03:29:02.592 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.592 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.592 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.592 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.592 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.592 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.593 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.593 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:29:02.593 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.593 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.593 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.593 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.593 INFO:teuthology.orchestra.run.smithi006.stdout: "daemon_cache_timeout": { 2024-09-12T03:29:02.593 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:29:02.593 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.593 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.593 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.593 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-09-12T03:29:02.593 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.594 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.594 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.594 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:29:02.594 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.594 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.594 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.594 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.594 INFO:teuthology.orchestra.run.smithi006.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:29:02.594 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:29:02.594 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.594 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.594 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.594 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "900", 2024-09-12T03:29:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:29:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: "default_registry": { 2024-09-12T03:29:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_registry", 2024-09-12T03:29:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.595 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io", 2024-09-12T03:29:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:29:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: "device_cache_timeout": { 2024-09-12T03:29:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_cache_timeout", 2024-09-12T03:29:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.596 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.597 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.597 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1800", 2024-09-12T03:29:02.597 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.597 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.597 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.597 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:29:02.597 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.597 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.597 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.597 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.597 INFO:teuthology.orchestra.run.smithi006.stdout: "device_enhanced_scan": { 2024-09-12T03:29:02.597 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_enhanced_scan", 2024-09-12T03:29:02.597 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.597 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.598 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.598 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.598 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.598 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.598 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.598 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:29:02.598 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.598 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.598 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.598 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.598 INFO:teuthology.orchestra.run.smithi006.stdout: "facts_cache_timeout": { 2024-09-12T03:29:02.598 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "facts_cache_timeout", 2024-09-12T03:29:02.598 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.599 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.599 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.599 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-09-12T03:29:02.599 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.599 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.599 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.599 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:29:02.599 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.599 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.599 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.599 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.599 INFO:teuthology.orchestra.run.smithi006.stdout: "grafana_dashboards_path": { 2024-09-12T03:29:02.599 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:29:02.600 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.600 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.600 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.600 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:29:02.600 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.600 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.600 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.600 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:29:02.600 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.600 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.600 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.600 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.600 INFO:teuthology.orchestra.run.smithi006.stdout: "host_check_interval": { 2024-09-12T03:29:02.601 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "host_check_interval", 2024-09-12T03:29:02.601 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.601 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.601 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.601 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-09-12T03:29:02.601 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.601 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.601 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.601 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:29:02.601 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.601 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.601 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.602 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.602 INFO:teuthology.orchestra.run.smithi006.stdout: "hw_monitoring": { 2024-09-12T03:29:02.602 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "hw_monitoring", 2024-09-12T03:29:02.602 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.602 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.602 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.602 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.602 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.602 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.602 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.602 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:29:02.602 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.602 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.603 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.603 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.603 INFO:teuthology.orchestra.run.smithi006.stdout: "inventory_list_all": { 2024-09-12T03:29:02.603 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "inventory_list_all", 2024-09-12T03:29:02.603 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.603 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.603 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.603 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.603 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.603 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.603 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.603 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:29:02.603 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.603 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.604 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.604 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.604 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.604 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.604 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.604 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.604 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.604 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.604 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.604 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.604 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.604 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.604 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.605 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.605 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.605 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.605 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.605 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.605 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.605 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.605 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.605 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.605 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.605 INFO:teuthology.orchestra.run.smithi006.stdout: "log_refresh_metadata": { 2024-09-12T03:29:02.605 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_refresh_metadata", 2024-09-12T03:29:02.605 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.606 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.606 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.606 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.606 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.606 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.606 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.606 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:29:02.606 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.606 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.606 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.606 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.606 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.606 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.607 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.607 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.607 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.607 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.607 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.607 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.607 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.607 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:29:02.607 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.607 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.607 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.607 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.607 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.608 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.608 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.608 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.608 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.608 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.608 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.608 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.608 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.608 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.608 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.608 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.608 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.608 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.609 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.609 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.609 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.609 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.609 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.609 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.609 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.609 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.609 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.609 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.609 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.609 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.609 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.610 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.610 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.610 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.610 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.610 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.610 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.610 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.610 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.610 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:29:02.610 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:29:02.610 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.610 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.610 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:29:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:29:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:29:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.611 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "*", 2024-09-12T03:29:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:29:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: "max_count_per_host": { 2024-09-12T03:29:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_count_per_host", 2024-09-12T03:29:02.612 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:29:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: "max_osd_draining_count": { 2024-09-12T03:29:02.613 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_osd_draining_count", 2024-09-12T03:29:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:29:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: "migration_current": { 2024-09-12T03:29:02.614 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "migration_current", 2024-09-12T03:29:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "internal - do not modify", 2024-09-12T03:29:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.615 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2024-09-12T03:29:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2024-09-12T03:29:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "root", 2024-09-12T03:29:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm-package", 2024-09-12T03:29:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: "root" 2024-09-12T03:29:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:29:02.616 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: "oob_default_addr": { 2024-09-12T03:29:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "oob_default_addr", 2024-09-12T03:29:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "169.254.1.1", 2024-09-12T03:29:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.617 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:29:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: "prometheus_alerts_path": { 2024-09-12T03:29:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:29:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:29:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.618 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:29:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_insecure": { 2024-09-12T03:29:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_insecure", 2024-09-12T03:29:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.619 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:29:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_password": { 2024-09-12T03:29:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_password", 2024-09-12T03:29:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.620 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.621 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:29:02.621 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.621 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.621 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.621 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.621 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_url": { 2024-09-12T03:29:02.621 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_url", 2024-09-12T03:29:02.621 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.621 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.621 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.621 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.621 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.621 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.621 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.622 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:29:02.622 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.622 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.622 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.622 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.622 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_username": { 2024-09-12T03:29:02.622 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_username", 2024-09-12T03:29:02.622 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.622 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.622 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.622 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.622 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.622 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.623 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.623 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:29:02.623 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.623 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.623 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.623 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.623 INFO:teuthology.orchestra.run.smithi006.stdout: "secure_monitoring_stack": { 2024-09-12T03:29:02.623 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:29:02.623 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.623 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.623 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.623 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.623 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.623 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.624 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.624 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:29:02.624 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.624 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.624 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.624 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.624 INFO:teuthology.orchestra.run.smithi006.stdout: "service_discovery_port": { 2024-09-12T03:29:02.624 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "service_discovery_port", 2024-09-12T03:29:02.624 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.624 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.624 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.624 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8765", 2024-09-12T03:29:02.625 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.625 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.625 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.625 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:29:02.625 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.625 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.625 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.625 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.625 INFO:teuthology.orchestra.run.smithi006.stdout: "ssh_config_file": { 2024-09-12T03:29:02.625 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssh_config_file", 2024-09-12T03:29:02.625 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.625 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.625 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.626 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.626 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.626 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.626 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.626 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:29:02.626 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.626 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.626 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.626 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.626 INFO:teuthology.orchestra.run.smithi006.stdout: "use_agent": { 2024-09-12T03:29:02.626 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_agent", 2024-09-12T03:29:02.626 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.626 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.627 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.627 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.627 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.627 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.627 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.627 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:29:02.627 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.627 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.627 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.627 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.627 INFO:teuthology.orchestra.run.smithi006.stdout: "use_repo_digest": { 2024-09-12T03:29:02.627 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_repo_digest", 2024-09-12T03:29:02.627 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.627 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.628 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.628 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.628 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.628 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.628 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.628 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:29:02.628 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.628 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.628 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.628 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.628 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_failed_host_check": { 2024-09-12T03:29:02.628 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:29:02.628 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.629 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.629 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.629 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.629 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.629 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.629 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.629 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:29:02.629 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.629 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.629 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.629 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.629 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_daemons": { 2024-09-12T03:29:02.629 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:29:02.630 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.630 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.630 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.630 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.630 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.630 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.630 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.630 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:29:02.630 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.630 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.630 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.630 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.630 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_hosts": { 2024-09-12T03:29:02.631 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:29:02.631 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.631 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.631 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.631 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.631 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.631 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.631 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.631 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:29:02.631 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.631 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.631 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.631 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.631 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.632 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.632 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.632 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crash", 2024-09-12T03:29:02.632 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.632 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.632 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.632 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.632 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.632 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.632 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.632 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.632 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.633 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.633 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.633 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.633 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.633 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.633 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.633 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.633 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.633 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.633 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.633 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.633 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.633 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.634 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.634 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.634 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.634 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.634 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.634 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.634 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.634 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.634 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.634 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.634 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.634 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.634 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.635 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.635 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.635 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.635 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.635 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.635 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.635 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.635 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.635 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.635 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.635 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.635 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.635 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.635 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.636 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.636 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.636 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.636 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.636 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.636 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.636 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.636 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.636 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.636 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.636 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.636 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.636 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.637 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.637 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.637 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.637 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.637 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.637 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.637 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.637 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.637 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.637 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.637 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.637 INFO:teuthology.orchestra.run.smithi006.stdout: "retain_interval": { 2024-09-12T03:29:02.637 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retain_interval", 2024-09-12T03:29:02.637 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.638 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.638 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.638 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "31536000", 2024-09-12T03:29:02.638 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.638 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.638 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.638 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:29:02.638 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.638 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.638 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.638 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.638 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_recent_interval": { 2024-09-12T03:29:02.638 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_recent_interval", 2024-09-12T03:29:02.639 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.639 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.639 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.639 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1209600", 2024-09-12T03:29:02.639 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.639 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.639 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.639 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:29:02.639 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.639 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.639 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.639 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.639 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.640 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.640 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.640 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dashboard", 2024-09-12T03:29:02.640 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.640 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.640 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.640 INFO:teuthology.orchestra.run.smithi006.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:29:02.640 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:29:02.640 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.640 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.640 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.640 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:02.640 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.641 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.641 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.641 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.641 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.641 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.641 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.641 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.641 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:29:02.641 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:29:02.641 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.641 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.641 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.641 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.641 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.642 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.642 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.642 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.642 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.642 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.642 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.642 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.642 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:29:02.642 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:29:02.642 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.642 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.642 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.643 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.643 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.643 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.643 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.643 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.643 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.643 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.643 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.643 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.643 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:29:02.643 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:29:02.643 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.643 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.643 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.644 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.644 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.644 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.644 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.644 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.644 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.644 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.644 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.644 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.644 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:29:02.644 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:29:02.644 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.644 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.645 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.645 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.645 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.645 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.645 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.645 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.645 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.645 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.645 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.645 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.645 INFO:teuthology.orchestra.run.smithi006.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:29:02.645 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:29:02.645 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.646 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.646 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.646 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.646 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.646 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.646 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.646 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.646 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.646 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.646 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.646 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.646 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:29:02.646 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:29:02.647 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.647 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.647 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.647 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.647 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.647 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.647 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.647 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.647 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.647 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.647 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.647 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.647 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:29:02.647 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:29:02.648 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.648 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.648 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.648 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.648 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.648 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.648 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.648 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.648 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.648 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.648 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.648 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.648 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:29:02.649 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:29:02.649 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.649 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.649 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.649 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.649 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.649 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.649 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.649 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.649 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.649 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.649 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.649 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.650 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:29:02.650 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:29:02.650 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.650 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.650 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.650 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.650 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.650 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.650 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.650 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.650 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.650 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.650 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.650 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.651 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:29:02.651 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:29:02.651 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.651 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.651 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.651 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.651 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.651 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.651 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.651 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.651 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.651 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.651 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.652 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.652 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:29:02.652 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:29:02.652 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.652 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.652 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.652 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.652 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.652 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.652 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.652 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.652 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.652 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.653 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.653 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.653 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:29:02.653 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:29:02.653 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.653 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.653 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.653 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.653 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.653 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.653 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.653 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.653 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.653 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.654 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.654 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.654 INFO:teuthology.orchestra.run.smithi006.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:29:02.654 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:29:02.654 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.654 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.654 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.654 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.654 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.654 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.654 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.654 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.654 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.655 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.655 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.655 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.655 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:29:02.655 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:29:02.655 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.655 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.655 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.655 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-09-12T03:29:02.655 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.655 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.655 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.655 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.655 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.656 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.656 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.656 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.656 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:29:02.656 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:29:02.656 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.656 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.656 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.656 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.656 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.656 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.656 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.656 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.657 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.657 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.657 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.657 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.657 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_URL": { 2024-09-12T03:29:02.657 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:29:02.657 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.657 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.657 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.657 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.657 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.657 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.657 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.658 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.658 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.658 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.658 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.658 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.658 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:29:02.658 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:29:02.658 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.658 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.658 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.658 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-09-12T03:29:02.658 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.658 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.659 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.659 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.659 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.659 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.659 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.659 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.659 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:29:02.659 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:29:02.659 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.659 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.659 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.659 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.659 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.660 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.660 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.660 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.660 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.660 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.660 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.660 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.660 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:29:02.660 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:29:02.660 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.660 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.660 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.660 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.660 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.661 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.661 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.661 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.661 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.661 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.661 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.661 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.661 INFO:teuthology.orchestra.run.smithi006.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:29:02.661 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:29:02.661 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.661 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.661 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.661 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.662 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.662 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.662 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.662 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.662 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.662 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.662 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.662 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.662 INFO:teuthology.orchestra.run.smithi006.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:29:02.662 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:29:02.662 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.662 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.662 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.663 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.663 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.663 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.663 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.663 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.663 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.663 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.663 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.663 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.663 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:29:02.663 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:29:02.663 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.663 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.664 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.664 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.664 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.664 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.664 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.664 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.664 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.664 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.664 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.664 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.664 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:29:02.664 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:29:02.665 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.665 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.665 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.665 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.665 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.665 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.665 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.665 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.665 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.665 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.665 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.665 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.665 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:29:02.665 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:29:02.666 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.666 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.666 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.666 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.666 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.666 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.666 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.666 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.666 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.666 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.666 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.666 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.666 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:29:02.667 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:29:02.667 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.667 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.667 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.667 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.667 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.667 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.667 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.667 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.667 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.667 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.667 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.667 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.668 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:29:02.668 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:29:02.668 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.668 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.668 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.668 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.668 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.668 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.668 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.668 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.668 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.668 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.668 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.668 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.669 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:29:02.669 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:29:02.669 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.669 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.669 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.669 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.669 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.669 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.669 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.669 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.669 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.669 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.669 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.670 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.670 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:29:02.670 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:29:02.670 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.670 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.670 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.670 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.670 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.670 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.670 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.670 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.670 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.670 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.670 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.671 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.671 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:29:02.671 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:29:02.671 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.671 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.671 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.671 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.671 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.671 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.671 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.671 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.671 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.671 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.671 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.672 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.672 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:29:02.672 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:29:02.672 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.672 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.672 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.672 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.672 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.672 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.672 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.672 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.672 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.672 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.673 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.673 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.673 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:29:02.673 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:29:02.673 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.673 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.673 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.673 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.673 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.673 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.673 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.673 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.673 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.674 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.674 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.674 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.674 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:29:02.674 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:29:02.674 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.674 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.674 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.674 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:29:02.674 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.674 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.674 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.674 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.675 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.675 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.675 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.675 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.675 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:29:02.675 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:29:02.675 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.675 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.675 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.675 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:02.675 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.675 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.675 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.675 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.676 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.676 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.676 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.676 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.676 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:29:02.676 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:29:02.676 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.676 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.676 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.676 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8", 2024-09-12T03:29:02.676 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.676 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.676 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.677 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.677 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.677 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.677 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.677 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.677 INFO:teuthology.orchestra.run.smithi006.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:29:02.677 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:29:02.677 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.677 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.677 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.677 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "45", 2024-09-12T03:29:02.677 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.677 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.678 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.678 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.678 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.678 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.678 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.678 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.678 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:29:02.678 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:29:02.678 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.678 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.678 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.678 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.678 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.679 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.679 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.679 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.679 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.679 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.679 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.679 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.679 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:29:02.679 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:29:02.679 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.679 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.679 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.680 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-09-12T03:29:02.680 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.680 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.680 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.680 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.680 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.680 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.680 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.680 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.680 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:29:02.680 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:29:02.680 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.681 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.681 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.681 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.681 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.681 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.681 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.681 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.681 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.681 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.681 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.681 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.681 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:29:02.681 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:29:02.681 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.682 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.682 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.682 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.682 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.682 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.682 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.682 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.682 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.682 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.682 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.682 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.682 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:29:02.682 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:29:02.683 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.683 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.683 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.683 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-09-12T03:29:02.683 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.683 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.683 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.683 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.683 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.683 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.683 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.683 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.683 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:29:02.684 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:29:02.684 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.684 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.684 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.684 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:02.684 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.684 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.684 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.684 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.684 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.684 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.684 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.684 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.684 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:29:02.685 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:29:02.685 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.685 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.685 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.685 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-09-12T03:29:02.685 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.685 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.685 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.685 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.685 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.685 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.685 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.685 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.686 INFO:teuthology.orchestra.run.smithi006.stdout: "cross_origin_url": { 2024-09-12T03:29:02.686 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cross_origin_url", 2024-09-12T03:29:02.686 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.686 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.686 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.686 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.686 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.686 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.686 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.686 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.686 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.686 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.686 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.687 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.687 INFO:teuthology.orchestra.run.smithi006.stdout: "crt_file": { 2024-09-12T03:29:02.687 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crt_file", 2024-09-12T03:29:02.687 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.687 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.687 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.687 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.687 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.687 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.687 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.687 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.687 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.687 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.687 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.688 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.688 INFO:teuthology.orchestra.run.smithi006.stdout: "debug": { 2024-09-12T03:29:02.688 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "debug", 2024-09-12T03:29:02.688 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.688 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.688 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.688 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.688 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.688 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.688 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.688 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:29:02.688 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.688 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.689 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.689 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.689 INFO:teuthology.orchestra.run.smithi006.stdout: "jwt_token_ttl": { 2024-09-12T03:29:02.689 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "jwt_token_ttl", 2024-09-12T03:29:02.689 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.689 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.689 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.689 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "28800", 2024-09-12T03:29:02.689 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.689 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.689 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.689 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.689 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.689 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.690 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.690 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.690 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2024-09-12T03:29:02.690 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2024-09-12T03:29:02.690 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.690 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.690 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.690 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.690 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.690 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.690 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.690 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.690 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.691 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.691 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.691 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.691 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.691 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.691 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.691 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.691 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.691 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.691 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.691 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.691 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.692 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.692 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.692 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.692 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.692 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.692 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.692 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.692 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.692 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.692 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.692 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.692 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.692 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.693 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.693 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.693 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.693 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.693 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.693 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.693 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.693 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.693 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.693 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.693 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.693 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.693 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.693 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.694 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.694 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.694 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.694 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.694 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.694 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.694 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.694 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.694 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.694 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.694 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.694 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.694 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.695 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.695 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.695 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.695 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.695 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.695 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.695 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.695 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.695 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.695 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.695 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.695 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.695 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.696 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.696 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.696 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.696 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.696 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.696 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.696 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.696 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.696 INFO:teuthology.orchestra.run.smithi006.stdout: "motd": { 2024-09-12T03:29:02.696 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "motd", 2024-09-12T03:29:02.696 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.696 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.697 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.697 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.697 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.697 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.697 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.697 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "The message of the day", 2024-09-12T03:29:02.697 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.697 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.697 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.697 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.697 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:29:02.697 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:29:02.697 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.698 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.698 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.698 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.698 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.698 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.698 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.698 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.698 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.698 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.698 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.698 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.698 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-09-12T03:29:02.698 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-09-12T03:29:02.699 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.699 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.699 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.699 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2024-09-12T03:29:02.699 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.699 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.699 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.699 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.699 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.699 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.699 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.699 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.699 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-09-12T03:29:02.699 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-09-12T03:29:02.700 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.700 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.700 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.700 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8080", 2024-09-12T03:29:02.700 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.700 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.700 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.700 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.700 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.700 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.700 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.700 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.700 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2024-09-12T03:29:02.701 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2024-09-12T03:29:02.701 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.701 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.701 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.701 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.701 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.701 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.701 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.701 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.701 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.701 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.701 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.701 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.702 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl_server_port": { 2024-09-12T03:29:02.702 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl_server_port", 2024-09-12T03:29:02.702 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.702 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.702 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.702 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8443", 2024-09-12T03:29:02.702 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.702 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.702 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.702 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.702 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.702 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.702 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.702 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.703 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2024-09-12T03:29:02.703 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2024-09-12T03:29:02.703 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.703 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.703 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.703 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "redirect", 2024-09-12T03:29:02.703 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.703 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.703 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.703 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.703 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect" 2024-09-12T03:29:02.703 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.703 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.704 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.704 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.704 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.704 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.704 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2024-09-12T03:29:02.704 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2024-09-12T03:29:02.704 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.704 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.704 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.704 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2024-09-12T03:29:02.704 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2024-09-12T03:29:02.704 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2024-09-12T03:29:02.704 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.705 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.705 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.705 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.705 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.705 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.705 INFO:teuthology.orchestra.run.smithi006.stdout: "url_prefix": { 2024-09-12T03:29:02.705 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url_prefix", 2024-09-12T03:29:02.705 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.705 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.705 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.705 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.705 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.705 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.705 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.706 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.706 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.706 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.706 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.706 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.706 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.706 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.706 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.706 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "devicehealth", 2024-09-12T03:29:02.706 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.706 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.706 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.706 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_monitoring": { 2024-09-12T03:29:02.706 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_monitoring", 2024-09-12T03:29:02.707 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.707 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.707 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.707 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.707 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.707 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.707 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.707 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "monitor device health metrics", 2024-09-12T03:29:02.707 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.707 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.707 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.707 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.707 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.708 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.708 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.708 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.708 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.708 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.708 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.708 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.708 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.708 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.708 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.708 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.708 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.708 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.709 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.709 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.709 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.709 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.709 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.709 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.709 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.709 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.709 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.709 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.709 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.709 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.709 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.710 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.710 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.710 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.710 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.710 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.710 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.710 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.710 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.710 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.710 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.710 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.710 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.710 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.711 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.711 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.711 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.711 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.711 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.711 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.711 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.711 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.711 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.711 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.711 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.711 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.711 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.711 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.712 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.712 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.712 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.712 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.712 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.712 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.712 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.712 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.712 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.712 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.712 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.712 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.712 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.713 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.713 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.713 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.713 INFO:teuthology.orchestra.run.smithi006.stdout: "mark_out_threshold": { 2024-09-12T03:29:02.713 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mark_out_threshold", 2024-09-12T03:29:02.713 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.713 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.713 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.713 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2419200", 2024-09-12T03:29:02.714 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.714 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.714 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.714 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:29:02.714 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.714 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.714 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.714 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.716 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_name": { 2024-09-12T03:29:02.717 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_name", 2024-09-12T03:29:02.717 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.717 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.717 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.717 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "device_health_metrics", 2024-09-12T03:29:02.717 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.718 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.718 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.718 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:29:02.718 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.718 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.718 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.719 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.719 INFO:teuthology.orchestra.run.smithi006.stdout: "retention_period": { 2024-09-12T03:29:02.719 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retention_period", 2024-09-12T03:29:02.719 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.719 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.719 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.720 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15552000", 2024-09-12T03:29:02.720 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.720 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.720 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.720 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:29:02.720 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.720 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.720 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.720 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.720 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_frequency": { 2024-09-12T03:29:02.721 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_frequency", 2024-09-12T03:29:02.721 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.721 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.721 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.721 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2024-09-12T03:29:02.721 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.721 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.721 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.721 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:29:02.722 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.722 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.722 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.722 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.722 INFO:teuthology.orchestra.run.smithi006.stdout: "self_heal": { 2024-09-12T03:29:02.722 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "self_heal", 2024-09-12T03:29:02.722 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.722 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.723 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.723 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.723 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.723 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.723 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.723 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:29:02.723 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.723 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.724 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.724 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.724 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-09-12T03:29:02.724 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-09-12T03:29:02.724 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.724 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.724 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.724 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-09-12T03:29:02.724 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.725 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.725 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.725 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:29:02.725 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.725 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.725 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.725 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.725 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_threshold": { 2024-09-12T03:29:02.726 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_threshold", 2024-09-12T03:29:02.726 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.726 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.726 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.726 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7257600", 2024-09-12T03:29:02.726 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.726 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.726 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.727 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:29:02.727 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.727 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.727 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.727 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.727 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.727 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.727 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.727 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "diskprediction_local", 2024-09-12T03:29:02.727 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.727 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.727 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.728 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.728 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.728 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.728 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.728 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.728 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.728 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.728 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.728 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.729 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.729 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.729 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.729 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.729 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.729 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.729 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.729 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.729 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.729 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.729 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.730 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.730 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.730 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.730 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.730 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.730 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.730 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.731 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.731 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.731 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.731 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.731 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.731 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.732 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.732 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.732 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.732 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.732 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.732 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.732 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.733 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.733 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.733 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.733 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.733 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.733 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.733 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.733 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.733 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.733 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.733 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.733 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.734 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.734 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.734 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.734 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.734 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.734 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.734 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.734 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.734 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.734 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.734 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.734 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.734 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.735 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.735 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.735 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.735 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.735 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.735 INFO:teuthology.orchestra.run.smithi006.stdout: "predict_interval": { 2024-09-12T03:29:02.735 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predict_interval", 2024-09-12T03:29:02.735 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.735 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.735 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.736 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2024-09-12T03:29:02.736 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.736 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.736 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.736 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.736 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.736 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.736 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.736 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.736 INFO:teuthology.orchestra.run.smithi006.stdout: "predictor_model": { 2024-09-12T03:29:02.736 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predictor_model", 2024-09-12T03:29:02.737 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.737 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.737 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.737 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "prophetstor", 2024-09-12T03:29:02.737 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.737 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.737 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.737 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.737 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.737 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.737 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.737 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.738 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-09-12T03:29:02.738 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-09-12T03:29:02.738 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.738 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.738 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.738 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-09-12T03:29:02.738 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.738 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.738 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.738 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.738 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.738 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.738 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.739 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.739 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.739 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.739 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.739 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "influx", 2024-09-12T03:29:02.739 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": false, 2024-09-12T03:29:02.739 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:29:02.739 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.739 INFO:teuthology.orchestra.run.smithi006.stdout: "batch_size": { 2024-09-12T03:29:02.739 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "batch_size", 2024-09-12T03:29:02.739 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.739 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.739 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.740 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5000", 2024-09-12T03:29:02.740 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.740 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.740 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.740 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:29:02.740 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.740 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.740 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.740 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.740 INFO:teuthology.orchestra.run.smithi006.stdout: "database": { 2024-09-12T03:29:02.740 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "database", 2024-09-12T03:29:02.740 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.741 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.741 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.741 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "ceph", 2024-09-12T03:29:02.741 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.741 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.741 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.741 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:29:02.741 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.741 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.741 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.742 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.742 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": { 2024-09-12T03:29:02.742 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "hostname", 2024-09-12T03:29:02.742 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.742 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.742 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.742 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.742 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.742 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.742 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.742 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:29:02.742 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.743 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.743 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.743 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.743 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-09-12T03:29:02.743 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-09-12T03:29:02.743 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.744 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.744 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.744 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "30", 2024-09-12T03:29:02.744 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "5", 2024-09-12T03:29:02.744 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.744 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.745 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:29:02.745 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.745 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.745 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.745 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.745 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.745 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.745 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.745 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.745 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.745 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.745 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.745 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.746 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.746 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.746 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.746 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.746 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.746 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.746 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.746 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.746 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.747 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.747 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.747 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.747 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.747 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.747 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.747 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.747 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.748 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.748 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.748 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.748 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.748 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.748 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.748 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.748 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.748 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.749 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.749 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.749 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.749 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.749 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.749 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.749 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.750 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.750 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.750 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.750 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.750 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.750 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.750 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.751 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.751 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.751 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.751 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.751 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.751 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.751 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.751 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.751 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.751 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.751 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.751 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.751 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.752 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.752 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.752 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.752 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.752 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.752 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.752 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.752 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.752 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.752 INFO:teuthology.orchestra.run.smithi006.stdout: "password": { 2024-09-12T03:29:02.753 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "password", 2024-09-12T03:29:02.753 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.753 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.753 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.753 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.753 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.753 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.753 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.753 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:29:02.754 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.754 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.754 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.754 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.754 INFO:teuthology.orchestra.run.smithi006.stdout: "port": { 2024-09-12T03:29:02.754 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "port", 2024-09-12T03:29:02.755 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.755 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.755 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.755 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8086", 2024-09-12T03:29:02.755 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.755 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.755 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.755 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server port", 2024-09-12T03:29:02.755 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.755 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.755 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.756 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.756 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2024-09-12T03:29:02.756 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2024-09-12T03:29:02.756 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.756 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.756 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.756 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "false", 2024-09-12T03:29:02.756 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.756 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.756 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.756 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:29:02.756 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.757 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.757 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.757 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.757 INFO:teuthology.orchestra.run.smithi006.stdout: "threads": { 2024-09-12T03:29:02.757 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threads", 2024-09-12T03:29:02.757 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.757 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.757 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.757 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-09-12T03:29:02.757 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-09-12T03:29:02.757 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "32", 2024-09-12T03:29:02.758 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.758 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:29:02.758 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.758 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.758 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.758 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.758 INFO:teuthology.orchestra.run.smithi006.stdout: "username": { 2024-09-12T03:29:02.758 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "username", 2024-09-12T03:29:02.758 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.758 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.758 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.758 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.759 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.759 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.759 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.759 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:29:02.759 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.759 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.759 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.759 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.759 INFO:teuthology.orchestra.run.smithi006.stdout: "verify_ssl": { 2024-09-12T03:29:02.759 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "verify_ssl", 2024-09-12T03:29:02.759 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.759 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.759 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.760 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "true", 2024-09-12T03:29:02.760 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.760 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.760 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.760 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:29:02.760 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.760 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.760 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.760 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.760 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.760 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.760 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.760 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "insights", 2024-09-12T03:29:02.761 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.761 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.761 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.761 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.761 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.761 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.761 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.761 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.761 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.761 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.761 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.762 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.762 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.762 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.762 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.762 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.762 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.762 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.762 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.762 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.762 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.762 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.762 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.762 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.763 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.763 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.763 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.763 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.763 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.763 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.763 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.763 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.763 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.763 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.763 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.763 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.763 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.763 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.764 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.764 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.764 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.764 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.764 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.764 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.764 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.764 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.764 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.764 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.764 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.764 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.765 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.765 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.765 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.765 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.765 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.765 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.765 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.766 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.766 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.766 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.766 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.766 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.767 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.767 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.767 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.767 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.767 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.767 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.768 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.768 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.768 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.768 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.768 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.768 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.769 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.769 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.769 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "iostat", 2024-09-12T03:29:02.769 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.769 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.769 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.769 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.770 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.770 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.770 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.770 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.770 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.770 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.771 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.771 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.771 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.771 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.771 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.771 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.772 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.772 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.772 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.772 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.772 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.772 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.773 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.773 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.773 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.773 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.773 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.773 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.773 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.774 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.774 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.774 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.774 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.774 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.774 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.774 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.774 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.775 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.775 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.775 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.775 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.775 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.775 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.775 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.776 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.776 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.776 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.776 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.776 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.777 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.777 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.777 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.777 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.777 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.777 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.777 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.778 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.778 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.778 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.778 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.778 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.778 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.778 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.778 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.778 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.778 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.778 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.778 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.779 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.779 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.779 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.779 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.779 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.779 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.779 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.779 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.779 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "k8sevents", 2024-09-12T03:29:02.779 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.779 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.780 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.780 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_event_retention_days": { 2024-09-12T03:29:02.780 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:29:02.780 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.780 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.780 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.780 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7", 2024-09-12T03:29:02.780 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.780 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.780 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.780 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:29:02.780 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.780 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.781 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.781 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.781 INFO:teuthology.orchestra.run.smithi006.stdout: "config_check_secs": { 2024-09-12T03:29:02.781 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_check_secs", 2024-09-12T03:29:02.781 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.781 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.781 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.781 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:02.781 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "10", 2024-09-12T03:29:02.781 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.781 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.781 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:29:02.781 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.782 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.782 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.782 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.782 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.782 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.782 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.782 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.782 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.782 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.783 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.783 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.783 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.783 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.783 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.783 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.783 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.783 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.783 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.783 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.783 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.783 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.783 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.783 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.784 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.784 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.784 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.784 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.784 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.784 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.784 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.784 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.784 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.784 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.784 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.784 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.785 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.785 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.785 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.785 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.785 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.785 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.785 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.785 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.785 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.785 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.785 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.785 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.786 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.786 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.786 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.786 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.786 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.786 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.786 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.786 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.786 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.786 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.786 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.786 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.786 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.787 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.787 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.787 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.787 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.787 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.787 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.787 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.787 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.787 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.787 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.787 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.787 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.788 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.788 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.788 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.788 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.788 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "localpool", 2024-09-12T03:29:02.788 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.788 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.788 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.789 INFO:teuthology.orchestra.run.smithi006.stdout: "failure_domain": { 2024-09-12T03:29:02.789 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "failure_domain", 2024-09-12T03:29:02.789 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.789 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.789 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.789 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "host", 2024-09-12T03:29:02.789 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.789 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.789 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.789 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:29:02.789 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:29:02.789 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.789 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.790 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.790 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.790 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.790 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.790 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.790 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.790 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.790 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.790 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.790 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.790 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.790 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.790 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.791 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.791 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.791 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.791 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.791 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.791 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.791 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.791 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.791 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.791 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.791 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.791 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.791 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.791 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.792 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.792 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.792 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.792 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.792 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.792 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.792 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.792 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.792 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.792 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.792 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.792 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.792 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.793 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.793 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.793 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.793 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.793 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.793 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.793 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.793 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.793 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.793 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.793 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.793 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.793 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.794 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.794 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.794 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.794 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.794 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.794 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.794 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.794 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.794 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.794 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.794 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.794 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.794 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.794 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.795 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.795 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.795 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.795 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.795 INFO:teuthology.orchestra.run.smithi006.stdout: "min_size": { 2024-09-12T03:29:02.795 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_size", 2024-09-12T03:29:02.795 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.795 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.795 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.795 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.795 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.795 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.795 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.795 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:29:02.796 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:29:02.796 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.796 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.796 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.796 INFO:teuthology.orchestra.run.smithi006.stdout: "num_rep": { 2024-09-12T03:29:02.796 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "num_rep", 2024-09-12T03:29:02.796 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.796 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.796 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.796 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3", 2024-09-12T03:29:02.796 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.796 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.796 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.797 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:29:02.797 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.797 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.797 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.797 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.797 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_num": { 2024-09-12T03:29:02.797 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_num", 2024-09-12T03:29:02.797 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.797 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.797 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.797 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "128", 2024-09-12T03:29:02.797 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.797 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.798 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.798 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:29:02.798 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.798 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.798 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.798 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.798 INFO:teuthology.orchestra.run.smithi006.stdout: "prefix": { 2024-09-12T03:29:02.798 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prefix", 2024-09-12T03:29:02.798 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.798 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.798 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.798 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.798 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.798 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.799 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.799 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:29:02.799 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.799 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.799 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.799 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.799 INFO:teuthology.orchestra.run.smithi006.stdout: "subtree": { 2024-09-12T03:29:02.799 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "subtree", 2024-09-12T03:29:02.799 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.799 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.799 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.799 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "rack", 2024-09-12T03:29:02.799 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.799 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.800 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.800 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:29:02.800 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:29:02.800 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.800 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.800 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.800 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.800 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.800 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.800 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mds_autoscaler", 2024-09-12T03:29:02.800 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.800 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.800 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.800 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.801 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.801 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.801 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.801 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.801 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.801 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.801 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.801 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.801 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.801 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.801 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.801 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.801 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.802 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.802 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.802 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.802 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.802 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.802 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.802 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.802 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.802 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.802 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.802 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.802 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.802 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.802 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.803 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.803 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.803 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.803 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.803 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.803 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.803 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.803 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.803 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.803 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.803 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.803 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.803 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.804 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.804 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.804 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.804 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.804 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.804 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.804 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.804 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.804 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.804 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.804 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.804 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.804 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.805 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.805 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.805 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.805 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.805 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.805 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.805 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.805 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.805 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.805 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.805 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.805 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.805 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.806 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.806 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.806 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.806 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.806 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.806 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.806 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirroring", 2024-09-12T03:29:02.806 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.806 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.806 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.806 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.806 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.806 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.806 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.807 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.807 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.807 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.807 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.807 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.807 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.807 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.807 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.807 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.807 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.807 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.808 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.808 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.808 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.808 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.808 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.808 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.808 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.808 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.808 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.808 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.808 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.808 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.808 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.808 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.809 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.809 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.809 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.809 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.809 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.809 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.809 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.809 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.809 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.809 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.809 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.809 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.809 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.809 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.810 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.810 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.810 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.810 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.810 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.810 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.810 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.810 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.810 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.810 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.810 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.810 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.810 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.811 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.811 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.811 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.811 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.811 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.811 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.811 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.811 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.811 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.811 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.811 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.811 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.811 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.811 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.812 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.812 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.812 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.812 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "nfs", 2024-09-12T03:29:02.812 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.812 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.812 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.812 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.812 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.812 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.812 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.812 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.812 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.813 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.813 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.813 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.813 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.813 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.813 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.813 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.813 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.813 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.813 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.813 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.813 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.813 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.813 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.814 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.814 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.814 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.814 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.814 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.814 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.814 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.814 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.814 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.814 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.814 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.814 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.814 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.814 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.815 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.815 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.815 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.815 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.815 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.815 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.815 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.815 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.815 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.815 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.815 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.815 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.815 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.815 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.816 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.816 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.816 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.816 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.816 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.816 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.816 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.816 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.816 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.816 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.816 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.816 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.816 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.816 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.817 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.817 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.817 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.817 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.817 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.817 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.817 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.817 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.817 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.817 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.817 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.817 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2024-09-12T03:29:02.817 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.817 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.818 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.818 INFO:teuthology.orchestra.run.smithi006.stdout: "fail_fs": { 2024-09-12T03:29:02.818 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "fail_fs", 2024-09-12T03:29:02.818 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.818 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.818 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.818 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.818 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.818 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.818 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.818 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:29:02.818 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.819 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.819 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.819 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.819 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.819 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.819 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.819 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.819 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.819 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.819 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.819 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.819 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.819 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.819 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.820 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.820 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.820 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.820 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.820 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.820 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.820 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.820 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.820 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.820 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.820 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.820 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.820 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.821 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.821 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.821 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.821 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.821 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.821 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.821 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.821 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.821 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.821 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.821 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.821 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.822 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.822 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.822 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.822 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.822 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.822 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.822 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.822 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.822 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.822 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.822 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.822 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.822 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.822 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.823 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.823 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.823 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.823 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.823 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.823 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.823 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.823 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.823 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.823 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.823 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.823 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.823 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.823 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.824 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.824 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.824 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.824 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.824 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.824 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.824 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator": { 2024-09-12T03:29:02.824 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2024-09-12T03:29:02.824 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.824 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.824 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.824 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.824 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.825 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.825 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.825 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm", 2024-09-12T03:29:02.825 INFO:teuthology.orchestra.run.smithi006.stdout: "rook", 2024-09-12T03:29:02.825 INFO:teuthology.orchestra.run.smithi006.stdout: "test_orchestrator" 2024-09-12T03:29:02.825 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.825 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Orchestrator backend", 2024-09-12T03:29:02.825 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.825 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.825 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.825 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.825 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.825 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.826 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.826 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_perf_query", 2024-09-12T03:29:02.826 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.826 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.826 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.826 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.826 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.826 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.826 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.826 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.826 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.826 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.826 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.826 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.827 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.827 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.827 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.827 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.827 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.827 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.827 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.827 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.827 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.827 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.827 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.827 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.827 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.828 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.828 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.828 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.828 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.828 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.828 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.828 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.828 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.828 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.828 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.828 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.828 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.828 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.829 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.829 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.829 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.829 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.829 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.829 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.829 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.829 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.829 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.829 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.829 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.829 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.829 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.830 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.830 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.830 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.830 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.830 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.830 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.830 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.830 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.830 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.830 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.830 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.830 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.830 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.830 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.831 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.831 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.831 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.831 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.831 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.831 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.831 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.831 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.831 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.831 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.831 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.831 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_support", 2024-09-12T03:29:02.831 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.831 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.832 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.832 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.832 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.832 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.832 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.832 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.832 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.832 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.832 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.832 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.832 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.832 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.833 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.833 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.833 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.833 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.833 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.833 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.833 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.833 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.833 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.833 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.833 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.833 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.833 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.834 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.834 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.834 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.834 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.834 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.834 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.834 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.834 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.834 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.834 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.834 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.834 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.834 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.834 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.835 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.835 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.835 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.835 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.835 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.835 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.835 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.835 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.835 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.835 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.835 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.835 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.835 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.835 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.836 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.836 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.836 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.836 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.836 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.836 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.836 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.836 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.836 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.836 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.836 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.836 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.836 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.837 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.837 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.837 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.837 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.837 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.837 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.837 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.837 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.837 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_autoscaler", 2024-09-12T03:29:02.837 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.837 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.837 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.837 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.837 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.838 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.838 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.838 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.838 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.838 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.838 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.838 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.838 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.838 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.838 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.838 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.838 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.838 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.839 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.839 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.839 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.839 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.839 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.839 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.839 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.839 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.839 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.839 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.839 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.839 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.839 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.840 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.840 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.840 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.840 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.840 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.840 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.840 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.840 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.840 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.840 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.840 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.840 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.840 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.840 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.841 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.841 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.841 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.841 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.841 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.841 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.841 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.841 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.841 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.841 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.841 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.841 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.841 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.842 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.842 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.842 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.842 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.842 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.842 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.842 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.842 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.842 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.842 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.842 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.842 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.842 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.842 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.843 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.843 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-09-12T03:29:02.843 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-09-12T03:29:02.843 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.843 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.843 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.843 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-09-12T03:29:02.843 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.843 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.843 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.843 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.843 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.843 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.844 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.844 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.844 INFO:teuthology.orchestra.run.smithi006.stdout: "threshold": { 2024-09-12T03:29:02.844 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threshold", 2024-09-12T03:29:02.844 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-09-12T03:29:02.844 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.844 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.844 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2024-09-12T03:29:02.844 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1.0", 2024-09-12T03:29:02.844 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.844 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.844 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "scaling threshold", 2024-09-12T03:29:02.844 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:29:02.845 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.845 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.845 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.845 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.845 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.845 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.845 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "progress", 2024-09-12T03:29:02.845 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.845 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.845 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.845 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_pg_recovery_event": { 2024-09-12T03:29:02.845 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:29:02.845 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.846 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.846 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.846 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.846 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.846 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.846 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.846 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:29:02.846 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.846 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.846 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.846 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.846 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2024-09-12T03:29:02.846 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2024-09-12T03:29:02.847 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.847 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.847 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.847 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.847 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.847 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.847 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.847 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.847 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.847 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.847 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.847 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.847 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.848 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.848 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.848 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.848 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.848 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.848 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.848 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.848 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.848 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.848 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.848 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.848 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.848 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.849 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.849 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.849 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.849 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.849 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.849 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.849 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.849 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.849 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.849 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.849 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.849 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.849 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.849 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.850 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.850 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.850 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.850 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.850 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.850 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.850 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.850 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.850 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.850 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.850 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.850 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.850 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.851 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.851 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.851 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.851 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.851 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.851 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.851 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.851 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.851 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.851 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.851 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.851 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.851 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.852 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.852 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.852 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.852 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.852 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.852 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.852 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.852 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.852 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.852 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.852 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.852 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.852 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.853 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.853 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.853 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.853 INFO:teuthology.orchestra.run.smithi006.stdout: "max_completed_events": { 2024-09-12T03:29:02.853 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_completed_events", 2024-09-12T03:29:02.853 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.853 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.853 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.853 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "50", 2024-09-12T03:29:02.853 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.853 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.853 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.853 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:29:02.853 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.854 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.854 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.854 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.854 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-09-12T03:29:02.854 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-09-12T03:29:02.854 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.854 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.854 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.854 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-09-12T03:29:02.854 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.854 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.854 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.854 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:29:02.854 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.855 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.855 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.855 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.855 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.855 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.855 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.855 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus", 2024-09-12T03:29:02.855 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.855 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.855 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.855 INFO:teuthology.orchestra.run.smithi006.stdout: "cache": { 2024-09-12T03:29:02.855 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cache", 2024-09-12T03:29:02.855 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.856 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.856 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.856 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.856 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.856 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.856 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.856 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.856 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.856 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.856 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.856 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.856 INFO:teuthology.orchestra.run.smithi006.stdout: "exclude_perf_counters": { 2024-09-12T03:29:02.856 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "exclude_perf_counters", 2024-09-12T03:29:02.857 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.857 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.857 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.857 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.857 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.857 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.857 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.857 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:29:02.857 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:29:02.857 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.857 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.857 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.857 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.858 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.858 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.858 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.858 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.858 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.858 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.858 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.858 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.858 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.858 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.858 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.858 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.858 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.858 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.859 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.859 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.859 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.859 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.859 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.859 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.859 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.859 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.859 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.859 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.859 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.859 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.859 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.859 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.860 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.860 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.860 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.860 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.860 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.860 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.860 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.860 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.860 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.860 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.860 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.860 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.860 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.861 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.861 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.861 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.861 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.861 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.861 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.861 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.861 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.861 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.861 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.861 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.861 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.862 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.862 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.862 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.862 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.862 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.862 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.862 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.862 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.862 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.862 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.862 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.862 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.862 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.863 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.863 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.863 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.863 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools": { 2024-09-12T03:29:02.863 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools", 2024-09-12T03:29:02.863 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.863 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.863 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.863 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.863 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.863 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.863 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.863 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.864 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.864 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.864 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.864 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.864 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:29:02.864 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:29:02.864 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.864 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.864 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.864 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "300", 2024-09-12T03:29:02.864 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.864 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.864 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.865 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.865 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.865 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.865 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.865 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.865 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_interval": { 2024-09-12T03:29:02.865 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_interval", 2024-09-12T03:29:02.865 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-09-12T03:29:02.865 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.865 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.865 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15.0", 2024-09-12T03:29:02.865 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.865 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.866 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.866 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.866 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.866 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.866 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.866 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.866 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-09-12T03:29:02.866 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-09-12T03:29:02.866 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.866 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.866 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.866 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2024-09-12T03:29:02.866 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.866 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.867 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.867 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:29:02.867 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.867 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.867 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.867 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.867 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-09-12T03:29:02.867 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-09-12T03:29:02.867 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.867 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.867 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.867 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "9283", 2024-09-12T03:29:02.867 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.867 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.867 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.868 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:29:02.868 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.868 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.868 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.868 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.868 INFO:teuthology.orchestra.run.smithi006.stdout: "stale_cache_strategy": { 2024-09-12T03:29:02.868 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stale_cache_strategy", 2024-09-12T03:29:02.868 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.868 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.868 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.868 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "log", 2024-09-12T03:29:02.868 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.868 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.869 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.869 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.869 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.869 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.869 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.869 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.869 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2024-09-12T03:29:02.869 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2024-09-12T03:29:02.869 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.869 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.869 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.869 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "default", 2024-09-12T03:29:02.869 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.869 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.870 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.870 INFO:teuthology.orchestra.run.smithi006.stdout: "default", 2024-09-12T03:29:02.870 INFO:teuthology.orchestra.run.smithi006.stdout: "error" 2024-09-12T03:29:02.870 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.870 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.870 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.870 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.870 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.870 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.870 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2024-09-12T03:29:02.870 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2024-09-12T03:29:02.870 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.870 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.870 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.871 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2024-09-12T03:29:02.871 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2024-09-12T03:29:02.871 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2024-09-12T03:29:02.871 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.871 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.871 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.871 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.871 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.871 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.871 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.871 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.871 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.871 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_support", 2024-09-12T03:29:02.871 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.872 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.872 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.872 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.872 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.872 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.872 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.872 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.872 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.872 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.872 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.872 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.872 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.872 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.873 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.873 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.873 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.873 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.873 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.873 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.873 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.873 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.873 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.873 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.873 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.873 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.873 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.874 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.874 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.874 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.874 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.874 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.874 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.874 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.874 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.874 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.874 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.874 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.874 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.874 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.874 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.875 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.875 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.875 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.875 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.875 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.875 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.875 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.875 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.875 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.875 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.875 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.875 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.875 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.875 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.876 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.876 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.876 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.876 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.876 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.876 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.876 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.876 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.876 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.876 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.876 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.876 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.876 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.877 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.877 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.877 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.877 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.877 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.877 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_snap_create": { 2024-09-12T03:29:02.877 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:29:02.877 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.877 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.877 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.877 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-09-12T03:29:02.877 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.877 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.878 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.878 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.878 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.878 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.878 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.878 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.878 INFO:teuthology.orchestra.run.smithi006.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:29:02.878 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:29:02.878 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.878 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.878 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.878 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.878 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.878 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.879 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.879 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.879 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.879 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.879 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.879 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.879 INFO:teuthology.orchestra.run.smithi006.stdout: "trash_purge_schedule": { 2024-09-12T03:29:02.879 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "trash_purge_schedule", 2024-09-12T03:29:02.879 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.879 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.879 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.879 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.879 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.880 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.880 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.880 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.880 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.880 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.880 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.880 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.880 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.880 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.880 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.880 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "restful", 2024-09-12T03:29:02.880 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.880 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.880 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.881 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_auth": { 2024-09-12T03:29:02.881 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_auth", 2024-09-12T03:29:02.881 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.881 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.881 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.881 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.881 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.881 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.881 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.881 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.881 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.881 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.881 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.881 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.882 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2024-09-12T03:29:02.882 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2024-09-12T03:29:02.882 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.882 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.882 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.882 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.882 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.882 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.882 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.882 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.882 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.882 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.882 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.883 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.883 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.883 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.883 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.883 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.883 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.883 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.883 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.883 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.883 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.883 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.883 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.883 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.883 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.884 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.884 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.884 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.884 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.884 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.884 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.884 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.884 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.884 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.884 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.884 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.884 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.884 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.885 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.885 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.885 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.885 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.885 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.885 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.885 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.885 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.885 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.885 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.885 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.885 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.885 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.886 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.886 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.886 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.886 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.886 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.886 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.886 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.886 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.886 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.886 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.886 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.886 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.886 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.886 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.887 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.887 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.887 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.887 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.887 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.887 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.887 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.887 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.887 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.887 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.887 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.887 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.887 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.887 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.888 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.888 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.888 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.888 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-09-12T03:29:02.888 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-09-12T03:29:02.888 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.888 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.888 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.888 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.888 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.889 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.889 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.889 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.889 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.889 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.889 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.889 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.889 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-09-12T03:29:02.889 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-09-12T03:29:02.889 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.889 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.889 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.890 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.890 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.890 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.890 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.890 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.890 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.890 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.890 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.890 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.890 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.890 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.891 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.891 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rgw", 2024-09-12T03:29:02.891 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.891 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.891 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.891 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.891 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.891 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.891 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.891 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.891 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.892 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.892 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.892 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.892 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.892 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.892 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.892 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.892 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.892 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.893 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.893 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.893 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.893 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.893 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.893 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.893 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.893 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.893 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.894 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.894 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.894 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.894 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.894 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.894 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.894 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.894 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.894 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.894 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.894 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.895 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.895 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.895 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.895 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.895 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.895 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.895 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.895 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.895 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.895 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.895 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.895 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.895 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.896 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.896 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.896 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.896 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.896 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.896 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.896 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.896 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.896 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.896 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.896 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.896 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.896 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.896 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.897 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.897 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.897 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.897 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.897 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.897 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.897 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.897 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.897 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.897 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.897 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.897 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rook", 2024-09-12T03:29:02.897 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.898 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.898 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.898 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.898 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.898 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.898 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.898 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.898 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.898 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.898 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.898 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.898 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.898 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.898 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.899 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.899 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.899 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.899 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.899 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.899 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.899 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.899 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.899 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.899 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.899 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.899 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.899 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.899 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.900 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.900 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.900 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.900 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.900 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.900 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.900 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.900 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.900 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.900 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.900 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.900 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.900 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.900 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.901 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.901 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.901 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.901 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.901 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.901 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.901 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.901 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.901 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.901 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.901 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.901 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.901 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.902 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.902 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.902 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.902 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.902 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.902 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.902 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.902 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.902 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.902 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.902 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.902 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.902 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.902 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.903 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.903 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.903 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.903 INFO:teuthology.orchestra.run.smithi006.stdout: "storage_class": { 2024-09-12T03:29:02.903 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "storage_class", 2024-09-12T03:29:02.903 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.903 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.903 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.903 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "local", 2024-09-12T03:29:02.903 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.903 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.903 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.903 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:29:02.903 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.904 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.904 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.904 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.904 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.904 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.904 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.904 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "selftest", 2024-09-12T03:29:02.904 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.904 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.904 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.904 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.904 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.904 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.905 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.905 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.905 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.905 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.905 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.905 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.905 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.905 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.905 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.905 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.905 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.905 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.905 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.905 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.906 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.906 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.906 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.906 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.906 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.906 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.906 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.906 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.906 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.906 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.906 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.906 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.906 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.906 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.907 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.907 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.907 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.907 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.907 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.907 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.907 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.907 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.907 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.907 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.907 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.907 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.907 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.907 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.908 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.908 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.908 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.908 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.908 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.908 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.908 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.908 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.908 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.908 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.908 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.908 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.908 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.908 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.909 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.909 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.909 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.909 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.909 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.909 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.909 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.909 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.909 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.909 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.909 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.909 INFO:teuthology.orchestra.run.smithi006.stdout: "roption1": { 2024-09-12T03:29:02.909 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption1", 2024-09-12T03:29:02.910 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.910 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.910 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.910 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.910 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.910 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.910 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.910 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.910 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.910 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.910 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.910 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.910 INFO:teuthology.orchestra.run.smithi006.stdout: "roption2": { 2024-09-12T03:29:02.911 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption2", 2024-09-12T03:29:02.911 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.911 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.911 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.911 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "xyz", 2024-09-12T03:29:02.911 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.911 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.911 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.911 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.911 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.911 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.911 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.911 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.911 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption1": { 2024-09-12T03:29:02.912 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption1", 2024-09-12T03:29:02.912 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.912 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.912 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.912 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.912 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.912 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.912 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.912 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.912 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.912 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.912 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.912 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.913 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption2": { 2024-09-12T03:29:02.913 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption2", 2024-09-12T03:29:02.913 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.913 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.913 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.913 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.913 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.913 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.913 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.913 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.913 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.913 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.913 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.913 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.914 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption3": { 2024-09-12T03:29:02.914 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption3", 2024-09-12T03:29:02.914 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-09-12T03:29:02.914 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.914 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.914 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.914 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.914 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.914 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.914 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.914 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.914 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.914 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.915 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.915 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption4": { 2024-09-12T03:29:02.915 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption4", 2024-09-12T03:29:02.915 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.915 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.916 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.916 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.916 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.916 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption5": { 2024-09-12T03:29:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption5", 2024-09-12T03:29:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.918 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.918 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.918 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.918 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.918 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.918 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.918 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption6": { 2024-09-12T03:29:02.918 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption6", 2024-09-12T03:29:02.918 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.918 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.919 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.919 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.919 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.919 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.919 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.919 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.919 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.919 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.919 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.919 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.919 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption7": { 2024-09-12T03:29:02.919 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption7", 2024-09-12T03:29:02.919 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.920 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.920 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.920 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.920 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-09-12T03:29:02.920 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "42", 2024-09-12T03:29:02.920 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.920 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.920 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.920 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.920 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.920 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.920 INFO:teuthology.orchestra.run.smithi006.stdout: "testkey": { 2024-09-12T03:29:02.921 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testkey", 2024-09-12T03:29:02.921 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.921 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.921 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.921 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.921 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.921 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.921 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.921 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.921 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.921 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.922 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.922 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.922 INFO:teuthology.orchestra.run.smithi006.stdout: "testlkey": { 2024-09-12T03:29:02.922 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testlkey", 2024-09-12T03:29:02.922 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.922 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.922 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.922 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.922 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.922 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.922 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.923 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.923 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.923 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.923 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.923 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.923 INFO:teuthology.orchestra.run.smithi006.stdout: "testnewline": { 2024-09-12T03:29:02.923 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testnewline", 2024-09-12T03:29:02.923 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.923 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.923 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.923 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.923 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.924 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.924 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.924 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.924 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.924 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.924 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.924 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.924 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.924 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.924 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.924 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snap_schedule", 2024-09-12T03:29:02.925 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.925 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.925 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.925 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_m_granularity": { 2024-09-12T03:29:02.925 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_m_granularity", 2024-09-12T03:29:02.925 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.925 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.925 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.925 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.925 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.925 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.926 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.926 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:29:02.926 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.926 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.926 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.926 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.926 INFO:teuthology.orchestra.run.smithi006.stdout: "dump_on_update": { 2024-09-12T03:29:02.926 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dump_on_update", 2024-09-12T03:29:02.926 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.926 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.926 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.926 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.927 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.927 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.927 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.927 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:29:02.927 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.927 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.927 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.927 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.927 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.927 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.927 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.927 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.928 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.928 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.928 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.928 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.928 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.928 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.928 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.928 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.928 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.928 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.928 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.929 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.929 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.929 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.929 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.929 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.929 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.929 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.929 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.929 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.929 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.929 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.930 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.930 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.930 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.930 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.930 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.930 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.930 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.930 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.931 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.931 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.931 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.931 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.931 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.931 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.931 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.931 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.931 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.932 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.932 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.932 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.932 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.932 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.932 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.932 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.932 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.932 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.932 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.932 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.933 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.933 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.933 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.933 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.933 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.933 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.933 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.933 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.933 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.933 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.933 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.933 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.933 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.934 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.934 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.934 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.934 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.934 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.934 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.934 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stats", 2024-09-12T03:29:02.934 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.934 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.934 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.934 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.934 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.934 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.934 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.935 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.935 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.935 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.935 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.935 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.935 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.935 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.935 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.935 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.935 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.935 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.935 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.935 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.936 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.936 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.936 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.936 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.936 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.936 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.936 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.936 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.936 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.936 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.936 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.936 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.937 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.937 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.937 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.937 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.937 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.937 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.937 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.937 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.937 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.937 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.937 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.937 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.937 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.938 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.938 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.938 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.938 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.938 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.938 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.938 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.938 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.938 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.938 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.938 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.938 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.938 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.939 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.939 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.939 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.939 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.939 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.939 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.939 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.939 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.939 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.939 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.939 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.939 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.939 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.940 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.940 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.940 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.940 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.940 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.940 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "status", 2024-09-12T03:29:02.940 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.940 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.940 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.940 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.940 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.940 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.941 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.941 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.941 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.941 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.941 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.941 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.941 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.941 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.941 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.941 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.941 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.941 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.941 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.942 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.942 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.942 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.942 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.942 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.942 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.942 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.942 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.942 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.942 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.942 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.942 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.942 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.942 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.943 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.943 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.943 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.943 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.943 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.943 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.943 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.943 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.943 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.943 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.943 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.943 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.944 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.944 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.944 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.944 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.944 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.944 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.944 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.944 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.944 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.944 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.944 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.944 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.944 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.945 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.945 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.945 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.945 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.945 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.945 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.945 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.945 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.945 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.945 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.945 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.945 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.945 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.945 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.946 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.946 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.946 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.946 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.946 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telegraf", 2024-09-12T03:29:02.946 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.946 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.946 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.946 INFO:teuthology.orchestra.run.smithi006.stdout: "address": { 2024-09-12T03:29:02.946 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "address", 2024-09-12T03:29:02.946 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.946 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.946 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.947 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:29:02.947 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.947 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.947 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.947 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.947 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.947 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.947 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.947 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.947 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-09-12T03:29:02.947 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-09-12T03:29:02.947 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.947 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.947 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.948 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15", 2024-09-12T03:29:02.948 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.948 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.948 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.948 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.948 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.948 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.948 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.948 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.948 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.948 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.948 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.948 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.949 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.949 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.949 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.949 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.949 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.949 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.949 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.949 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.949 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.949 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.949 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.949 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.949 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.950 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.950 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.950 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.950 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.950 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.950 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.950 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.950 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.950 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.950 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.950 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.950 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.950 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.950 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.951 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.951 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.951 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.951 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.951 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.951 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.951 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.951 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.951 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.951 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.951 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.951 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.951 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.951 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.952 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.952 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.952 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.952 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.952 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.952 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.952 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.952 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.952 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.952 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.952 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.952 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.952 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.953 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.953 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.953 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.953 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.953 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.953 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.953 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.953 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.953 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.953 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.953 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.953 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.953 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.953 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.954 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.954 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telemetry", 2024-09-12T03:29:02.954 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.954 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.954 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.954 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_basic": { 2024-09-12T03:29:02.954 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_basic", 2024-09-12T03:29:02.954 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.954 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.954 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.954 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.954 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.954 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.955 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.955 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:29:02.955 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.955 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.955 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.955 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.955 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_crash": { 2024-09-12T03:29:02.955 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_crash", 2024-09-12T03:29:02.955 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.955 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.955 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.955 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.955 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.955 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.956 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.956 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:29:02.956 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.956 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.956 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.956 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.956 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_device": { 2024-09-12T03:29:02.956 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_device", 2024-09-12T03:29:02.956 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.956 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.956 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.956 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.956 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.957 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.957 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.957 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:29:02.957 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.957 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.957 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.957 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.957 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_ident": { 2024-09-12T03:29:02.957 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_ident", 2024-09-12T03:29:02.957 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.957 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.957 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.957 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.957 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.958 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.958 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.958 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:29:02.958 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.958 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.958 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.958 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.958 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_perf": { 2024-09-12T03:29:02.958 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_perf", 2024-09-12T03:29:02.958 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.958 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.958 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.958 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.959 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.959 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.959 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.959 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:29:02.959 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.959 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.959 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.959 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.959 INFO:teuthology.orchestra.run.smithi006.stdout: "contact": { 2024-09-12T03:29:02.959 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "contact", 2024-09-12T03:29:02.959 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.959 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.959 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.959 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.960 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.960 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.960 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.960 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.960 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.960 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.960 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.960 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.960 INFO:teuthology.orchestra.run.smithi006.stdout: "description": { 2024-09-12T03:29:02.960 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "description", 2024-09-12T03:29:02.960 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.961 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.961 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.961 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.961 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.961 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.961 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.961 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.961 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.961 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.961 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.961 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.962 INFO:teuthology.orchestra.run.smithi006.stdout: "device_url": { 2024-09-12T03:29:02.962 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_url", 2024-09-12T03:29:02.962 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.962 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.962 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.962 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:29:02.962 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.962 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.962 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.962 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.962 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.962 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.962 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.962 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.963 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2024-09-12T03:29:02.963 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2024-09-12T03:29:02.963 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.963 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.963 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.963 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.963 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.963 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.963 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.963 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.963 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.963 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.964 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.964 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.964 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-09-12T03:29:02.964 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-09-12T03:29:02.964 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.964 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.964 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.964 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "24", 2024-09-12T03:29:02.964 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "8", 2024-09-12T03:29:02.964 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.964 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.965 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.965 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.965 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.965 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.965 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.965 INFO:teuthology.orchestra.run.smithi006.stdout: "last_opt_revision": { 2024-09-12T03:29:02.965 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "last_opt_revision", 2024-09-12T03:29:02.965 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.965 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.965 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.965 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1", 2024-09-12T03:29:02.966 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.966 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.966 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.966 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.966 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.966 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.966 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.966 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.966 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard": { 2024-09-12T03:29:02.966 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard", 2024-09-12T03:29:02.966 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.966 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.966 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.967 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.967 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.967 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.967 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.967 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.967 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.967 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.967 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.967 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.967 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard_description": { 2024-09-12T03:29:02.967 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard_description", 2024-09-12T03:29:02.968 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.968 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.968 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.968 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.968 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.968 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.968 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.968 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.968 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.968 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.968 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.969 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.969 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.969 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.969 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.969 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.969 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.969 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.969 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.969 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.969 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.970 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.970 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.970 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.970 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.970 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.970 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.970 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.970 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.970 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.970 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.970 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.970 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.970 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.970 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.971 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.971 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.971 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.971 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.971 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.971 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.971 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.971 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.971 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.971 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.971 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.971 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.971 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.972 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.972 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.972 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.972 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.972 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.972 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.972 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.972 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.972 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.972 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.972 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.972 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.972 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.972 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.973 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.973 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.973 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.973 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.973 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.973 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.973 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.973 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.973 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.973 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.973 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.974 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.974 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.974 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.974 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.974 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.974 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.974 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.974 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.974 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.974 INFO:teuthology.orchestra.run.smithi006.stdout: "organization": { 2024-09-12T03:29:02.975 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "organization", 2024-09-12T03:29:02.975 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.975 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.975 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.975 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.975 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.975 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.975 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.975 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.975 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.976 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.976 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.976 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.976 INFO:teuthology.orchestra.run.smithi006.stdout: "proxy": { 2024-09-12T03:29:02.976 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "proxy", 2024-09-12T03:29:02.976 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.976 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.976 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.976 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.976 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.977 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.977 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.977 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.977 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.977 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.977 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.977 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.977 INFO:teuthology.orchestra.run.smithi006.stdout: "url": { 2024-09-12T03:29:02.977 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url", 2024-09-12T03:29:02.977 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.977 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.978 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.978 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:29:02.978 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.978 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.978 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.978 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.978 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.978 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.978 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.978 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.978 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.978 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.979 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.979 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "test_orchestrator", 2024-09-12T03:29:02.979 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.979 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.979 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.979 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.979 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.979 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.979 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.979 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.979 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.979 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.979 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.979 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.980 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.980 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.980 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.980 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.980 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.980 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.980 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.980 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.980 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.980 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.980 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.980 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.980 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.981 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.981 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.981 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.981 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.981 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.981 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.981 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.981 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.981 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.981 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.981 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.981 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.981 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.981 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.982 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.982 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.982 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.982 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.982 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.982 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.982 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.982 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.982 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.982 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.982 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.982 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.982 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.982 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.983 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.983 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.983 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.983 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.983 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.983 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.983 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.983 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.983 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.983 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.983 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.983 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.983 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.983 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.984 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.984 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.984 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.984 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.984 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.984 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.984 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.984 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.984 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.984 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "volumes", 2024-09-12T03:29:02.984 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.984 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.984 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.984 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.985 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.985 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.985 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.985 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.985 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.985 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.985 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.985 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.985 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.985 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.985 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.985 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.985 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.985 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.986 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.986 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.986 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.986 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.986 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.986 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.986 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.986 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.986 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.986 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.986 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.986 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.986 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.987 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.987 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.987 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.987 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.987 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.987 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.987 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.987 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:02.987 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:02.987 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.987 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.987 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.987 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:02.988 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.988 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.988 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.988 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.988 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.988 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.988 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.988 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.988 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.988 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.988 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.988 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.988 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.989 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.989 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.989 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:02.989 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:02.989 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.989 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.989 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.989 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.989 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.989 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.989 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.989 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.989 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.989 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.990 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.990 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.990 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_clones": { 2024-09-12T03:29:02.990 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_clones", 2024-09-12T03:29:02.990 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.990 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.990 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.990 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4", 2024-09-12T03:29:02.990 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.990 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.990 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.990 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:29:02.990 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.990 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.991 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.991 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.991 INFO:teuthology.orchestra.run.smithi006.stdout: "periodic_async_work": { 2024-09-12T03:29:02.991 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "periodic_async_work", 2024-09-12T03:29:02.991 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.991 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.991 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.991 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.991 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.991 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.991 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.991 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Periodically check for async work", 2024-09-12T03:29:02.991 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.991 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.992 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.992 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.992 INFO:teuthology.orchestra.run.smithi006.stdout: "snapshot_clone_delay": { 2024-09-12T03:29:02.992 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:29:02.992 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:02.992 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.992 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.992 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-09-12T03:29:02.992 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.992 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.992 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.992 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:29:02.992 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.993 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.993 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.993 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.993 INFO:teuthology.orchestra.run.smithi006.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:29:02.993 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:29:02.993 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.993 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.993 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.993 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-09-12T03:29:02.993 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.993 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.993 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.993 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:29:02.993 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.994 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.994 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.994 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.994 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:02.994 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.994 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:02.994 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix", 2024-09-12T03:29:02.994 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-09-12T03:29:02.994 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-09-12T03:29:02.994 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-09-12T03:29:02.994 INFO:teuthology.orchestra.run.smithi006.stdout: "discovery_interval": { 2024-09-12T03:29:02.994 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "discovery_interval", 2024-09-12T03:29:02.994 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-09-12T03:29:02.995 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.995 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.995 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "100", 2024-09-12T03:29:02.995 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.995 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.995 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.995 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.995 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.995 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.995 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.995 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.995 INFO:teuthology.orchestra.run.smithi006.stdout: "identifier": { 2024-09-12T03:29:02.995 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "identifier", 2024-09-12T03:29:02.995 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.996 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.996 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.996 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.996 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.996 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.996 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.996 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.996 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.996 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.996 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.996 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.996 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-09-12T03:29:02.996 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-09-12T03:29:02.996 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-09-12T03:29:02.997 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.997 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:02.997 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-09-12T03:29:02.997 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.997 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.997 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:02.997 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.997 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.997 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.997 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.997 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.997 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-09-12T03:29:02.997 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-09-12T03:29:02.998 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:02.998 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.998 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.998 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:02.998 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.998 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:02.998 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:02.998 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:02.998 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:02.998 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:02.998 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:02.998 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:02.998 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:02.998 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:02.999 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:02.999 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:02.999 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:02.999 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:02.999 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:02.999 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-09-12T03:29:02.999 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-09-12T03:29:02.999 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:02.999 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:02.999 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:02.999 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:02.999 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:02.999 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:03.000 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:03.000 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:03.000 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:03.000 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:03.000 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:03.000 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:03.000 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-09-12T03:29:03.000 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-09-12T03:29:03.000 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:03.000 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:03.000 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:03.000 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-09-12T03:29:03.000 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:03.001 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:03.001 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-09-12T03:29:03.001 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-09-12T03:29:03.001 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-09-12T03:29:03.001 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-09-12T03:29:03.001 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-09-12T03:29:03.001 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-09-12T03:29:03.001 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-09-12T03:29:03.001 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:03.001 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:03.001 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:03.001 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:03.001 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:03.002 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:03.002 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-09-12T03:29:03.002 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-09-12T03:29:03.002 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-09-12T03:29:03.002 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:03.002 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-09-12T03:29:03.002 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-09-12T03:29:03.002 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:03.002 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:03.002 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:03.002 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:03.002 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:03.002 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:03.002 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:03.003 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:03.003 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_host": { 2024-09-12T03:29:03.003 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_host", 2024-09-12T03:29:03.003 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:03.003 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:03.003 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:03.003 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-09-12T03:29:03.003 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:03.003 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:03.003 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:03.003 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:03.003 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:03.003 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:03.003 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:03.004 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:03.004 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_port": { 2024-09-12T03:29:03.004 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_port", 2024-09-12T03:29:03.004 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-09-12T03:29:03.004 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:03.004 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:03.004 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10051", 2024-09-12T03:29:03.004 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:03.004 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:03.004 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:03.004 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:03.004 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:03.004 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:03.005 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:03.005 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:03.005 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_sender": { 2024-09-12T03:29:03.005 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_sender", 2024-09-12T03:29:03.005 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-09-12T03:29:03.005 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-09-12T03:29:03.005 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-09-12T03:29:03.005 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:29:03.005 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-09-12T03:29:03.006 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-09-12T03:29:03.006 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-09-12T03:29:03.006 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-09-12T03:29:03.006 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-09-12T03:29:03.006 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-09-12T03:29:03.006 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-09-12T03:29:03.006 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:03.006 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:03.006 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:03.006 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:03.006 INFO:teuthology.orchestra.run.smithi006.stdout: "services": { 2024-09-12T03:29:03.006 INFO:teuthology.orchestra.run.smithi006.stdout: "dashboard": "https://172.21.15.6:8443/" 2024-09-12T03:29:03.006 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:03.007 INFO:teuthology.orchestra.run.smithi006.stdout: "always_on_modules": { 2024-09-12T03:29:03.007 INFO:teuthology.orchestra.run.smithi006.stdout: "octopus": [ 2024-09-12T03:29:03.007 INFO:teuthology.orchestra.run.smithi006.stdout: "balancer", 2024-09-12T03:29:03.007 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-09-12T03:29:03.007 INFO:teuthology.orchestra.run.smithi006.stdout: "devicehealth", 2024-09-12T03:29:03.007 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator", 2024-09-12T03:29:03.007 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_autoscaler", 2024-09-12T03:29:03.007 INFO:teuthology.orchestra.run.smithi006.stdout: "progress", 2024-09-12T03:29:03.007 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_support", 2024-09-12T03:29:03.007 INFO:teuthology.orchestra.run.smithi006.stdout: "status", 2024-09-12T03:29:03.007 INFO:teuthology.orchestra.run.smithi006.stdout: "telemetry", 2024-09-12T03:29:03.007 INFO:teuthology.orchestra.run.smithi006.stdout: "volumes" 2024-09-12T03:29:03.007 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:03.008 INFO:teuthology.orchestra.run.smithi006.stdout: "pacific": [ 2024-09-12T03:29:03.008 INFO:teuthology.orchestra.run.smithi006.stdout: "balancer", 2024-09-12T03:29:03.008 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-09-12T03:29:03.008 INFO:teuthology.orchestra.run.smithi006.stdout: "devicehealth", 2024-09-12T03:29:03.008 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator", 2024-09-12T03:29:03.008 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_autoscaler", 2024-09-12T03:29:03.008 INFO:teuthology.orchestra.run.smithi006.stdout: "progress", 2024-09-12T03:29:03.008 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_support", 2024-09-12T03:29:03.008 INFO:teuthology.orchestra.run.smithi006.stdout: "status", 2024-09-12T03:29:03.009 INFO:teuthology.orchestra.run.smithi006.stdout: "telemetry", 2024-09-12T03:29:03.009 INFO:teuthology.orchestra.run.smithi006.stdout: "volumes" 2024-09-12T03:29:03.009 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:03.009 INFO:teuthology.orchestra.run.smithi006.stdout: "quincy": [ 2024-09-12T03:29:03.009 INFO:teuthology.orchestra.run.smithi006.stdout: "balancer", 2024-09-12T03:29:03.009 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-09-12T03:29:03.009 INFO:teuthology.orchestra.run.smithi006.stdout: "devicehealth", 2024-09-12T03:29:03.009 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator", 2024-09-12T03:29:03.009 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_autoscaler", 2024-09-12T03:29:03.009 INFO:teuthology.orchestra.run.smithi006.stdout: "progress", 2024-09-12T03:29:03.009 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_support", 2024-09-12T03:29:03.009 INFO:teuthology.orchestra.run.smithi006.stdout: "status", 2024-09-12T03:29:03.009 INFO:teuthology.orchestra.run.smithi006.stdout: "telemetry", 2024-09-12T03:29:03.009 INFO:teuthology.orchestra.run.smithi006.stdout: "volumes" 2024-09-12T03:29:03.010 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:03.010 INFO:teuthology.orchestra.run.smithi006.stdout: "reef": [ 2024-09-12T03:29:03.010 INFO:teuthology.orchestra.run.smithi006.stdout: "balancer", 2024-09-12T03:29:03.010 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-09-12T03:29:03.010 INFO:teuthology.orchestra.run.smithi006.stdout: "devicehealth", 2024-09-12T03:29:03.010 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator", 2024-09-12T03:29:03.010 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_autoscaler", 2024-09-12T03:29:03.010 INFO:teuthology.orchestra.run.smithi006.stdout: "progress", 2024-09-12T03:29:03.010 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_support", 2024-09-12T03:29:03.010 INFO:teuthology.orchestra.run.smithi006.stdout: "status", 2024-09-12T03:29:03.011 INFO:teuthology.orchestra.run.smithi006.stdout: "telemetry", 2024-09-12T03:29:03.011 INFO:teuthology.orchestra.run.smithi006.stdout: "volumes" 2024-09-12T03:29:03.011 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-09-12T03:29:03.011 INFO:teuthology.orchestra.run.smithi006.stdout: "squid": [ 2024-09-12T03:29:03.011 INFO:teuthology.orchestra.run.smithi006.stdout: "balancer", 2024-09-12T03:29:03.011 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-09-12T03:29:03.011 INFO:teuthology.orchestra.run.smithi006.stdout: "devicehealth", 2024-09-12T03:29:03.011 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator", 2024-09-12T03:29:03.011 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_autoscaler", 2024-09-12T03:29:03.011 INFO:teuthology.orchestra.run.smithi006.stdout: "progress", 2024-09-12T03:29:03.011 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_support", 2024-09-12T03:29:03.012 INFO:teuthology.orchestra.run.smithi006.stdout: "status", 2024-09-12T03:29:03.012 INFO:teuthology.orchestra.run.smithi006.stdout: "telemetry", 2024-09-12T03:29:03.012 INFO:teuthology.orchestra.run.smithi006.stdout: "volumes" 2024-09-12T03:29:03.012 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-09-12T03:29:03.012 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:03.012 INFO:teuthology.orchestra.run.smithi006.stdout: "last_failure_osd_epoch": 3, 2024-09-12T03:29:03.012 INFO:teuthology.orchestra.run.smithi006.stdout: "active_clients": [ 2024-09-12T03:29:03.012 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:03.012 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "devicehealth", 2024-09-12T03:29:03.012 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2024-09-12T03:29:03.012 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:03.013 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2024-09-12T03:29:03.013 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2024-09-12T03:29:03.013 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3076856827 2024-09-12T03:29:03.013 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:03.013 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-09-12T03:29:03.013 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:03.013 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:03.013 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "libcephsqlite", 2024-09-12T03:29:03.013 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2024-09-12T03:29:03.013 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:03.014 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2024-09-12T03:29:03.014 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2024-09-12T03:29:03.014 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 933258147 2024-09-12T03:29:03.014 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:03.014 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-09-12T03:29:03.014 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:03.014 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:03.014 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_support", 2024-09-12T03:29:03.014 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2024-09-12T03:29:03.014 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:03.014 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2024-09-12T03:29:03.015 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2024-09-12T03:29:03.015 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 2549978381 2024-09-12T03:29:03.015 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:03.015 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-09-12T03:29:03.015 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:03.015 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:03.015 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "volumes", 2024-09-12T03:29:03.015 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2024-09-12T03:29:03.015 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-09-12T03:29:03.015 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2024-09-12T03:29:03.016 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2024-09-12T03:29:03.016 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 1720356888 2024-09-12T03:29:03.016 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:03.016 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-09-12T03:29:03.016 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:03.016 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-09-12T03:29:03.016 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:29:03.018 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:01 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/685143016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:29:03.018 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:01 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3114284010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:29:03.018 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:01 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2724961351' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:29:03.018 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:01 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/685143016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:29:03.019 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:01 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3114284010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:29:03.019 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:01 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2724961351' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:29:03.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.597+0000 7f5bcbb6d640 1 -- 172.21.15.6:0/2724961351 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5bb00a2b90 msgr2=0x7f5bb00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:03.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.597+0000 7f5bcbb6d640 1 --2- 172.21.15.6:0/2724961351 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5bb00a2b90 0x7f5bb00a5050 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5bb40058c0 tx=0x7f5bb4005850 comp rx=0 tx=0).stop 2024-09-12T03:29:03.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.597+0000 7f5bcbb6d640 1 -- 172.21.15.6:0/2724961351 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bc4102650 msgr2=0x7f5bc4101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:03.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.597+0000 7f5bcbb6d640 1 --2- 172.21.15.6:0/2724961351 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bc4102650 0x7f5bc4101fc0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000c920 tx=0x7f5bc000cde0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.598+0000 7f5bcbb6d640 1 -- 172.21.15.6:0/2724961351 shutdown_connections 2024-09-12T03:29:03.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.598+0000 7f5bcbb6d640 1 --2- 172.21.15.6:0/2724961351 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f5bb00a2b90 0x7f5bb00a5050 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.598+0000 7f5bcbb6d640 1 --2- 172.21.15.6:0/2724961351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bc41078b0 0x7f5bc4100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.598+0000 7f5bcbb6d640 1 --2- 172.21.15.6:0/2724961351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bc4104f80 0x7f5bc4100600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.598+0000 7f5bcbb6d640 1 --2- 172.21.15.6:0/2724961351 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bc4102650 0x7f5bc4101fc0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.598+0000 7f5bcbb6d640 1 -- 172.21.15.6:0/2724961351 >> 172.21.15.6:0/2724961351 conn(0x7f5bc40fc170 msgr2=0x7f5bc4102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:03.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.598+0000 7f5bcbb6d640 1 -- 172.21.15.6:0/2724961351 shutdown_connections 2024-09-12T03:29:03.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:01.598+0000 7f5bcbb6d640 1 -- 172.21.15.6:0/2724961351 wait complete. 2024-09-12T03:29:03.077 DEBUG:tasks.check_counter:Getting stats from a 2024-09-12T03:29:03.077 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-12T03:29:03.077 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:01 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/685143016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:29:03.078 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:01 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3114284010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:29:03.078 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:01 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2724961351' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:29:03.100 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:02 smithi006 ceph-mon[30991]: pgmap v573: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-12T03:29:03.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:02 smithi161 ceph-mon[35485]: pgmap v573: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-12T03:29:03.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:02 smithi118 ceph-mon[35048]: pgmap v573: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-12T03:29:03.336 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:03.894 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:29:03.894 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:29:03.894 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 4, 2024-09-12T03:29:03.894 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 4, 2024-09-12T03:29:03.894 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 3072, 2024-09-12T03:29:03.894 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 448, 2024-09-12T03:29:03.895 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 4, 2024-09-12T03:29:03.895 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 0, 2024-09-12T03:29:03.895 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.003609589, 2024-09-12T03:29:03.895 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.000040971, 2024-09-12T03:29:03.895 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.002464080, 2024-09-12T03:29:03.895 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:29:03.895 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:03.895 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 3, 2024-09-12T03:29:03.895 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.004271060, 2024-09-12T03:29:03.895 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.001423686 2024-09-12T03:29:03.895 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:03.896 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:03.896 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-09-12T03:29:03.896 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-09-12T03:29:03.896 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-09-12T03:29:03.896 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:03.896 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-12T03:29:03.896 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:29:03.896 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:03.896 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:29:03.896 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 170, 2024-09-12T03:29:03.897 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 165, 2024-09-12T03:29:03.897 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 739744, 2024-09-12T03:29:03.897 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 100688, 2024-09-12T03:29:03.897 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 2, 2024-09-12T03:29:03.897 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 1, 2024-09-12T03:29:03.897 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.028855455, 2024-09-12T03:29:03.897 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.010954808, 2024-09-12T03:29:03.897 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.022990637, 2024-09-12T03:29:03.897 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:29:03.898 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:03.898 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 164, 2024-09-12T03:29:03.898 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.012834036, 2024-09-12T03:29:03.898 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000078256 2024-09-12T03:29:03.898 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:03.898 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:03.898 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-09-12T03:29:03.898 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-09-12T03:29:03.899 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-09-12T03:29:03.899 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:03.899 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 739744, 2024-09-12T03:29:03.899 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 100688 2024-09-12T03:29:03.899 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:03.899 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:29:03.899 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 1, 2024-09-12T03:29:03.899 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 125, 2024-09-12T03:29:03.899 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 160, 2024-09-12T03:29:03.900 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 100592, 2024-09-12T03:29:03.900 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 3, 2024-09-12T03:29:03.900 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 2, 2024-09-12T03:29:03.900 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.017432830, 2024-09-12T03:29:03.900 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.014020495, 2024-09-12T03:29:03.900 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.001795512, 2024-09-12T03:29:03.900 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:29:03.900 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:03.900 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 123, 2024-09-12T03:29:03.901 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.025324727, 2024-09-12T03:29:03.901 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000205892 2024-09-12T03:29:03.901 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:03.901 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:03.901 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-09-12T03:29:03.901 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-09-12T03:29:03.901 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-09-12T03:29:03.902 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:03.902 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T03:29:03.902 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 100592 2024-09-12T03:29:03.902 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:03.902 INFO:teuthology.orchestra.run.smithi006.stdout: "cct": { 2024-09-12T03:29:03.902 INFO:teuthology.orchestra.run.smithi006.stdout: "total_workers": 0, 2024-09-12T03:29:03.902 INFO:teuthology.orchestra.run.smithi006.stdout: "unhealthy_workers": 0 2024-09-12T03:29:03.902 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:03.903 INFO:teuthology.orchestra.run.smithi006.stdout: "mempool": { 2024-09-12T03:29:03.903 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:29:03.903 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_items": 0, 2024-09-12T03:29:03.903 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:29:03.903 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:29:03.903 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:29:03.903 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:29:03.903 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:29:03.903 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:29:03.903 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:29:03.904 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:29:03.904 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:29:03.904 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:29:03.904 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:29:03.904 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:29:03.904 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:29:03.904 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_items": 0, 2024-09-12T03:29:03.905 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:29:03.905 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_items": 0, 2024-09-12T03:29:03.905 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:29:03.905 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:29:03.905 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:29:03.905 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:29:03.905 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:29:03.905 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:29:03.905 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:29:03.905 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_items": 0, 2024-09-12T03:29:03.906 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:29:03.906 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:29:03.906 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:29:03.906 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_items": 0, 2024-09-12T03:29:03.906 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_bytes": 0, 2024-09-12T03:29:03.906 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_items": 0, 2024-09-12T03:29:03.906 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:29:03.906 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:29:03.906 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:29:03.906 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:29:03.907 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_bytes": 149001, 2024-09-12T03:29:03.907 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_items": 31, 2024-09-12T03:29:03.907 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:29:03.907 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_items": 0, 2024-09-12T03:29:03.907 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_bytes": 0, 2024-09-12T03:29:03.907 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_items": 0, 2024-09-12T03:29:03.907 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:29:03.907 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_items": 0, 2024-09-12T03:29:03.907 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:29:03.907 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_items": 0, 2024-09-12T03:29:03.908 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_bytes": 0, 2024-09-12T03:29:03.908 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_items": 0, 2024-09-12T03:29:03.908 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:29:03.908 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:29:03.908 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_bytes": 0, 2024-09-12T03:29:03.908 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_items": 0, 2024-09-12T03:29:03.908 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_bytes": 928, 2024-09-12T03:29:03.908 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_items": 2, 2024-09-12T03:29:03.908 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_bytes": 0, 2024-09-12T03:29:03.908 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_items": 0, 2024-09-12T03:29:03.908 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_bytes": 0, 2024-09-12T03:29:03.909 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_items": 0 2024-09-12T03:29:03.909 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:03.909 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:29:03.909 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-09-12T03:29:03.909 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 104857600, 2024-09-12T03:29:03.909 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-09-12T03:29:03.909 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 171, 2024-09-12T03:29:03.910 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 715759, 2024-09-12T03:29:03.910 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:03.910 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 171, 2024-09-12T03:29:03.910 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2024-09-12T03:29:03.910 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2024-09-12T03:29:03.910 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 171, 2024-09-12T03:29:03.910 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 715759, 2024-09-12T03:29:03.910 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-09-12T03:29:03.910 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-09-12T03:29:03.910 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-09-12T03:29:03.911 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-09-12T03:29:03.911 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:03.911 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:03.911 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:29:04.691 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-12T03:29:04.691 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-12T03:29:04.691 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-09-12T03:29:04.692 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-09-12T03:29:04.692 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-09-12T03:29:04.692 DEBUG:tasks.check_counter:Getting stats from d 2024-09-12T03:29:04.692 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-12T03:29:04.995 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:05.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:04 smithi006 ceph-mon[30991]: pgmap v574: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:29:05.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:04 smithi161 ceph-mon[35485]: pgmap v574: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:29:05.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:04 smithi118 ceph-mon[35048]: pgmap v574: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:29:05.532 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:29:05.532 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:29:05.532 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 0, 2024-09-12T03:29:05.532 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 2, 2024-09-12T03:29:05.532 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:29:05.532 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 0, 2024-09-12T03:29:05.533 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 4, 2024-09-12T03:29:05.533 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 0, 2024-09-12T03:29:05.533 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.001418344, 2024-09-12T03:29:05.533 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:29:05.533 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.000259836, 2024-09-12T03:29:05.533 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:29:05.533 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:05.533 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-09-12T03:29:05.533 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-09-12T03:29:05.533 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-09-12T03:29:05.534 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:05.534 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:05.534 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-09-12T03:29:05.534 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-09-12T03:29:05.534 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-09-12T03:29:05.534 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:05.534 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:29:05.534 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:29:05.534 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:05.534 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:29:05.534 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 5, 2024-09-12T03:29:05.535 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 131, 2024-09-12T03:29:05.535 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 3232, 2024-09-12T03:29:05.535 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 101744, 2024-09-12T03:29:05.535 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 3, 2024-09-12T03:29:05.535 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 2, 2024-09-12T03:29:05.535 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.012335674, 2024-09-12T03:29:05.535 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.007013569, 2024-09-12T03:29:05.535 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.003581656, 2024-09-12T03:29:05.535 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:29:05.535 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:05.535 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 130, 2024-09-12T03:29:05.536 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.010261097, 2024-09-12T03:29:05.536 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000078931 2024-09-12T03:29:05.536 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:05.536 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:05.536 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-09-12T03:29:05.536 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-09-12T03:29:05.536 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-09-12T03:29:05.536 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:05.536 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-12T03:29:05.536 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 101744 2024-09-12T03:29:05.536 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:05.537 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:29:05.537 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 175, 2024-09-12T03:29:05.537 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 170, 2024-09-12T03:29:05.537 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 741104, 2024-09-12T03:29:05.537 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 103888, 2024-09-12T03:29:05.537 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 2, 2024-09-12T03:29:05.537 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 1, 2024-09-12T03:29:05.537 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.031648635, 2024-09-12T03:29:05.537 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.011291487, 2024-09-12T03:29:05.537 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.026452053, 2024-09-12T03:29:05.537 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:29:05.538 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:05.538 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 169, 2024-09-12T03:29:05.538 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.019767263, 2024-09-12T03:29:05.538 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000116966 2024-09-12T03:29:05.538 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:05.538 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:05.538 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-09-12T03:29:05.538 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-09-12T03:29:05.538 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-09-12T03:29:05.538 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:05.538 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 741104, 2024-09-12T03:29:05.538 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 103888 2024-09-12T03:29:05.539 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:05.539 INFO:teuthology.orchestra.run.smithi006.stdout: "cct": { 2024-09-12T03:29:05.539 INFO:teuthology.orchestra.run.smithi006.stdout: "total_workers": 0, 2024-09-12T03:29:05.539 INFO:teuthology.orchestra.run.smithi006.stdout: "unhealthy_workers": 0 2024-09-12T03:29:05.539 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:05.539 INFO:teuthology.orchestra.run.smithi006.stdout: "mempool": { 2024-09-12T03:29:05.539 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:29:05.539 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_items": 0, 2024-09-12T03:29:05.539 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:29:05.539 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:29:05.539 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:29:05.540 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:29:05.540 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:29:05.540 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:29:05.540 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:29:05.540 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:29:05.540 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:29:05.540 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:29:05.540 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:29:05.540 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:29:05.540 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:29:05.540 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_items": 0, 2024-09-12T03:29:05.541 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:29:05.541 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_items": 0, 2024-09-12T03:29:05.541 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:29:05.541 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:29:05.541 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:29:05.541 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:29:05.541 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:29:05.541 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:29:05.541 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:29:05.541 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_items": 0, 2024-09-12T03:29:05.542 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:29:05.542 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:29:05.542 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:29:05.542 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_items": 0, 2024-09-12T03:29:05.542 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_bytes": 0, 2024-09-12T03:29:05.542 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_items": 0, 2024-09-12T03:29:05.542 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:29:05.542 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:29:05.542 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:29:05.542 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:29:05.542 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_bytes": 148967, 2024-09-12T03:29:05.543 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_items": 29, 2024-09-12T03:29:05.543 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:29:05.543 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_items": 0, 2024-09-12T03:29:05.543 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_bytes": 0, 2024-09-12T03:29:05.543 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_items": 0, 2024-09-12T03:29:05.543 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:29:05.543 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_items": 0, 2024-09-12T03:29:05.543 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:29:05.543 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_items": 0, 2024-09-12T03:29:05.543 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_bytes": 0, 2024-09-12T03:29:05.543 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_items": 0, 2024-09-12T03:29:05.543 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:29:05.544 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:29:05.544 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_bytes": 0, 2024-09-12T03:29:05.544 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_items": 0, 2024-09-12T03:29:05.544 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_bytes": 928, 2024-09-12T03:29:05.544 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_items": 2, 2024-09-12T03:29:05.544 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_bytes": 0, 2024-09-12T03:29:05.544 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_items": 0, 2024-09-12T03:29:05.544 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_bytes": 0, 2024-09-12T03:29:05.544 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_items": 0 2024-09-12T03:29:05.544 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:05.544 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:29:05.545 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-09-12T03:29:05.545 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 104857600, 2024-09-12T03:29:05.545 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-09-12T03:29:05.545 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 176, 2024-09-12T03:29:05.545 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 716409, 2024-09-12T03:29:05.545 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:05.545 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 176, 2024-09-12T03:29:05.545 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2024-09-12T03:29:05.545 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2024-09-12T03:29:05.545 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 176, 2024-09-12T03:29:05.545 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 716409, 2024-09-12T03:29:05.546 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-09-12T03:29:05.546 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-09-12T03:29:05.546 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-09-12T03:29:05.546 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-09-12T03:29:05.546 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:05.546 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:05.546 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:29:06.399 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-12T03:29:06.400 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-12T03:29:06.400 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-12T03:29:06.400 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-12T03:29:06.400 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-12T03:29:06.400 DEBUG:tasks.check_counter:Getting stats from g 2024-09-12T03:29:06.400 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-12T03:29:06.658 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:07.066 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:06 smithi006 ceph-mon[30991]: pgmap v575: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:29:07.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:06 smithi161 ceph-mon[35485]: pgmap v575: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:29:07.170 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:29:07.170 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:29:07.170 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 171, 2024-09-12T03:29:07.170 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 166, 2024-09-12T03:29:07.171 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 740016, 2024-09-12T03:29:07.171 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 101328, 2024-09-12T03:29:07.171 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 4, 2024-09-12T03:29:07.171 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 0, 2024-09-12T03:29:07.171 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.031761124, 2024-09-12T03:29:07.171 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.010761841, 2024-09-12T03:29:07.171 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.026006441, 2024-09-12T03:29:07.171 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:29:07.171 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:07.171 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 165, 2024-09-12T03:29:07.171 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.017940724, 2024-09-12T03:29:07.171 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000108731 2024-09-12T03:29:07.172 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:07.172 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:07.172 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-09-12T03:29:07.172 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-09-12T03:29:07.172 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-09-12T03:29:07.172 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:07.172 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 740016, 2024-09-12T03:29:07.172 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 101328 2024-09-12T03:29:07.172 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:07.172 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:29:07.172 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 1, 2024-09-12T03:29:07.172 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 126, 2024-09-12T03:29:07.172 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 160, 2024-09-12T03:29:07.173 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 100768, 2024-09-12T03:29:07.173 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 3, 2024-09-12T03:29:07.173 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 2, 2024-09-12T03:29:07.173 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.010356931, 2024-09-12T03:29:07.173 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.007606002, 2024-09-12T03:29:07.173 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.001246379, 2024-09-12T03:29:07.173 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:29:07.173 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 124, 2024-09-12T03:29:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.028063185, 2024-09-12T03:29:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000226316 2024-09-12T03:29:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-09-12T03:29:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-09-12T03:29:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-09-12T03:29:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T03:29:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 100768 2024-09-12T03:29:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:29:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 4, 2024-09-12T03:29:07.175 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 4, 2024-09-12T03:29:07.175 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 3072, 2024-09-12T03:29:07.175 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 448, 2024-09-12T03:29:07.175 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 2, 2024-09-12T03:29:07.175 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 1, 2024-09-12T03:29:07.175 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.002460505, 2024-09-12T03:29:07.175 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.000043263, 2024-09-12T03:29:07.175 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.002414969, 2024-09-12T03:29:07.175 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:29:07.175 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:07.175 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 3, 2024-09-12T03:29:07.175 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.004445337, 2024-09-12T03:29:07.175 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.001481779 2024-09-12T03:29:07.176 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:07.176 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:07.176 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-09-12T03:29:07.176 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-09-12T03:29:07.176 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-09-12T03:29:07.176 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:07.176 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-12T03:29:07.176 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:29:07.176 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:07.176 INFO:teuthology.orchestra.run.smithi006.stdout: "cct": { 2024-09-12T03:29:07.176 INFO:teuthology.orchestra.run.smithi006.stdout: "total_workers": 0, 2024-09-12T03:29:07.176 INFO:teuthology.orchestra.run.smithi006.stdout: "unhealthy_workers": 0 2024-09-12T03:29:07.176 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "mempool": { 2024-09-12T03:29:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:29:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_items": 0, 2024-09-12T03:29:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:29:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:29:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:29:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:29:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:29:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:29:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:29:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:29:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:29:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:29:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:29:07.178 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:29:07.178 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:29:07.178 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_items": 0, 2024-09-12T03:29:07.178 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:29:07.178 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_items": 0, 2024-09-12T03:29:07.178 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:29:07.178 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:29:07.178 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:29:07.178 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:29:07.178 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:29:07.179 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:29:07.179 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:29:07.179 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_items": 0, 2024-09-12T03:29:07.179 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:29:07.179 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:29:07.179 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:29:07.179 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_items": 0, 2024-09-12T03:29:07.179 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_bytes": 0, 2024-09-12T03:29:07.180 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_items": 0, 2024-09-12T03:29:07.180 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:29:07.180 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:29:07.180 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:29:07.180 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:29:07.180 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_bytes": 148967, 2024-09-12T03:29:07.180 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_items": 29, 2024-09-12T03:29:07.180 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:29:07.180 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_items": 0, 2024-09-12T03:29:07.180 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_bytes": 0, 2024-09-12T03:29:07.180 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_items": 0, 2024-09-12T03:29:07.180 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:29:07.180 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_items": 0, 2024-09-12T03:29:07.181 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:29:07.181 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_items": 0, 2024-09-12T03:29:07.181 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_bytes": 0, 2024-09-12T03:29:07.181 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_items": 0, 2024-09-12T03:29:07.181 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:29:07.181 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:29:07.181 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_bytes": 0, 2024-09-12T03:29:07.181 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_items": 0, 2024-09-12T03:29:07.181 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_bytes": 928, 2024-09-12T03:29:07.181 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_items": 2, 2024-09-12T03:29:07.181 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_bytes": 0, 2024-09-12T03:29:07.181 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_items": 0, 2024-09-12T03:29:07.181 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_bytes": 0, 2024-09-12T03:29:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_items": 0 2024-09-12T03:29:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:29:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-09-12T03:29:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 104857600, 2024-09-12T03:29:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-09-12T03:29:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 172, 2024-09-12T03:29:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 715889, 2024-09-12T03:29:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 172, 2024-09-12T03:29:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2024-09-12T03:29:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2024-09-12T03:29:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 172, 2024-09-12T03:29:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 715889, 2024-09-12T03:29:07.183 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-09-12T03:29:07.183 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-09-12T03:29:07.183 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-09-12T03:29:07.183 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-09-12T03:29:07.183 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:07.183 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:07.183 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:29:07.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:06 smithi118 ceph-mon[35048]: pgmap v575: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:29:07.866 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-12T03:29:07.866 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-12T03:29:07.867 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-09-12T03:29:07.867 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-09-12T03:29:07.867 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-09-12T03:29:07.867 DEBUG:tasks.check_counter:Getting stats from j 2024-09-12T03:29:07.867 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-12T03:29:08.124 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:08.638 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-09-12T03:29:08.638 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:29:08.638 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 177, 2024-09-12T03:29:08.638 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 172, 2024-09-12T03:29:08.638 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 741648, 2024-09-12T03:29:08.638 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 105168, 2024-09-12T03:29:08.638 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 4, 2024-09-12T03:29:08.638 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 0, 2024-09-12T03:29:08.638 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.032281681, 2024-09-12T03:29:08.639 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.011552223, 2024-09-12T03:29:08.639 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.026497999, 2024-09-12T03:29:08.639 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:29:08.639 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:08.639 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 171, 2024-09-12T03:29:08.639 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.012613874, 2024-09-12T03:29:08.639 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000073765 2024-09-12T03:29:08.639 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:08.639 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:08.639 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-09-12T03:29:08.639 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-09-12T03:29:08.639 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-09-12T03:29:08.640 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:08.640 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 741648, 2024-09-12T03:29:08.640 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 105168 2024-09-12T03:29:08.640 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:08.640 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:29:08.640 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 1, 2024-09-12T03:29:08.640 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 130, 2024-09-12T03:29:08.640 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 160, 2024-09-12T03:29:08.640 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 101472, 2024-09-12T03:29:08.640 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 3, 2024-09-12T03:29:08.640 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 2, 2024-09-12T03:29:08.640 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.010172888, 2024-09-12T03:29:08.641 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.007415214, 2024-09-12T03:29:08.641 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.001268004, 2024-09-12T03:29:08.641 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:29:08.641 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:08.641 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 128, 2024-09-12T03:29:08.641 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.005255588, 2024-09-12T03:29:08.641 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000041059 2024-09-12T03:29:08.641 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:08.641 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:08.641 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-09-12T03:29:08.642 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-09-12T03:29:08.642 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-09-12T03:29:08.642 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:08.643 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T03:29:08.643 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 101472 2024-09-12T03:29:08.643 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:08.644 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:29:08.644 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 4, 2024-09-12T03:29:08.644 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 4, 2024-09-12T03:29:08.644 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 3072, 2024-09-12T03:29:08.644 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 448, 2024-09-12T03:29:08.644 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 2, 2024-09-12T03:29:08.644 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 1, 2024-09-12T03:29:08.644 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.002300862, 2024-09-12T03:29:08.644 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.000052616, 2024-09-12T03:29:08.644 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.002212886, 2024-09-12T03:29:08.644 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:29:08.645 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:08.645 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 3, 2024-09-12T03:29:08.645 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.004063618, 2024-09-12T03:29:08.645 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.001354539 2024-09-12T03:29:08.645 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:08.645 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:08.645 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-09-12T03:29:08.645 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-09-12T03:29:08.645 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-09-12T03:29:08.645 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:08.645 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-12T03:29:08.645 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:29:08.645 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:08.646 INFO:teuthology.orchestra.run.smithi006.stdout: "cct": { 2024-09-12T03:29:08.646 INFO:teuthology.orchestra.run.smithi006.stdout: "total_workers": 0, 2024-09-12T03:29:08.646 INFO:teuthology.orchestra.run.smithi006.stdout: "unhealthy_workers": 0 2024-09-12T03:29:08.646 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:08.646 INFO:teuthology.orchestra.run.smithi006.stdout: "mempool": { 2024-09-12T03:29:08.646 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:29:08.646 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_items": 0, 2024-09-12T03:29:08.646 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:29:08.646 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:29:08.646 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:29:08.646 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:29:08.646 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:29:08.646 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:29:08.647 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:29:08.647 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:29:08.647 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:29:08.647 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:29:08.647 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:29:08.647 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:29:08.647 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:29:08.647 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_items": 0, 2024-09-12T03:29:08.647 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:29:08.647 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_items": 0, 2024-09-12T03:29:08.647 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:29:08.647 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:29:08.647 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:29:08.648 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:29:08.648 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:29:08.648 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:29:08.648 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:29:08.648 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_items": 0, 2024-09-12T03:29:08.648 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:29:08.648 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:29:08.648 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:29:08.648 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_items": 0, 2024-09-12T03:29:08.648 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_bytes": 0, 2024-09-12T03:29:08.648 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_items": 0, 2024-09-12T03:29:08.648 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:29:08.648 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:29:08.649 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:29:08.649 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:29:08.649 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_bytes": 149001, 2024-09-12T03:29:08.649 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_items": 31, 2024-09-12T03:29:08.649 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:29:08.649 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_items": 0, 2024-09-12T03:29:08.649 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_bytes": 0, 2024-09-12T03:29:08.649 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_items": 0, 2024-09-12T03:29:08.649 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:29:08.649 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_items": 0, 2024-09-12T03:29:08.649 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:29:08.649 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_items": 0, 2024-09-12T03:29:08.649 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_bytes": 0, 2024-09-12T03:29:08.650 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_items": 0, 2024-09-12T03:29:08.650 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:29:08.650 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:29:08.650 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_bytes": 0, 2024-09-12T03:29:08.650 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_items": 0, 2024-09-12T03:29:08.650 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_bytes": 928, 2024-09-12T03:29:08.650 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_items": 2, 2024-09-12T03:29:08.650 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_bytes": 0, 2024-09-12T03:29:08.650 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_items": 0, 2024-09-12T03:29:08.650 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_bytes": 0, 2024-09-12T03:29:08.650 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_items": 0 2024-09-12T03:29:08.650 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-09-12T03:29:08.650 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:29:08.651 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-09-12T03:29:08.651 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 104857600, 2024-09-12T03:29:08.651 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-09-12T03:29:08.651 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 178, 2024-09-12T03:29:08.651 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 716669, 2024-09-12T03:29:08.651 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:08.651 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 178, 2024-09-12T03:29:08.651 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2024-09-12T03:29:08.651 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2024-09-12T03:29:08.651 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 178, 2024-09-12T03:29:08.651 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 716669, 2024-09-12T03:29:08.651 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-09-12T03:29:08.651 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-09-12T03:29:08.652 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-09-12T03:29:08.652 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-09-12T03:29:08.652 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:08.652 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-09-12T03:29:08.652 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-09-12T03:29:09.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:08 smithi118 ceph-mon[35048]: pgmap v576: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:29:09.218 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:08 smithi006 ceph-mon[30991]: pgmap v576: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:29:09.246 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-12T03:29:09.246 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-12T03:29:09.246 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-12T03:29:09.246 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-12T03:29:09.247 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-12T03:29:09.247 DEBUG:tasks.check_counter:Getting stats from b 2024-09-12T03:29:09.247 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-12T03:29:09.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:08 smithi161 ceph-mon[35485]: pgmap v576: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:29:09.507 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:29:09.977 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-12T03:29:09.977 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:29:09.977 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 6480, 2024-09-12T03:29:09.977 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 8508, 2024-09-12T03:29:09.978 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 1559439, 2024-09-12T03:29:09.978 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 72437612, 2024-09-12T03:29:09.978 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 89, 2024-09-12T03:29:09.978 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 4, 2024-09-12T03:29:09.978 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 0.835937513, 2024-09-12T03:29:09.978 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 0.387957644, 2024-09-12T03:29:09.978 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 0.241872077, 2024-09-12T03:29:09.978 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.176193930, 2024-09-12T03:29:09.978 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:09.978 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 8506, 2024-09-12T03:29:09.978 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 2.632413018, 2024-09-12T03:29:09.978 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000309477 2024-09-12T03:29:09.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.979 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:09.979 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 6393, 2024-09-12T03:29:09.979 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.002330798, 2024-09-12T03:29:09.979 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000364 2024-09-12T03:29:09.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.979 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 42816, 2024-09-12T03:29:09.979 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 5136 2024-09-12T03:29:09.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.979 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:29:09.979 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 83943, 2024-09-12T03:29:09.979 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 89913, 2024-09-12T03:29:09.979 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 33743883, 2024-09-12T03:29:09.979 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 77784895, 2024-09-12T03:29:09.980 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 27, 2024-09-12T03:29:09.980 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 5, 2024-09-12T03:29:09.980 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 5.697969795, 2024-09-12T03:29:09.980 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 2.222154502, 2024-09-12T03:29:09.980 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 9.674715578, 2024-09-12T03:29:09.980 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.189273623, 2024-09-12T03:29:09.980 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:09.980 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 89912, 2024-09-12T03:29:09.980 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 98.514666295, 2024-09-12T03:29:09.980 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.001095678 2024-09-12T03:29:09.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.980 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:09.980 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 115772, 2024-09-12T03:29:09.980 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.044953146, 2024-09-12T03:29:09.981 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000388 2024-09-12T03:29:09.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.981 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 32739808, 2024-09-12T03:29:09.981 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 37878896 2024-09-12T03:29:09.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.981 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:29:09.981 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 5100, 2024-09-12T03:29:09.981 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 7390, 2024-09-12T03:29:09.981 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 1224728, 2024-09-12T03:29:09.981 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 38244703, 2024-09-12T03:29:09.981 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 12, 2024-09-12T03:29:09.981 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 6, 2024-09-12T03:29:09.981 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 0.857008116, 2024-09-12T03:29:09.982 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 0.544230965, 2024-09-12T03:29:09.982 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 0.179568043, 2024-09-12T03:29:09.982 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.133476590, 2024-09-12T03:29:09.982 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:09.982 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 7389, 2024-09-12T03:29:09.982 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 2.344078105, 2024-09-12T03:29:09.982 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000317238 2024-09-12T03:29:09.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.982 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:09.982 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 6179, 2024-09-12T03:29:09.982 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.001782678, 2024-09-12T03:29:09.982 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000288 2024-09-12T03:29:09.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.983 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-12T03:29:09.983 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 308048 2024-09-12T03:29:09.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.983 INFO:teuthology.orchestra.run.smithi118.stdout: "cct": { 2024-09-12T03:29:09.983 INFO:teuthology.orchestra.run.smithi118.stdout: "total_workers": 1, 2024-09-12T03:29:09.983 INFO:teuthology.orchestra.run.smithi118.stdout: "unhealthy_workers": 0 2024-09-12T03:29:09.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.983 INFO:teuthology.orchestra.run.smithi118.stdout: "finisher-MDSRank": { 2024-09-12T03:29:09.983 INFO:teuthology.orchestra.run.smithi118.stdout: "queue_len": 0, 2024-09-12T03:29:09.983 INFO:teuthology.orchestra.run.smithi118.stdout: "complete_latency": { 2024-09-12T03:29:09.984 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 5737, 2024-09-12T03:29:09.984 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 46.316497159, 2024-09-12T03:29:09.984 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.008073295 2024-09-12T03:29:09.984 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:09.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.984 INFO:teuthology.orchestra.run.smithi118.stdout: "finisher-PurgeQueue": { 2024-09-12T03:29:09.984 INFO:teuthology.orchestra.run.smithi118.stdout: "queue_len": 0, 2024-09-12T03:29:09.984 INFO:teuthology.orchestra.run.smithi118.stdout: "complete_latency": { 2024-09-12T03:29:09.984 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 4516, 2024-09-12T03:29:09.984 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.208463913, 2024-09-12T03:29:09.984 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000046161 2024-09-12T03:29:09.984 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:09.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.984 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-09-12T03:29:09.985 INFO:teuthology.orchestra.run.smithi118.stdout: "request": 3470, 2024-09-12T03:29:09.985 INFO:teuthology.orchestra.run.smithi118.stdout: "reply": 3468, 2024-09-12T03:29:09.985 INFO:teuthology.orchestra.run.smithi118.stdout: "reply_latency": { 2024-09-12T03:29:09.985 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 3468, 2024-09-12T03:29:09.985 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 88.940207325, 2024-09-12T03:29:09.985 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.025645965 2024-09-12T03:29:09.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.985 INFO:teuthology.orchestra.run.smithi118.stdout: "slow_reply": 0, 2024-09-12T03:29:09.985 INFO:teuthology.orchestra.run.smithi118.stdout: "forward": 2, 2024-09-12T03:29:09.985 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_fetch_complete": 0, 2024-09-12T03:29:09.985 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_fetch_keys": 0, 2024-09-12T03:29:09.985 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_commit": 21, 2024-09-12T03:29:09.985 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_split": 1, 2024-09-12T03:29:09.986 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_merge": 0, 2024-09-12T03:29:09.986 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes": 166, 2024-09-12T03:29:09.986 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_top": 49, 2024-09-12T03:29:09.986 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_bottom": 95, 2024-09-12T03:29:09.986 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_pin_tail": 22, 2024-09-12T03:29:09.986 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_pinned": 95, 2024-09-12T03:29:09.986 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_expired": 8, 2024-09-12T03:29:09.986 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_with_caps": 2, 2024-09-12T03:29:09.986 INFO:teuthology.orchestra.run.smithi118.stdout: "caps": 2, 2024-09-12T03:29:09.986 INFO:teuthology.orchestra.run.smithi118.stdout: "subtrees": 3, 2024-09-12T03:29:09.987 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse": 4920, 2024-09-12T03:29:09.987 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_hit": 4641, 2024-09-12T03:29:09.987 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_forward": 0, 2024-09-12T03:29:09.987 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_discover": 0, 2024-09-12T03:29:09.987 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:29:09.987 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_remote_ino": 0, 2024-09-12T03:29:09.987 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_lock": 48, 2024-09-12T03:29:09.987 INFO:teuthology.orchestra.run.smithi118.stdout: "load_cent": 0, 2024-09-12T03:29:09.988 INFO:teuthology.orchestra.run.smithi118.stdout: "q": 0, 2024-09-12T03:29:09.988 INFO:teuthology.orchestra.run.smithi118.stdout: "exported": 2, 2024-09-12T03:29:09.988 INFO:teuthology.orchestra.run.smithi118.stdout: "exported_inodes": 4, 2024-09-12T03:29:09.988 INFO:teuthology.orchestra.run.smithi118.stdout: "imported": 2, 2024-09-12T03:29:09.988 INFO:teuthology.orchestra.run.smithi118.stdout: "imported_inodes": 0, 2024-09-12T03:29:09.988 INFO:teuthology.orchestra.run.smithi118.stdout: "openino_dir_fetch": 0, 2024-09-12T03:29:09.988 INFO:teuthology.orchestra.run.smithi118.stdout: "openino_backtrace_fetch": 2050, 2024-09-12T03:29:09.988 INFO:teuthology.orchestra.run.smithi118.stdout: "openino_peer_discover": 0, 2024-09-12T03:29:09.988 INFO:teuthology.orchestra.run.smithi118.stdout: "root_rfiles": 4, 2024-09-12T03:29:09.989 INFO:teuthology.orchestra.run.smithi118.stdout: "root_rbytes": 234, 2024-09-12T03:29:09.989 INFO:teuthology.orchestra.run.smithi118.stdout: "root_rsnaps": 0, 2024-09-12T03:29:09.989 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_backtrace_fetch": 2050, 2024-09-12T03:29:09.989 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_set_tag": 0, 2024-09-12T03:29:09.989 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:29:09.989 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:29:09.989 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_dir_inodes": 465, 2024-09-12T03:29:09.989 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_dir_base_inodes": 7, 2024-09-12T03:29:09.989 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_dirfrag_rstats": 458, 2024-09-12T03:29:09.989 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_file_inodes": 1585, 2024-09-12T03:29:09.989 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_inode_file_caps": 8, 2024-09-12T03:29:09.989 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_revoke": 1568, 2024-09-12T03:29:09.989 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_grant": 4730, 2024-09-12T03:29:09.990 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_trunc": 6, 2024-09-12T03:29:09.990 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:29:09.990 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:29:09.990 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_caps": 75953, 2024-09-12T03:29:09.990 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_caps_dirty": 74418, 2024-09-12T03:29:09.990 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_cap_release": 98, 2024-09-12T03:29:09.990 INFO:teuthology.orchestra.run.smithi118.stdout: "process_request_cap_release": 1479 2024-09-12T03:29:09.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.990 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_cache": { 2024-09-12T03:29:09.990 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_update": 179, 2024-09-12T03:29:09.990 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_update_receipt": 0, 2024-09-12T03:29:09.990 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_try_discover": 0, 2024-09-12T03:29:09.990 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_send_discover": 0, 2024-09-12T03:29:09.990 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_handle_discover": 345, 2024-09-12T03:29:09.991 INFO:teuthology.orchestra.run.smithi118.stdout: "num_strays": 11, 2024-09-12T03:29:09.991 INFO:teuthology.orchestra.run.smithi118.stdout: "num_strays_delayed": 0, 2024-09-12T03:29:09.991 INFO:teuthology.orchestra.run.smithi118.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:29:09.991 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_created": 932, 2024-09-12T03:29:09.991 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_enqueued": 860, 2024-09-12T03:29:09.991 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_reintegrated": 61, 2024-09-12T03:29:09.991 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_migrated": 0, 2024-09-12T03:29:09.991 INFO:teuthology.orchestra.run.smithi118.stdout: "num_recovering_processing": 0, 2024-09-12T03:29:09.991 INFO:teuthology.orchestra.run.smithi118.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:29:09.991 INFO:teuthology.orchestra.run.smithi118.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:29:09.991 INFO:teuthology.orchestra.run.smithi118.stdout: "recovery_started": 0, 2024-09-12T03:29:09.991 INFO:teuthology.orchestra.run.smithi118.stdout: "recovery_completed": 0, 2024-09-12T03:29:09.991 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_quiesce_path": 7, 2024-09-12T03:29:09.991 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_quiesce_inode": 2496, 2024-09-12T03:29:09.992 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_enqueue_scrub": 7, 2024-09-12T03:29:09.992 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_exportdir": 2, 2024-09-12T03:29:09.992 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_flush": 0, 2024-09-12T03:29:09.992 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_fragmentdir": 1, 2024-09-12T03:29:09.992 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_fragstats": 0, 2024-09-12T03:29:09.992 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_inodestats": 0 2024-09-12T03:29:09.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.992 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_log": { 2024-09-12T03:29:09.992 INFO:teuthology.orchestra.run.smithi118.stdout: "evlrg": 0, 2024-09-12T03:29:09.992 INFO:teuthology.orchestra.run.smithi118.stdout: "evadd": 78419, 2024-09-12T03:29:09.992 INFO:teuthology.orchestra.run.smithi118.stdout: "evex": 0, 2024-09-12T03:29:09.992 INFO:teuthology.orchestra.run.smithi118.stdout: "evtrm": 0, 2024-09-12T03:29:09.992 INFO:teuthology.orchestra.run.smithi118.stdout: "ev": 78419, 2024-09-12T03:29:09.993 INFO:teuthology.orchestra.run.smithi118.stdout: "evexg": 0, 2024-09-12T03:29:09.993 INFO:teuthology.orchestra.run.smithi118.stdout: "evexd": 0, 2024-09-12T03:29:09.993 INFO:teuthology.orchestra.run.smithi118.stdout: "segadd": 87, 2024-09-12T03:29:09.993 INFO:teuthology.orchestra.run.smithi118.stdout: "segex": 0, 2024-09-12T03:29:09.993 INFO:teuthology.orchestra.run.smithi118.stdout: "segtrm": 0, 2024-09-12T03:29:09.993 INFO:teuthology.orchestra.run.smithi118.stdout: "seg": 87, 2024-09-12T03:29:09.993 INFO:teuthology.orchestra.run.smithi118.stdout: "segmjr": 8, 2024-09-12T03:29:09.993 INFO:teuthology.orchestra.run.smithi118.stdout: "segexg": 0, 2024-09-12T03:29:09.993 INFO:teuthology.orchestra.run.smithi118.stdout: "segexd": 0, 2024-09-12T03:29:09.993 INFO:teuthology.orchestra.run.smithi118.stdout: "expos": 4194304, 2024-09-12T03:29:09.993 INFO:teuthology.orchestra.run.smithi118.stdout: "wrpos": 170471362, 2024-09-12T03:29:09.993 INFO:teuthology.orchestra.run.smithi118.stdout: "rdpos": 0, 2024-09-12T03:29:09.993 INFO:teuthology.orchestra.run.smithi118.stdout: "jlat": { 2024-09-12T03:29:09.993 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 3334, 2024-09-12T03:29:09.994 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 544.134150823, 2024-09-12T03:29:09.994 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.163207603 2024-09-12T03:29:09.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.994 INFO:teuthology.orchestra.run.smithi118.stdout: "replayed": 0 2024-09-12T03:29:09.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.994 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_mem": { 2024-09-12T03:29:09.994 INFO:teuthology.orchestra.run.smithi118.stdout: "ino": 36, 2024-09-12T03:29:09.994 INFO:teuthology.orchestra.run.smithi118.stdout: "ino+": 905, 2024-09-12T03:29:09.994 INFO:teuthology.orchestra.run.smithi118.stdout: "ino-": 869, 2024-09-12T03:29:09.994 INFO:teuthology.orchestra.run.smithi118.stdout: "dir": 31, 2024-09-12T03:29:09.994 INFO:teuthology.orchestra.run.smithi118.stdout: "dir+": 127, 2024-09-12T03:29:09.994 INFO:teuthology.orchestra.run.smithi118.stdout: "dir-": 96, 2024-09-12T03:29:09.994 INFO:teuthology.orchestra.run.smithi118.stdout: "dn": 166, 2024-09-12T03:29:09.994 INFO:teuthology.orchestra.run.smithi118.stdout: "dn+": 1581, 2024-09-12T03:29:09.995 INFO:teuthology.orchestra.run.smithi118.stdout: "dn-": 1415, 2024-09-12T03:29:09.995 INFO:teuthology.orchestra.run.smithi118.stdout: "cap": 2, 2024-09-12T03:29:09.995 INFO:teuthology.orchestra.run.smithi118.stdout: "cap+": 1333, 2024-09-12T03:29:09.995 INFO:teuthology.orchestra.run.smithi118.stdout: "cap-": 1331, 2024-09-12T03:29:09.995 INFO:teuthology.orchestra.run.smithi118.stdout: "rss": 75784, 2024-09-12T03:29:09.995 INFO:teuthology.orchestra.run.smithi118.stdout: "heap": 231712 2024-09-12T03:29:09.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.995 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_server": { 2024-09-12T03:29:09.995 INFO:teuthology.orchestra.run.smithi118.stdout: "dispatch_client_request": 4560, 2024-09-12T03:29:09.995 INFO:teuthology.orchestra.run.smithi118.stdout: "dispatch_server_request": 5, 2024-09-12T03:29:09.995 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_request": 3470, 2024-09-12T03:29:09.995 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_session": 19, 2024-09-12T03:29:09.995 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_peer_request": 23, 2024-09-12T03:29:09.995 INFO:teuthology.orchestra.run.smithi118.stdout: "req_create_latency": { 2024-09-12T03:29:09.996 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 765, 2024-09-12T03:29:09.996 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.759629866, 2024-09-12T03:29:09.996 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000992980 2024-09-12T03:29:09.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.996 INFO:teuthology.orchestra.run.smithi118.stdout: "req_getattr_latency": { 2024-09-12T03:29:09.996 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 111, 2024-09-12T03:29:09.996 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 10.752242057, 2024-09-12T03:29:09.996 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.096867045 2024-09-12T03:29:09.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.996 INFO:teuthology.orchestra.run.smithi118.stdout: "req_getfilelock_latency": { 2024-09-12T03:29:09.996 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:09.996 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:09.996 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:09.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.997 INFO:teuthology.orchestra.run.smithi118.stdout: "req_link_latency": { 2024-09-12T03:29:09.997 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 61, 2024-09-12T03:29:09.997 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.043835323, 2024-09-12T03:29:09.997 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000718611 2024-09-12T03:29:09.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.997 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookup_latency": { 2024-09-12T03:29:09.997 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 742, 2024-09-12T03:29:09.997 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 31.415300525, 2024-09-12T03:29:09.997 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.042338679 2024-09-12T03:29:09.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.997 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookuphash_latency": { 2024-09-12T03:29:09.998 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:09.998 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:09.998 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:09.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.998 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupino_latency": { 2024-09-12T03:29:09.998 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 1, 2024-09-12T03:29:09.998 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000126768, 2024-09-12T03:29:09.998 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000126768 2024-09-12T03:29:09.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.998 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupname_latency": { 2024-09-12T03:29:09.998 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:09.998 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:09.998 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:09.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.999 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupparent_latency": { 2024-09-12T03:29:09.999 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:09.999 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:09.999 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:09.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.999 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupsnap_latency": { 2024-09-12T03:29:09.999 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:09.999 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:09.999 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:09.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:09.999 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lssnap_latency": { 2024-09-12T03:29:10.000 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:10.000 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:10.000 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:10.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.000 INFO:teuthology.orchestra.run.smithi118.stdout: "req_mkdir_latency": { 2024-09-12T03:29:10.000 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 105, 2024-09-12T03:29:10.001 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.098268112, 2024-09-12T03:29:10.001 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000935886 2024-09-12T03:29:10.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.001 INFO:teuthology.orchestra.run.smithi118.stdout: "req_mknod_latency": { 2024-09-12T03:29:10.001 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:10.001 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:10.001 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:10.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.001 INFO:teuthology.orchestra.run.smithi118.stdout: "req_mksnap_latency": { 2024-09-12T03:29:10.001 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:10.001 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:10.001 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:10.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.002 INFO:teuthology.orchestra.run.smithi118.stdout: "req_open_latency": { 2024-09-12T03:29:10.002 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 397, 2024-09-12T03:29:10.002 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 27.789419660, 2024-09-12T03:29:10.002 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.069998538 2024-09-12T03:29:10.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.002 INFO:teuthology.orchestra.run.smithi118.stdout: "req_readdir_latency": { 2024-09-12T03:29:10.002 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 242, 2024-09-12T03:29:10.002 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.936115700, 2024-09-12T03:29:10.002 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.003868246 2024-09-12T03:29:10.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.002 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rename_latency": { 2024-09-12T03:29:10.002 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 113, 2024-09-12T03:29:10.002 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 9.809220524, 2024-09-12T03:29:10.003 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.086807261 2024-09-12T03:29:10.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.003 INFO:teuthology.orchestra.run.smithi118.stdout: "req_renamesnap_latency": { 2024-09-12T03:29:10.003 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:10.003 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:10.003 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:10.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.003 INFO:teuthology.orchestra.run.smithi118.stdout: "req_snapdiff_latency": { 2024-09-12T03:29:10.003 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:10.003 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:10.003 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:10.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.003 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rmdir_latency": { 2024-09-12T03:29:10.003 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 99, 2024-09-12T03:29:10.004 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 6.194204987, 2024-09-12T03:29:10.004 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.062567727 2024-09-12T03:29:10.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.004 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rmsnap_latency": { 2024-09-12T03:29:10.004 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:10.004 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:10.004 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:10.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.004 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rmxattr_latency": { 2024-09-12T03:29:10.004 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:10.004 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:10.004 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:10.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.005 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setattr_latency": { 2024-09-12T03:29:10.005 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 7, 2024-09-12T03:29:10.005 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.035591465, 2024-09-12T03:29:10.005 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.005084495 2024-09-12T03:29:10.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.005 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setdirlayout_latency": { 2024-09-12T03:29:10.006 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:10.006 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:10.006 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:10.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.006 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setfilelock_latency": { 2024-09-12T03:29:10.006 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 4, 2024-09-12T03:29:10.006 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000860451, 2024-09-12T03:29:10.006 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000215112 2024-09-12T03:29:10.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.006 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setlayout_latency": { 2024-09-12T03:29:10.006 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:10.006 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:10.006 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:10.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.007 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setxattr_latency": { 2024-09-12T03:29:10.007 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 8, 2024-09-12T03:29:10.007 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.020469421, 2024-09-12T03:29:10.007 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.002558677 2024-09-12T03:29:10.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.007 INFO:teuthology.orchestra.run.smithi118.stdout: "req_symlink_latency": { 2024-09-12T03:29:10.007 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 13, 2024-09-12T03:29:10.007 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.007083049, 2024-09-12T03:29:10.007 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000544849 2024-09-12T03:29:10.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.007 INFO:teuthology.orchestra.run.smithi118.stdout: "req_unlink_latency": { 2024-09-12T03:29:10.007 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 800, 2024-09-12T03:29:10.007 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 1.077839417, 2024-09-12T03:29:10.008 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.001347299 2024-09-12T03:29:10.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.008 INFO:teuthology.orchestra.run.smithi118.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:29:10.008 INFO:teuthology.orchestra.run.smithi118.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:29:10.008 INFO:teuthology.orchestra.run.smithi118.stdout: "req_getvxattr_latency": { 2024-09-12T03:29:10.008 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:10.008 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:10.008 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:10.008 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:10.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.008 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_sessions": { 2024-09-12T03:29:10.008 INFO:teuthology.orchestra.run.smithi118.stdout: "session_count": 1, 2024-09-12T03:29:10.008 INFO:teuthology.orchestra.run.smithi118.stdout: "session_add": 3, 2024-09-12T03:29:10.009 INFO:teuthology.orchestra.run.smithi118.stdout: "session_remove": 2, 2024-09-12T03:29:10.009 INFO:teuthology.orchestra.run.smithi118.stdout: "sessions_open": 1, 2024-09-12T03:29:10.009 INFO:teuthology.orchestra.run.smithi118.stdout: "sessions_stale": 0, 2024-09-12T03:29:10.009 INFO:teuthology.orchestra.run.smithi118.stdout: "total_load": 932, 2024-09-12T03:29:10.009 INFO:teuthology.orchestra.run.smithi118.stdout: "average_load": 932, 2024-09-12T03:29:10.009 INFO:teuthology.orchestra.run.smithi118.stdout: "avg_session_uptime": 494, 2024-09-12T03:29:10.009 INFO:teuthology.orchestra.run.smithi118.stdout: "mdthresh_evicted": 0 2024-09-12T03:29:10.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.009 INFO:teuthology.orchestra.run.smithi118.stdout: "mempool": { 2024-09-12T03:29:10.009 INFO:teuthology.orchestra.run.smithi118.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:29:10.009 INFO:teuthology.orchestra.run.smithi118.stdout: "bloom_filter_items": 0, 2024-09-12T03:29:10.009 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:29:10.009 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:29:10.009 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:29:10.010 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:29:10.010 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:29:10.010 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:29:10.010 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:29:10.010 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:29:10.010 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:29:10.010 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:29:10.010 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:29:10.010 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:29:10.010 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:29:10.010 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_extent_items": 0, 2024-09-12T03:29:10.010 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:29:10.010 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_blob_items": 0, 2024-09-12T03:29:10.010 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:29:10.011 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:29:10.011 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:29:10.011 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:29:10.011 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:29:10.011 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:29:10.011 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:29:10.011 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_txc_items": 0, 2024-09-12T03:29:10.011 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:29:10.011 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:29:10.011 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:29:10.011 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_items": 0, 2024-09-12T03:29:10.011 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_bytes": 0, 2024-09-12T03:29:10.011 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_items": 0, 2024-09-12T03:29:10.012 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:29:10.012 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:29:10.012 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:29:10.012 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:29:10.012 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_anon_bytes": 302910, 2024-09-12T03:29:10.012 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_anon_items": 111, 2024-09-12T03:29:10.012 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:29:10.012 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_meta_items": 0, 2024-09-12T03:29:10.012 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_bytes": 0, 2024-09-12T03:29:10.012 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_items": 0, 2024-09-12T03:29:10.012 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:29:10.012 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mapbl_items": 0, 2024-09-12T03:29:10.012 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:29:10.012 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pglog_items": 0, 2024-09-12T03:29:10.013 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_bytes": 7400, 2024-09-12T03:29:10.013 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_items": 152, 2024-09-12T03:29:10.013 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:29:10.013 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:29:10.013 INFO:teuthology.orchestra.run.smithi118.stdout: "pgmap_bytes": 0, 2024-09-12T03:29:10.013 INFO:teuthology.orchestra.run.smithi118.stdout: "pgmap_items": 0, 2024-09-12T03:29:10.013 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_co_bytes": 254196, 2024-09-12T03:29:10.013 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_co_items": 3616, 2024-09-12T03:29:10.013 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_1_bytes": 0, 2024-09-12T03:29:10.013 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_1_items": 0, 2024-09-12T03:29:10.013 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_2_bytes": 0, 2024-09-12T03:29:10.013 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_2_items": 0 2024-09-12T03:29:10.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.013 INFO:teuthology.orchestra.run.smithi118.stdout: "objecter": { 2024-09-12T03:29:10.014 INFO:teuthology.orchestra.run.smithi118.stdout: "op_active": 0, 2024-09-12T03:29:10.014 INFO:teuthology.orchestra.run.smithi118.stdout: "op_laggy": 0, 2024-09-12T03:29:10.014 INFO:teuthology.orchestra.run.smithi118.stdout: "op_send": 10271, 2024-09-12T03:29:10.014 INFO:teuthology.orchestra.run.smithi118.stdout: "op_send_bytes": 166653603, 2024-09-12T03:29:10.014 INFO:teuthology.orchestra.run.smithi118.stdout: "op_resend": 34, 2024-09-12T03:29:10.014 INFO:teuthology.orchestra.run.smithi118.stdout: "op_reply": 10271, 2024-09-12T03:29:10.014 INFO:teuthology.orchestra.run.smithi118.stdout: "op_latency": { 2024-09-12T03:29:10.014 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 10271, 2024-09-12T03:29:10.014 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 1209.391897960, 2024-09-12T03:29:10.014 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.117748213 2024-09-12T03:29:10.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.014 INFO:teuthology.orchestra.run.smithi118.stdout: "op_inflight": 0, 2024-09-12T03:29:10.014 INFO:teuthology.orchestra.run.smithi118.stdout: "oplen_avg": { 2024-09-12T03:29:10.014 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 10271, 2024-09-12T03:29:10.015 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 10523 2024-09-12T03:29:10.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.015 INFO:teuthology.orchestra.run.smithi118.stdout: "op": 10271, 2024-09-12T03:29:10.015 INFO:teuthology.orchestra.run.smithi118.stdout: "op_r": 2090, 2024-09-12T03:29:10.015 INFO:teuthology.orchestra.run.smithi118.stdout: "op_w": 8181, 2024-09-12T03:29:10.015 INFO:teuthology.orchestra.run.smithi118.stdout: "op_rmw": 0, 2024-09-12T03:29:10.015 INFO:teuthology.orchestra.run.smithi118.stdout: "op_pg": 0, 2024-09-12T03:29:10.015 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_stat": 0, 2024-09-12T03:29:10.015 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_create": 11, 2024-09-12T03:29:10.015 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_read": 33, 2024-09-12T03:29:10.015 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_write": 3360, 2024-09-12T03:29:10.015 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_writefull": 49, 2024-09-12T03:29:10.015 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_writesame": 0, 2024-09-12T03:29:10.015 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_append": 0, 2024-09-12T03:29:10.016 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_zero": 0, 2024-09-12T03:29:10.016 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_truncate": 1, 2024-09-12T03:29:10.016 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_delete": 4491, 2024-09-12T03:29:10.016 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_mapext": 0, 2024-09-12T03:29:10.016 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_sparse_read": 0, 2024-09-12T03:29:10.016 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_clonerange": 0, 2024-09-12T03:29:10.016 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_getxattr": 2057, 2024-09-12T03:29:10.016 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_setxattr": 22, 2024-09-12T03:29:10.016 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:29:10.016 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_rmxattr": 0, 2024-09-12T03:29:10.016 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:29:10.016 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_call": 0, 2024-09-12T03:29:10.016 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_watch": 0, 2024-09-12T03:29:10.016 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_notify": 0, 2024-09-12T03:29:10.017 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:29:10.017 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_pgls": 0, 2024-09-12T03:29:10.017 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:29:10.017 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_other": 6, 2024-09-12T03:29:10.017 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_active": 0, 2024-09-12T03:29:10.017 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_send": 0, 2024-09-12T03:29:10.017 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_resend": 0, 2024-09-12T03:29:10.017 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_ping": 0, 2024-09-12T03:29:10.017 INFO:teuthology.orchestra.run.smithi118.stdout: "poolop_active": 0, 2024-09-12T03:29:10.017 INFO:teuthology.orchestra.run.smithi118.stdout: "poolop_send": 0, 2024-09-12T03:29:10.017 INFO:teuthology.orchestra.run.smithi118.stdout: "poolop_resend": 0, 2024-09-12T03:29:10.017 INFO:teuthology.orchestra.run.smithi118.stdout: "poolstat_active": 0, 2024-09-12T03:29:10.017 INFO:teuthology.orchestra.run.smithi118.stdout: "poolstat_send": 0, 2024-09-12T03:29:10.017 INFO:teuthology.orchestra.run.smithi118.stdout: "poolstat_resend": 0, 2024-09-12T03:29:10.018 INFO:teuthology.orchestra.run.smithi118.stdout: "statfs_active": 0, 2024-09-12T03:29:10.018 INFO:teuthology.orchestra.run.smithi118.stdout: "statfs_send": 0, 2024-09-12T03:29:10.018 INFO:teuthology.orchestra.run.smithi118.stdout: "statfs_resend": 0, 2024-09-12T03:29:10.018 INFO:teuthology.orchestra.run.smithi118.stdout: "command_active": 0, 2024-09-12T03:29:10.018 INFO:teuthology.orchestra.run.smithi118.stdout: "command_send": 0, 2024-09-12T03:29:10.018 INFO:teuthology.orchestra.run.smithi118.stdout: "command_resend": 0, 2024-09-12T03:29:10.018 INFO:teuthology.orchestra.run.smithi118.stdout: "map_epoch": 82, 2024-09-12T03:29:10.018 INFO:teuthology.orchestra.run.smithi118.stdout: "map_full": 0, 2024-09-12T03:29:10.018 INFO:teuthology.orchestra.run.smithi118.stdout: "map_inc": 2, 2024-09-12T03:29:10.018 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_sessions": 12, 2024-09-12T03:29:10.018 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_session_open": 12, 2024-09-12T03:29:10.018 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_session_close": 0, 2024-09-12T03:29:10.018 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_laggy": 0, 2024-09-12T03:29:10.018 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_wr": 372, 2024-09-12T03:29:10.019 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_rd": 0, 2024-09-12T03:29:10.019 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_del": 121 2024-09-12T03:29:10.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.019 INFO:teuthology.orchestra.run.smithi118.stdout: "oft": { 2024-09-12T03:29:10.019 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_objs": 1, 2024-09-12T03:29:10.019 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:29:10.019 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_updates": 0, 2024-09-12T03:29:10.019 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_removes": 0 2024-09-12T03:29:10.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.019 INFO:teuthology.orchestra.run.smithi118.stdout: "osd-slow-ops": { 2024-09-12T03:29:10.019 INFO:teuthology.orchestra.run.smithi118.stdout: "slow_ops_count": 2088 2024-09-12T03:29:10.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.019 INFO:teuthology.orchestra.run.smithi118.stdout: "purge_queue": { 2024-09-12T03:29:10.019 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing_ops": 0, 2024-09-12T03:29:10.020 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing_ops_high_water": 26, 2024-09-12T03:29:10.020 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing": 0, 2024-09-12T03:29:10.020 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing_high_water": 10, 2024-09-12T03:29:10.020 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executed_ops": 3940, 2024-09-12T03:29:10.020 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executed": 860, 2024-09-12T03:29:10.020 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_item_in_journal": 0 2024-09-12T03:29:10.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.020 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:29:10.020 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2024-09-12T03:29:10.020 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 104857600, 2024-09-12T03:29:10.020 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2024-09-12T03:29:10.020 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 95519, 2024-09-12T03:29:10.020 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 23925034, 2024-09-12T03:29:10.021 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:10.021 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 95519, 2024-09-12T03:29:10.021 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2024-09-12T03:29:10.021 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2024-09-12T03:29:10.021 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 95519, 2024-09-12T03:29:10.021 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 23925034, 2024-09-12T03:29:10.021 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2024-09-12T03:29:10.021 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:10.021 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:10.021 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:10.021 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:10.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.021 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-objecter_bytes": { 2024-09-12T03:29:10.021 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2024-09-12T03:29:10.022 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 104857600, 2024-09-12T03:29:10.022 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2024-09-12T03:29:10.022 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 0, 2024-09-12T03:29:10.022 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 0, 2024-09-12T03:29:10.022 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:10.022 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 0, 2024-09-12T03:29:10.022 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 10271, 2024-09-12T03:29:10.022 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 166740471, 2024-09-12T03:29:10.022 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 10271, 2024-09-12T03:29:10.022 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 166740471, 2024-09-12T03:29:10.022 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2024-09-12T03:29:10.022 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:10.022 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:10.022 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:10.023 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:10.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.023 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-objecter_ops": { 2024-09-12T03:29:10.023 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2024-09-12T03:29:10.023 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 1024, 2024-09-12T03:29:10.023 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2024-09-12T03:29:10.023 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 0, 2024-09-12T03:29:10.023 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 0, 2024-09-12T03:29:10.023 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:10.023 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 0, 2024-09-12T03:29:10.023 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 10271, 2024-09-12T03:29:10.023 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 10271, 2024-09-12T03:29:10.023 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 10271, 2024-09-12T03:29:10.023 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 10271, 2024-09-12T03:29:10.024 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2024-09-12T03:29:10.024 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:10.024 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:10.024 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:10.024 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:10.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.024 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:29:10.024 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2024-09-12T03:29:10.024 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 3758096384, 2024-09-12T03:29:10.024 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2024-09-12T03:29:10.024 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 860, 2024-09-12T03:29:10.024 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 86868, 2024-09-12T03:29:10.024 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:10.025 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 860, 2024-09-12T03:29:10.025 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2024-09-12T03:29:10.025 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2024-09-12T03:29:10.025 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 26, 2024-09-12T03:29:10.025 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 86868, 2024-09-12T03:29:10.025 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2024-09-12T03:29:10.025 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:10.025 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:10.025 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:10.025 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:10.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:10.025 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-write_buf_throttle-0x55a7a1866960": { 2024-09-12T03:29:10.025 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2024-09-12T03:29:10.025 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 3758096384, 2024-09-12T03:29:10.026 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2024-09-12T03:29:10.026 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 78419, 2024-09-12T03:29:10.026 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 166277058, 2024-09-12T03:29:10.026 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:10.026 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 78419, 2024-09-12T03:29:10.026 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2024-09-12T03:29:10.026 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2024-09-12T03:29:10.026 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 3334, 2024-09-12T03:29:10.026 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 166277058, 2024-09-12T03:29:10.026 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2024-09-12T03:29:10.026 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:10.026 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:10.026 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:10.026 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:10.027 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:10.027 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-12T03:29:10.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:10 smithi118 ceph-mon[35048]: pgmap v577: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:29:11.042 INFO:tasks.check_counter:Daemon mds.b mds.exported=2 2024-09-12T03:29:11.042 INFO:tasks.check_counter:Daemon mds.b mds.imported=2 2024-09-12T03:29:11.043 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=179 2024-09-12T03:29:11.043 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-12T03:29:11.043 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=1 2024-09-12T03:29:11.043 DEBUG:tasks.check_counter:Getting stats from e 2024-09-12T03:29:11.043 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-12T03:29:11.298 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:29:11.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:10 smithi006 ceph-mon[30991]: pgmap v577: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:29:11.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:10 smithi161 ceph-mon[35485]: pgmap v577: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:29:11.823 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-12T03:29:11.823 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:29:11.823 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 1690, 2024-09-12T03:29:11.823 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 1690, 2024-09-12T03:29:11.823 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 504944, 2024-09-12T03:29:11.824 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 536603, 2024-09-12T03:29:11.824 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 6, 2024-09-12T03:29:11.824 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 2, 2024-09-12T03:29:11.824 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 0.181010620, 2024-09-12T03:29:11.824 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 0.038984036, 2024-09-12T03:29:11.824 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 0.036070836, 2024-09-12T03:29:11.824 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.096187056, 2024-09-12T03:29:11.824 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:11.824 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 1689, 2024-09-12T03:29:11.825 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.076506519, 2024-09-12T03:29:11.825 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000045296 2024-09-12T03:29:11.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.825 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:11.825 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.825 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.825 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.825 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-12T03:29:11.826 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:29:11.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.826 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:29:11.826 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 190, 2024-09-12T03:29:11.826 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 179, 2024-09-12T03:29:11.826 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 781660, 2024-09-12T03:29:11.826 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 106234, 2024-09-12T03:29:11.826 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 3, 2024-09-12T03:29:11.826 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 2, 2024-09-12T03:29:11.827 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 0.028124432, 2024-09-12T03:29:11.827 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 0.011436846, 2024-09-12T03:29:11.827 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 0.023775199, 2024-09-12T03:29:11.827 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.000049267, 2024-09-12T03:29:11.827 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:11.827 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 178, 2024-09-12T03:29:11.827 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.010446715, 2024-09-12T03:29:11.827 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000058689 2024-09-12T03:29:11.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.827 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:11.827 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.827 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.827 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.828 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 781104, 2024-09-12T03:29:11.828 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 105536 2024-09-12T03:29:11.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.828 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:29:11.828 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 2, 2024-09-12T03:29:11.828 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 130, 2024-09-12T03:29:11.828 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 428, 2024-09-12T03:29:11.828 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 301582, 2024-09-12T03:29:11.829 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 4, 2024-09-12T03:29:11.829 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 3, 2024-09-12T03:29:11.829 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 0.007615120, 2024-09-12T03:29:11.829 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 0.005378744, 2024-09-12T03:29:11.829 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 0.001034589, 2024-09-12T03:29:11.829 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.000023919, 2024-09-12T03:29:11.829 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:11.829 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 128, 2024-09-12T03:29:11.829 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.016680117, 2024-09-12T03:29:11.829 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000130313 2024-09-12T03:29:11.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.829 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:11.829 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.830 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.830 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.830 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T03:29:11.830 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 301264 2024-09-12T03:29:11.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.830 INFO:teuthology.orchestra.run.smithi118.stdout: "cct": { 2024-09-12T03:29:11.830 INFO:teuthology.orchestra.run.smithi118.stdout: "total_workers": 1, 2024-09-12T03:29:11.830 INFO:teuthology.orchestra.run.smithi118.stdout: "unhealthy_workers": 0 2024-09-12T03:29:11.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.830 INFO:teuthology.orchestra.run.smithi118.stdout: "finisher-MDSRank": { 2024-09-12T03:29:11.830 INFO:teuthology.orchestra.run.smithi118.stdout: "queue_len": 0, 2024-09-12T03:29:11.830 INFO:teuthology.orchestra.run.smithi118.stdout: "complete_latency": { 2024-09-12T03:29:11.831 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 2851, 2024-09-12T03:29:11.831 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.069100990, 2024-09-12T03:29:11.831 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000024237 2024-09-12T03:29:11.831 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:11.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.831 INFO:teuthology.orchestra.run.smithi118.stdout: "finisher-PurgeQueue": { 2024-09-12T03:29:11.831 INFO:teuthology.orchestra.run.smithi118.stdout: "queue_len": 0, 2024-09-12T03:29:11.831 INFO:teuthology.orchestra.run.smithi118.stdout: "complete_latency": { 2024-09-12T03:29:11.831 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.831 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.831 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.831 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:11.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.832 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-09-12T03:29:11.832 INFO:teuthology.orchestra.run.smithi118.stdout: "request": 0, 2024-09-12T03:29:11.832 INFO:teuthology.orchestra.run.smithi118.stdout: "reply": 0, 2024-09-12T03:29:11.832 INFO:teuthology.orchestra.run.smithi118.stdout: "reply_latency": { 2024-09-12T03:29:11.832 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.832 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.832 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.832 INFO:teuthology.orchestra.run.smithi118.stdout: "slow_reply": 0, 2024-09-12T03:29:11.832 INFO:teuthology.orchestra.run.smithi118.stdout: "forward": 0, 2024-09-12T03:29:11.832 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_fetch_complete": 0, 2024-09-12T03:29:11.832 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_fetch_keys": 0, 2024-09-12T03:29:11.832 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_commit": 0, 2024-09-12T03:29:11.833 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_split": 0, 2024-09-12T03:29:11.833 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_merge": 0, 2024-09-12T03:29:11.833 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes": 2, 2024-09-12T03:29:11.833 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_top": 0, 2024-09-12T03:29:11.833 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_bottom": 2, 2024-09-12T03:29:11.833 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_pin_tail": 0, 2024-09-12T03:29:11.833 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_pinned": 2, 2024-09-12T03:29:11.833 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_expired": 0, 2024-09-12T03:29:11.833 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_with_caps": 0, 2024-09-12T03:29:11.833 INFO:teuthology.orchestra.run.smithi118.stdout: "caps": 0, 2024-09-12T03:29:11.833 INFO:teuthology.orchestra.run.smithi118.stdout: "subtrees": 1, 2024-09-12T03:29:11.833 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse": 0, 2024-09-12T03:29:11.834 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_hit": 0, 2024-09-12T03:29:11.834 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_forward": 0, 2024-09-12T03:29:11.834 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_discover": 0, 2024-09-12T03:29:11.834 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:29:11.834 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_remote_ino": 0, 2024-09-12T03:29:11.834 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_lock": 0, 2024-09-12T03:29:11.834 INFO:teuthology.orchestra.run.smithi118.stdout: "load_cent": 0, 2024-09-12T03:29:11.834 INFO:teuthology.orchestra.run.smithi118.stdout: "q": 0, 2024-09-12T03:29:11.834 INFO:teuthology.orchestra.run.smithi118.stdout: "exported": 0, 2024-09-12T03:29:11.834 INFO:teuthology.orchestra.run.smithi118.stdout: "exported_inodes": 0, 2024-09-12T03:29:11.834 INFO:teuthology.orchestra.run.smithi118.stdout: "imported": 0, 2024-09-12T03:29:11.834 INFO:teuthology.orchestra.run.smithi118.stdout: "imported_inodes": 0, 2024-09-12T03:29:11.834 INFO:teuthology.orchestra.run.smithi118.stdout: "openino_dir_fetch": 0, 2024-09-12T03:29:11.834 INFO:teuthology.orchestra.run.smithi118.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:29:11.835 INFO:teuthology.orchestra.run.smithi118.stdout: "openino_peer_discover": 0, 2024-09-12T03:29:11.835 INFO:teuthology.orchestra.run.smithi118.stdout: "root_rfiles": 0, 2024-09-12T03:29:11.835 INFO:teuthology.orchestra.run.smithi118.stdout: "root_rbytes": 0, 2024-09-12T03:29:11.835 INFO:teuthology.orchestra.run.smithi118.stdout: "root_rsnaps": 0, 2024-09-12T03:29:11.835 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:29:11.835 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_set_tag": 0, 2024-09-12T03:29:11.835 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:29:11.835 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:29:11.835 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:29:11.835 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:29:11.835 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:29:11.835 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_file_inodes": 0, 2024-09-12T03:29:11.835 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:29:11.836 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:29:11.836 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:29:11.836 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:29:11.836 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:29:11.836 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:29:11.836 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_caps": 0, 2024-09-12T03:29:11.836 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:29:11.836 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_cap_release": 0, 2024-09-12T03:29:11.836 INFO:teuthology.orchestra.run.smithi118.stdout: "process_request_cap_release": 0 2024-09-12T03:29:11.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.836 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_cache": { 2024-09-12T03:29:11.836 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_update": 0, 2024-09-12T03:29:11.837 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_update_receipt": 0, 2024-09-12T03:29:11.837 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_try_discover": 0, 2024-09-12T03:29:11.837 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_send_discover": 0, 2024-09-12T03:29:11.837 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_handle_discover": 0, 2024-09-12T03:29:11.837 INFO:teuthology.orchestra.run.smithi118.stdout: "num_strays": 0, 2024-09-12T03:29:11.837 INFO:teuthology.orchestra.run.smithi118.stdout: "num_strays_delayed": 0, 2024-09-12T03:29:11.837 INFO:teuthology.orchestra.run.smithi118.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:29:11.837 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_created": 0, 2024-09-12T03:29:11.837 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_enqueued": 0, 2024-09-12T03:29:11.837 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_reintegrated": 0, 2024-09-12T03:29:11.837 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_migrated": 0, 2024-09-12T03:29:11.837 INFO:teuthology.orchestra.run.smithi118.stdout: "num_recovering_processing": 0, 2024-09-12T03:29:11.837 INFO:teuthology.orchestra.run.smithi118.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:29:11.838 INFO:teuthology.orchestra.run.smithi118.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:29:11.838 INFO:teuthology.orchestra.run.smithi118.stdout: "recovery_started": 0, 2024-09-12T03:29:11.838 INFO:teuthology.orchestra.run.smithi118.stdout: "recovery_completed": 0, 2024-09-12T03:29:11.838 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:29:11.838 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:29:11.838 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:29:11.838 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_exportdir": 0, 2024-09-12T03:29:11.838 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_flush": 0, 2024-09-12T03:29:11.838 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:29:11.838 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_fragstats": 0, 2024-09-12T03:29:11.838 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_inodestats": 0 2024-09-12T03:29:11.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.838 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_log": { 2024-09-12T03:29:11.838 INFO:teuthology.orchestra.run.smithi118.stdout: "evlrg": 0, 2024-09-12T03:29:11.839 INFO:teuthology.orchestra.run.smithi118.stdout: "evadd": 0, 2024-09-12T03:29:11.839 INFO:teuthology.orchestra.run.smithi118.stdout: "evex": 0, 2024-09-12T03:29:11.839 INFO:teuthology.orchestra.run.smithi118.stdout: "evtrm": 0, 2024-09-12T03:29:11.839 INFO:teuthology.orchestra.run.smithi118.stdout: "ev": 18, 2024-09-12T03:29:11.839 INFO:teuthology.orchestra.run.smithi118.stdout: "evexg": 0, 2024-09-12T03:29:11.839 INFO:teuthology.orchestra.run.smithi118.stdout: "evexd": 0, 2024-09-12T03:29:11.839 INFO:teuthology.orchestra.run.smithi118.stdout: "segadd": 0, 2024-09-12T03:29:11.839 INFO:teuthology.orchestra.run.smithi118.stdout: "segex": 0, 2024-09-12T03:29:11.839 INFO:teuthology.orchestra.run.smithi118.stdout: "segtrm": 0, 2024-09-12T03:29:11.839 INFO:teuthology.orchestra.run.smithi118.stdout: "seg": 2, 2024-09-12T03:29:11.839 INFO:teuthology.orchestra.run.smithi118.stdout: "segmjr": 2, 2024-09-12T03:29:11.839 INFO:teuthology.orchestra.run.smithi118.stdout: "segexg": 0, 2024-09-12T03:29:11.839 INFO:teuthology.orchestra.run.smithi118.stdout: "segexd": 0, 2024-09-12T03:29:11.839 INFO:teuthology.orchestra.run.smithi118.stdout: "expos": 4194304, 2024-09-12T03:29:11.840 INFO:teuthology.orchestra.run.smithi118.stdout: "wrpos": 4218575, 2024-09-12T03:29:11.840 INFO:teuthology.orchestra.run.smithi118.stdout: "rdpos": 4216924, 2024-09-12T03:29:11.840 INFO:teuthology.orchestra.run.smithi118.stdout: "jlat": { 2024-09-12T03:29:11.840 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.840 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.840 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.840 INFO:teuthology.orchestra.run.smithi118.stdout: "replayed": 18 2024-09-12T03:29:11.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.840 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_mem": { 2024-09-12T03:29:11.840 INFO:teuthology.orchestra.run.smithi118.stdout: "ino": 4, 2024-09-12T03:29:11.840 INFO:teuthology.orchestra.run.smithi118.stdout: "ino+": 17, 2024-09-12T03:29:11.840 INFO:teuthology.orchestra.run.smithi118.stdout: "ino-": 13, 2024-09-12T03:29:11.841 INFO:teuthology.orchestra.run.smithi118.stdout: "dir": 2, 2024-09-12T03:29:11.841 INFO:teuthology.orchestra.run.smithi118.stdout: "dir+": 15, 2024-09-12T03:29:11.841 INFO:teuthology.orchestra.run.smithi118.stdout: "dir-": 13, 2024-09-12T03:29:11.841 INFO:teuthology.orchestra.run.smithi118.stdout: "dn": 2, 2024-09-12T03:29:11.841 INFO:teuthology.orchestra.run.smithi118.stdout: "dn+": 16, 2024-09-12T03:29:11.841 INFO:teuthology.orchestra.run.smithi118.stdout: "dn-": 14, 2024-09-12T03:29:11.841 INFO:teuthology.orchestra.run.smithi118.stdout: "cap": 0, 2024-09-12T03:29:11.841 INFO:teuthology.orchestra.run.smithi118.stdout: "cap+": 0, 2024-09-12T03:29:11.841 INFO:teuthology.orchestra.run.smithi118.stdout: "cap-": 0, 2024-09-12T03:29:11.841 INFO:teuthology.orchestra.run.smithi118.stdout: "rss": 49020, 2024-09-12T03:29:11.841 INFO:teuthology.orchestra.run.smithi118.stdout: "heap": 215328 2024-09-12T03:29:11.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.841 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_server": { 2024-09-12T03:29:11.841 INFO:teuthology.orchestra.run.smithi118.stdout: "dispatch_client_request": 0, 2024-09-12T03:29:11.842 INFO:teuthology.orchestra.run.smithi118.stdout: "dispatch_server_request": 0, 2024-09-12T03:29:11.842 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_request": 0, 2024-09-12T03:29:11.842 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_session": 0, 2024-09-12T03:29:11.842 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_peer_request": 0, 2024-09-12T03:29:11.842 INFO:teuthology.orchestra.run.smithi118.stdout: "req_create_latency": { 2024-09-12T03:29:11.842 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.842 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.842 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.842 INFO:teuthology.orchestra.run.smithi118.stdout: "req_getattr_latency": { 2024-09-12T03:29:11.842 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.842 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.842 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.843 INFO:teuthology.orchestra.run.smithi118.stdout: "req_getfilelock_latency": { 2024-09-12T03:29:11.843 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.843 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.843 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.843 INFO:teuthology.orchestra.run.smithi118.stdout: "req_link_latency": { 2024-09-12T03:29:11.843 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.843 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.843 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.843 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookup_latency": { 2024-09-12T03:29:11.843 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.843 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.843 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.844 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookuphash_latency": { 2024-09-12T03:29:11.844 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.844 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.844 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.844 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupino_latency": { 2024-09-12T03:29:11.844 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.844 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.844 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.844 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupname_latency": { 2024-09-12T03:29:11.844 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.845 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.845 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.845 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupparent_latency": { 2024-09-12T03:29:11.845 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.845 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.845 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.845 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupsnap_latency": { 2024-09-12T03:29:11.845 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.846 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.846 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.846 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lssnap_latency": { 2024-09-12T03:29:11.846 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.846 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.846 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.846 INFO:teuthology.orchestra.run.smithi118.stdout: "req_mkdir_latency": { 2024-09-12T03:29:11.846 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.847 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.847 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.847 INFO:teuthology.orchestra.run.smithi118.stdout: "req_mknod_latency": { 2024-09-12T03:29:11.847 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.847 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.847 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.848 INFO:teuthology.orchestra.run.smithi118.stdout: "req_mksnap_latency": { 2024-09-12T03:29:11.848 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.848 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.848 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.848 INFO:teuthology.orchestra.run.smithi118.stdout: "req_open_latency": { 2024-09-12T03:29:11.848 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.848 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.848 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.848 INFO:teuthology.orchestra.run.smithi118.stdout: "req_readdir_latency": { 2024-09-12T03:29:11.849 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.849 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.849 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.849 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rename_latency": { 2024-09-12T03:29:11.849 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.849 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.849 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.850 INFO:teuthology.orchestra.run.smithi118.stdout: "req_renamesnap_latency": { 2024-09-12T03:29:11.850 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.850 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.850 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.850 INFO:teuthology.orchestra.run.smithi118.stdout: "req_snapdiff_latency": { 2024-09-12T03:29:11.850 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.850 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.851 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.851 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rmdir_latency": { 2024-09-12T03:29:11.851 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.851 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.851 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.851 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rmsnap_latency": { 2024-09-12T03:29:11.851 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.851 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.851 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.851 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rmxattr_latency": { 2024-09-12T03:29:11.852 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.852 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.852 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.852 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setattr_latency": { 2024-09-12T03:29:11.852 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.852 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.852 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.853 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setdirlayout_latency": { 2024-09-12T03:29:11.853 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.853 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.853 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.853 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setfilelock_latency": { 2024-09-12T03:29:11.853 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.853 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.853 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.854 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setlayout_latency": { 2024-09-12T03:29:11.854 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.854 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.854 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.854 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setxattr_latency": { 2024-09-12T03:29:11.854 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.854 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.854 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.854 INFO:teuthology.orchestra.run.smithi118.stdout: "req_symlink_latency": { 2024-09-12T03:29:11.855 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.855 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.855 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.855 INFO:teuthology.orchestra.run.smithi118.stdout: "req_unlink_latency": { 2024-09-12T03:29:11.855 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.855 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.855 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.855 INFO:teuthology.orchestra.run.smithi118.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:29:11.855 INFO:teuthology.orchestra.run.smithi118.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:29:11.855 INFO:teuthology.orchestra.run.smithi118.stdout: "req_getvxattr_latency": { 2024-09-12T03:29:11.855 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.856 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.856 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.856 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:11.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.856 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_sessions": { 2024-09-12T03:29:11.856 INFO:teuthology.orchestra.run.smithi118.stdout: "session_count": 1, 2024-09-12T03:29:11.856 INFO:teuthology.orchestra.run.smithi118.stdout: "session_add": 3, 2024-09-12T03:29:11.856 INFO:teuthology.orchestra.run.smithi118.stdout: "session_remove": 2, 2024-09-12T03:29:11.856 INFO:teuthology.orchestra.run.smithi118.stdout: "sessions_open": 2, 2024-09-12T03:29:11.856 INFO:teuthology.orchestra.run.smithi118.stdout: "sessions_stale": 0, 2024-09-12T03:29:11.856 INFO:teuthology.orchestra.run.smithi118.stdout: "total_load": 0, 2024-09-12T03:29:11.856 INFO:teuthology.orchestra.run.smithi118.stdout: "average_load": 0, 2024-09-12T03:29:11.856 INFO:teuthology.orchestra.run.smithi118.stdout: "avg_session_uptime": 3039, 2024-09-12T03:29:11.857 INFO:teuthology.orchestra.run.smithi118.stdout: "mdthresh_evicted": 0 2024-09-12T03:29:11.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.857 INFO:teuthology.orchestra.run.smithi118.stdout: "mempool": { 2024-09-12T03:29:11.857 INFO:teuthology.orchestra.run.smithi118.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:29:11.857 INFO:teuthology.orchestra.run.smithi118.stdout: "bloom_filter_items": 0, 2024-09-12T03:29:11.857 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:29:11.857 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:29:11.857 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:29:11.857 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:29:11.857 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:29:11.857 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:29:11.857 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:29:11.857 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:29:11.858 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:29:11.858 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:29:11.858 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:29:11.858 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:29:11.858 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:29:11.858 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_extent_items": 0, 2024-09-12T03:29:11.858 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:29:11.858 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_blob_items": 0, 2024-09-12T03:29:11.858 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:29:11.858 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:29:11.858 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:29:11.858 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:29:11.858 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:29:11.859 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:29:11.859 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:29:11.859 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_txc_items": 0, 2024-09-12T03:29:11.859 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:29:11.859 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:29:11.859 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:29:11.859 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_items": 0, 2024-09-12T03:29:11.859 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_bytes": 0, 2024-09-12T03:29:11.859 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_items": 0, 2024-09-12T03:29:11.859 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:29:11.859 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:29:11.859 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:29:11.859 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:29:11.860 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_anon_bytes": 181303, 2024-09-12T03:29:11.860 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_anon_items": 46, 2024-09-12T03:29:11.860 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:29:11.860 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_meta_items": 0, 2024-09-12T03:29:11.860 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_bytes": 0, 2024-09-12T03:29:11.860 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_items": 0, 2024-09-12T03:29:11.860 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:29:11.860 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mapbl_items": 0, 2024-09-12T03:29:11.860 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:29:11.860 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pglog_items": 0, 2024-09-12T03:29:11.860 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_bytes": 7400, 2024-09-12T03:29:11.860 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_items": 152, 2024-09-12T03:29:11.860 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:29:11.861 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:29:11.861 INFO:teuthology.orchestra.run.smithi118.stdout: "pgmap_bytes": 0, 2024-09-12T03:29:11.861 INFO:teuthology.orchestra.run.smithi118.stdout: "pgmap_items": 0, 2024-09-12T03:29:11.861 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_co_bytes": 13456, 2024-09-12T03:29:11.861 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_co_items": 36, 2024-09-12T03:29:11.861 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_1_bytes": 0, 2024-09-12T03:29:11.861 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_1_items": 0, 2024-09-12T03:29:11.861 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_2_bytes": 0, 2024-09-12T03:29:11.861 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_2_items": 0 2024-09-12T03:29:11.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.861 INFO:teuthology.orchestra.run.smithi118.stdout: "objecter": { 2024-09-12T03:29:11.861 INFO:teuthology.orchestra.run.smithi118.stdout: "op_active": 0, 2024-09-12T03:29:11.861 INFO:teuthology.orchestra.run.smithi118.stdout: "op_laggy": 0, 2024-09-12T03:29:11.862 INFO:teuthology.orchestra.run.smithi118.stdout: "op_send": 1689, 2024-09-12T03:29:11.862 INFO:teuthology.orchestra.run.smithi118.stdout: "op_send_bytes": 21, 2024-09-12T03:29:11.862 INFO:teuthology.orchestra.run.smithi118.stdout: "op_resend": 3, 2024-09-12T03:29:11.862 INFO:teuthology.orchestra.run.smithi118.stdout: "op_reply": 1689, 2024-09-12T03:29:11.862 INFO:teuthology.orchestra.run.smithi118.stdout: "op_latency": { 2024-09-12T03:29:11.862 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 1689, 2024-09-12T03:29:11.862 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 8.688805697, 2024-09-12T03:29:11.862 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.005144349 2024-09-12T03:29:11.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.862 INFO:teuthology.orchestra.run.smithi118.stdout: "op_inflight": 0, 2024-09-12T03:29:11.862 INFO:teuthology.orchestra.run.smithi118.stdout: "oplen_avg": { 2024-09-12T03:29:11.862 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 1689, 2024-09-12T03:29:11.862 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 1690 2024-09-12T03:29:11.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.863 INFO:teuthology.orchestra.run.smithi118.stdout: "op": 1689, 2024-09-12T03:29:11.863 INFO:teuthology.orchestra.run.smithi118.stdout: "op_r": 1689, 2024-09-12T03:29:11.863 INFO:teuthology.orchestra.run.smithi118.stdout: "op_w": 0, 2024-09-12T03:29:11.863 INFO:teuthology.orchestra.run.smithi118.stdout: "op_rmw": 0, 2024-09-12T03:29:11.863 INFO:teuthology.orchestra.run.smithi118.stdout: "op_pg": 0, 2024-09-12T03:29:11.863 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_stat": 1084, 2024-09-12T03:29:11.863 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_create": 0, 2024-09-12T03:29:11.863 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_read": 603, 2024-09-12T03:29:11.863 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_write": 0, 2024-09-12T03:29:11.863 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_writefull": 0, 2024-09-12T03:29:11.863 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_writesame": 0, 2024-09-12T03:29:11.863 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_append": 0, 2024-09-12T03:29:11.863 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_zero": 0, 2024-09-12T03:29:11.864 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_truncate": 0, 2024-09-12T03:29:11.864 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_delete": 0, 2024-09-12T03:29:11.864 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_mapext": 0, 2024-09-12T03:29:11.864 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_sparse_read": 0, 2024-09-12T03:29:11.864 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_clonerange": 0, 2024-09-12T03:29:11.864 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_getxattr": 1, 2024-09-12T03:29:11.864 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_setxattr": 0, 2024-09-12T03:29:11.864 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:29:11.864 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_rmxattr": 0, 2024-09-12T03:29:11.864 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:29:11.864 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_call": 0, 2024-09-12T03:29:11.864 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_watch": 0, 2024-09-12T03:29:11.864 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_notify": 0, 2024-09-12T03:29:11.865 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:29:11.865 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_pgls": 0, 2024-09-12T03:29:11.865 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:29:11.865 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_other": 0, 2024-09-12T03:29:11.865 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_active": 0, 2024-09-12T03:29:11.865 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_send": 0, 2024-09-12T03:29:11.865 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_resend": 0, 2024-09-12T03:29:11.865 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_ping": 0, 2024-09-12T03:29:11.865 INFO:teuthology.orchestra.run.smithi118.stdout: "poolop_active": 0, 2024-09-12T03:29:11.865 INFO:teuthology.orchestra.run.smithi118.stdout: "poolop_send": 0, 2024-09-12T03:29:11.865 INFO:teuthology.orchestra.run.smithi118.stdout: "poolop_resend": 0, 2024-09-12T03:29:11.865 INFO:teuthology.orchestra.run.smithi118.stdout: "poolstat_active": 0, 2024-09-12T03:29:11.865 INFO:teuthology.orchestra.run.smithi118.stdout: "poolstat_send": 0, 2024-09-12T03:29:11.865 INFO:teuthology.orchestra.run.smithi118.stdout: "poolstat_resend": 0, 2024-09-12T03:29:11.866 INFO:teuthology.orchestra.run.smithi118.stdout: "statfs_active": 0, 2024-09-12T03:29:11.866 INFO:teuthology.orchestra.run.smithi118.stdout: "statfs_send": 0, 2024-09-12T03:29:11.866 INFO:teuthology.orchestra.run.smithi118.stdout: "statfs_resend": 0, 2024-09-12T03:29:11.866 INFO:teuthology.orchestra.run.smithi118.stdout: "command_active": 0, 2024-09-12T03:29:11.866 INFO:teuthology.orchestra.run.smithi118.stdout: "command_send": 0, 2024-09-12T03:29:11.866 INFO:teuthology.orchestra.run.smithi118.stdout: "command_resend": 0, 2024-09-12T03:29:11.866 INFO:teuthology.orchestra.run.smithi118.stdout: "map_epoch": 82, 2024-09-12T03:29:11.866 INFO:teuthology.orchestra.run.smithi118.stdout: "map_full": 0, 2024-09-12T03:29:11.866 INFO:teuthology.orchestra.run.smithi118.stdout: "map_inc": 1, 2024-09-12T03:29:11.866 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_sessions": 4, 2024-09-12T03:29:11.866 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_session_open": 4, 2024-09-12T03:29:11.866 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_session_close": 0, 2024-09-12T03:29:11.866 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_laggy": 0, 2024-09-12T03:29:11.867 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_wr": 0, 2024-09-12T03:29:11.867 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_rd": 2, 2024-09-12T03:29:11.867 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_del": 0 2024-09-12T03:29:11.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.867 INFO:teuthology.orchestra.run.smithi118.stdout: "oft": { 2024-09-12T03:29:11.867 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_objs": 0, 2024-09-12T03:29:11.867 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:29:11.867 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_updates": 0, 2024-09-12T03:29:11.867 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_removes": 0 2024-09-12T03:29:11.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.867 INFO:teuthology.orchestra.run.smithi118.stdout: "osd-slow-ops": { 2024-09-12T03:29:11.867 INFO:teuthology.orchestra.run.smithi118.stdout: "slow_ops_count": 0 2024-09-12T03:29:11.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.867 INFO:teuthology.orchestra.run.smithi118.stdout: "purge_queue": { 2024-09-12T03:29:11.868 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing_ops": 0, 2024-09-12T03:29:11.868 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:29:11.868 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing": 0, 2024-09-12T03:29:11.868 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing_high_water": 0, 2024-09-12T03:29:11.868 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executed_ops": 0, 2024-09-12T03:29:11.868 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executed": 0, 2024-09-12T03:29:11.868 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_item_in_journal": 0 2024-09-12T03:29:11.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.868 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:29:11.868 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2024-09-12T03:29:11.868 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 104857600, 2024-09-12T03:29:11.868 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2024-09-12T03:29:11.868 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 1878, 2024-09-12T03:29:11.869 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 1105298, 2024-09-12T03:29:11.869 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:11.869 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 1878, 2024-09-12T03:29:11.869 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2024-09-12T03:29:11.869 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2024-09-12T03:29:11.869 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 1878, 2024-09-12T03:29:11.869 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 1105298, 2024-09-12T03:29:11.869 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2024-09-12T03:29:11.869 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.869 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.869 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.869 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:11.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.870 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-objecter_bytes": { 2024-09-12T03:29:11.870 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2024-09-12T03:29:11.870 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 104857600, 2024-09-12T03:29:11.870 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2024-09-12T03:29:11.870 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 0, 2024-09-12T03:29:11.870 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 0, 2024-09-12T03:29:11.870 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:11.870 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 0, 2024-09-12T03:29:11.870 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 1689, 2024-09-12T03:29:11.870 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 24276, 2024-09-12T03:29:11.870 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 1689, 2024-09-12T03:29:11.870 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 24276, 2024-09-12T03:29:11.870 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2024-09-12T03:29:11.871 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.871 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.871 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.871 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:11.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.871 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-objecter_ops": { 2024-09-12T03:29:11.871 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2024-09-12T03:29:11.871 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 1024, 2024-09-12T03:29:11.871 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2024-09-12T03:29:11.871 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 0, 2024-09-12T03:29:11.871 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 0, 2024-09-12T03:29:11.871 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:11.872 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 0, 2024-09-12T03:29:11.872 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 1689, 2024-09-12T03:29:11.872 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 1689, 2024-09-12T03:29:11.872 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 1689, 2024-09-12T03:29:11.872 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 1689, 2024-09-12T03:29:11.872 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2024-09-12T03:29:11.872 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.872 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.872 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.872 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:11.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.872 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:29:11.872 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2024-09-12T03:29:11.873 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 3758096384, 2024-09-12T03:29:11.873 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2024-09-12T03:29:11.873 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 0, 2024-09-12T03:29:11.873 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 0, 2024-09-12T03:29:11.873 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:11.873 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 0, 2024-09-12T03:29:11.873 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2024-09-12T03:29:11.873 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2024-09-12T03:29:11.873 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 0, 2024-09-12T03:29:11.873 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 0, 2024-09-12T03:29:11.873 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2024-09-12T03:29:11.873 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.873 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.873 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.874 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:11.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:11.874 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-write_buf_throttle-0x55cb6a8dab40": { 2024-09-12T03:29:11.874 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2024-09-12T03:29:11.874 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 3758096384, 2024-09-12T03:29:11.874 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2024-09-12T03:29:11.874 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 0, 2024-09-12T03:29:11.874 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 0, 2024-09-12T03:29:11.874 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:11.874 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 0, 2024-09-12T03:29:11.874 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2024-09-12T03:29:11.874 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2024-09-12T03:29:11.874 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 0, 2024-09-12T03:29:11.874 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 0, 2024-09-12T03:29:11.875 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2024-09-12T03:29:11.875 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:11.875 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:11.875 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:11.875 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:11.875 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:11.875 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-12T03:29:12.682 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-12T03:29:12.682 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-12T03:29:12.682 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-12T03:29:12.682 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-12T03:29:12.682 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-09-12T03:29:12.683 DEBUG:tasks.check_counter:Getting stats from h 2024-09-12T03:29:12.683 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-12T03:29:12.930 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:29:12.962 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:12 smithi118 ceph-mon[35048]: pgmap v578: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-12T03:29:13.165 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:12 smithi161 ceph-mon[35485]: pgmap v578: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-12T03:29:13.365 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:12 smithi006 ceph-mon[30991]: pgmap v578: 97 pgs: 97 active+clean; 159 MiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-12T03:29:13.422 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-12T03:29:13.422 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:29:13.422 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 311, 2024-09-12T03:29:13.422 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 301, 2024-09-12T03:29:13.422 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 827945, 2024-09-12T03:29:13.422 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 221203, 2024-09-12T03:29:13.422 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 14, 2024-09-12T03:29:13.422 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 5, 2024-09-12T03:29:13.422 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 0.046908347, 2024-09-12T03:29:13.423 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 0.015251707, 2024-09-12T03:29:13.423 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 0.036726629, 2024-09-12T03:29:13.423 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.000804604, 2024-09-12T03:29:13.423 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:13.423 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 299, 2024-09-12T03:29:13.423 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.031810033, 2024-09-12T03:29:13.423 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000106388 2024-09-12T03:29:13.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.423 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:13.423 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 84, 2024-09-12T03:29:13.423 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000014422, 2024-09-12T03:29:13.423 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000171 2024-09-12T03:29:13.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.424 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 809728, 2024-09-12T03:29:13.424 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 126912 2024-09-12T03:29:13.424 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.424 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:29:13.424 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 4514, 2024-09-12T03:29:13.424 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 2510, 2024-09-12T03:29:13.424 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 1449662, 2024-09-12T03:29:13.424 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 835658, 2024-09-12T03:29:13.424 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 7, 2024-09-12T03:29:13.424 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 4, 2024-09-12T03:29:13.424 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 0.223763922, 2024-09-12T03:29:13.424 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 0.100369825, 2024-09-12T03:29:13.424 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 0.173653212, 2024-09-12T03:29:13.425 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.003280828, 2024-09-12T03:29:13.425 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:13.425 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 2510, 2024-09-12T03:29:13.425 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.142475913, 2024-09-12T03:29:13.425 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000056763 2024-09-12T03:29:13.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.425 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:13.425 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 6358, 2024-09-12T03:29:13.425 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000889597, 2024-09-12T03:29:13.425 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000139 2024-09-12T03:29:13.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.425 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T03:29:13.425 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 302960 2024-09-12T03:29:13.426 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.426 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:29:13.426 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 47, 2024-09-12T03:29:13.426 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 49, 2024-09-12T03:29:13.426 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 10634, 2024-09-12T03:29:13.426 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 214591, 2024-09-12T03:29:13.426 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 23, 2024-09-12T03:29:13.426 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 4, 2024-09-12T03:29:13.426 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 0.011626107, 2024-09-12T03:29:13.426 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 0.001179136, 2024-09-12T03:29:13.426 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 0.009025496, 2024-09-12T03:29:13.426 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.000792302, 2024-09-12T03:29:13.427 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:13.427 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 47, 2024-09-12T03:29:13.427 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.019066630, 2024-09-12T03:29:13.427 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000405672 2024-09-12T03:29:13.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.427 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:13.427 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 23, 2024-09-12T03:29:13.427 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000003281, 2024-09-12T03:29:13.427 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000142 2024-09-12T03:29:13.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.427 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 928, 2024-09-12T03:29:13.427 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 736 2024-09-12T03:29:13.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.427 INFO:teuthology.orchestra.run.smithi118.stdout: "cct": { 2024-09-12T03:29:13.428 INFO:teuthology.orchestra.run.smithi118.stdout: "total_workers": 1, 2024-09-12T03:29:13.428 INFO:teuthology.orchestra.run.smithi118.stdout: "unhealthy_workers": 0 2024-09-12T03:29:13.428 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.428 INFO:teuthology.orchestra.run.smithi118.stdout: "finisher-MDSRank": { 2024-09-12T03:29:13.428 INFO:teuthology.orchestra.run.smithi118.stdout: "queue_len": 0, 2024-09-12T03:29:13.428 INFO:teuthology.orchestra.run.smithi118.stdout: "complete_latency": { 2024-09-12T03:29:13.428 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 69, 2024-09-12T03:29:13.428 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.005568045, 2024-09-12T03:29:13.428 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000080696 2024-09-12T03:29:13.429 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:13.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.429 INFO:teuthology.orchestra.run.smithi118.stdout: "finisher-PurgeQueue": { 2024-09-12T03:29:13.429 INFO:teuthology.orchestra.run.smithi118.stdout: "queue_len": 0, 2024-09-12T03:29:13.429 INFO:teuthology.orchestra.run.smithi118.stdout: "complete_latency": { 2024-09-12T03:29:13.429 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 16, 2024-09-12T03:29:13.429 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000350321, 2024-09-12T03:29:13.429 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000021895 2024-09-12T03:29:13.429 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:13.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.429 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-09-12T03:29:13.429 INFO:teuthology.orchestra.run.smithi118.stdout: "request": 16, 2024-09-12T03:29:13.430 INFO:teuthology.orchestra.run.smithi118.stdout: "reply": 5, 2024-09-12T03:29:13.430 INFO:teuthology.orchestra.run.smithi118.stdout: "reply_latency": { 2024-09-12T03:29:13.430 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 5, 2024-09-12T03:29:13.430 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.010217642, 2024-09-12T03:29:13.430 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.002043528 2024-09-12T03:29:13.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.430 INFO:teuthology.orchestra.run.smithi118.stdout: "slow_reply": 0, 2024-09-12T03:29:13.430 INFO:teuthology.orchestra.run.smithi118.stdout: "forward": 11, 2024-09-12T03:29:13.430 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_fetch_complete": 0, 2024-09-12T03:29:13.430 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_fetch_keys": 0, 2024-09-12T03:29:13.430 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_commit": 11, 2024-09-12T03:29:13.430 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_split": 0, 2024-09-12T03:29:13.430 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_merge": 0, 2024-09-12T03:29:13.430 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes": 20, 2024-09-12T03:29:13.431 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_top": 4, 2024-09-12T03:29:13.431 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_bottom": 16, 2024-09-12T03:29:13.431 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_pin_tail": 0, 2024-09-12T03:29:13.431 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_pinned": 13, 2024-09-12T03:29:13.431 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_expired": 172, 2024-09-12T03:29:13.431 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_with_caps": 0, 2024-09-12T03:29:13.431 INFO:teuthology.orchestra.run.smithi118.stdout: "caps": 0, 2024-09-12T03:29:13.431 INFO:teuthology.orchestra.run.smithi118.stdout: "subtrees": 3, 2024-09-12T03:29:13.431 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse": 224, 2024-09-12T03:29:13.432 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_hit": 114, 2024-09-12T03:29:13.432 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_forward": 10, 2024-09-12T03:29:13.432 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_discover": 96, 2024-09-12T03:29:13.432 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:29:13.432 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_remote_ino": 0, 2024-09-12T03:29:13.432 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_lock": 0, 2024-09-12T03:29:13.432 INFO:teuthology.orchestra.run.smithi118.stdout: "load_cent": 0, 2024-09-12T03:29:13.432 INFO:teuthology.orchestra.run.smithi118.stdout: "q": 0, 2024-09-12T03:29:13.432 INFO:teuthology.orchestra.run.smithi118.stdout: "exported": 1, 2024-09-12T03:29:13.432 INFO:teuthology.orchestra.run.smithi118.stdout: "exported_inodes": 0, 2024-09-12T03:29:13.432 INFO:teuthology.orchestra.run.smithi118.stdout: "imported": 1, 2024-09-12T03:29:13.432 INFO:teuthology.orchestra.run.smithi118.stdout: "imported_inodes": 2, 2024-09-12T03:29:13.432 INFO:teuthology.orchestra.run.smithi118.stdout: "openino_dir_fetch": 0, 2024-09-12T03:29:13.433 INFO:teuthology.orchestra.run.smithi118.stdout: "openino_backtrace_fetch": 5, 2024-09-12T03:29:13.433 INFO:teuthology.orchestra.run.smithi118.stdout: "openino_peer_discover": 0, 2024-09-12T03:29:13.433 INFO:teuthology.orchestra.run.smithi118.stdout: "root_rfiles": 325, 2024-09-12T03:29:13.433 INFO:teuthology.orchestra.run.smithi118.stdout: "root_rbytes": 13424230450, 2024-09-12T03:29:13.433 INFO:teuthology.orchestra.run.smithi118.stdout: "root_rsnaps": 0, 2024-09-12T03:29:13.433 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_backtrace_fetch": 5, 2024-09-12T03:29:13.433 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_set_tag": 0, 2024-09-12T03:29:13.433 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:29:13.433 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:29:13.433 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:29:13.433 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:29:13.433 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:29:13.433 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_file_inodes": 5, 2024-09-12T03:29:13.434 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:29:13.434 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_revoke": 5, 2024-09-12T03:29:13.434 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_grant": 4, 2024-09-12T03:29:13.434 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:29:13.434 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:29:13.434 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:29:13.434 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_caps": 5, 2024-09-12T03:29:13.434 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:29:13.434 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_cap_release": 2, 2024-09-12T03:29:13.434 INFO:teuthology.orchestra.run.smithi118.stdout: "process_request_cap_release": 4 2024-09-12T03:29:13.434 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.434 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_cache": { 2024-09-12T03:29:13.434 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_update": 1, 2024-09-12T03:29:13.435 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_update_receipt": 276, 2024-09-12T03:29:13.435 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_try_discover": 185, 2024-09-12T03:29:13.435 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_send_discover": 175, 2024-09-12T03:29:13.435 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_handle_discover": 1, 2024-09-12T03:29:13.435 INFO:teuthology.orchestra.run.smithi118.stdout: "num_strays": 0, 2024-09-12T03:29:13.435 INFO:teuthology.orchestra.run.smithi118.stdout: "num_strays_delayed": 0, 2024-09-12T03:29:13.435 INFO:teuthology.orchestra.run.smithi118.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:29:13.435 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_created": 1, 2024-09-12T03:29:13.435 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_enqueued": 1, 2024-09-12T03:29:13.435 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_reintegrated": 0, 2024-09-12T03:29:13.435 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_migrated": 0, 2024-09-12T03:29:13.435 INFO:teuthology.orchestra.run.smithi118.stdout: "num_recovering_processing": 0, 2024-09-12T03:29:13.435 INFO:teuthology.orchestra.run.smithi118.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:29:13.436 INFO:teuthology.orchestra.run.smithi118.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:29:13.436 INFO:teuthology.orchestra.run.smithi118.stdout: "recovery_started": 0, 2024-09-12T03:29:13.436 INFO:teuthology.orchestra.run.smithi118.stdout: "recovery_completed": 0, 2024-09-12T03:29:13.436 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_quiesce_path": 7, 2024-09-12T03:29:13.436 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_quiesce_inode": 509, 2024-09-12T03:29:13.436 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:29:13.436 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_exportdir": 1, 2024-09-12T03:29:13.436 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_flush": 0, 2024-09-12T03:29:13.436 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:29:13.436 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_fragstats": 0, 2024-09-12T03:29:13.436 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_inodestats": 0 2024-09-12T03:29:13.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.436 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_log": { 2024-09-12T03:29:13.437 INFO:teuthology.orchestra.run.smithi118.stdout: "evlrg": 0, 2024-09-12T03:29:13.437 INFO:teuthology.orchestra.run.smithi118.stdout: "evadd": 18, 2024-09-12T03:29:13.437 INFO:teuthology.orchestra.run.smithi118.stdout: "evex": 0, 2024-09-12T03:29:13.437 INFO:teuthology.orchestra.run.smithi118.stdout: "evtrm": 0, 2024-09-12T03:29:13.437 INFO:teuthology.orchestra.run.smithi118.stdout: "ev": 18, 2024-09-12T03:29:13.437 INFO:teuthology.orchestra.run.smithi118.stdout: "evexg": 0, 2024-09-12T03:29:13.437 INFO:teuthology.orchestra.run.smithi118.stdout: "evexd": 0, 2024-09-12T03:29:13.437 INFO:teuthology.orchestra.run.smithi118.stdout: "segadd": 2, 2024-09-12T03:29:13.437 INFO:teuthology.orchestra.run.smithi118.stdout: "segex": 0, 2024-09-12T03:29:13.437 INFO:teuthology.orchestra.run.smithi118.stdout: "segtrm": 0, 2024-09-12T03:29:13.437 INFO:teuthology.orchestra.run.smithi118.stdout: "seg": 2, 2024-09-12T03:29:13.437 INFO:teuthology.orchestra.run.smithi118.stdout: "segmjr": 2, 2024-09-12T03:29:13.437 INFO:teuthology.orchestra.run.smithi118.stdout: "segexg": 0, 2024-09-12T03:29:13.437 INFO:teuthology.orchestra.run.smithi118.stdout: "segexd": 0, 2024-09-12T03:29:13.438 INFO:teuthology.orchestra.run.smithi118.stdout: "expos": 4194304, 2024-09-12T03:29:13.438 INFO:teuthology.orchestra.run.smithi118.stdout: "wrpos": 4218575, 2024-09-12T03:29:13.438 INFO:teuthology.orchestra.run.smithi118.stdout: "rdpos": 0, 2024-09-12T03:29:13.438 INFO:teuthology.orchestra.run.smithi118.stdout: "jlat": { 2024-09-12T03:29:13.438 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 15, 2024-09-12T03:29:13.438 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.127615888, 2024-09-12T03:29:13.438 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.008507725 2024-09-12T03:29:13.438 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.438 INFO:teuthology.orchestra.run.smithi118.stdout: "replayed": 0 2024-09-12T03:29:13.438 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.438 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_mem": { 2024-09-12T03:29:13.438 INFO:teuthology.orchestra.run.smithi118.stdout: "ino": 24, 2024-09-12T03:29:13.438 INFO:teuthology.orchestra.run.smithi118.stdout: "ino+": 111, 2024-09-12T03:29:13.439 INFO:teuthology.orchestra.run.smithi118.stdout: "ino-": 87, 2024-09-12T03:29:13.439 INFO:teuthology.orchestra.run.smithi118.stdout: "dir": 23, 2024-09-12T03:29:13.439 INFO:teuthology.orchestra.run.smithi118.stdout: "dir+": 119, 2024-09-12T03:29:13.439 INFO:teuthology.orchestra.run.smithi118.stdout: "dir-": 96, 2024-09-12T03:29:13.439 INFO:teuthology.orchestra.run.smithi118.stdout: "dn": 20, 2024-09-12T03:29:13.439 INFO:teuthology.orchestra.run.smithi118.stdout: "dn+": 195, 2024-09-12T03:29:13.439 INFO:teuthology.orchestra.run.smithi118.stdout: "dn-": 175, 2024-09-12T03:29:13.439 INFO:teuthology.orchestra.run.smithi118.stdout: "cap": 0, 2024-09-12T03:29:13.439 INFO:teuthology.orchestra.run.smithi118.stdout: "cap+": 5, 2024-09-12T03:29:13.439 INFO:teuthology.orchestra.run.smithi118.stdout: "cap-": 5, 2024-09-12T03:29:13.439 INFO:teuthology.orchestra.run.smithi118.stdout: "rss": 54532, 2024-09-12T03:29:13.439 INFO:teuthology.orchestra.run.smithi118.stdout: "heap": 223520 2024-09-12T03:29:13.439 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.439 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_server": { 2024-09-12T03:29:13.440 INFO:teuthology.orchestra.run.smithi118.stdout: "dispatch_client_request": 23, 2024-09-12T03:29:13.440 INFO:teuthology.orchestra.run.smithi118.stdout: "dispatch_server_request": 3, 2024-09-12T03:29:13.440 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_request": 16, 2024-09-12T03:29:13.440 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_session": 15, 2024-09-12T03:29:13.440 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_peer_request": 9, 2024-09-12T03:29:13.440 INFO:teuthology.orchestra.run.smithi118.stdout: "req_create_latency": { 2024-09-12T03:29:13.440 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.440 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.440 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.440 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.440 INFO:teuthology.orchestra.run.smithi118.stdout: "req_getattr_latency": { 2024-09-12T03:29:13.440 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 1, 2024-09-12T03:29:13.441 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000192833, 2024-09-12T03:29:13.441 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000192833 2024-09-12T03:29:13.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.441 INFO:teuthology.orchestra.run.smithi118.stdout: "req_getfilelock_latency": { 2024-09-12T03:29:13.441 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.441 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.441 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.441 INFO:teuthology.orchestra.run.smithi118.stdout: "req_link_latency": { 2024-09-12T03:29:13.441 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.441 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.441 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.442 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookup_latency": { 2024-09-12T03:29:13.442 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 1, 2024-09-12T03:29:13.442 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000386080, 2024-09-12T03:29:13.442 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000386080 2024-09-12T03:29:13.442 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.442 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookuphash_latency": { 2024-09-12T03:29:13.442 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.442 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.442 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.442 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.442 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupino_latency": { 2024-09-12T03:29:13.442 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.442 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.442 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.443 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.443 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupname_latency": { 2024-09-12T03:29:13.443 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.443 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.443 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.443 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.443 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupparent_latency": { 2024-09-12T03:29:13.443 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.443 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.443 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.443 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.443 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupsnap_latency": { 2024-09-12T03:29:13.443 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.444 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.444 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.444 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.444 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lssnap_latency": { 2024-09-12T03:29:13.444 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.444 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.444 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.444 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.445 INFO:teuthology.orchestra.run.smithi118.stdout: "req_mkdir_latency": { 2024-09-12T03:29:13.445 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.445 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.445 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.445 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.445 INFO:teuthology.orchestra.run.smithi118.stdout: "req_mknod_latency": { 2024-09-12T03:29:13.445 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.445 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.445 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.445 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.445 INFO:teuthology.orchestra.run.smithi118.stdout: "req_mksnap_latency": { 2024-09-12T03:29:13.446 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.446 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.446 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.446 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.446 INFO:teuthology.orchestra.run.smithi118.stdout: "req_open_latency": { 2024-09-12T03:29:13.446 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.446 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.446 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.446 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.446 INFO:teuthology.orchestra.run.smithi118.stdout: "req_readdir_latency": { 2024-09-12T03:29:13.446 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 2, 2024-09-12T03:29:13.446 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.001403258, 2024-09-12T03:29:13.446 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000701629 2024-09-12T03:29:13.447 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.447 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rename_latency": { 2024-09-12T03:29:13.447 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.447 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.447 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.447 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.447 INFO:teuthology.orchestra.run.smithi118.stdout: "req_renamesnap_latency": { 2024-09-12T03:29:13.447 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.447 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.447 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.447 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.447 INFO:teuthology.orchestra.run.smithi118.stdout: "req_snapdiff_latency": { 2024-09-12T03:29:13.448 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.448 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.448 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.448 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.448 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rmdir_latency": { 2024-09-12T03:29:13.448 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.448 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.448 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.448 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.448 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rmsnap_latency": { 2024-09-12T03:29:13.448 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.448 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.448 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.449 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.449 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rmxattr_latency": { 2024-09-12T03:29:13.449 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.449 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.449 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.449 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.449 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setattr_latency": { 2024-09-12T03:29:13.449 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.449 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.449 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.449 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.449 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setdirlayout_latency": { 2024-09-12T03:29:13.449 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.450 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.450 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.450 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setfilelock_latency": { 2024-09-12T03:29:13.450 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.450 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.450 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.450 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setlayout_latency": { 2024-09-12T03:29:13.450 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.450 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.450 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.451 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setxattr_latency": { 2024-09-12T03:29:13.451 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.451 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.451 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.451 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.451 INFO:teuthology.orchestra.run.smithi118.stdout: "req_symlink_latency": { 2024-09-12T03:29:13.451 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.451 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.451 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.451 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.451 INFO:teuthology.orchestra.run.smithi118.stdout: "req_unlink_latency": { 2024-09-12T03:29:13.451 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 1, 2024-09-12T03:29:13.451 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.008235471, 2024-09-12T03:29:13.452 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.008235471 2024-09-12T03:29:13.452 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.452 INFO:teuthology.orchestra.run.smithi118.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:29:13.452 INFO:teuthology.orchestra.run.smithi118.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:29:13.452 INFO:teuthology.orchestra.run.smithi118.stdout: "req_getvxattr_latency": { 2024-09-12T03:29:13.452 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.453 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.453 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.453 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:13.453 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.453 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_sessions": { 2024-09-12T03:29:13.453 INFO:teuthology.orchestra.run.smithi118.stdout: "session_count": 1, 2024-09-12T03:29:13.453 INFO:teuthology.orchestra.run.smithi118.stdout: "session_add": 3, 2024-09-12T03:29:13.453 INFO:teuthology.orchestra.run.smithi118.stdout: "session_remove": 2, 2024-09-12T03:29:13.453 INFO:teuthology.orchestra.run.smithi118.stdout: "sessions_open": 1, 2024-09-12T03:29:13.453 INFO:teuthology.orchestra.run.smithi118.stdout: "sessions_stale": 0, 2024-09-12T03:29:13.453 INFO:teuthology.orchestra.run.smithi118.stdout: "total_load": 2, 2024-09-12T03:29:13.454 INFO:teuthology.orchestra.run.smithi118.stdout: "average_load": 2, 2024-09-12T03:29:13.454 INFO:teuthology.orchestra.run.smithi118.stdout: "avg_session_uptime": 498, 2024-09-12T03:29:13.454 INFO:teuthology.orchestra.run.smithi118.stdout: "mdthresh_evicted": 0 2024-09-12T03:29:13.454 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.454 INFO:teuthology.orchestra.run.smithi118.stdout: "mempool": { 2024-09-12T03:29:13.454 INFO:teuthology.orchestra.run.smithi118.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:29:13.454 INFO:teuthology.orchestra.run.smithi118.stdout: "bloom_filter_items": 0, 2024-09-12T03:29:13.454 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:29:13.454 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:29:13.454 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:29:13.454 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:29:13.454 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:29:13.455 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:29:13.455 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:29:13.455 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:29:13.455 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:29:13.455 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:29:13.455 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:29:13.455 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:29:13.455 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:29:13.455 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_extent_items": 0, 2024-09-12T03:29:13.455 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:29:13.455 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_blob_items": 0, 2024-09-12T03:29:13.455 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:29:13.456 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:29:13.456 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:29:13.456 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:29:13.456 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:29:13.456 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:29:13.456 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:29:13.456 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_txc_items": 0, 2024-09-12T03:29:13.456 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:29:13.456 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:29:13.456 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:29:13.456 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_items": 0, 2024-09-12T03:29:13.456 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_bytes": 0, 2024-09-12T03:29:13.456 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_items": 0, 2024-09-12T03:29:13.457 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:29:13.457 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:29:13.457 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:29:13.457 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:29:13.457 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_anon_bytes": 262823, 2024-09-12T03:29:13.457 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_anon_items": 82, 2024-09-12T03:29:13.457 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:29:13.457 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_meta_items": 0, 2024-09-12T03:29:13.457 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_bytes": 0, 2024-09-12T03:29:13.457 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_items": 0, 2024-09-12T03:29:13.457 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:29:13.457 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mapbl_items": 0, 2024-09-12T03:29:13.458 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:29:13.458 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pglog_items": 0, 2024-09-12T03:29:13.458 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_bytes": 7400, 2024-09-12T03:29:13.458 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_items": 152, 2024-09-12T03:29:13.458 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:29:13.458 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:29:13.458 INFO:teuthology.orchestra.run.smithi118.stdout: "pgmap_bytes": 0, 2024-09-12T03:29:13.458 INFO:teuthology.orchestra.run.smithi118.stdout: "pgmap_items": 0, 2024-09-12T03:29:13.458 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_co_bytes": 97781, 2024-09-12T03:29:13.458 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_co_items": 385, 2024-09-12T03:29:13.458 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_1_bytes": 0, 2024-09-12T03:29:13.458 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_1_items": 0, 2024-09-12T03:29:13.458 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_2_bytes": 0, 2024-09-12T03:29:13.458 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_2_items": 0 2024-09-12T03:29:13.459 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.459 INFO:teuthology.orchestra.run.smithi118.stdout: "objecter": { 2024-09-12T03:29:13.459 INFO:teuthology.orchestra.run.smithi118.stdout: "op_active": 0, 2024-09-12T03:29:13.459 INFO:teuthology.orchestra.run.smithi118.stdout: "op_laggy": 0, 2024-09-12T03:29:13.459 INFO:teuthology.orchestra.run.smithi118.stdout: "op_send": 74, 2024-09-12T03:29:13.459 INFO:teuthology.orchestra.run.smithi118.stdout: "op_send_bytes": 36537, 2024-09-12T03:29:13.459 INFO:teuthology.orchestra.run.smithi118.stdout: "op_resend": 32, 2024-09-12T03:29:13.459 INFO:teuthology.orchestra.run.smithi118.stdout: "op_reply": 74, 2024-09-12T03:29:13.459 INFO:teuthology.orchestra.run.smithi118.stdout: "op_latency": { 2024-09-12T03:29:13.459 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 74, 2024-09-12T03:29:13.459 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.442990080, 2024-09-12T03:29:13.459 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.005986352 2024-09-12T03:29:13.459 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.460 INFO:teuthology.orchestra.run.smithi118.stdout: "op_inflight": 0, 2024-09-12T03:29:13.460 INFO:teuthology.orchestra.run.smithi118.stdout: "oplen_avg": { 2024-09-12T03:29:13.460 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 74, 2024-09-12T03:29:13.460 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 101 2024-09-12T03:29:13.460 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.460 INFO:teuthology.orchestra.run.smithi118.stdout: "op": 74, 2024-09-12T03:29:13.460 INFO:teuthology.orchestra.run.smithi118.stdout: "op_r": 6, 2024-09-12T03:29:13.460 INFO:teuthology.orchestra.run.smithi118.stdout: "op_w": 68, 2024-09-12T03:29:13.460 INFO:teuthology.orchestra.run.smithi118.stdout: "op_rmw": 0, 2024-09-12T03:29:13.460 INFO:teuthology.orchestra.run.smithi118.stdout: "op_pg": 0, 2024-09-12T03:29:13.460 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_stat": 0, 2024-09-12T03:29:13.460 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_create": 10, 2024-09-12T03:29:13.460 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_read": 1, 2024-09-12T03:29:13.461 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_write": 16, 2024-09-12T03:29:13.461 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_writefull": 11, 2024-09-12T03:29:13.461 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_writesame": 0, 2024-09-12T03:29:13.461 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_append": 0, 2024-09-12T03:29:13.461 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_zero": 0, 2024-09-12T03:29:13.461 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_truncate": 0, 2024-09-12T03:29:13.461 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_delete": 13, 2024-09-12T03:29:13.461 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_mapext": 0, 2024-09-12T03:29:13.461 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_sparse_read": 0, 2024-09-12T03:29:13.461 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_clonerange": 0, 2024-09-12T03:29:13.461 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_getxattr": 5, 2024-09-12T03:29:13.461 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_setxattr": 20, 2024-09-12T03:29:13.461 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:29:13.461 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_rmxattr": 0, 2024-09-12T03:29:13.462 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:29:13.462 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_call": 0, 2024-09-12T03:29:13.462 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_watch": 0, 2024-09-12T03:29:13.462 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_notify": 0, 2024-09-12T03:29:13.462 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:29:13.462 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_pgls": 0, 2024-09-12T03:29:13.462 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:29:13.462 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_other": 0, 2024-09-12T03:29:13.462 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_active": 0, 2024-09-12T03:29:13.462 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_send": 0, 2024-09-12T03:29:13.462 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_resend": 0, 2024-09-12T03:29:13.462 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_ping": 0, 2024-09-12T03:29:13.462 INFO:teuthology.orchestra.run.smithi118.stdout: "poolop_active": 0, 2024-09-12T03:29:13.463 INFO:teuthology.orchestra.run.smithi118.stdout: "poolop_send": 0, 2024-09-12T03:29:13.463 INFO:teuthology.orchestra.run.smithi118.stdout: "poolop_resend": 0, 2024-09-12T03:29:13.463 INFO:teuthology.orchestra.run.smithi118.stdout: "poolstat_active": 0, 2024-09-12T03:29:13.463 INFO:teuthology.orchestra.run.smithi118.stdout: "poolstat_send": 0, 2024-09-12T03:29:13.463 INFO:teuthology.orchestra.run.smithi118.stdout: "poolstat_resend": 0, 2024-09-12T03:29:13.463 INFO:teuthology.orchestra.run.smithi118.stdout: "statfs_active": 0, 2024-09-12T03:29:13.463 INFO:teuthology.orchestra.run.smithi118.stdout: "statfs_send": 0, 2024-09-12T03:29:13.463 INFO:teuthology.orchestra.run.smithi118.stdout: "statfs_resend": 0, 2024-09-12T03:29:13.463 INFO:teuthology.orchestra.run.smithi118.stdout: "command_active": 0, 2024-09-12T03:29:13.463 INFO:teuthology.orchestra.run.smithi118.stdout: "command_send": 0, 2024-09-12T03:29:13.463 INFO:teuthology.orchestra.run.smithi118.stdout: "command_resend": 0, 2024-09-12T03:29:13.463 INFO:teuthology.orchestra.run.smithi118.stdout: "map_epoch": 82, 2024-09-12T03:29:13.463 INFO:teuthology.orchestra.run.smithi118.stdout: "map_full": 0, 2024-09-12T03:29:13.463 INFO:teuthology.orchestra.run.smithi118.stdout: "map_inc": 2, 2024-09-12T03:29:13.464 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_sessions": 9, 2024-09-12T03:29:13.464 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_session_open": 9, 2024-09-12T03:29:13.464 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_session_close": 0, 2024-09-12T03:29:13.464 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_laggy": 0, 2024-09-12T03:29:13.464 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_wr": 21, 2024-09-12T03:29:13.464 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_rd": 0, 2024-09-12T03:29:13.464 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_del": 4 2024-09-12T03:29:13.464 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.464 INFO:teuthology.orchestra.run.smithi118.stdout: "oft": { 2024-09-12T03:29:13.464 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_objs": 1, 2024-09-12T03:29:13.464 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:29:13.464 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_updates": 0, 2024-09-12T03:29:13.465 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_removes": 0 2024-09-12T03:29:13.465 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.465 INFO:teuthology.orchestra.run.smithi118.stdout: "osd-slow-ops": { 2024-09-12T03:29:13.465 INFO:teuthology.orchestra.run.smithi118.stdout: "slow_ops_count": 431 2024-09-12T03:29:13.465 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.465 INFO:teuthology.orchestra.run.smithi118.stdout: "purge_queue": { 2024-09-12T03:29:13.465 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing_ops": 0, 2024-09-12T03:29:13.465 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing_ops_high_water": 1, 2024-09-12T03:29:13.466 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing": 0, 2024-09-12T03:29:13.466 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing_high_water": 1, 2024-09-12T03:29:13.466 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executed_ops": 1, 2024-09-12T03:29:13.466 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executed": 1, 2024-09-12T03:29:13.466 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_item_in_journal": 0 2024-09-12T03:29:13.466 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.466 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:29:13.466 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2024-09-12T03:29:13.466 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 104857600, 2024-09-12T03:29:13.466 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2024-09-12T03:29:13.466 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 4868, 2024-09-12T03:29:13.466 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 1833790, 2024-09-12T03:29:13.466 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:13.467 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 4868, 2024-09-12T03:29:13.467 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2024-09-12T03:29:13.467 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2024-09-12T03:29:13.467 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 4868, 2024-09-12T03:29:13.467 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 1833790, 2024-09-12T03:29:13.467 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2024-09-12T03:29:13.467 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.467 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.467 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.467 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:13.467 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.467 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-objecter_bytes": { 2024-09-12T03:29:13.467 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2024-09-12T03:29:13.467 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 104857600, 2024-09-12T03:29:13.468 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2024-09-12T03:29:13.468 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 0, 2024-09-12T03:29:13.468 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 0, 2024-09-12T03:29:13.468 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:13.468 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 0, 2024-09-12T03:29:13.468 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 74, 2024-09-12T03:29:13.468 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 36638, 2024-09-12T03:29:13.468 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 74, 2024-09-12T03:29:13.468 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 36638, 2024-09-12T03:29:13.468 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2024-09-12T03:29:13.468 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.468 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.468 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.469 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:13.469 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.469 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-objecter_ops": { 2024-09-12T03:29:13.469 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2024-09-12T03:29:13.469 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 1024, 2024-09-12T03:29:13.469 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2024-09-12T03:29:13.469 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 0, 2024-09-12T03:29:13.469 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 0, 2024-09-12T03:29:13.469 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:13.469 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 0, 2024-09-12T03:29:13.469 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 74, 2024-09-12T03:29:13.469 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 74, 2024-09-12T03:29:13.469 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 74, 2024-09-12T03:29:13.470 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 74, 2024-09-12T03:29:13.470 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2024-09-12T03:29:13.470 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.470 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.470 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.470 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:13.470 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.470 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:29:13.470 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2024-09-12T03:29:13.470 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 3758096384, 2024-09-12T03:29:13.470 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2024-09-12T03:29:13.470 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 1, 2024-09-12T03:29:13.470 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 101, 2024-09-12T03:29:13.471 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:13.471 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 1, 2024-09-12T03:29:13.471 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2024-09-12T03:29:13.471 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2024-09-12T03:29:13.471 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 1, 2024-09-12T03:29:13.471 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 101, 2024-09-12T03:29:13.471 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2024-09-12T03:29:13.471 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.471 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.471 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.471 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:13.471 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:13.471 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-write_buf_throttle-0x558e89404960": { 2024-09-12T03:29:13.472 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2024-09-12T03:29:13.472 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 3758096384, 2024-09-12T03:29:13.472 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2024-09-12T03:29:13.472 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 18, 2024-09-12T03:29:13.472 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 24271, 2024-09-12T03:29:13.472 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:13.472 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 18, 2024-09-12T03:29:13.472 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2024-09-12T03:29:13.472 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2024-09-12T03:29:13.472 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 15, 2024-09-12T03:29:13.472 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 24271, 2024-09-12T03:29:13.472 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2024-09-12T03:29:13.472 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:13.472 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:13.473 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:13.473 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:13.473 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:13.473 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-12T03:29:14.251 INFO:tasks.check_counter:Daemon mds.h mds.exported=1 2024-09-12T03:29:14.252 INFO:tasks.check_counter:Daemon mds.h mds.imported=1 2024-09-12T03:29:14.252 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=1 2024-09-12T03:29:14.252 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=276 2024-09-12T03:29:14.252 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-09-12T03:29:14.252 DEBUG:tasks.check_counter:Getting stats from k 2024-09-12T03:29:14.252 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-12T03:29:14.500 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:29:14.988 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-12T03:29:14.988 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:29:14.988 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 0, 2024-09-12T03:29:14.988 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 2, 2024-09-12T03:29:14.988 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:29:14.989 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 0, 2024-09-12T03:29:14.989 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 4, 2024-09-12T03:29:14.989 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 0, 2024-09-12T03:29:14.989 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 0.001522971, 2024-09-12T03:29:14.989 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:29:14.989 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 0.000228652, 2024-09-12T03:29:14.989 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:29:14.989 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:14.989 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:14.989 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:14.989 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:14.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:14.989 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:14.990 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:14.990 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:14.990 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:14.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:14.990 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:29:14.990 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:29:14.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:14.990 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:29:14.990 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 179, 2024-09-12T03:29:14.990 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 174, 2024-09-12T03:29:14.990 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 742192, 2024-09-12T03:29:14.990 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 106448, 2024-09-12T03:29:14.990 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 2, 2024-09-12T03:29:14.991 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 1, 2024-09-12T03:29:14.991 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 0.027525378, 2024-09-12T03:29:14.991 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 0.011026827, 2024-09-12T03:29:14.991 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 0.024084900, 2024-09-12T03:29:14.991 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:29:14.991 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:14.991 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 173, 2024-09-12T03:29:14.991 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.010595858, 2024-09-12T03:29:14.991 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000061247 2024-09-12T03:29:14.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:14.991 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:14.991 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:14.991 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:14.991 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:14.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:14.992 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 742192, 2024-09-12T03:29:14.992 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 106448 2024-09-12T03:29:14.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:14.992 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:29:14.992 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 5, 2024-09-12T03:29:14.992 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 134, 2024-09-12T03:29:14.992 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 3232, 2024-09-12T03:29:14.992 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 102272, 2024-09-12T03:29:14.992 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 3, 2024-09-12T03:29:14.992 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 2, 2024-09-12T03:29:14.992 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 0.010825423, 2024-09-12T03:29:14.992 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 0.005532650, 2024-09-12T03:29:14.993 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 0.003571535, 2024-09-12T03:29:14.993 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:29:14.993 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:14.993 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 133, 2024-09-12T03:29:14.993 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.011584032, 2024-09-12T03:29:14.993 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000087097 2024-09-12T03:29:14.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:14.993 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:14.996 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:14.996 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:14.996 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:14.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:14.997 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-12T03:29:14.997 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 102272 2024-09-12T03:29:14.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:14.997 INFO:teuthology.orchestra.run.smithi118.stdout: "cct": { 2024-09-12T03:29:14.997 INFO:teuthology.orchestra.run.smithi118.stdout: "total_workers": 0, 2024-09-12T03:29:14.997 INFO:teuthology.orchestra.run.smithi118.stdout: "unhealthy_workers": 0 2024-09-12T03:29:14.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:14.997 INFO:teuthology.orchestra.run.smithi118.stdout: "mempool": { 2024-09-12T03:29:14.997 INFO:teuthology.orchestra.run.smithi118.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:29:14.997 INFO:teuthology.orchestra.run.smithi118.stdout: "bloom_filter_items": 0, 2024-09-12T03:29:14.997 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:29:14.997 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:29:14.997 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:29:14.997 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:29:14.998 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:29:14.998 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:29:14.998 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:29:14.998 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:29:14.998 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:29:14.998 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:29:14.998 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:29:14.998 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:29:14.998 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:29:14.998 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_extent_items": 0, 2024-09-12T03:29:14.998 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:29:14.998 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_blob_items": 0, 2024-09-12T03:29:14.998 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:29:14.999 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:29:14.999 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:29:14.999 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:29:14.999 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:29:14.999 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:29:14.999 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:29:14.999 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_txc_items": 0, 2024-09-12T03:29:14.999 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:29:14.999 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:29:14.999 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:29:14.999 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_items": 0, 2024-09-12T03:29:14.999 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_bytes": 0, 2024-09-12T03:29:14.999 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_items": 0, 2024-09-12T03:29:14.999 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:29:15.000 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:29:15.000 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:29:15.000 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:29:15.000 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_anon_bytes": 149007, 2024-09-12T03:29:15.000 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_anon_items": 30, 2024-09-12T03:29:15.000 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:29:15.000 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_meta_items": 0, 2024-09-12T03:29:15.000 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_bytes": 0, 2024-09-12T03:29:15.000 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_items": 0, 2024-09-12T03:29:15.000 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:29:15.000 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mapbl_items": 0, 2024-09-12T03:29:15.000 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:29:15.000 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pglog_items": 0, 2024-09-12T03:29:15.000 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_bytes": 0, 2024-09-12T03:29:15.001 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_items": 0, 2024-09-12T03:29:15.001 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:29:15.001 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:29:15.001 INFO:teuthology.orchestra.run.smithi118.stdout: "pgmap_bytes": 0, 2024-09-12T03:29:15.001 INFO:teuthology.orchestra.run.smithi118.stdout: "pgmap_items": 0, 2024-09-12T03:29:15.001 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_co_bytes": 928, 2024-09-12T03:29:15.001 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_co_items": 2, 2024-09-12T03:29:15.001 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_1_bytes": 0, 2024-09-12T03:29:15.001 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_1_items": 0, 2024-09-12T03:29:15.001 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_2_bytes": 0, 2024-09-12T03:29:15.001 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_2_items": 0 2024-09-12T03:29:15.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-12T03:29:15.001 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:29:15.001 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2024-09-12T03:29:15.002 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 104857600, 2024-09-12T03:29:15.002 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2024-09-12T03:29:15.002 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 180, 2024-09-12T03:29:15.002 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 716929, 2024-09-12T03:29:15.002 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:15.002 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 180, 2024-09-12T03:29:15.002 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2024-09-12T03:29:15.002 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2024-09-12T03:29:15.002 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 180, 2024-09-12T03:29:15.002 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 716929, 2024-09-12T03:29:15.002 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2024-09-12T03:29:15.002 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2024-09-12T03:29:15.002 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2024-09-12T03:29:15.002 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2024-09-12T03:29:15.003 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:15.003 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-12T03:29:15.003 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-12T03:29:15.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:14 smithi118 ceph-mon[35048]: pgmap v579: 97 pgs: 97 active+clean; 159 MiB data, 110 GiB used, 963 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:29:15.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:14 smithi006 ceph-mon[30991]: pgmap v579: 97 pgs: 97 active+clean; 159 MiB data, 110 GiB used, 963 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:29:15.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:14 smithi161 ceph-mon[35485]: pgmap v579: 97 pgs: 97 active+clean; 159 MiB data, 110 GiB used, 963 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:29:15.611 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-12T03:29:15.611 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-12T03:29:15.611 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-09-12T03:29:15.611 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-09-12T03:29:15.612 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-09-12T03:29:15.612 DEBUG:tasks.check_counter:Getting stats from c 2024-09-12T03:29:15.612 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-12T03:29:15.867 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:29:16.442 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-12T03:29:16.442 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:29:16.442 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1166, 2024-09-12T03:29:16.442 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1167, 2024-09-12T03:29:16.442 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 75858946, 2024-09-12T03:29:16.442 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 370480, 2024-09-12T03:29:16.442 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 8, 2024-09-12T03:29:16.442 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-09-12T03:29:16.443 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.165299521, 2024-09-12T03:29:16.443 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.031958424, 2024-09-12T03:29:16.443 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.084791882, 2024-09-12T03:29:16.443 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.038504747, 2024-09-12T03:29:16.443 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:16.443 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1165, 2024-09-12T03:29:16.443 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.064333119, 2024-09-12T03:29:16.443 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000055221 2024-09-12T03:29:16.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.443 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:16.443 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.443 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.443 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.444 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:29:16.444 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:29:16.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.444 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:29:16.444 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 459, 2024-09-12T03:29:16.444 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 587, 2024-09-12T03:29:16.444 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 53171079, 2024-09-12T03:29:16.444 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 448590, 2024-09-12T03:29:16.444 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 7, 2024-09-12T03:29:16.444 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 6, 2024-09-12T03:29:16.444 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.083742677, 2024-09-12T03:29:16.444 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.015376891, 2024-09-12T03:29:16.445 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.050788076, 2024-09-12T03:29:16.445 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.013705241, 2024-09-12T03:29:16.445 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:16.445 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 585, 2024-09-12T03:29:16.445 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.049469391, 2024-09-12T03:29:16.445 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000084563 2024-09-12T03:29:16.445 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.445 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:16.445 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.445 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.445 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.445 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.446 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T03:29:16.446 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 302880 2024-09-12T03:29:16.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.446 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:29:16.446 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 606, 2024-09-12T03:29:16.446 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 593, 2024-09-12T03:29:16.446 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 38648186, 2024-09-12T03:29:16.446 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 237479, 2024-09-12T03:29:16.446 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 5, 2024-09-12T03:29:16.446 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-09-12T03:29:16.446 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.092254065, 2024-09-12T03:29:16.446 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.023296098, 2024-09-12T03:29:16.446 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.059789496, 2024-09-12T03:29:16.447 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.013396205, 2024-09-12T03:29:16.447 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:16.447 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 593, 2024-09-12T03:29:16.447 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.051193078, 2024-09-12T03:29:16.447 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000086328 2024-09-12T03:29:16.447 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.447 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:16.447 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.447 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.447 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.447 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.447 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 784336, 2024-09-12T03:29:16.447 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 106144 2024-09-12T03:29:16.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.448 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-09-12T03:29:16.448 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-09-12T03:29:16.448 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-09-12T03:29:16.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.448 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-09-12T03:29:16.448 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-12T03:29:16.448 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-12T03:29:16.448 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3313, 2024-09-12T03:29:16.448 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.085854179, 2024-09-12T03:29:16.448 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000025914 2024-09-12T03:29:16.449 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:16.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.449 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-09-12T03:29:16.449 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-12T03:29:16.449 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-12T03:29:16.449 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.449 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.449 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.449 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:16.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.449 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-09-12T03:29:16.449 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2024-09-12T03:29:16.450 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2024-09-12T03:29:16.450 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-09-12T03:29:16.450 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.450 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.450 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.450 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.450 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-09-12T03:29:16.450 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-09-12T03:29:16.450 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-09-12T03:29:16.450 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-09-12T03:29:16.450 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2024-09-12T03:29:16.450 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-09-12T03:29:16.450 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-09-12T03:29:16.451 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 1180, 2024-09-12T03:29:16.451 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2024-09-12T03:29:16.451 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 1180, 2024-09-12T03:29:16.451 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-09-12T03:29:16.451 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 1180, 2024-09-12T03:29:16.451 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 0, 2024-09-12T03:29:16.451 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-09-12T03:29:16.451 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-09-12T03:29:16.451 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 2, 2024-09-12T03:29:16.451 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2024-09-12T03:29:16.451 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2024-09-12T03:29:16.451 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-09-12T03:29:16.451 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2024-09-12T03:29:16.451 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:29:16.452 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-09-12T03:29:16.452 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-09-12T03:29:16.452 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-09-12T03:29:16.452 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-09-12T03:29:16.452 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-09-12T03:29:16.452 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-09-12T03:29:16.452 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-09-12T03:29:16.452 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-09-12T03:29:16.452 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-09-12T03:29:16.452 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:29:16.452 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-09-12T03:29:16.452 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 4, 2024-09-12T03:29:16.452 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 234, 2024-09-12T03:29:16.453 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-09-12T03:29:16.453 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:29:16.453 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-09-12T03:29:16.453 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:29:16.453 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:29:16.453 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:29:16.453 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:29:16.453 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:29:16.453 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-09-12T03:29:16.453 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:29:16.453 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:29:16.453 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:29:16.454 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:29:16.454 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:29:16.454 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:29:16.454 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-09-12T03:29:16.454 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:29:16.454 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-09-12T03:29:16.454 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2024-09-12T03:29:16.454 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.454 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-09-12T03:29:16.454 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-09-12T03:29:16.454 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-09-12T03:29:16.454 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-09-12T03:29:16.454 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2024-09-12T03:29:16.454 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-09-12T03:29:16.455 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-09-12T03:29:16.455 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-09-12T03:29:16.455 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:29:16.455 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-09-12T03:29:16.455 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-09-12T03:29:16.455 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-09-12T03:29:16.455 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-09-12T03:29:16.455 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-09-12T03:29:16.455 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:29:16.455 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:29:16.455 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-09-12T03:29:16.455 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-09-12T03:29:16.455 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:29:16.455 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:29:16.456 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:29:16.456 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-09-12T03:29:16.456 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-09-12T03:29:16.456 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:29:16.456 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-09-12T03:29:16.456 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-09-12T03:29:16.456 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.456 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-09-12T03:29:16.456 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-09-12T03:29:16.456 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2024-09-12T03:29:16.456 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-09-12T03:29:16.456 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-09-12T03:29:16.456 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 78419, 2024-09-12T03:29:16.457 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-09-12T03:29:16.457 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-09-12T03:29:16.457 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2024-09-12T03:29:16.457 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-09-12T03:29:16.457 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-09-12T03:29:16.457 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 87, 2024-09-12T03:29:16.457 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 8, 2024-09-12T03:29:16.457 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-09-12T03:29:16.457 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-09-12T03:29:16.457 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-09-12T03:29:16.457 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 170471362, 2024-09-12T03:29:16.457 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 170466725, 2024-09-12T03:29:16.457 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-09-12T03:29:16.457 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.458 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.458 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.458 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.458 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 78419 2024-09-12T03:29:16.458 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.458 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-09-12T03:29:16.458 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 29, 2024-09-12T03:29:16.458 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 891, 2024-09-12T03:29:16.458 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 862, 2024-09-12T03:29:16.458 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 24, 2024-09-12T03:29:16.458 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 120, 2024-09-12T03:29:16.458 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 96, 2024-09-12T03:29:16.458 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 1180, 2024-09-12T03:29:16.458 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 1536, 2024-09-12T03:29:16.458 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 356, 2024-09-12T03:29:16.459 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-09-12T03:29:16.459 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2024-09-12T03:29:16.459 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2024-09-12T03:29:16.459 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 53360, 2024-09-12T03:29:16.459 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 215328 2024-09-12T03:29:16.459 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.459 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-09-12T03:29:16.459 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2024-09-12T03:29:16.459 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-09-12T03:29:16.459 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2024-09-12T03:29:16.459 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2024-09-12T03:29:16.459 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-09-12T03:29:16.459 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-09-12T03:29:16.459 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.460 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.460 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.460 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.460 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-09-12T03:29:16.460 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.460 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.460 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.460 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.460 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-09-12T03:29:16.460 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.460 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.460 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.460 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.460 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-09-12T03:29:16.461 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.461 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.461 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.461 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.461 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-09-12T03:29:16.461 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.461 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.461 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.461 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.461 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-09-12T03:29:16.461 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.461 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.462 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.462 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.462 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-09-12T03:29:16.462 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.462 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.462 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.462 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.462 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-09-12T03:29:16.462 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.462 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.462 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.462 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.462 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-09-12T03:29:16.462 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.462 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.463 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.463 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.463 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-09-12T03:29:16.463 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.463 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.463 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.463 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.463 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-09-12T03:29:16.463 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.463 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.463 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.463 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.464 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-09-12T03:29:16.464 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.464 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.464 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.464 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.464 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-09-12T03:29:16.464 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.464 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.464 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.464 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.464 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-09-12T03:29:16.464 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.465 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.465 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.465 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.465 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-09-12T03:29:16.465 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.465 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.465 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.465 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.465 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-09-12T03:29:16.465 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.465 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.465 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.466 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.466 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-09-12T03:29:16.466 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.466 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.466 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.466 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.466 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-09-12T03:29:16.466 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.466 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.466 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.466 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.466 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-09-12T03:29:16.466 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.467 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.467 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.467 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.467 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-09-12T03:29:16.467 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.467 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.467 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.467 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.467 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-09-12T03:29:16.467 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.467 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.467 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.467 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.467 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-09-12T03:29:16.468 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.468 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.468 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.468 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.468 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-09-12T03:29:16.468 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.468 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.468 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.468 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.468 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-09-12T03:29:16.468 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.468 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.468 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.469 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-09-12T03:29:16.469 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.469 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.469 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.469 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-09-12T03:29:16.469 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.469 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.469 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.469 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-09-12T03:29:16.470 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.470 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.470 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.470 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.470 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-09-12T03:29:16.470 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.470 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.470 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.470 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.470 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-09-12T03:29:16.470 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.470 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.470 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.470 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.470 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:29:16.471 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:29:16.471 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-09-12T03:29:16.471 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.471 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.471 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.471 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:16.471 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.471 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-09-12T03:29:16.471 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-09-12T03:29:16.471 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 3, 2024-09-12T03:29:16.471 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 2, 2024-09-12T03:29:16.471 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 2, 2024-09-12T03:29:16.471 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-09-12T03:29:16.471 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2024-09-12T03:29:16.472 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2024-09-12T03:29:16.472 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 3079, 2024-09-12T03:29:16.472 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-09-12T03:29:16.472 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.472 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-09-12T03:29:16.472 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:29:16.472 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-09-12T03:29:16.472 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:29:16.472 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:29:16.472 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:29:16.472 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:29:16.472 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:29:16.472 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:29:16.473 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:29:16.473 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:29:16.473 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:29:16.473 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:29:16.473 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:29:16.473 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:29:16.473 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:29:16.473 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-09-12T03:29:16.473 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:29:16.473 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-09-12T03:29:16.473 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:29:16.473 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:29:16.473 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:29:16.474 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:29:16.474 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:29:16.474 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:29:16.474 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:29:16.474 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-09-12T03:29:16.474 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:29:16.474 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:29:16.474 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:29:16.474 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-09-12T03:29:16.474 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-09-12T03:29:16.474 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-09-12T03:29:16.474 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:29:16.474 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:29:16.475 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:29:16.475 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:29:16.475 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 239855, 2024-09-12T03:29:16.475 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 90, 2024-09-12T03:29:16.475 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:29:16.475 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-09-12T03:29:16.475 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-09-12T03:29:16.475 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-09-12T03:29:16.475 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:29:16.475 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-09-12T03:29:16.475 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:29:16.475 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-09-12T03:29:16.475 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 7400, 2024-09-12T03:29:16.475 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 152, 2024-09-12T03:29:16.476 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:29:16.476 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:29:16.476 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-09-12T03:29:16.476 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-09-12T03:29:16.476 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 929479, 2024-09-12T03:29:16.476 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 6326, 2024-09-12T03:29:16.476 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-09-12T03:29:16.476 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-09-12T03:29:16.476 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-09-12T03:29:16.476 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-09-12T03:29:16.476 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.476 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-09-12T03:29:16.476 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-09-12T03:29:16.476 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-09-12T03:29:16.477 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 2036, 2024-09-12T03:29:16.477 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 26, 2024-09-12T03:29:16.477 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 4, 2024-09-12T03:29:16.477 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 2036, 2024-09-12T03:29:16.477 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-09-12T03:29:16.477 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2036, 2024-09-12T03:29:16.477 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 42.066061822, 2024-09-12T03:29:16.477 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.020661130 2024-09-12T03:29:16.477 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.477 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-09-12T03:29:16.477 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-09-12T03:29:16.477 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2036, 2024-09-12T03:29:16.477 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2037 2024-09-12T03:29:16.477 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.478 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 2036, 2024-09-12T03:29:16.478 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 2036, 2024-09-12T03:29:16.478 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2024-09-12T03:29:16.478 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-09-12T03:29:16.478 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-09-12T03:29:16.478 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 1225, 2024-09-12T03:29:16.478 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2024-09-12T03:29:16.478 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 808, 2024-09-12T03:29:16.478 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2024-09-12T03:29:16.478 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2024-09-12T03:29:16.478 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-09-12T03:29:16.478 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-09-12T03:29:16.478 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-09-12T03:29:16.479 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-09-12T03:29:16.479 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2024-09-12T03:29:16.479 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-09-12T03:29:16.479 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-09-12T03:29:16.479 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-09-12T03:29:16.479 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 2, 2024-09-12T03:29:16.479 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2024-09-12T03:29:16.479 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:29:16.479 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-09-12T03:29:16.479 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:29:16.479 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-09-12T03:29:16.479 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-09-12T03:29:16.479 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-09-12T03:29:16.480 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:29:16.480 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-09-12T03:29:16.480 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:29:16.480 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-09-12T03:29:16.480 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-09-12T03:29:16.480 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-09-12T03:29:16.480 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-09-12T03:29:16.480 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-09-12T03:29:16.480 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-09-12T03:29:16.480 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-09-12T03:29:16.480 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-09-12T03:29:16.480 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-09-12T03:29:16.480 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-09-12T03:29:16.481 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-09-12T03:29:16.481 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-09-12T03:29:16.481 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-09-12T03:29:16.481 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-09-12T03:29:16.481 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-09-12T03:29:16.481 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-09-12T03:29:16.481 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-09-12T03:29:16.481 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 82, 2024-09-12T03:29:16.481 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-09-12T03:29:16.481 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 1, 2024-09-12T03:29:16.481 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 11, 2024-09-12T03:29:16.481 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 11, 2024-09-12T03:29:16.481 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-09-12T03:29:16.481 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-09-12T03:29:16.482 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2024-09-12T03:29:16.482 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2024-09-12T03:29:16.482 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2024-09-12T03:29:16.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.482 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-09-12T03:29:16.482 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2024-09-12T03:29:16.482 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:29:16.482 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-09-12T03:29:16.482 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-09-12T03:29:16.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.482 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-09-12T03:29:16.482 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-09-12T03:29:16.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.482 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-09-12T03:29:16.483 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-09-12T03:29:16.483 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:29:16.483 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-09-12T03:29:16.483 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-09-12T03:29:16.483 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 0, 2024-09-12T03:29:16.483 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-09-12T03:29:16.483 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-09-12T03:29:16.483 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.483 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:29:16.483 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-12T03:29:16.483 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-12T03:29:16.483 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-12T03:29:16.483 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 2227, 2024-09-12T03:29:16.483 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 167465021, 2024-09-12T03:29:16.484 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:16.484 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 2227, 2024-09-12T03:29:16.484 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-12T03:29:16.484 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-12T03:29:16.484 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2227, 2024-09-12T03:29:16.484 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 167465021, 2024-09-12T03:29:16.484 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-12T03:29:16.484 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.484 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.484 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.484 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:16.484 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.484 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-09-12T03:29:16.484 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-12T03:29:16.485 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-12T03:29:16.485 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-12T03:29:16.485 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-12T03:29:16.485 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-12T03:29:16.485 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:16.485 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-12T03:29:16.485 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 2036, 2024-09-12T03:29:16.485 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 166328281, 2024-09-12T03:29:16.485 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2036, 2024-09-12T03:29:16.485 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 166328281, 2024-09-12T03:29:16.485 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-12T03:29:16.485 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.485 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.485 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.486 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:16.486 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.486 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-09-12T03:29:16.486 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-12T03:29:16.486 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-09-12T03:29:16.486 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-12T03:29:16.486 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-12T03:29:16.486 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-12T03:29:16.486 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:16.486 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-12T03:29:16.486 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 2036, 2024-09-12T03:29:16.486 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 2036, 2024-09-12T03:29:16.486 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2036, 2024-09-12T03:29:16.487 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 2036, 2024-09-12T03:29:16.487 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-12T03:29:16.487 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.487 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.487 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.487 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:16.487 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.487 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:29:16.487 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-12T03:29:16.487 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-12T03:29:16.487 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-12T03:29:16.487 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-12T03:29:16.487 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-12T03:29:16.487 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:16.487 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-12T03:29:16.488 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-12T03:29:16.488 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-12T03:29:16.488 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-09-12T03:29:16.488 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-09-12T03:29:16.488 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-12T03:29:16.488 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.488 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.488 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.488 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:16.488 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:16.488 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x5620ec48ab40": { 2024-09-12T03:29:16.488 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-12T03:29:16.488 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-12T03:29:16.489 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-12T03:29:16.489 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-12T03:29:16.489 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-12T03:29:16.489 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:16.489 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-12T03:29:16.489 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-12T03:29:16.489 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-12T03:29:16.489 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-09-12T03:29:16.489 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-09-12T03:29:16.489 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-12T03:29:16.489 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:16.489 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:16.489 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:16.489 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:16.490 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:16.490 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-12T03:29:17.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:16 smithi118 ceph-mon[35048]: pgmap v580: 97 pgs: 97 active+clean; 159 MiB data, 110 GiB used, 963 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:29:17.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:16 smithi161 ceph-mon[35485]: pgmap v580: 97 pgs: 97 active+clean; 159 MiB data, 110 GiB used, 963 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:29:17.257 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-12T03:29:17.258 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-12T03:29:17.258 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-12T03:29:17.258 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-09-12T03:29:17.258 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-09-12T03:29:17.258 DEBUG:tasks.check_counter:Getting stats from f 2024-09-12T03:29:17.258 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-12T03:29:17.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:16 smithi006 ceph-mon[30991]: pgmap v580: 97 pgs: 97 active+clean; 159 MiB data, 110 GiB used, 963 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:29:17.521 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:29:18.050 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-12T03:29:18.050 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:29:18.051 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 527, 2024-09-12T03:29:18.051 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 528, 2024-09-12T03:29:18.051 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 162564, 2024-09-12T03:29:18.051 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 167331, 2024-09-12T03:29:18.051 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 6, 2024-09-12T03:29:18.051 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2024-09-12T03:29:18.051 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.049571040, 2024-09-12T03:29:18.051 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.013130682, 2024-09-12T03:29:18.051 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.014628657, 2024-09-12T03:29:18.051 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.016488908, 2024-09-12T03:29:18.051 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:18.051 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 526, 2024-09-12T03:29:18.051 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.023460561, 2024-09-12T03:29:18.052 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000044601 2024-09-12T03:29:18.052 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.052 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:18.052 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.052 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.052 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.052 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.052 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:29:18.052 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:29:18.052 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.052 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:29:18.052 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 191, 2024-09-12T03:29:18.053 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 180, 2024-09-12T03:29:18.053 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 782478, 2024-09-12T03:29:18.053 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 106818, 2024-09-12T03:29:18.053 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-09-12T03:29:18.053 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-09-12T03:29:18.053 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.027459277, 2024-09-12T03:29:18.053 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.010646532, 2024-09-12T03:29:18.053 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.024522619, 2024-09-12T03:29:18.053 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000042311, 2024-09-12T03:29:18.053 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:18.053 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 179, 2024-09-12T03:29:18.053 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.010342156, 2024-09-12T03:29:18.053 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000057777 2024-09-12T03:29:18.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.054 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:18.054 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.054 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.054 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.054 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 781376, 2024-09-12T03:29:18.054 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 106176 2024-09-12T03:29:18.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.054 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:29:18.054 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 936, 2024-09-12T03:29:18.054 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1063, 2024-09-12T03:29:18.054 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 278338, 2024-09-12T03:29:18.055 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 601178, 2024-09-12T03:29:18.055 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 5, 2024-09-12T03:29:18.055 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-09-12T03:29:18.055 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.104309415, 2024-09-12T03:29:18.055 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.030382650, 2024-09-12T03:29:18.055 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.028677943, 2024-09-12T03:29:18.055 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.036466357, 2024-09-12T03:29:18.055 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:18.055 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1062, 2024-09-12T03:29:18.055 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.059551631, 2024-09-12T03:29:18.055 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000056074 2024-09-12T03:29:18.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.055 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:18.056 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.056 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.056 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.056 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-12T03:29:18.056 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 305120 2024-09-12T03:29:18.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.056 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-09-12T03:29:18.056 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-09-12T03:29:18.056 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-09-12T03:29:18.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.056 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-09-12T03:29:18.056 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-12T03:29:18.056 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-12T03:29:18.057 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2478, 2024-09-12T03:29:18.057 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.071482237, 2024-09-12T03:29:18.057 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000028846 2024-09-12T03:29:18.057 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:18.057 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.057 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-09-12T03:29:18.057 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-12T03:29:18.057 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-12T03:29:18.057 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.057 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.057 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.057 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:18.057 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.058 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-09-12T03:29:18.058 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2024-09-12T03:29:18.058 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2024-09-12T03:29:18.058 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-09-12T03:29:18.058 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.058 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.058 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.058 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.058 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-09-12T03:29:18.058 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-09-12T03:29:18.058 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-09-12T03:29:18.058 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-09-12T03:29:18.058 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2024-09-12T03:29:18.059 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-09-12T03:29:18.059 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-09-12T03:29:18.059 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 2, 2024-09-12T03:29:18.059 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2024-09-12T03:29:18.059 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 2, 2024-09-12T03:29:18.059 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-09-12T03:29:18.059 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 2, 2024-09-12T03:29:18.059 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 0, 2024-09-12T03:29:18.059 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-09-12T03:29:18.059 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-09-12T03:29:18.059 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 1, 2024-09-12T03:29:18.059 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2024-09-12T03:29:18.059 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2024-09-12T03:29:18.060 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-09-12T03:29:18.060 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2024-09-12T03:29:18.060 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:29:18.060 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-09-12T03:29:18.060 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-09-12T03:29:18.060 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-09-12T03:29:18.060 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-09-12T03:29:18.060 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-09-12T03:29:18.060 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-09-12T03:29:18.060 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-09-12T03:29:18.060 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-09-12T03:29:18.060 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-09-12T03:29:18.060 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:29:18.061 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-09-12T03:29:18.061 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 0, 2024-09-12T03:29:18.061 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 0, 2024-09-12T03:29:18.061 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-09-12T03:29:18.061 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:29:18.061 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-09-12T03:29:18.061 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:29:18.061 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:29:18.061 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:29:18.061 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:29:18.061 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:29:18.061 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-09-12T03:29:18.061 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:29:18.062 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:29:18.062 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:29:18.062 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:29:18.062 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:29:18.062 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:29:18.062 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-09-12T03:29:18.062 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:29:18.062 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-09-12T03:29:18.062 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2024-09-12T03:29:18.062 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.062 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-09-12T03:29:18.062 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-09-12T03:29:18.062 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-09-12T03:29:18.062 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-09-12T03:29:18.063 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2024-09-12T03:29:18.063 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-09-12T03:29:18.063 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-09-12T03:29:18.063 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-09-12T03:29:18.063 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:29:18.063 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-09-12T03:29:18.063 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-09-12T03:29:18.063 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-09-12T03:29:18.063 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-09-12T03:29:18.063 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-09-12T03:29:18.063 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:29:18.063 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:29:18.063 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-09-12T03:29:18.064 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-09-12T03:29:18.064 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:29:18.064 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:29:18.064 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:29:18.064 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-09-12T03:29:18.064 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-09-12T03:29:18.064 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:29:18.064 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-09-12T03:29:18.064 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-09-12T03:29:18.064 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.064 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-09-12T03:29:18.064 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-09-12T03:29:18.064 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2024-09-12T03:29:18.064 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-09-12T03:29:18.065 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-09-12T03:29:18.065 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 16, 2024-09-12T03:29:18.065 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-09-12T03:29:18.065 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-09-12T03:29:18.065 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2024-09-12T03:29:18.065 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-09-12T03:29:18.065 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-09-12T03:29:18.065 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 2, 2024-09-12T03:29:18.065 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 2, 2024-09-12T03:29:18.065 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-09-12T03:29:18.065 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-09-12T03:29:18.065 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-09-12T03:29:18.065 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 4218755, 2024-09-12T03:29:18.066 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 4217104, 2024-09-12T03:29:18.066 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-09-12T03:29:18.066 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.066 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.066 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.066 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.066 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 16 2024-09-12T03:29:18.066 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.066 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-09-12T03:29:18.066 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 4, 2024-09-12T03:29:18.066 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 17, 2024-09-12T03:29:18.066 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 13, 2024-09-12T03:29:18.066 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 2, 2024-09-12T03:29:18.066 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 15, 2024-09-12T03:29:18.067 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 13, 2024-09-12T03:29:18.067 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 2, 2024-09-12T03:29:18.067 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 16, 2024-09-12T03:29:18.067 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 14, 2024-09-12T03:29:18.067 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-09-12T03:29:18.067 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2024-09-12T03:29:18.067 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2024-09-12T03:29:18.067 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 48772, 2024-09-12T03:29:18.067 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 215328 2024-09-12T03:29:18.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.067 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-09-12T03:29:18.067 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2024-09-12T03:29:18.067 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-09-12T03:29:18.067 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2024-09-12T03:29:18.068 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2024-09-12T03:29:18.068 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-09-12T03:29:18.068 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-09-12T03:29:18.068 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.068 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.068 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.068 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-09-12T03:29:18.068 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.068 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.068 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.068 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-09-12T03:29:18.068 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.068 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.069 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.069 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-09-12T03:29:18.069 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.069 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.069 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.069 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-09-12T03:29:18.069 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.069 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.069 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.069 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-09-12T03:29:18.070 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.070 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.070 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.070 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.070 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-09-12T03:29:18.070 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.070 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.070 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.070 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.070 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-09-12T03:29:18.070 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.070 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.070 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.071 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-09-12T03:29:18.071 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.071 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.071 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.071 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-09-12T03:29:18.071 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.071 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.071 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.071 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-09-12T03:29:18.071 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.072 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.072 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.072 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-09-12T03:29:18.072 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.072 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.072 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.072 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-09-12T03:29:18.072 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.072 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.072 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.073 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-09-12T03:29:18.073 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.073 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.073 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.073 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-09-12T03:29:18.073 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.073 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.073 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.073 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-09-12T03:29:18.073 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.073 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.073 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.074 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-09-12T03:29:18.074 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.074 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.074 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.074 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-09-12T03:29:18.074 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.074 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.074 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.074 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-09-12T03:29:18.074 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.074 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.075 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.075 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.075 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-09-12T03:29:18.075 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.075 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.075 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.075 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.075 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-09-12T03:29:18.075 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.075 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.075 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.075 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.075 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-09-12T03:29:18.075 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.076 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.076 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.076 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-09-12T03:29:18.076 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.076 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.076 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.076 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-09-12T03:29:18.076 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.076 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.076 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.077 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-09-12T03:29:18.077 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.077 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.077 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.077 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-09-12T03:29:18.077 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.077 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.077 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.077 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-09-12T03:29:18.077 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.078 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.078 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.078 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-09-12T03:29:18.078 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.078 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.078 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.078 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-09-12T03:29:18.078 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.078 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.078 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.078 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:29:18.079 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:29:18.079 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-09-12T03:29:18.079 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.079 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.079 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.079 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:18.079 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.079 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-09-12T03:29:18.079 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-09-12T03:29:18.079 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 2, 2024-09-12T03:29:18.079 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 1, 2024-09-12T03:29:18.079 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 2, 2024-09-12T03:29:18.079 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-09-12T03:29:18.079 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2024-09-12T03:29:18.080 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2024-09-12T03:29:18.080 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 3081, 2024-09-12T03:29:18.080 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-09-12T03:29:18.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.080 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-09-12T03:29:18.080 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:29:18.080 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-09-12T03:29:18.080 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:29:18.080 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:29:18.080 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:29:18.080 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:29:18.080 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:29:18.080 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:29:18.081 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:29:18.081 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:29:18.081 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:29:18.081 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:29:18.081 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:29:18.081 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:29:18.081 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:29:18.081 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-09-12T03:29:18.081 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:29:18.081 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-09-12T03:29:18.081 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:29:18.081 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:29:18.081 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:29:18.082 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:29:18.082 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:29:18.082 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:29:18.082 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:29:18.082 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-09-12T03:29:18.082 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:29:18.082 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:29:18.082 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:29:18.082 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-09-12T03:29:18.082 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-09-12T03:29:18.082 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-09-12T03:29:18.082 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:29:18.082 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:29:18.082 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:29:18.083 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:29:18.083 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 197480, 2024-09-12T03:29:18.083 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 55, 2024-09-12T03:29:18.083 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:29:18.083 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-09-12T03:29:18.083 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-09-12T03:29:18.083 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-09-12T03:29:18.083 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:29:18.083 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-09-12T03:29:18.083 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:29:18.084 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-09-12T03:29:18.084 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 7400, 2024-09-12T03:29:18.084 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 152, 2024-09-12T03:29:18.084 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:29:18.084 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:29:18.084 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-09-12T03:29:18.084 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-09-12T03:29:18.084 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 13456, 2024-09-12T03:29:18.084 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 36, 2024-09-12T03:29:18.084 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-09-12T03:29:18.084 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-09-12T03:29:18.084 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-09-12T03:29:18.084 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-09-12T03:29:18.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.085 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-09-12T03:29:18.085 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-09-12T03:29:18.085 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-09-12T03:29:18.085 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 1459, 2024-09-12T03:29:18.085 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 21, 2024-09-12T03:29:18.085 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 3, 2024-09-12T03:29:18.085 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 1459, 2024-09-12T03:29:18.085 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-09-12T03:29:18.085 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1459, 2024-09-12T03:29:18.085 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 90.157987380, 2024-09-12T03:29:18.085 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.061794371 2024-09-12T03:29:18.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.085 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-09-12T03:29:18.086 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-09-12T03:29:18.086 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1459, 2024-09-12T03:29:18.086 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1460 2024-09-12T03:29:18.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.086 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 1459, 2024-09-12T03:29:18.086 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 1459, 2024-09-12T03:29:18.086 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2024-09-12T03:29:18.086 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-09-12T03:29:18.086 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-09-12T03:29:18.086 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 931, 2024-09-12T03:29:18.086 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2024-09-12T03:29:18.086 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 526, 2024-09-12T03:29:18.086 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2024-09-12T03:29:18.086 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2024-09-12T03:29:18.087 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-09-12T03:29:18.087 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-09-12T03:29:18.087 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-09-12T03:29:18.087 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-09-12T03:29:18.087 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2024-09-12T03:29:18.087 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-09-12T03:29:18.087 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-09-12T03:29:18.087 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-09-12T03:29:18.087 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 1, 2024-09-12T03:29:18.087 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2024-09-12T03:29:18.087 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:29:18.087 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-09-12T03:29:18.087 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:29:18.087 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-09-12T03:29:18.087 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-09-12T03:29:18.088 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-09-12T03:29:18.088 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:29:18.088 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-09-12T03:29:18.088 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:29:18.088 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-09-12T03:29:18.088 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-09-12T03:29:18.088 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-09-12T03:29:18.088 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-09-12T03:29:18.088 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-09-12T03:29:18.088 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-09-12T03:29:18.088 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-09-12T03:29:18.088 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-09-12T03:29:18.088 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-09-12T03:29:18.089 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-09-12T03:29:18.089 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-09-12T03:29:18.089 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-09-12T03:29:18.089 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-09-12T03:29:18.089 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-09-12T03:29:18.089 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-09-12T03:29:18.089 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-09-12T03:29:18.089 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-09-12T03:29:18.089 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 82, 2024-09-12T03:29:18.089 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-09-12T03:29:18.089 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 1, 2024-09-12T03:29:18.089 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 6, 2024-09-12T03:29:18.089 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 6, 2024-09-12T03:29:18.089 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-09-12T03:29:18.090 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-09-12T03:29:18.090 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2024-09-12T03:29:18.090 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2024-09-12T03:29:18.090 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2024-09-12T03:29:18.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.090 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-09-12T03:29:18.090 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2024-09-12T03:29:18.090 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:29:18.090 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-09-12T03:29:18.090 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-09-12T03:29:18.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.090 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-09-12T03:29:18.090 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-09-12T03:29:18.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.090 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-09-12T03:29:18.091 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-09-12T03:29:18.091 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:29:18.091 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-09-12T03:29:18.091 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-09-12T03:29:18.091 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 0, 2024-09-12T03:29:18.091 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-09-12T03:29:18.091 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-09-12T03:29:18.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.091 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:29:18.091 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-12T03:29:18.091 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-12T03:29:18.091 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-12T03:29:18.091 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 1650, 2024-09-12T03:29:18.091 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1062114, 2024-09-12T03:29:18.092 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:18.092 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 1650, 2024-09-12T03:29:18.092 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-12T03:29:18.092 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-12T03:29:18.092 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 1650, 2024-09-12T03:29:18.092 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1062114, 2024-09-12T03:29:18.092 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-12T03:29:18.092 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.092 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.092 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.092 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:18.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.092 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-09-12T03:29:18.093 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-12T03:29:18.093 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-12T03:29:18.093 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-12T03:29:18.093 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-12T03:29:18.093 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-12T03:29:18.093 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:18.093 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-12T03:29:18.093 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 1459, 2024-09-12T03:29:18.093 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 24456, 2024-09-12T03:29:18.093 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 1459, 2024-09-12T03:29:18.093 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 24456, 2024-09-12T03:29:18.093 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-12T03:29:18.093 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.094 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.094 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.094 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:18.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.094 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-09-12T03:29:18.094 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-12T03:29:18.094 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-09-12T03:29:18.094 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-12T03:29:18.094 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-12T03:29:18.094 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-12T03:29:18.094 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:18.094 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-12T03:29:18.094 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 1459, 2024-09-12T03:29:18.094 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 1459, 2024-09-12T03:29:18.094 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 1459, 2024-09-12T03:29:18.095 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1459, 2024-09-12T03:29:18.095 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-12T03:29:18.095 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.095 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.095 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.095 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:18.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.095 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:29:18.095 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-12T03:29:18.095 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-12T03:29:18.095 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-12T03:29:18.095 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-12T03:29:18.095 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-12T03:29:18.095 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:18.095 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-12T03:29:18.095 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-12T03:29:18.096 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-12T03:29:18.096 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-09-12T03:29:18.096 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-09-12T03:29:18.096 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-12T03:29:18.096 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.096 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.096 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.096 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:18.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:18.096 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x55aa0e24ab40": { 2024-09-12T03:29:18.096 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-12T03:29:18.096 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-12T03:29:18.096 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-12T03:29:18.096 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-12T03:29:18.096 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-12T03:29:18.097 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:18.097 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-12T03:29:18.097 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-12T03:29:18.097 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-12T03:29:18.097 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-09-12T03:29:18.097 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-09-12T03:29:18.097 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-12T03:29:18.097 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:18.097 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:18.097 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:18.097 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:18.097 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:18.097 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-12T03:29:18.870 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-12T03:29:18.870 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-12T03:29:18.870 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-12T03:29:18.870 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-12T03:29:18.870 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-09-12T03:29:18.870 DEBUG:tasks.check_counter:Getting stats from i 2024-09-12T03:29:18.870 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-12T03:29:19.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:18 smithi161 ceph-mon[35485]: pgmap v581: 97 pgs: 97 active+clean; 159 MiB data, 110 GiB used, 963 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:29:19.132 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:29:19.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:18 smithi118 ceph-mon[35048]: pgmap v581: 97 pgs: 97 active+clean; 159 MiB data, 110 GiB used, 963 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:29:19.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:18 smithi006 ceph-mon[30991]: pgmap v581: 97 pgs: 97 active+clean; 159 MiB data, 110 GiB used, 963 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:29:19.651 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-12T03:29:19.651 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:29:19.651 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 61, 2024-09-12T03:29:19.651 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 65, 2024-09-12T03:29:19.651 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 13995, 2024-09-12T03:29:19.651 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 213044, 2024-09-12T03:29:19.651 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 25, 2024-09-12T03:29:19.652 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 7, 2024-09-12T03:29:19.652 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.013919304, 2024-09-12T03:29:19.652 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.001256508, 2024-09-12T03:29:19.652 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.008351474, 2024-09-12T03:29:19.652 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.001370527, 2024-09-12T03:29:19.652 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:19.652 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 61, 2024-09-12T03:29:19.652 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.020311952, 2024-09-12T03:29:19.652 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000332982 2024-09-12T03:29:19.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.652 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:19.652 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 21, 2024-09-12T03:29:19.653 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000004107, 2024-09-12T03:29:19.653 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000195 2024-09-12T03:29:19.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.653 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 928, 2024-09-12T03:29:19.653 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 736 2024-09-12T03:29:19.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.653 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:29:19.653 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 59, 2024-09-12T03:29:19.653 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 192, 2024-09-12T03:29:19.653 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 16759, 2024-09-12T03:29:19.653 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 360941, 2024-09-12T03:29:19.653 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 9, 2024-09-12T03:29:19.654 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 7, 2024-09-12T03:29:19.654 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.018823750, 2024-09-12T03:29:19.654 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.007998156, 2024-09-12T03:29:19.654 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.009069865, 2024-09-12T03:29:19.654 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000797336, 2024-09-12T03:29:19.654 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:19.654 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 190, 2024-09-12T03:29:19.654 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.022166048, 2024-09-12T03:29:19.654 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000116663 2024-09-12T03:29:19.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.654 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:19.654 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 61, 2024-09-12T03:29:19.654 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000011309, 2024-09-12T03:29:19.654 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000185 2024-09-12T03:29:19.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.655 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 9856, 2024-09-12T03:29:19.655 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 319120 2024-09-12T03:29:19.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.655 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:29:19.655 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 4759, 2024-09-12T03:29:19.655 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 2636, 2024-09-12T03:29:19.655 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 2255429, 2024-09-12T03:29:19.655 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 700845, 2024-09-12T03:29:19.655 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 10, 2024-09-12T03:29:19.655 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-09-12T03:29:19.655 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.228560599, 2024-09-12T03:29:19.655 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.082280693, 2024-09-12T03:29:19.656 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.221223822, 2024-09-12T03:29:19.656 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.003873101, 2024-09-12T03:29:19.656 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:19.656 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2638, 2024-09-12T03:29:19.656 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.233965506, 2024-09-12T03:29:19.656 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000088690 2024-09-12T03:29:19.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.656 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:19.656 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 6482, 2024-09-12T03:29:19.656 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.001502352, 2024-09-12T03:29:19.657 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000231 2024-09-12T03:29:19.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.657 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 795392, 2024-09-12T03:29:19.657 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 114496 2024-09-12T03:29:19.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.657 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-09-12T03:29:19.657 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-09-12T03:29:19.657 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-09-12T03:29:19.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.657 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-09-12T03:29:19.657 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-12T03:29:19.657 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-12T03:29:19.657 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 81, 2024-09-12T03:29:19.658 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.005824930, 2024-09-12T03:29:19.658 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000071912 2024-09-12T03:29:19.658 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:19.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.658 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-09-12T03:29:19.658 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-12T03:29:19.658 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-12T03:29:19.658 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 16, 2024-09-12T03:29:19.658 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000312101, 2024-09-12T03:29:19.658 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000019506 2024-09-12T03:29:19.658 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:19.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.659 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-09-12T03:29:19.659 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 13, 2024-09-12T03:29:19.659 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 4, 2024-09-12T03:29:19.659 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-09-12T03:29:19.659 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4, 2024-09-12T03:29:19.659 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.004673813, 2024-09-12T03:29:19.659 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001168453 2024-09-12T03:29:19.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.659 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-09-12T03:29:19.659 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 9, 2024-09-12T03:29:19.659 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-09-12T03:29:19.659 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-09-12T03:29:19.660 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 11, 2024-09-12T03:29:19.660 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-09-12T03:29:19.660 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-09-12T03:29:19.660 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 20, 2024-09-12T03:29:19.660 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 4, 2024-09-12T03:29:19.660 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 16, 2024-09-12T03:29:19.660 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-09-12T03:29:19.660 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 13, 2024-09-12T03:29:19.660 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 172, 2024-09-12T03:29:19.660 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 1, 2024-09-12T03:29:19.660 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 1, 2024-09-12T03:29:19.660 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 3, 2024-09-12T03:29:19.660 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 218, 2024-09-12T03:29:19.661 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 110, 2024-09-12T03:29:19.661 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 9, 2024-09-12T03:29:19.661 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 95, 2024-09-12T03:29:19.661 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:29:19.661 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-09-12T03:29:19.661 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-09-12T03:29:19.661 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-09-12T03:29:19.661 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-09-12T03:29:19.661 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 1, 2024-09-12T03:29:19.661 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-09-12T03:29:19.662 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 1, 2024-09-12T03:29:19.662 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 2, 2024-09-12T03:29:19.662 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-09-12T03:29:19.662 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 5, 2024-09-12T03:29:19.662 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-09-12T03:29:19.662 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 325, 2024-09-12T03:29:19.662 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 13424230450, 2024-09-12T03:29:19.662 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-09-12T03:29:19.662 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 5, 2024-09-12T03:29:19.662 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-09-12T03:29:19.662 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:29:19.662 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:29:19.662 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:29:19.662 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:29:19.663 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:29:19.663 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 5, 2024-09-12T03:29:19.663 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 2, 2024-09-12T03:29:19.663 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 9, 2024-09-12T03:29:19.663 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 6, 2024-09-12T03:29:19.663 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:29:19.663 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:29:19.663 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:29:19.663 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 9, 2024-09-12T03:29:19.663 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:29:19.663 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 3, 2024-09-12T03:29:19.663 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 2 2024-09-12T03:29:19.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.663 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-09-12T03:29:19.664 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 2, 2024-09-12T03:29:19.664 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 276, 2024-09-12T03:29:19.664 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 185, 2024-09-12T03:29:19.664 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 173, 2024-09-12T03:29:19.664 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 2, 2024-09-12T03:29:19.664 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-09-12T03:29:19.664 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-09-12T03:29:19.664 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:29:19.664 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 1, 2024-09-12T03:29:19.664 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 1, 2024-09-12T03:29:19.664 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-09-12T03:29:19.664 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-09-12T03:29:19.664 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-09-12T03:29:19.665 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:29:19.665 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:29:19.665 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-09-12T03:29:19.665 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-09-12T03:29:19.665 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 7, 2024-09-12T03:29:19.665 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 509, 2024-09-12T03:29:19.665 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:29:19.665 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 1, 2024-09-12T03:29:19.665 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-09-12T03:29:19.665 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:29:19.665 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-09-12T03:29:19.665 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-09-12T03:29:19.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.666 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-09-12T03:29:19.666 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-09-12T03:29:19.666 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 16, 2024-09-12T03:29:19.666 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-09-12T03:29:19.666 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-09-12T03:29:19.666 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 16, 2024-09-12T03:29:19.666 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-09-12T03:29:19.666 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-09-12T03:29:19.666 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 2, 2024-09-12T03:29:19.666 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-09-12T03:29:19.666 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-09-12T03:29:19.666 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 2, 2024-09-12T03:29:19.666 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 2, 2024-09-12T03:29:19.667 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-09-12T03:29:19.667 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-09-12T03:29:19.667 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-09-12T03:29:19.667 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 4218755, 2024-09-12T03:29:19.667 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2024-09-12T03:29:19.667 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-09-12T03:29:19.667 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 14, 2024-09-12T03:29:19.667 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.503514574, 2024-09-12T03:29:19.667 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.035965326 2024-09-12T03:29:19.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.667 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2024-09-12T03:29:19.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.668 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-09-12T03:29:19.668 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 24, 2024-09-12T03:29:19.668 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 111, 2024-09-12T03:29:19.668 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 87, 2024-09-12T03:29:19.668 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 23, 2024-09-12T03:29:19.668 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 119, 2024-09-12T03:29:19.668 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 96, 2024-09-12T03:29:19.668 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 20, 2024-09-12T03:29:19.668 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 195, 2024-09-12T03:29:19.668 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 175, 2024-09-12T03:29:19.668 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 1, 2024-09-12T03:29:19.668 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 5, 2024-09-12T03:29:19.668 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 4, 2024-09-12T03:29:19.669 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 53016, 2024-09-12T03:29:19.669 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 223520 2024-09-12T03:29:19.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.669 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-09-12T03:29:19.669 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 19, 2024-09-12T03:29:19.669 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 4, 2024-09-12T03:29:19.669 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 13, 2024-09-12T03:29:19.669 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 11, 2024-09-12T03:29:19.669 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 10, 2024-09-12T03:29:19.669 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-09-12T03:29:19.669 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.669 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.670 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.670 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-09-12T03:29:19.670 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.670 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.670 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.670 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-09-12T03:29:19.670 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.670 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.670 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.670 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-09-12T03:29:19.671 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.671 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.671 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.671 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-09-12T03:29:19.671 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.671 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.671 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.671 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-09-12T03:29:19.671 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.671 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.671 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.671 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-09-12T03:29:19.672 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1, 2024-09-12T03:29:19.672 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000118547, 2024-09-12T03:29:19.672 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000118547 2024-09-12T03:29:19.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.672 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-09-12T03:29:19.672 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.672 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.672 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.672 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-09-12T03:29:19.672 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.672 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.672 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.673 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-09-12T03:29:19.673 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.673 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.673 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.673 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-09-12T03:29:19.673 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.673 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.673 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.673 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-09-12T03:29:19.674 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.674 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.674 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.674 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-09-12T03:29:19.674 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.674 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.674 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.674 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-09-12T03:29:19.674 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.674 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.674 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.675 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-09-12T03:29:19.675 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.675 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.675 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.675 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-09-12T03:29:19.675 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2024-09-12T03:29:19.675 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000755069, 2024-09-12T03:29:19.675 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000377534 2024-09-12T03:29:19.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.676 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-09-12T03:29:19.676 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.676 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.676 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.676 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-09-12T03:29:19.676 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.676 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.677 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.677 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-09-12T03:29:19.677 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.677 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.677 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.677 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-09-12T03:29:19.677 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.677 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.677 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.677 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-09-12T03:29:19.678 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.678 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.678 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.678 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-09-12T03:29:19.678 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.678 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.678 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.678 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-09-12T03:29:19.678 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.678 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.678 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.679 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-09-12T03:29:19.679 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.679 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.679 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.679 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-09-12T03:29:19.679 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.679 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.679 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.679 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-09-12T03:29:19.679 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.679 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.680 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.680 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-09-12T03:29:19.680 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.680 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.680 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.680 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-09-12T03:29:19.680 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.680 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.680 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.680 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-09-12T03:29:19.680 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1, 2024-09-12T03:29:19.681 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.003800197, 2024-09-12T03:29:19.681 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.003800197 2024-09-12T03:29:19.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.681 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:29:19.681 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:29:19.681 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-09-12T03:29:19.681 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.681 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.681 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.681 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:19.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.681 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-09-12T03:29:19.681 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-09-12T03:29:19.682 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 2, 2024-09-12T03:29:19.682 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 1, 2024-09-12T03:29:19.682 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 1, 2024-09-12T03:29:19.682 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-09-12T03:29:19.682 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 2, 2024-09-12T03:29:19.682 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 2, 2024-09-12T03:29:19.682 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 482, 2024-09-12T03:29:19.682 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-09-12T03:29:19.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.682 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-09-12T03:29:19.682 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:29:19.682 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-09-12T03:29:19.682 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:29:19.683 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:29:19.683 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:29:19.683 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:29:19.683 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:29:19.683 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:29:19.683 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:29:19.683 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:29:19.683 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:29:19.683 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:29:19.683 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:29:19.683 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:29:19.683 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:29:19.683 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-09-12T03:29:19.684 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:29:19.684 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-09-12T03:29:19.684 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:29:19.684 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:29:19.684 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:29:19.684 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:29:19.684 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:29:19.684 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:29:19.684 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:29:19.684 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-09-12T03:29:19.684 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:29:19.684 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:29:19.684 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:29:19.685 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-09-12T03:29:19.685 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-09-12T03:29:19.685 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-09-12T03:29:19.685 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:29:19.685 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:29:19.685 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:29:19.685 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:29:19.685 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 263559, 2024-09-12T03:29:19.685 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 93, 2024-09-12T03:29:19.685 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:29:19.685 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-09-12T03:29:19.685 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-09-12T03:29:19.685 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-09-12T03:29:19.685 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:29:19.686 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-09-12T03:29:19.686 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:29:19.686 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-09-12T03:29:19.686 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 7400, 2024-09-12T03:29:19.686 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 152, 2024-09-12T03:29:19.686 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:29:19.686 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:29:19.686 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-09-12T03:29:19.686 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-09-12T03:29:19.686 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 98365, 2024-09-12T03:29:19.686 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 394, 2024-09-12T03:29:19.686 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-09-12T03:29:19.686 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-09-12T03:29:19.686 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-09-12T03:29:19.687 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-09-12T03:29:19.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.687 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-09-12T03:29:19.687 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-09-12T03:29:19.687 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-09-12T03:29:19.687 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 76, 2024-09-12T03:29:19.687 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 36989, 2024-09-12T03:29:19.687 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 28, 2024-09-12T03:29:19.687 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 76, 2024-09-12T03:29:19.687 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-09-12T03:29:19.687 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 76, 2024-09-12T03:29:19.687 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1.027977067, 2024-09-12T03:29:19.687 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.013526014 2024-09-12T03:29:19.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.688 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-09-12T03:29:19.688 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-09-12T03:29:19.688 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 76, 2024-09-12T03:29:19.688 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 106 2024-09-12T03:29:19.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.688 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 76, 2024-09-12T03:29:19.688 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 6, 2024-09-12T03:29:19.688 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 70, 2024-09-12T03:29:19.688 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-09-12T03:29:19.688 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-09-12T03:29:19.688 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 0, 2024-09-12T03:29:19.688 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 10, 2024-09-12T03:29:19.689 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 1, 2024-09-12T03:29:19.689 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 15, 2024-09-12T03:29:19.689 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 12, 2024-09-12T03:29:19.689 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-09-12T03:29:19.689 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-09-12T03:29:19.689 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-09-12T03:29:19.689 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-09-12T03:29:19.689 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 13, 2024-09-12T03:29:19.689 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-09-12T03:29:19.689 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-09-12T03:29:19.689 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-09-12T03:29:19.689 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 5, 2024-09-12T03:29:19.689 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 20, 2024-09-12T03:29:19.689 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:29:19.690 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-09-12T03:29:19.690 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:29:19.690 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-09-12T03:29:19.690 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-09-12T03:29:19.690 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-09-12T03:29:19.690 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:29:19.690 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-09-12T03:29:19.690 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:29:19.690 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-09-12T03:29:19.690 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-09-12T03:29:19.690 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-09-12T03:29:19.690 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-09-12T03:29:19.690 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-09-12T03:29:19.690 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-09-12T03:29:19.691 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-09-12T03:29:19.691 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-09-12T03:29:19.691 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-09-12T03:29:19.691 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-09-12T03:29:19.691 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-09-12T03:29:19.691 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-09-12T03:29:19.691 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-09-12T03:29:19.691 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-09-12T03:29:19.691 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-09-12T03:29:19.691 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-09-12T03:29:19.691 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-09-12T03:29:19.691 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 82, 2024-09-12T03:29:19.692 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-09-12T03:29:19.692 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 2, 2024-09-12T03:29:19.692 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 10, 2024-09-12T03:29:19.692 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 10, 2024-09-12T03:29:19.692 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-09-12T03:29:19.692 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-09-12T03:29:19.692 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 26, 2024-09-12T03:29:19.692 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 0, 2024-09-12T03:29:19.692 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 4 2024-09-12T03:29:19.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.692 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-09-12T03:29:19.692 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2024-09-12T03:29:19.692 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:29:19.693 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-09-12T03:29:19.693 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-09-12T03:29:19.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.693 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-09-12T03:29:19.693 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 431 2024-09-12T03:29:19.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.693 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-09-12T03:29:19.693 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-09-12T03:29:19.693 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 1, 2024-09-12T03:29:19.693 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-09-12T03:29:19.693 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 1, 2024-09-12T03:29:19.693 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 1, 2024-09-12T03:29:19.693 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 1, 2024-09-12T03:29:19.694 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-09-12T03:29:19.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.694 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:29:19.694 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-12T03:29:19.694 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-12T03:29:19.694 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-12T03:29:19.694 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 4875, 2024-09-12T03:29:19.694 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1831409, 2024-09-12T03:29:19.694 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:19.694 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 4875, 2024-09-12T03:29:19.694 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-12T03:29:19.694 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-12T03:29:19.694 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 4875, 2024-09-12T03:29:19.694 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1831409, 2024-09-12T03:29:19.695 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-12T03:29:19.695 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.695 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.695 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.695 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:19.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.695 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-09-12T03:29:19.695 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-12T03:29:19.695 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-12T03:29:19.695 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-12T03:29:19.695 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-12T03:29:19.695 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-12T03:29:19.695 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:19.695 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-12T03:29:19.696 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 76, 2024-09-12T03:29:19.696 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 37090, 2024-09-12T03:29:19.696 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 76, 2024-09-12T03:29:19.696 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 37090, 2024-09-12T03:29:19.696 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-12T03:29:19.696 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.696 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.696 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.696 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:19.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.696 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-09-12T03:29:19.696 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-12T03:29:19.697 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-09-12T03:29:19.697 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-12T03:29:19.697 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-12T03:29:19.697 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-12T03:29:19.697 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:19.697 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-12T03:29:19.697 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 76, 2024-09-12T03:29:19.697 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 76, 2024-09-12T03:29:19.697 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 76, 2024-09-12T03:29:19.697 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 76, 2024-09-12T03:29:19.697 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-12T03:29:19.697 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.697 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.698 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.698 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:19.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.698 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:29:19.698 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-12T03:29:19.698 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-12T03:29:19.698 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-12T03:29:19.698 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 1, 2024-09-12T03:29:19.698 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 101, 2024-09-12T03:29:19.698 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:19.698 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 1, 2024-09-12T03:29:19.698 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-12T03:29:19.698 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-12T03:29:19.698 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 1, 2024-09-12T03:29:19.699 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 101, 2024-09-12T03:29:19.699 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-12T03:29:19.699 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.699 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.699 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.699 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:19.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:19.699 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x55b79382e960": { 2024-09-12T03:29:19.699 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-12T03:29:19.699 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-12T03:29:19.699 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-12T03:29:19.699 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 16, 2024-09-12T03:29:19.699 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 24451, 2024-09-12T03:29:19.700 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:19.700 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 16, 2024-09-12T03:29:19.700 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-12T03:29:19.700 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-12T03:29:19.700 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 14, 2024-09-12T03:29:19.700 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 24451, 2024-09-12T03:29:19.700 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-12T03:29:19.700 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:19.700 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:19.700 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:19.700 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:19.700 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:19.700 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-12T03:29:20.475 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-09-12T03:29:20.476 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-09-12T03:29:20.476 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=2 2024-09-12T03:29:20.476 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=276 2024-09-12T03:29:20.476 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-09-12T03:29:20.476 DEBUG:tasks.check_counter:Getting stats from l 2024-09-12T03:29:20.477 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-12T03:29:20.736 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:29:21.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:20 smithi161 ceph-mon[35485]: pgmap v582: 97 pgs: 97 active+clean; 159 MiB data, 110 GiB used, 963 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:29:21.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:20 smithi118 ceph-mon[35048]: pgmap v582: 97 pgs: 97 active+clean; 159 MiB data, 110 GiB used, 963 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:29:21.287 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-12T03:29:21.288 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:29:21.288 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 0, 2024-09-12T03:29:21.288 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 2, 2024-09-12T03:29:21.288 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:29:21.288 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 0, 2024-09-12T03:29:21.288 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-09-12T03:29:21.288 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 0, 2024-09-12T03:29:21.288 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.001021131, 2024-09-12T03:29:21.288 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:29:21.288 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.000181132, 2024-09-12T03:29:21.288 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:29:21.289 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:21.289 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:21.289 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:21.289 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:21.289 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:21.289 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:21.289 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:21.289 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:21.289 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:21.289 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:21.289 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:29:21.289 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:29:21.290 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:21.290 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:29:21.290 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 5, 2024-09-12T03:29:21.290 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 136, 2024-09-12T03:29:21.290 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 3232, 2024-09-12T03:29:21.290 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 102624, 2024-09-12T03:29:21.290 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 3, 2024-09-12T03:29:21.290 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2024-09-12T03:29:21.290 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.009423858, 2024-09-12T03:29:21.290 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.005373354, 2024-09-12T03:29:21.291 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.002675152, 2024-09-12T03:29:21.291 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:29:21.291 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:21.291 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 135, 2024-09-12T03:29:21.291 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.011175324, 2024-09-12T03:29:21.291 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000082780 2024-09-12T03:29:21.291 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:21.291 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:21.291 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:21.291 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:21.291 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:21.291 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:21.291 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-12T03:29:21.291 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 102624 2024-09-12T03:29:21.292 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:21.292 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:29:21.292 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 181, 2024-09-12T03:29:21.292 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 176, 2024-09-12T03:29:21.292 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 742736, 2024-09-12T03:29:21.292 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 107728, 2024-09-12T03:29:21.292 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 2, 2024-09-12T03:29:21.292 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 1, 2024-09-12T03:29:21.292 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.030528845, 2024-09-12T03:29:21.292 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.011259622, 2024-09-12T03:29:21.292 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.025711705, 2024-09-12T03:29:21.292 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:29:21.293 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:29:21.293 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 175, 2024-09-12T03:29:21.293 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.009504422, 2024-09-12T03:29:21.293 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000054310 2024-09-12T03:29:21.293 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:21.293 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:29:21.293 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:21.293 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:21.293 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:21.293 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:21.293 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 742736, 2024-09-12T03:29:21.293 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 107728 2024-09-12T03:29:21.293 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:21.294 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-09-12T03:29:21.294 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 0, 2024-09-12T03:29:21.294 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-09-12T03:29:21.294 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:21.294 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-09-12T03:29:21.294 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:29:21.294 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-09-12T03:29:21.294 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:29:21.294 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:29:21.294 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:29:21.294 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:29:21.295 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:29:21.295 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:29:21.295 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:29:21.295 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:29:21.295 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:29:21.296 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:29:21.296 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:29:21.296 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:29:21.296 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:29:21.296 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-09-12T03:29:21.296 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:29:21.296 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-09-12T03:29:21.296 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:29:21.296 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:29:21.296 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:29:21.296 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:29:21.296 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:29:21.297 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:29:21.297 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:29:21.297 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-09-12T03:29:21.297 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:29:21.297 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:29:21.297 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:29:21.297 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-09-12T03:29:21.297 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-09-12T03:29:21.297 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-09-12T03:29:21.297 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:29:21.297 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:29:21.297 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:29:21.298 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:29:21.298 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 148967, 2024-09-12T03:29:21.298 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 29, 2024-09-12T03:29:21.298 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:29:21.298 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-09-12T03:29:21.298 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-09-12T03:29:21.298 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-09-12T03:29:21.298 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:29:21.298 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-09-12T03:29:21.298 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:29:21.299 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-09-12T03:29:21.299 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 0, 2024-09-12T03:29:21.299 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 0, 2024-09-12T03:29:21.299 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:29:21.299 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:29:21.299 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-09-12T03:29:21.299 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-09-12T03:29:21.299 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 928, 2024-09-12T03:29:21.299 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 2, 2024-09-12T03:29:21.299 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-09-12T03:29:21.300 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-09-12T03:29:21.300 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-09-12T03:29:21.300 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-09-12T03:29:21.300 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-12T03:29:21.300 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:29:21.300 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-12T03:29:21.300 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-12T03:29:21.300 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-12T03:29:21.301 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 182, 2024-09-12T03:29:21.301 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 717189, 2024-09-12T03:29:21.301 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-12T03:29:21.301 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 182, 2024-09-12T03:29:21.301 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-12T03:29:21.301 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-12T03:29:21.301 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 182, 2024-09-12T03:29:21.301 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 717189, 2024-09-12T03:29:21.301 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-12T03:29:21.301 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-12T03:29:21.301 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-12T03:29:21.302 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-12T03:29:21.302 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:21.302 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-12T03:29:21.302 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-12T03:29:21.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:20 smithi006 ceph-mon[30991]: pgmap v582: 97 pgs: 97 active+clean; 159 MiB data, 110 GiB used, 963 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:29:22.034 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-12T03:29:22.034 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-12T03:29:22.034 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-09-12T03:29:22.034 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-09-12T03:29:22.034 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-09-12T03:29:22.035 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-12T03:29:22.045 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-12T03:29:22.045 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-09-12T03:29:22.046 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:29:22.063 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-09-12T03:29:22.064 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:29:22.120 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T03:29:22.418 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:22.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.890+0000 7f032651b640 1 -- 172.21.15.6:0/3766278438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0320102790 msgr2=0x7f03201067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:22.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.890+0000 7f032651b640 1 --2- 172.21.15.6:0/3766278438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0320102790 0x7f03201067e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0314009900 tx=0x7f031402a410 comp rx=0 tx=0).stop 2024-09-12T03:29:22.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.890+0000 7f032651b640 1 -- 172.21.15.6:0/3766278438 shutdown_connections 2024-09-12T03:29:22.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.890+0000 7f032651b640 1 --2- 172.21.15.6:0/3766278438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0320106f10 0x7f032010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:22.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.890+0000 7f032651b640 1 --2- 172.21.15.6:0/3766278438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0320102790 0x7f03201067e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:22.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.890+0000 7f032651b640 1 --2- 172.21.15.6:0/3766278438 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0320101de0 0x7f03201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:22.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.890+0000 7f032651b640 1 -- 172.21.15.6:0/3766278438 >> 172.21.15.6:0/3766278438 conn(0x7f03200fd8c0 msgr2=0x7f03200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:22.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.890+0000 7f032651b640 1 -- 172.21.15.6:0/3766278438 shutdown_connections 2024-09-12T03:29:22.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.890+0000 7f032651b640 1 -- 172.21.15.6:0/3766278438 wait complete. 2024-09-12T03:29:22.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.891+0000 7f032651b640 1 Processor -- start 2024-09-12T03:29:22.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.891+0000 7f032651b640 1 -- start start 2024-09-12T03:29:22.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.891+0000 7f032651b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0320101de0 0x7f03200744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:22.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.891+0000 7f032651b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0320102790 0x7f0320074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:22.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.891+0000 7f032651b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0320106f10 0x7f0320074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:22.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.892+0000 7f032651b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0320111f30 con 0x7f0320106f10 2024-09-12T03:29:22.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.892+0000 7f032651b640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f0320111db0 con 0x7f0320102790 2024-09-12T03:29:22.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.892+0000 7f032651b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f03201120b0 con 0x7f0320101de0 2024-09-12T03:29:22.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.892+0000 7f0324ab8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0320106f10 0x7f0320074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:22.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.892+0000 7f031ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0320101de0 0x7f03200744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:22.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.892+0000 7f0324ab8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0320106f10 0x7f0320074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44766/0 (socket says 172.21.15.6:44766) 2024-09-12T03:29:22.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.892+0000 7f0324ab8640 1 -- 172.21.15.6:0/236192772 learned_addr learned my addr 172.21.15.6:0/236192772 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:29:22.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.892+0000 7f031f7fe640 1 --2- 172.21.15.6:0/236192772 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0320102790 0x7f0320074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:22.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.892+0000 7f0324ab8640 1 -- 172.21.15.6:0/236192772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0320101de0 msgr2=0x7f03200744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:22.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.892+0000 7f0324ab8640 1 --2- 172.21.15.6:0/236192772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0320101de0 0x7f03200744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:22.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.892+0000 7f0324ab8640 1 -- 172.21.15.6:0/236192772 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0320102790 msgr2=0x7f0320074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:22.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.892+0000 7f0324ab8640 1 --2- 172.21.15.6:0/236192772 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0320102790 0x7f0320074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:22.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.892+0000 7f0324ab8640 1 -- 172.21.15.6:0/236192772 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03201a6860 con 0x7f0320106f10 2024-09-12T03:29:22.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.892+0000 7f0324ab8640 1 --2- 172.21.15.6:0/236192772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0320106f10 0x7f0320074f70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f031800eb10 tx=0x7f031800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:22.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.897+0000 7f031d7fa640 1 -- 172.21.15.6:0/236192772 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031800ce20 con 0x7f0320106f10 2024-09-12T03:29:22.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.897+0000 7f031d7fa640 1 -- 172.21.15.6:0/236192772 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0318005020 con 0x7f0320106f10 2024-09-12T03:29:22.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.897+0000 7f032651b640 1 -- 172.21.15.6:0/236192772 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03201a6b50 con 0x7f0320106f10 2024-09-12T03:29:22.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.898+0000 7f031d7fa640 1 -- 172.21.15.6:0/236192772 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318010690 con 0x7f0320106f10 2024-09-12T03:29:22.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.898+0000 7f032651b640 1 -- 172.21.15.6:0/236192772 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03201a7000 con 0x7f0320106f10 2024-09-12T03:29:22.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.899+0000 7f032651b640 1 -- 172.21.15.6:0/236192772 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03201021c0 con 0x7f0320106f10 2024-09-12T03:29:22.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.900+0000 7f031d7fa640 1 -- 172.21.15.6:0/236192772 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0318005540 con 0x7f0320106f10 2024-09-12T03:29:22.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.901+0000 7f031d7fa640 1 --2- 172.21.15.6:0/236192772 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f03080a2d50 0x7f03080a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:22.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.901+0000 7f031ffff640 1 --2- 172.21.15.6:0/236192772 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f03080a2d50 0x7f03080a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:22.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.901+0000 7f031d7fa640 1 -- 172.21.15.6:0/236192772 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f03180cf440 con 0x7f0320106f10 2024-09-12T03:29:22.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.901+0000 7f031ffff640 1 --2- 172.21.15.6:0/236192772 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f03080a2d50 0x7f03080a5210 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f031000abe0 tx=0x7f031000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:22.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:22.905+0000 7f031d7fa640 1 -- 172.21.15.6:0/236192772 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f031809c370 con 0x7f0320106f10 2024-09-12T03:29:23.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:23.093+0000 7f032651b640 1 -- 172.21.15.6:0/236192772 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f032010a0f0 con 0x7f0320106f10 2024-09-12T03:29:23.094 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:22 smithi006 ceph-mon[30991]: pgmap v583: 97 pgs: 97 active+clean; 159 MiB data, 110 GiB used, 963 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-12T03:29:23.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:23.093+0000 7f031d7fa640 1 -- 172.21.15.6:0/236192772 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v82) ==== 81+0+579 (secure 0 0 0) 0x7f031809bc70 con 0x7f0320106f10 2024-09-12T03:29:23.094 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/1684477653 2024-09-13T03:12:19.155264+0000 2024-09-12T03:29:23.094 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2620491916 2024-09-13T03:12:19.155264+0000 2024-09-12T03:29:23.094 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/4037656392 2024-09-13T03:12:19.155264+0000 2024-09-12T03:29:23.095 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/1377116573 2024-09-13T03:12:19.155264+0000 2024-09-12T03:29:23.095 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3062800125 2024-09-13T03:11:55.244257+0000 2024-09-12T03:29:23.095 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/1684477653 2024-09-13T03:12:19.155264+0000 2024-09-12T03:29:23.095 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/836933784 2024-09-13T03:11:55.244257+0000 2024-09-12T03:29:23.095 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/836933784 2024-09-13T03:11:55.244257+0000 2024-09-12T03:29:23.095 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/4145338968 2024-09-13T03:11:55.244257+0000 2024-09-12T03:29:23.095 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/825701128 2024-09-13T03:11:55.244257+0000 2024-09-12T03:29:23.096 INFO:teuthology.orchestra.run.smithi006.stderr:listed 10 entries 2024-09-12T03:29:23.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:23.095+0000 7f032651b640 1 -- 172.21.15.6:0/236192772 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f03080a2d50 msgr2=0x7f03080a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:23.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:23.095+0000 7f032651b640 1 --2- 172.21.15.6:0/236192772 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f03080a2d50 0x7f03080a5210 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f031000abe0 tx=0x7f031000a2d0 comp rx=0 tx=0).stop 2024-09-12T03:29:23.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:23.095+0000 7f032651b640 1 -- 172.21.15.6:0/236192772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0320106f10 msgr2=0x7f0320074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:23.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:23.095+0000 7f032651b640 1 --2- 172.21.15.6:0/236192772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0320106f10 0x7f0320074f70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f031800eb10 tx=0x7f031800ef10 comp rx=0 tx=0).stop 2024-09-12T03:29:23.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:23.096+0000 7f032651b640 1 -- 172.21.15.6:0/236192772 shutdown_connections 2024-09-12T03:29:23.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:23.096+0000 7f032651b640 1 --2- 172.21.15.6:0/236192772 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f03080a2d50 0x7f03080a5210 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:23.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:23.096+0000 7f032651b640 1 --2- 172.21.15.6:0/236192772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0320106f10 0x7f0320074f70 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:23.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:23.096+0000 7f032651b640 1 --2- 172.21.15.6:0/236192772 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0320102790 0x7f0320074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:23.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:23.096+0000 7f032651b640 1 --2- 172.21.15.6:0/236192772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0320101de0 0x7f03200744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:23.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:23.096+0000 7f032651b640 1 -- 172.21.15.6:0/236192772 >> 172.21.15.6:0/236192772 conn(0x7f03200fd8c0 msgr2=0x7f03200fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:23.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:23.096+0000 7f032651b640 1 -- 172.21.15.6:0/236192772 shutdown_connections 2024-09-12T03:29:23.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:23.096+0000 7f032651b640 1 -- 172.21.15.6:0/236192772 wait complete. 2024-09-12T03:29:23.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:22 smithi118 ceph-mon[35048]: pgmap v583: 97 pgs: 97 active+clean; 159 MiB data, 110 GiB used, 963 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-12T03:29:23.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:22 smithi161 ceph-mon[35485]: pgmap v583: 97 pgs: 97 active+clean; 159 MiB data, 110 GiB used, 963 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-12T03:29:23.828 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T03:29:23.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:29:23.829 DEBUG:teuthology.orchestra.run.smithi006:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T03:29:23.871 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2024-09-12T03:29:23.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:29:23.871 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T03:29:23.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:29:23.952 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2024-09-12T03:29:24.029 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T03:29:24.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:29:24.030 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:29:24.093 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:23 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/236192772' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:29:24.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:29:24.098 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link delete ceph-brx 2024-09-12T03:29:24.201 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-12T03:29:24.211 INFO:tasks.cephadm:Teardown begin 2024-09-12T03:29:24.211 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T03:29:24.211 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:29:24.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:23 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/236192772' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:29:24.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:23 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/236192772' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:29:24.477 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:25.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.014+0000 7f2b8e2b4640 1 -- 172.21.15.6:0/527975034 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2b88101de0 msgr2=0x7f2b881021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:25.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.014+0000 7f2b8e2b4640 1 --2- 172.21.15.6:0/527975034 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2b88101de0 0x7f2b881021c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2b78009900 tx=0x7f2b7802a410 comp rx=0 tx=0).stop 2024-09-12T03:29:25.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.014+0000 7f2b8e2b4640 1 -- 172.21.15.6:0/527975034 shutdown_connections 2024-09-12T03:29:25.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.014+0000 7f2b8e2b4640 1 --2- 172.21.15.6:0/527975034 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b88106f10 0x7f2b8810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:25.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.014+0000 7f2b8e2b4640 1 --2- 172.21.15.6:0/527975034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b88102790 0x7f2b881067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:25.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.014+0000 7f2b8e2b4640 1 --2- 172.21.15.6:0/527975034 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2b88101de0 0x7f2b881021c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:25.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.014+0000 7f2b8e2b4640 1 -- 172.21.15.6:0/527975034 >> 172.21.15.6:0/527975034 conn(0x7f2b880fd8e0 msgr2=0x7f2b880ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:25.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.014+0000 7f2b8e2b4640 1 -- 172.21.15.6:0/527975034 shutdown_connections 2024-09-12T03:29:25.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.014+0000 7f2b8e2b4640 1 -- 172.21.15.6:0/527975034 wait complete. 2024-09-12T03:29:25.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.015+0000 7f2b8e2b4640 1 Processor -- start 2024-09-12T03:29:25.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.015+0000 7f2b8e2b4640 1 -- start start 2024-09-12T03:29:25.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.015+0000 7f2b8e2b4640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b88101de0 0x7f2b88198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:25.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.015+0000 7f2b8e2b4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2b88102790 0x7f2b88198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:25.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.015+0000 7f2b8e2b4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b88106f10 0x7f2b8819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:25.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.015+0000 7f2b8e2b4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2b88111f70 con 0x7f2b88102790 2024-09-12T03:29:25.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.015+0000 7f2b8e2b4640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f2b88111df0 con 0x7f2b88101de0 2024-09-12T03:29:25.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.015+0000 7f2b8e2b4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2b881120f0 con 0x7f2b88106f10 2024-09-12T03:29:25.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.015+0000 7f2b7ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2b88102790 0x7f2b88198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:25.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.015+0000 7f2b7ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2b88102790 0x7f2b88198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44786/0 (socket says 172.21.15.6:44786) 2024-09-12T03:29:25.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.015+0000 7f2b7ffff640 1 -- 172.21.15.6:0/3203768536 learned_addr learned my addr 172.21.15.6:0/3203768536 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:29:25.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.015+0000 7f2b8c851640 1 --2- 172.21.15.6:0/3203768536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b88106f10 0x7f2b8819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:25.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.016+0000 7f2b87fff640 1 --2- 172.21.15.6:0/3203768536 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b88101de0 0x7f2b88198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:25.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.016+0000 7f2b7ffff640 1 -- 172.21.15.6:0/3203768536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b88106f10 msgr2=0x7f2b8819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:25.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.016+0000 7f2b7ffff640 1 --2- 172.21.15.6:0/3203768536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b88106f10 0x7f2b8819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:25.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.016+0000 7f2b7ffff640 1 -- 172.21.15.6:0/3203768536 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b88101de0 msgr2=0x7f2b88198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:25.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.016+0000 7f2b7ffff640 1 --2- 172.21.15.6:0/3203768536 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b88101de0 0x7f2b88198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:25.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.016+0000 7f2b7ffff640 1 -- 172.21.15.6:0/3203768536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8819d4b0 con 0x7f2b88102790 2024-09-12T03:29:25.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.016+0000 7f2b7ffff640 1 --2- 172.21.15.6:0/3203768536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2b88102790 0x7f2b88198a40 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000d950 tx=0x7f2b7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:25.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.016+0000 7f2b85ffb640 1 -- 172.21.15.6:0/3203768536 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70017070 con 0x7f2b88102790 2024-09-12T03:29:25.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.016+0000 7f2b8e2b4640 1 -- 172.21.15.6:0/3203768536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8819d7a0 con 0x7f2b88102790 2024-09-12T03:29:25.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.016+0000 7f2b85ffb640 1 -- 172.21.15.6:0/3203768536 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b70009c30 con 0x7f2b88102790 2024-09-12T03:29:25.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.016+0000 7f2b85ffb640 1 -- 172.21.15.6:0/3203768536 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7000f6b0 con 0x7f2b88102790 2024-09-12T03:29:25.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.016+0000 7f2b8e2b4640 1 -- 172.21.15.6:0/3203768536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b88074420 con 0x7f2b88102790 2024-09-12T03:29:25.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.018+0000 7f2b8e2b4640 1 -- 172.21.15.6:0/3203768536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b881021c0 con 0x7f2b88102790 2024-09-12T03:29:25.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.019+0000 7f2b85ffb640 1 -- 172.21.15.6:0/3203768536 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2b700109d0 con 0x7f2b88102790 2024-09-12T03:29:25.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.020+0000 7f2b85ffb640 1 --2- 172.21.15.6:0/3203768536 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2b740a2c00 0x7f2b740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:25.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.020+0000 7f2b85ffb640 1 -- 172.21.15.6:0/3203768536 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f2b700ceea0 con 0x7f2b88102790 2024-09-12T03:29:25.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.020+0000 7f2b87fff640 1 --2- 172.21.15.6:0/3203768536 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2b740a2c00 0x7f2b740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:25.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.021+0000 7f2b87fff640 1 --2- 172.21.15.6:0/3203768536 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2b740a2c00 0x7f2b740a50c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2b78009230 tx=0x7f2b780058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:25.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.025+0000 7f2b85ffb640 1 -- 172.21.15.6:0/3203768536 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2b70009de0 con 0x7f2b88102790 2024-09-12T03:29:25.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:24 smithi118 ceph-mon[35048]: pgmap v584: 97 pgs: 97 active+clean; 159 MiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:29:25.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:24 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:29:25.262 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:24 smithi006 ceph-mon[30991]: pgmap v584: 97 pgs: 97 active+clean; 159 MiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:29:25.262 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:24 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:29:25.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.262+0000 7f2b8e2b4640 1 -- 172.21.15.6:0/3203768536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b8810a0f0 con 0x7f2b88102790 2024-09-12T03:29:25.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.263+0000 7f2b85ffb640 1 -- 172.21.15.6:0/3203768536 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11910 (secure 0 0 0) 0x7f2b7009b7e0 con 0x7f2b88102790 2024-09-12T03:29:25.263 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:29:25.264 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":29,"btime":"2024-09-12T03:28:30:753620+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:28:30.102891+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24463]},"id":1}]} 2024-09-12T03:29:25.264 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 29 2024-09-12T03:29:25.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.265+0000 7f2b8e2b4640 1 -- 172.21.15.6:0/3203768536 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2b740a2c00 msgr2=0x7f2b740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:25.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.265+0000 7f2b8e2b4640 1 --2- 172.21.15.6:0/3203768536 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2b740a2c00 0x7f2b740a50c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2b78009230 tx=0x7f2b780058d0 comp rx=0 tx=0).stop 2024-09-12T03:29:25.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.265+0000 7f2b8e2b4640 1 -- 172.21.15.6:0/3203768536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2b88102790 msgr2=0x7f2b88198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:25.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.265+0000 7f2b8e2b4640 1 --2- 172.21.15.6:0/3203768536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2b88102790 0x7f2b88198a40 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000d950 tx=0x7f2b7000de10 comp rx=0 tx=0).stop 2024-09-12T03:29:25.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.265+0000 7f2b8e2b4640 1 -- 172.21.15.6:0/3203768536 shutdown_connections 2024-09-12T03:29:25.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.265+0000 7f2b8e2b4640 1 --2- 172.21.15.6:0/3203768536 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f2b740a2c00 0x7f2b740a50c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:25.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.265+0000 7f2b8e2b4640 1 --2- 172.21.15.6:0/3203768536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b88106f10 0x7f2b8819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:25.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.265+0000 7f2b8e2b4640 1 --2- 172.21.15.6:0/3203768536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2b88102790 0x7f2b88198a40 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:25.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.265+0000 7f2b8e2b4640 1 --2- 172.21.15.6:0/3203768536 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b88101de0 0x7f2b88198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:25.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.265+0000 7f2b8e2b4640 1 -- 172.21.15.6:0/3203768536 >> 172.21.15.6:0/3203768536 conn(0x7f2b880fd8e0 msgr2=0x7f2b880fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:25.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.265+0000 7f2b8e2b4640 1 -- 172.21.15.6:0/3203768536 shutdown_connections 2024-09-12T03:29:25.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:25.265+0000 7f2b8e2b4640 1 -- 172.21.15.6:0/3203768536 wait complete. 2024-09-12T03:29:25.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:24 smithi161 ceph-mon[35485]: pgmap v584: 97 pgs: 97 active+clean; 159 MiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:29:25.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:24 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:29:25.997 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:29:26.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:25 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:29:26.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:25 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:29:26.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:25 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:29:26.212 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:25 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3203768536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:29:26.265 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:26.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:25 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:29:26.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:25 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:29:26.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:25 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:29:26.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:25 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3203768536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:29:26.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:25 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:29:26.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:25 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:29:26.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:25 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' 2024-09-12T03:29:26.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:25 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3203768536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:29:26.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.780+0000 7eff581c4640 1 -- 172.21.15.6:0/3958658274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff50106f10 msgr2=0x7eff5010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:26.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.780+0000 7eff581c4640 1 --2- 172.21.15.6:0/3958658274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff50106f10 0x7eff5010ea90 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7eff48009ca0 tx=0x7eff4802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:29:26.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.780+0000 7eff581c4640 1 -- 172.21.15.6:0/3958658274 shutdown_connections 2024-09-12T03:29:26.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.780+0000 7eff581c4640 1 --2- 172.21.15.6:0/3958658274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff50106f10 0x7eff5010ea90 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:26.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.780+0000 7eff581c4640 1 --2- 172.21.15.6:0/3958658274 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff50102790 0x7eff501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:26.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.780+0000 7eff581c4640 1 --2- 172.21.15.6:0/3958658274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff50101de0 0x7eff501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:26.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.780+0000 7eff581c4640 1 -- 172.21.15.6:0/3958658274 >> 172.21.15.6:0/3958658274 conn(0x7eff500fd8c0 msgr2=0x7eff500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:26.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.781+0000 7eff581c4640 1 -- 172.21.15.6:0/3958658274 shutdown_connections 2024-09-12T03:29:26.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.781+0000 7eff581c4640 1 -- 172.21.15.6:0/3958658274 wait complete. 2024-09-12T03:29:26.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.781+0000 7eff581c4640 1 Processor -- start 2024-09-12T03:29:26.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.782+0000 7eff581c4640 1 -- start start 2024-09-12T03:29:26.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.782+0000 7eff581c4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff50101de0 0x7eff50198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:26.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.782+0000 7eff581c4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff50102790 0x7eff50198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:26.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.782+0000 7eff581c4640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff50106f10 0x7eff5019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:26.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.782+0000 7eff581c4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7eff50111f70 con 0x7eff50101de0 2024-09-12T03:29:26.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.782+0000 7eff581c4640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7eff50111df0 con 0x7eff50106f10 2024-09-12T03:29:26.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.782+0000 7eff581c4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff501120f0 con 0x7eff50102790 2024-09-12T03:29:26.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.782+0000 7eff55f60640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff50101de0 0x7eff50198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:26.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.782+0000 7eff55f60640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff50101de0 0x7eff50198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44800/0 (socket says 172.21.15.6:44800) 2024-09-12T03:29:26.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.782+0000 7eff55f60640 1 -- 172.21.15.6:0/2784356156 learned_addr learned my addr 172.21.15.6:0/2784356156 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:29:26.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.782+0000 7eff5575f640 1 --2- 172.21.15.6:0/2784356156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff50102790 0x7eff50198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:26.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.782+0000 7eff56761640 1 --2- 172.21.15.6:0/2784356156 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff50106f10 0x7eff5019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:26.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.782+0000 7eff55f60640 1 -- 172.21.15.6:0/2784356156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff50102790 msgr2=0x7eff50198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:26.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.782+0000 7eff55f60640 1 --2- 172.21.15.6:0/2784356156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff50102790 0x7eff50198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:26.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.782+0000 7eff55f60640 1 -- 172.21.15.6:0/2784356156 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff50106f10 msgr2=0x7eff5019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:26.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.782+0000 7eff55f60640 1 --2- 172.21.15.6:0/2784356156 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff50106f10 0x7eff5019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:26.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.782+0000 7eff55f60640 1 -- 172.21.15.6:0/2784356156 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5019d300 con 0x7eff50101de0 2024-09-12T03:29:26.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.782+0000 7eff5575f640 1 --2- 172.21.15.6:0/2784356156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff50102790 0x7eff50198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:26.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.782+0000 7eff56761640 1 --2- 172.21.15.6:0/2784356156 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff50106f10 0x7eff5019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:26.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.783+0000 7eff55f60640 1 --2- 172.21.15.6:0/2784356156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff50101de0 0x7eff50198350 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00d950 tx=0x7eff4c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:26.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.783+0000 7eff46ffd640 1 -- 172.21.15.6:0/2784356156 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c017030 con 0x7eff50101de0 2024-09-12T03:29:26.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.783+0000 7eff46ffd640 1 -- 172.21.15.6:0/2784356156 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff4c004020 con 0x7eff50101de0 2024-09-12T03:29:26.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.783+0000 7eff46ffd640 1 -- 172.21.15.6:0/2784356156 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c0047a0 con 0x7eff50101de0 2024-09-12T03:29:26.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.783+0000 7eff581c4640 1 -- 172.21.15.6:0/2784356156 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5019d5f0 con 0x7eff50101de0 2024-09-12T03:29:26.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.783+0000 7eff581c4640 1 -- 172.21.15.6:0/2784356156 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff501a4ed0 con 0x7eff50101de0 2024-09-12T03:29:26.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.784+0000 7eff581c4640 1 -- 172.21.15.6:0/2784356156 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff501021c0 con 0x7eff50101de0 2024-09-12T03:29:26.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.785+0000 7eff46ffd640 1 -- 172.21.15.6:0/2784356156 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7eff4c0041d0 con 0x7eff50101de0 2024-09-12T03:29:26.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.786+0000 7eff46ffd640 1 --2- 172.21.15.6:0/2784356156 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7eff3c0a2b80 0x7eff3c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:26.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.786+0000 7eff5575f640 1 --2- 172.21.15.6:0/2784356156 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7eff3c0a2b80 0x7eff3c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:26.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.786+0000 7eff46ffd640 1 -- 172.21.15.6:0/2784356156 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7eff4c0ceab0 con 0x7eff50101de0 2024-09-12T03:29:26.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.786+0000 7eff5575f640 1 --2- 172.21.15.6:0/2784356156 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7eff3c0a2b80 0x7eff3c0a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7eff40005e10 tx=0x7eff40005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:26.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:26.790+0000 7eff46ffd640 1 -- 172.21.15.6:0/2784356156 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7eff4c09ba40 con 0x7eff50101de0 2024-09-12T03:29:27.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:27.001+0000 7eff581c4640 1 -- 172.21.15.6:0/2784356156 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff501a51c0 con 0x7eff50101de0 2024-09-12T03:29:27.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:27.002+0000 7eff46ffd640 1 -- 172.21.15.6:0/2784356156 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11910 (secure 0 0 0) 0x7eff4c09b340 con 0x7eff50101de0 2024-09-12T03:29:27.002 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:29:27.003 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":29,"btime":"2024-09-12T03:28:30:753620+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/4139960526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":4139960526},{"type":"v1","addr":"172.21.15.6:6835","nonce":4139960526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3369703756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3369703756},{"type":"v1","addr":"172.21.15.6:6837","nonce":3369703756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3301072029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3301072029},{"type":"v1","addr":"172.21.15.6:6839","nonce":3301072029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/2670411625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":2670411625},{"type":"v1","addr":"172.21.15.6:6841","nonce":2670411625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1749185252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1749185252},{"type":"v1","addr":"172.21.15.161:6833","nonce":1749185252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3592007490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3592007490},{"type":"v1","addr":"172.21.15.118:6833","nonce":3592007490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:19:09.851402+0000","modified":"2024-09-12T03:28:30.102891+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/955127607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":955127607},{"type":"v1","addr":"172.21.15.161:6835","nonce":955127607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6835/1389654073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1389654073},{"type":"v1","addr":"172.21.15.118:6835","nonce":1389654073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/283024944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":283024944},{"type":"v1","addr":"172.21.15.161:6837","nonce":283024944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/3645555254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3645555254},{"type":"v1","addr":"172.21.15.161:6839","nonce":3645555254}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6837/3879191727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":3879191727},{"type":"v1","addr":"172.21.15.118:6837","nonce":3879191727}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.118:6839/3035878957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":3035878957},{"type":"v1","addr":"172.21.15.118:6839","nonce":3035878957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24463]},"id":1}]} 2024-09-12T03:29:27.003 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 29 2024-09-12T03:29:27.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:27.005+0000 7eff581c4640 1 -- 172.21.15.6:0/2784356156 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7eff3c0a2b80 msgr2=0x7eff3c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:27.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:27.005+0000 7eff581c4640 1 --2- 172.21.15.6:0/2784356156 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7eff3c0a2b80 0x7eff3c0a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7eff40005e10 tx=0x7eff40005da0 comp rx=0 tx=0).stop 2024-09-12T03:29:27.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:27.005+0000 7eff581c4640 1 -- 172.21.15.6:0/2784356156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff50101de0 msgr2=0x7eff50198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:27.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:27.005+0000 7eff581c4640 1 --2- 172.21.15.6:0/2784356156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff50101de0 0x7eff50198350 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00d950 tx=0x7eff4c00de10 comp rx=0 tx=0).stop 2024-09-12T03:29:27.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:27.005+0000 7eff581c4640 1 -- 172.21.15.6:0/2784356156 shutdown_connections 2024-09-12T03:29:27.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:27.005+0000 7eff581c4640 1 --2- 172.21.15.6:0/2784356156 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7eff3c0a2b80 0x7eff3c0a5040 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:27.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:27.005+0000 7eff581c4640 1 --2- 172.21.15.6:0/2784356156 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff50106f10 0x7eff5019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:27.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:27.005+0000 7eff581c4640 1 --2- 172.21.15.6:0/2784356156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff50102790 0x7eff50198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:27.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:27.005+0000 7eff581c4640 1 --2- 172.21.15.6:0/2784356156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff50101de0 0x7eff50198350 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:27.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:27.005+0000 7eff581c4640 1 -- 172.21.15.6:0/2784356156 >> 172.21.15.6:0/2784356156 conn(0x7eff500fd8c0 msgr2=0x7eff500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:27.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:27.005+0000 7eff581c4640 1 -- 172.21.15.6:0/2784356156 shutdown_connections 2024-09-12T03:29:27.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:27.005+0000 7eff581c4640 1 -- 172.21.15.6:0/2784356156 wait complete. 2024-09-12T03:29:27.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:27 smithi006 ceph-mon[30991]: pgmap v585: 97 pgs: 97 active+clean; 159 MiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:29:27.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:27 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2784356156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:29:27.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:27 smithi161 ceph-mon[35485]: pgmap v585: 97 pgs: 97 active+clean; 159 MiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:29:27.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:27 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2784356156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:29:27.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:27 smithi118 ceph-mon[35048]: pgmap v585: 97 pgs: 97 active+clean; 159 MiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:29:27.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:27 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2784356156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:29:27.779 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:29:28.039 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:28.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.546+0000 7f9a38374640 1 -- 172.21.15.6:0/2548631706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a30102ef0 msgr2=0x7f9a3010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:28.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.546+0000 7f9a38374640 1 --2- 172.21.15.6:0/2548631706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a30102ef0 0x7f9a3010da40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009ca0 tx=0x7f9a2802a950 comp rx=0 tx=0).stop 2024-09-12T03:29:28.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.547+0000 7f9a38374640 1 -- 172.21.15.6:0/2548631706 shutdown_connections 2024-09-12T03:29:28.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.547+0000 7f9a38374640 1 --2- 172.21.15.6:0/2548631706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a30102ef0 0x7f9a3010da40 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:28.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.547+0000 7f9a38374640 1 --2- 172.21.15.6:0/2548631706 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a30102570 0x7f9a301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:28.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.547+0000 7f9a38374640 1 --2- 172.21.15.6:0/2548631706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a30100e20 0x7f9a30101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:28.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.547+0000 7f9a38374640 1 -- 172.21.15.6:0/2548631706 >> 172.21.15.6:0/2548631706 conn(0x7f9a300fc7d0 msgr2=0x7f9a300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:28.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.547+0000 7f9a38374640 1 -- 172.21.15.6:0/2548631706 shutdown_connections 2024-09-12T03:29:28.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.547+0000 7f9a38374640 1 -- 172.21.15.6:0/2548631706 wait complete. 2024-09-12T03:29:28.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.547+0000 7f9a38374640 1 Processor -- start 2024-09-12T03:29:28.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.547+0000 7f9a38374640 1 -- start start 2024-09-12T03:29:28.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.547+0000 7f9a38374640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a30100e20 0x7f9a300fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:28.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.547+0000 7f9a38374640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a30102570 0x7f9a300fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:28.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.547+0000 7f9a38374640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a30102ef0 0x7f9a300fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:28.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.547+0000 7f9a38374640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9a30110f80 con 0x7f9a30100e20 2024-09-12T03:29:28.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.547+0000 7f9a38374640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f9a30110e00 con 0x7f9a30102ef0 2024-09-12T03:29:28.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.547+0000 7f9a38374640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9a30111100 con 0x7f9a30102570 2024-09-12T03:29:28.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.548+0000 7f9a36110640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a30100e20 0x7f9a300fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:28.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.548+0000 7f9a36110640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a30100e20 0x7f9a300fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44816/0 (socket says 172.21.15.6:44816) 2024-09-12T03:29:28.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.548+0000 7f9a3590f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a30102570 0x7f9a300fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:28.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.548+0000 7f9a36110640 1 -- 172.21.15.6:0/1841245089 learned_addr learned my addr 172.21.15.6:0/1841245089 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:29:28.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.548+0000 7f9a36911640 1 --2- 172.21.15.6:0/1841245089 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a30102ef0 0x7f9a300fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:28.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.548+0000 7f9a36110640 1 -- 172.21.15.6:0/1841245089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a30102570 msgr2=0x7f9a300fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:28.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.548+0000 7f9a36110640 1 --2- 172.21.15.6:0/1841245089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a30102570 0x7f9a300fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:28.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.548+0000 7f9a36110640 1 -- 172.21.15.6:0/1841245089 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a30102ef0 msgr2=0x7f9a300fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:28.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.548+0000 7f9a36110640 1 --2- 172.21.15.6:0/1841245089 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a30102ef0 0x7f9a300fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:28.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.548+0000 7f9a36110640 1 -- 172.21.15.6:0/1841245089 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a301aae00 con 0x7f9a30100e20 2024-09-12T03:29:28.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.548+0000 7f9a3590f640 1 --2- 172.21.15.6:0/1841245089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a30102570 0x7f9a300fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:28.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.548+0000 7f9a36110640 1 --2- 172.21.15.6:0/1841245089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a30100e20 0x7f9a300fcc80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00e9f0 tx=0x7f9a2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:28.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.548+0000 7f9a277fe640 1 -- 172.21.15.6:0/1841245089 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c00cd80 con 0x7f9a30100e20 2024-09-12T03:29:28.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.548+0000 7f9a277fe640 1 -- 172.21.15.6:0/1841245089 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a2c005500 con 0x7f9a30100e20 2024-09-12T03:29:28.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.548+0000 7f9a38374640 1 -- 172.21.15.6:0/1841245089 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a301aafd0 con 0x7f9a30100e20 2024-09-12T03:29:28.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.548+0000 7f9a277fe640 1 -- 172.21.15.6:0/1841245089 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c010690 con 0x7f9a30100e20 2024-09-12T03:29:28.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.548+0000 7f9a38374640 1 -- 172.21.15.6:0/1841245089 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a301ab4b0 con 0x7f9a30100e20 2024-09-12T03:29:28.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.549+0000 7f9a38374640 1 -- 172.21.15.6:0/1841245089 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a301021b0 con 0x7f9a30100e20 2024-09-12T03:29:28.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.551+0000 7f9a277fe640 1 -- 172.21.15.6:0/1841245089 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9a2c0046c0 con 0x7f9a30100e20 2024-09-12T03:29:28.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.551+0000 7f9a277fe640 1 --2- 172.21.15.6:0/1841245089 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9a1c0a2c00 0x7f9a1c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:28.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.552+0000 7f9a3590f640 1 --2- 172.21.15.6:0/1841245089 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9a1c0a2c00 0x7f9a1c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:28.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.552+0000 7f9a277fe640 1 -- 172.21.15.6:0/1841245089 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f9a2c0cf3f0 con 0x7f9a30100e20 2024-09-12T03:29:28.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.552+0000 7f9a3590f640 1 --2- 172.21.15.6:0/1841245089 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9a1c0a2c00 0x7f9a1c0a50c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9a300fe440 tx=0x7f9a2000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:28.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.555+0000 7f9a277fe640 1 -- 172.21.15.6:0/1841245089 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9a2c09c380 con 0x7f9a30100e20 2024-09-12T03:29:28.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.732+0000 7f9a38374640 1 -- 172.21.15.6:0/1841245089 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a3010a0a0 con 0x7f9a30100e20 2024-09-12T03:29:28.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.734+0000 7f9a277fe640 1 -- 172.21.15.6:0/1841245089 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22607 (secure 0 0 0) 0x7f9a2c09bc80 con 0x7f9a30100e20 2024-09-12T03:29:28.734 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-09-12T03:29:28.734 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":82,"fsid":"89d99938-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:11:29.530545+0000","modified":"2024-09-12T03:19:20.366710+0000","last_up_change":"2024-09-12T03:18:21.292572+0000","last_in_change":"2024-09-12T03:17:58.013800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:14:37.662010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:06.143398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:08.305279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f55d68b-95a0-4ee2-a46f-bc422b38471e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6803","nonce":726729286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6805","nonce":726729286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6809","nonce":726729286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":726729286},{"type":"v1","addr":"172.21.15.6:6807","nonce":726729286}]},"public_addr":"172.21.15.6:6803/726729286","cluster_addr":"172.21.15.6:6805/726729286","heartbeat_back_addr":"172.21.15.6:6809/726729286","heartbeat_front_addr":"172.21.15.6:6807/726729286","state":["exists","up"]},{"osd":1,"uuid":"83afdb01-c731-4de1-96c6-3f798549b1eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6801","nonce":3377555415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6803","nonce":3377555415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6807","nonce":3377555415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":3377555415},{"type":"v1","addr":"172.21.15.118:6805","nonce":3377555415}]},"public_addr":"172.21.15.118:6801/3377555415","cluster_addr":"172.21.15.118:6803/3377555415","heartbeat_back_addr":"172.21.15.118:6807/3377555415","heartbeat_front_addr":"172.21.15.118:6805/3377555415","state":["exists","up"]},{"osd":2,"uuid":"aed718b4-b849-403a-b81c-3a140b5c380c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6801","nonce":392541009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6803","nonce":392541009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6807","nonce":392541009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":392541009},{"type":"v1","addr":"172.21.15.161:6805","nonce":392541009}]},"public_addr":"172.21.15.161:6801/392541009","cluster_addr":"172.21.15.161:6803/392541009","heartbeat_back_addr":"172.21.15.161:6807/392541009","heartbeat_front_addr":"172.21.15.161:6805/392541009","state":["exists","up"]},{"osd":3,"uuid":"25ced4c1-bf31-402d-8397-d42fd57f615f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6811","nonce":1892039151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6813","nonce":1892039151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6817","nonce":1892039151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1892039151},{"type":"v1","addr":"172.21.15.6:6815","nonce":1892039151}]},"public_addr":"172.21.15.6:6811/1892039151","cluster_addr":"172.21.15.6:6813/1892039151","heartbeat_back_addr":"172.21.15.6:6817/1892039151","heartbeat_front_addr":"172.21.15.6:6815/1892039151","state":["exists","up"]},{"osd":4,"uuid":"1d884604-920b-4c35-b41e-d629ad9f9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6809","nonce":150177024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6811","nonce":150177024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6815","nonce":150177024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":150177024},{"type":"v1","addr":"172.21.15.118:6813","nonce":150177024}]},"public_addr":"172.21.15.118:6809/150177024","cluster_addr":"172.21.15.118:6811/150177024","heartbeat_back_addr":"172.21.15.118:6815/150177024","heartbeat_front_addr":"172.21.15.118:6813/150177024","state":["exists","up"]},{"osd":5,"uuid":"1d40c0ef-99f6-426f-90c7-27579edc66af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6809","nonce":3758167854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6811","nonce":3758167854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6815","nonce":3758167854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3758167854},{"type":"v1","addr":"172.21.15.161:6813","nonce":3758167854}]},"public_addr":"172.21.15.161:6809/3758167854","cluster_addr":"172.21.15.161:6811/3758167854","heartbeat_back_addr":"172.21.15.161:6815/3758167854","heartbeat_front_addr":"172.21.15.161:6813/3758167854","state":["exists","up"]},{"osd":6,"uuid":"9d9f9510-b107-40c2-9a98-9012715944b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6819","nonce":2129224874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6821","nonce":2129224874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6825","nonce":2129224874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2129224874},{"type":"v1","addr":"172.21.15.6:6823","nonce":2129224874}]},"public_addr":"172.21.15.6:6819/2129224874","cluster_addr":"172.21.15.6:6821/2129224874","heartbeat_back_addr":"172.21.15.6:6825/2129224874","heartbeat_front_addr":"172.21.15.6:6823/2129224874","state":["exists","up"]},{"osd":7,"uuid":"84635ae1-3240-4954-af14-76bb3aa162e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6817","nonce":269980353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6819","nonce":269980353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6823","nonce":269980353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":269980353},{"type":"v1","addr":"172.21.15.118:6821","nonce":269980353}]},"public_addr":"172.21.15.118:6817/269980353","cluster_addr":"172.21.15.118:6819/269980353","heartbeat_back_addr":"172.21.15.118:6823/269980353","heartbeat_front_addr":"172.21.15.118:6821/269980353","state":["exists","up"]},{"osd":8,"uuid":"6fd58e55-1492-4595-8647-823ea3e3c5d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6817","nonce":1783416134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6819","nonce":1783416134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6823","nonce":1783416134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1783416134},{"type":"v1","addr":"172.21.15.161:6821","nonce":1783416134}]},"public_addr":"172.21.15.161:6817/1783416134","cluster_addr":"172.21.15.161:6819/1783416134","heartbeat_back_addr":"172.21.15.161:6823/1783416134","heartbeat_front_addr":"172.21.15.161:6821/1783416134","state":["exists","up"]},{"osd":9,"uuid":"90508a6d-a49c-43c6-a600-5bbccd8702ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6827","nonce":2987332365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6829","nonce":2987332365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6833","nonce":2987332365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2987332365},{"type":"v1","addr":"172.21.15.6:6831","nonce":2987332365}]},"public_addr":"172.21.15.6:6827/2987332365","cluster_addr":"172.21.15.6:6829/2987332365","heartbeat_back_addr":"172.21.15.6:6833/2987332365","heartbeat_front_addr":"172.21.15.6:6831/2987332365","state":["exists","up"]},{"osd":10,"uuid":"911e86d1-669a-4935-82c0-7da51d141df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6825","nonce":2167146341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6827","nonce":2167146341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6831","nonce":2167146341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":2167146341},{"type":"v1","addr":"172.21.15.118:6829","nonce":2167146341}]},"public_addr":"172.21.15.118:6825/2167146341","cluster_addr":"172.21.15.118:6827/2167146341","heartbeat_back_addr":"172.21.15.118:6831/2167146341","heartbeat_front_addr":"172.21.15.118:6829/2167146341","state":["exists","up"]},{"osd":11,"uuid":"5a15c6a7-ff43-473c-bb48-0c129c316df5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6825","nonce":2889795796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6827","nonce":2889795796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6831","nonce":2889795796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2889795796},{"type":"v1","addr":"172.21.15.161:6829","nonce":2889795796}]},"public_addr":"172.21.15.161:6825/2889795796","cluster_addr":"172.21.15.161:6827/2889795796","heartbeat_back_addr":"172.21.15.161:6831/2889795796","heartbeat_front_addr":"172.21.15.161:6829/2889795796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:42.912205+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:08.410669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:34.299134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:59.093181+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:24.116635+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:49.566433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:15.719320+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:40.541017+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:05.178178+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:31.163877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:17:55.405110+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:19.417156+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6800/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/2620491916":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/4037656392":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/1377116573":"2024-09-13T03:12:19.155264+0000","172.21.15.6:0/3062800125":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/1684477653":"2024-09-13T03:12:19.155264+0000","172.21.15.6:6800/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:6801/836933784":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/4145338968":"2024-09-13T03:11:55.244257+0000","172.21.15.6:0/825701128":"2024-09-13T03:11:55.244257+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:29:28.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.736+0000 7f9a38374640 1 -- 172.21.15.6:0/1841245089 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9a1c0a2c00 msgr2=0x7f9a1c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:28.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.736+0000 7f9a38374640 1 --2- 172.21.15.6:0/1841245089 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9a1c0a2c00 0x7f9a1c0a50c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9a300fe440 tx=0x7f9a2000b410 comp rx=0 tx=0).stop 2024-09-12T03:29:28.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.736+0000 7f9a38374640 1 -- 172.21.15.6:0/1841245089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a30100e20 msgr2=0x7f9a300fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:28.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.736+0000 7f9a38374640 1 --2- 172.21.15.6:0/1841245089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a30100e20 0x7f9a300fcc80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00e9f0 tx=0x7f9a2c00edf0 comp rx=0 tx=0).stop 2024-09-12T03:29:28.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.736+0000 7f9a38374640 1 -- 172.21.15.6:0/1841245089 shutdown_connections 2024-09-12T03:29:28.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.736+0000 7f9a38374640 1 --2- 172.21.15.6:0/1841245089 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f9a1c0a2c00 0x7f9a1c0a50c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:28.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.736+0000 7f9a38374640 1 --2- 172.21.15.6:0/1841245089 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a30102ef0 0x7f9a300fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:28.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.736+0000 7f9a38374640 1 --2- 172.21.15.6:0/1841245089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a30102570 0x7f9a300fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:28.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.736+0000 7f9a38374640 1 --2- 172.21.15.6:0/1841245089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a30100e20 0x7f9a300fcc80 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:28.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.736+0000 7f9a38374640 1 -- 172.21.15.6:0/1841245089 >> 172.21.15.6:0/1841245089 conn(0x7f9a300fc7d0 msgr2=0x7f9a3006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:28.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.736+0000 7f9a38374640 1 -- 172.21.15.6:0/1841245089 shutdown_connections 2024-09-12T03:29:28.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:28.736+0000 7f9a38374640 1 -- 172.21.15.6:0/1841245089 wait complete. 2024-09-12T03:29:29.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:29 smithi006 ceph-mon[30991]: pgmap v586: 97 pgs: 97 active+clean; 159 MiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-12T03:29:29.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:29 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/1841245089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:29:29.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:29 smithi161 ceph-mon[35485]: pgmap v586: 97 pgs: 97 active+clean; 159 MiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-12T03:29:29.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:29 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/1841245089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:29:29.411 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-12T03:29:29.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:29 smithi118 ceph-mon[35048]: pgmap v586: 97 pgs: 97 active+clean; 159 MiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-12T03:29:29.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:29 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/1841245089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:29:29.673 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:30.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.131+0000 7ff4e370d640 1 -- 172.21.15.6:0/4063872075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4dc106f20 msgr2=0x7ff4dc10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:30.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.131+0000 7ff4e370d640 1 --2- 172.21.15.6:0/4063872075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4dc106f20 0x7ff4dc10eaa0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4009ca0 tx=0x7ff4d402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.131+0000 7ff4e370d640 1 -- 172.21.15.6:0/4063872075 shutdown_connections 2024-09-12T03:29:30.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.131+0000 7ff4e370d640 1 --2- 172.21.15.6:0/4063872075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4dc106f20 0x7ff4dc10eaa0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.132+0000 7ff4e370d640 1 --2- 172.21.15.6:0/4063872075 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4dc1027a0 0x7ff4dc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.132+0000 7ff4e370d640 1 --2- 172.21.15.6:0/4063872075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4dc101df0 0x7ff4dc1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.132+0000 7ff4e370d640 1 -- 172.21.15.6:0/4063872075 >> 172.21.15.6:0/4063872075 conn(0x7ff4dc0fd8f0 msgr2=0x7ff4dc0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:30.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.132+0000 7ff4e370d640 1 -- 172.21.15.6:0/4063872075 shutdown_connections 2024-09-12T03:29:30.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.132+0000 7ff4e370d640 1 -- 172.21.15.6:0/4063872075 wait complete. 2024-09-12T03:29:30.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.132+0000 7ff4e370d640 1 Processor -- start 2024-09-12T03:29:30.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.132+0000 7ff4e370d640 1 -- start start 2024-09-12T03:29:30.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.132+0000 7ff4e370d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4dc101df0 0x7ff4dc19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.132+0000 7ff4e370d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4dc1027a0 0x7ff4dc19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.132+0000 7ff4e370d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4dc106f20 0x7ff4dc19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.132+0000 7ff4e370d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc111f80 con 0x7ff4dc106f20 2024-09-12T03:29:30.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.132+0000 7ff4e370d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc111e00 con 0x7ff4dc1027a0 2024-09-12T03:29:30.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.132+0000 7ff4e370d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc112100 con 0x7ff4dc101df0 2024-09-12T03:29:30.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.133+0000 7ff4e1caa640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4dc106f20 0x7ff4dc19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:30.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.133+0000 7ff4e1caa640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4dc106f20 0x7ff4dc19eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50304/0 (socket says 172.21.15.6:50304) 2024-09-12T03:29:30.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.133+0000 7ff4e1caa640 1 -- 172.21.15.6:0/2443469486 learned_addr learned my addr 172.21.15.6:0/2443469486 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:29:30.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.133+0000 7ff4e0ca8640 1 --2- 172.21.15.6:0/2443469486 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4dc1027a0 0x7ff4dc19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:30.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.133+0000 7ff4e14a9640 1 --2- 172.21.15.6:0/2443469486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4dc101df0 0x7ff4dc19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:30.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.133+0000 7ff4e1caa640 1 -- 172.21.15.6:0/2443469486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4dc101df0 msgr2=0x7ff4dc19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:30.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.133+0000 7ff4e1caa640 1 --2- 172.21.15.6:0/2443469486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4dc101df0 0x7ff4dc19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.133+0000 7ff4e1caa640 1 -- 172.21.15.6:0/2443469486 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4dc1027a0 msgr2=0x7ff4dc19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:30.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.133+0000 7ff4e1caa640 1 --2- 172.21.15.6:0/2443469486 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4dc1027a0 0x7ff4dc19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.133+0000 7ff4e1caa640 1 -- 172.21.15.6:0/2443469486 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4dc19f5a0 con 0x7ff4dc106f20 2024-09-12T03:29:30.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.133+0000 7ff4e1caa640 1 --2- 172.21.15.6:0/2443469486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4dc106f20 0x7ff4dc19eec0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4031a20 tx=0x7ff4d4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:30.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.133+0000 7ff4d27fc640 1 -- 172.21.15.6:0/2443469486 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4003c30 con 0x7ff4dc106f20 2024-09-12T03:29:30.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.133+0000 7ff4d27fc640 1 -- 172.21.15.6:0/2443469486 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4d4003dd0 con 0x7ff4dc106f20 2024-09-12T03:29:30.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.133+0000 7ff4e370d640 1 -- 172.21.15.6:0/2443469486 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4dc19f830 con 0x7ff4dc106f20 2024-09-12T03:29:30.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.133+0000 7ff4d27fc640 1 -- 172.21.15.6:0/2443469486 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d402fc40 con 0x7ff4dc106f20 2024-09-12T03:29:30.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.133+0000 7ff4e370d640 1 -- 172.21.15.6:0/2443469486 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4dc1a7170 con 0x7ff4dc106f20 2024-09-12T03:29:30.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.134+0000 7ff4e370d640 1 -- 172.21.15.6:0/2443469486 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4dc1021d0 con 0x7ff4dc106f20 2024-09-12T03:29:30.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.136+0000 7ff4d27fc640 1 -- 172.21.15.6:0/2443469486 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff4d4004050 con 0x7ff4dc106f20 2024-09-12T03:29:30.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.136+0000 7ff4d27fc640 1 --2- 172.21.15.6:0/2443469486 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff4c80a2c00 0x7ff4c80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.136+0000 7ff4d27fc640 1 -- 172.21.15.6:0/2443469486 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff4d40f02d0 con 0x7ff4dc106f20 2024-09-12T03:29:30.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.136+0000 7ff4e14a9640 1 --2- 172.21.15.6:0/2443469486 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff4c80a2c00 0x7ff4c80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:30.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.137+0000 7ff4e14a9640 1 --2- 172.21.15.6:0/2443469486 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff4c80a2c00 0x7ff4c80a50c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8009bd0 tx=0x7ff4d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:30.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.139+0000 7ff4d27fc640 1 -- 172.21.15.6:0/2443469486 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4d40bd2f0 con 0x7ff4dc106f20 2024-09-12T03:29:30.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.356+0000 7ff4e370d640 1 -- 172.21.15.6:0/2443469486 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff4dc1a73d0 con 0x7ff4dc106f20 2024-09-12T03:29:30.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.358+0000 7ff4d27fc640 1 -- 172.21.15.6:0/2443469486 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 161+0+0 (secure 0 0 0) 0x7ff4d40bcbf0 con 0x7ff4dc106f20 2024-09-12T03:29:30.358 INFO:teuthology.orchestra.run.smithi006.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T03:29:30.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.360+0000 7ff4e370d640 1 -- 172.21.15.6:0/2443469486 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff4c80a2c00 msgr2=0x7ff4c80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:30.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.360+0000 7ff4e370d640 1 --2- 172.21.15.6:0/2443469486 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff4c80a2c00 0x7ff4c80a50c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8009bd0 tx=0x7ff4d8005f90 comp rx=0 tx=0).stop 2024-09-12T03:29:30.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.361+0000 7ff4e370d640 1 -- 172.21.15.6:0/2443469486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4dc106f20 msgr2=0x7ff4dc19eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:30.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.361+0000 7ff4e370d640 1 --2- 172.21.15.6:0/2443469486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4dc106f20 0x7ff4dc19eec0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4031a20 tx=0x7ff4d4031a50 comp rx=0 tx=0).stop 2024-09-12T03:29:30.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.361+0000 7ff4e370d640 1 -- 172.21.15.6:0/2443469486 shutdown_connections 2024-09-12T03:29:30.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.361+0000 7ff4e370d640 1 --2- 172.21.15.6:0/2443469486 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7ff4c80a2c00 0x7ff4c80a50c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.361+0000 7ff4e370d640 1 --2- 172.21.15.6:0/2443469486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4dc106f20 0x7ff4dc19eec0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.361+0000 7ff4e370d640 1 --2- 172.21.15.6:0/2443469486 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4dc1027a0 0x7ff4dc19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.361+0000 7ff4e370d640 1 --2- 172.21.15.6:0/2443469486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4dc101df0 0x7ff4dc19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.361+0000 7ff4e370d640 1 -- 172.21.15.6:0/2443469486 >> 172.21.15.6:0/2443469486 conn(0x7ff4dc0fd8f0 msgr2=0x7ff4dc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:30.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.361+0000 7ff4e370d640 1 -- 172.21.15.6:0/2443469486 shutdown_connections 2024-09-12T03:29:30.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:30.361+0000 7ff4e370d640 1 -- 172.21.15.6:0/2443469486 wait complete. 2024-09-12T03:29:30.611 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:30 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-a[30987]: 2024-09-12T03:29:30.357+0000 7f70409b6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:29:30.712 INFO:journalctl@ceph.mds.e.smithi118.stdout:Sep 12 03:29:30 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-e[64900]: ignoring --setuser ceph since I am not root 2024-09-12T03:29:30.712 INFO:journalctl@ceph.mds.e.smithi118.stdout:Sep 12 03:29:30 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-e[64900]: ignoring --setgroup ceph since I am not root 2024-09-12T03:29:30.712 INFO:journalctl@ceph.mds.e.smithi118.stdout:Sep 12 03:29:30 smithi118 ceph-mds[64904]: -- 172.21.15.118:0/1359509586 <== mon.2 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e9e16e1860 con 0x55e9e16be800 2024-09-12T03:29:30.713 INFO:journalctl@ceph.mds.e.smithi118.stdout:Sep 12 03:29:30 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-e[64900]: starting mds.e at 2024-09-12T03:29:30.713 INFO:journalctl@ceph.mds.h.smithi118.stdout:Sep 12 03:29:30 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-h[65203]: ignoring --setuser ceph since I am not root 2024-09-12T03:29:30.713 INFO:journalctl@ceph.mds.h.smithi118.stdout:Sep 12 03:29:30 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-h[65203]: ignoring --setgroup ceph since I am not root 2024-09-12T03:29:30.713 INFO:journalctl@ceph.mds.h.smithi118.stdout:Sep 12 03:29:30 smithi118 ceph-mds[65207]: -- 172.21.15.118:0/1182088429 <== mon.2 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b3a25c3860 con 0x55b3a25a0800 2024-09-12T03:29:30.713 INFO:journalctl@ceph.mds.h.smithi118.stdout:Sep 12 03:29:30 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-h[65203]: starting mds.h at 2024-09-12T03:29:30.713 INFO:journalctl@ceph.mds.b.smithi118.stdout:Sep 12 03:29:30 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-b[65505]: ignoring --setuser ceph since I am not root 2024-09-12T03:29:30.713 INFO:journalctl@ceph.mds.b.smithi118.stdout:Sep 12 03:29:30 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-b[65505]: ignoring --setgroup ceph since I am not root 2024-09-12T03:29:30.713 INFO:journalctl@ceph.mds.b.smithi118.stdout:Sep 12 03:29:30 smithi118 ceph-mds[65509]: -- 172.21.15.118:0/1225940527 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558018107860 con 0x5580180e5400 2024-09-12T03:29:30.713 INFO:journalctl@ceph.mds.b.smithi118.stdout:Sep 12 03:29:30 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-b[65505]: starting mds.b at 2024-09-12T03:29:30.863 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 12 03:29:30 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-c[66065]: ignoring --setuser ceph since I am not root 2024-09-12T03:29:30.863 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 12 03:29:30 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-c[66065]: ignoring --setgroup ceph since I am not root 2024-09-12T03:29:30.863 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 12 03:29:30 smithi161 ceph-mds[66069]: -- 172.21.15.161:0/2401894066 <== mon.0 v2:172.21.15.6:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c6c5ad1860 con 0x55c6c5aaf000 2024-09-12T03:29:30.863 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 12 03:29:30 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-c[66065]: starting mds.c at 2024-09-12T03:29:30.864 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 12 03:29:30 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-f[65762]: ignoring --setuser ceph since I am not root 2024-09-12T03:29:30.864 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 12 03:29:30 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-f[65762]: ignoring --setgroup ceph since I am not root 2024-09-12T03:29:30.864 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 12 03:29:30 smithi161 ceph-mds[65766]: -- 172.21.15.161:0/4201639039 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a5d3c1860 con 0x557a5d39f000 2024-09-12T03:29:30.864 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 12 03:29:30 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-f[65762]: starting mds.f at 2024-09-12T03:29:30.864 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 12 03:29:30 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-i[66368]: ignoring --setuser ceph since I am not root 2024-09-12T03:29:30.864 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 12 03:29:30 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-i[66368]: ignoring --setgroup ceph since I am not root 2024-09-12T03:29:30.864 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 12 03:29:30 smithi161 ceph-mds[66372]: -- 172.21.15.161:0/3929024807 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5606a10e9860 con 0x5606a10c7400 2024-09-12T03:29:30.865 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 12 03:29:30 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-i[66368]: starting mds.i at 2024-09-12T03:29:30.865 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 12 03:29:30 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-i[66368]: 2024-09-12T03:29:30.402+0000 7f1a7f8c5640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6835/0: (98) Address already in use 2024-09-12T03:29:31.045 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T03:29:31.302 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:31.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:31 smithi006 ceph-mon[30991]: pgmap v587: 97 pgs: 97 active+clean; 159 MiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 2.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:29:31.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:31 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2443469486' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:29:31.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:31 smithi006 ceph-mon[30991]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:29:31.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:31 smithi006 ceph-mon[30991]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:29:31.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:31 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/2443469486' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:29:31.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:31 smithi006 ceph-mon[30991]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T03:29:31.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:31 smithi006 ceph-mon[30991]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T03:29:31.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:31 smithi161 ceph-mon[35485]: pgmap v587: 97 pgs: 97 active+clean; 159 MiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 2.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:29:31.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:31 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2443469486' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:29:31.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:31 smithi161 ceph-mon[35485]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:29:31.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:31 smithi161 ceph-mon[35485]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:29:31.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:31 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/2443469486' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:29:31.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:31 smithi161 ceph-mon[35485]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T03:29:31.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:31 smithi161 ceph-mon[35485]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T03:29:31.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:31 smithi118 ceph-mon[35048]: pgmap v587: 97 pgs: 97 active+clean; 159 MiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 2.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:29:31.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:31 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2443469486' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:29:31.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:31 smithi118 ceph-mon[35048]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:29:31.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:31 smithi118 ceph-mon[35048]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:29:31.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:31 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/2443469486' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:29:31.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:31 smithi118 ceph-mon[35048]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T03:29:31.463 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:31 smithi118 ceph-mon[35048]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T03:29:31.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.832+0000 7f50479ce640 1 -- 172.21.15.6:0/2844237649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5040102790 msgr2=0x7f50401067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:31.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.832+0000 7f50479ce640 1 --2- 172.21.15.6:0/2844237649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5040102790 0x7f50401067e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5030009900 tx=0x7f503002a410 comp rx=0 tx=0).stop 2024-09-12T03:29:31.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.832+0000 7f50479ce640 1 -- 172.21.15.6:0/2844237649 shutdown_connections 2024-09-12T03:29:31.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.832+0000 7f50479ce640 1 --2- 172.21.15.6:0/2844237649 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5040106f10 0x7f504010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.832+0000 7f50479ce640 1 --2- 172.21.15.6:0/2844237649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5040102790 0x7f50401067e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.832+0000 7f50479ce640 1 --2- 172.21.15.6:0/2844237649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5040101de0 0x7f50401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.832+0000 7f50479ce640 1 -- 172.21.15.6:0/2844237649 >> 172.21.15.6:0/2844237649 conn(0x7f50400fd8c0 msgr2=0x7f50400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:31.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.832+0000 7f50479ce640 1 -- 172.21.15.6:0/2844237649 shutdown_connections 2024-09-12T03:29:31.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.832+0000 7f50479ce640 1 -- 172.21.15.6:0/2844237649 wait complete. 2024-09-12T03:29:31.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.833+0000 7f50479ce640 1 Processor -- start 2024-09-12T03:29:31.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.833+0000 7f50479ce640 1 -- start start 2024-09-12T03:29:31.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.833+0000 7f50479ce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5040101de0 0x7f5040198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.833+0000 7f50479ce640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5040102790 0x7f5040198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.833+0000 7f50479ce640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5040106f10 0x7f504019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.833+0000 7f50479ce640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5040111f70 con 0x7f5040102790 2024-09-12T03:29:31.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.833+0000 7f50479ce640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f5040111df0 con 0x7f5040106f10 2024-09-12T03:29:31.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.833+0000 7f50479ce640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f50401120f0 con 0x7f5040101de0 2024-09-12T03:29:31.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.833+0000 7f5044f69640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5040102790 0x7f5040198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:31.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.833+0000 7f504576a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5040101de0 0x7f5040198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:31.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.833+0000 7f5044f69640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5040102790 0x7f5040198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50324/0 (socket says 172.21.15.6:50324) 2024-09-12T03:29:31.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.833+0000 7f5044f69640 1 -- 172.21.15.6:0/4091112821 learned_addr learned my addr 172.21.15.6:0/4091112821 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:29:31.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.833+0000 7f5045f6b640 1 --2- 172.21.15.6:0/4091112821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5040106f10 0x7f504019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:31.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.834+0000 7f5044f69640 1 -- 172.21.15.6:0/4091112821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5040101de0 msgr2=0x7f5040198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:31.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.834+0000 7f5044f69640 1 --2- 172.21.15.6:0/4091112821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5040101de0 0x7f5040198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.834+0000 7f5044f69640 1 -- 172.21.15.6:0/4091112821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5040106f10 msgr2=0x7f504019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:31.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.834+0000 7f5044f69640 1 --2- 172.21.15.6:0/4091112821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5040106f10 0x7f504019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.834+0000 7f5044f69640 1 -- 172.21.15.6:0/4091112821 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504019d400 con 0x7f5040102790 2024-09-12T03:29:31.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.834+0000 7f504576a640 1 --2- 172.21.15.6:0/4091112821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5040101de0 0x7f5040198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:31.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.834+0000 7f5044f69640 1 --2- 172.21.15.6:0/4091112821 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5040102790 0x7f5040198990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5030006010 tx=0x7f50300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:31.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.834+0000 7f50367fc640 1 -- 172.21.15.6:0/4091112821 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503003f030 con 0x7f5040102790 2024-09-12T03:29:31.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.834+0000 7f50479ce640 1 -- 172.21.15.6:0/4091112821 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f504019d690 con 0x7f5040102790 2024-09-12T03:29:31.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.834+0000 7f50367fc640 1 -- 172.21.15.6:0/4091112821 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5030003ce0 con 0x7f5040102790 2024-09-12T03:29:31.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.834+0000 7f50367fc640 1 -- 172.21.15.6:0/4091112821 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5030035070 con 0x7f5040102790 2024-09-12T03:29:31.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.834+0000 7f50479ce640 1 -- 172.21.15.6:0/4091112821 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50401a4fd0 con 0x7f5040102790 2024-09-12T03:29:31.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.835+0000 7f50479ce640 1 -- 172.21.15.6:0/4091112821 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50401021c0 con 0x7f5040102790 2024-09-12T03:29:31.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.836+0000 7f50367fc640 1 -- 172.21.15.6:0/4091112821 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5030042030 con 0x7f5040102790 2024-09-12T03:29:31.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.837+0000 7f50367fc640 1 --2- 172.21.15.6:0/4091112821 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f502c0a2b80 0x7f502c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.837+0000 7f50367fc640 1 -- 172.21.15.6:0/4091112821 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8440+0+0 (secure 0 0 0) 0x7f50300ef740 con 0x7f5040102790 2024-09-12T03:29:31.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.837+0000 7f504576a640 1 --2- 172.21.15.6:0/4091112821 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f502c0a2b80 0x7f502c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:31.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.838+0000 7f504576a640 1 --2- 172.21.15.6:0/4091112821 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f502c0a2b80 0x7f502c0a5040 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f503c00bfd0 tx=0x7f503c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:31.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:31.841+0000 7f50367fc640 1 -- 172.21.15.6:0/4091112821 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f50300bc690 con 0x7f5040102790 2024-09-12T03:29:32.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:32.072+0000 7f50479ce640 1 -- 172.21.15.6:0/4091112821 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f504010a180 con 0x7f5040102790 2024-09-12T03:29:32.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:32.369+0000 7f50367fc640 1 -- 172.21.15.6:0/4091112821 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) ==== 104+0+0 (secure 0 0 0) 0x7f50300bbf90 con 0x7f5040102790 2024-09-12T03:29:32.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:32.371+0000 7f50479ce640 1 -- 172.21.15.6:0/4091112821 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f502c0a2b80 msgr2=0x7f502c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:32.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:32.371+0000 7f50479ce640 1 --2- 172.21.15.6:0/4091112821 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f502c0a2b80 0x7f502c0a5040 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f503c00bfd0 tx=0x7f503c00b390 comp rx=0 tx=0).stop 2024-09-12T03:29:32.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:32.371+0000 7f50479ce640 1 -- 172.21.15.6:0/4091112821 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5040102790 msgr2=0x7f5040198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:32.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:32.371+0000 7f50479ce640 1 --2- 172.21.15.6:0/4091112821 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5040102790 0x7f5040198990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5030006010 tx=0x7f50300042f0 comp rx=0 tx=0).stop 2024-09-12T03:29:32.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:32.371+0000 7f50479ce640 1 -- 172.21.15.6:0/4091112821 shutdown_connections 2024-09-12T03:29:32.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:32.371+0000 7f50479ce640 1 --2- 172.21.15.6:0/4091112821 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f502c0a2b80 0x7f502c0a5040 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:32.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:32.371+0000 7f50479ce640 1 --2- 172.21.15.6:0/4091112821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5040106f10 0x7f504019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:32.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:32.371+0000 7f50479ce640 1 --2- 172.21.15.6:0/4091112821 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5040102790 0x7f5040198990 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:32.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:32.371+0000 7f50479ce640 1 --2- 172.21.15.6:0/4091112821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5040101de0 0x7f5040198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:32.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:32.371+0000 7f50479ce640 1 -- 172.21.15.6:0/4091112821 >> 172.21.15.6:0/4091112821 conn(0x7f50400fd8c0 msgr2=0x7f50400fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:32.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:32.372+0000 7f50479ce640 1 -- 172.21.15.6:0/4091112821 shutdown_connections 2024-09-12T03:29:32.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:32.372+0000 7f50479ce640 1 -- 172.21.15.6:0/4091112821 wait complete. 2024-09-12T03:29:32.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:32 smithi118 ceph-mon[35048]: mds.? [v2:172.21.15.161:6835/1168250036,v1:172.21.15.161:6838/1168250036] up:boot 2024-09-12T03:29:32.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:32 smithi118 ceph-mon[35048]: mds.? [v2:172.21.15.161:6834/1657492466,v1:172.21.15.161:6837/1657492466] up:boot 2024-09-12T03:29:32.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:32 smithi118 ceph-mon[35048]: mds.? [v2:172.21.15.161:6836/2540670629,v1:172.21.15.161:6839/2540670629] up:boot 2024-09-12T03:29:32.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:32 smithi118 ceph-mon[35048]: mds.? [v2:172.21.15.118:6834/3151494498,v1:172.21.15.118:6835/3151494498] up:boot 2024-09-12T03:29:32.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:32 smithi118 ceph-mon[35048]: mds.? [v2:172.21.15.118:6836/1206250161,v1:172.21.15.118:6837/1206250161] up:boot 2024-09-12T03:29:32.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:32 smithi118 ceph-mon[35048]: mds.? [v2:172.21.15.118:6838/1554635848,v1:172.21.15.118:6839/1554635848] up:boot 2024-09-12T03:29:32.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:32 smithi118 ceph-mon[35048]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T03:29:32.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:32 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:29:32.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:32 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:29:32.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:32 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:29:32.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:32 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:29:32.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:32 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:29:32.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:32 smithi118 ceph-mon[35048]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:29:32.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:32 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/4091112821' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:29:32.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:32 smithi006 ceph-mon[30991]: mds.? [v2:172.21.15.161:6835/1168250036,v1:172.21.15.161:6838/1168250036] up:boot 2024-09-12T03:29:32.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:32 smithi006 ceph-mon[30991]: mds.? [v2:172.21.15.161:6834/1657492466,v1:172.21.15.161:6837/1657492466] up:boot 2024-09-12T03:29:32.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:32 smithi006 ceph-mon[30991]: mds.? [v2:172.21.15.161:6836/2540670629,v1:172.21.15.161:6839/2540670629] up:boot 2024-09-12T03:29:32.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:32 smithi006 ceph-mon[30991]: mds.? [v2:172.21.15.118:6834/3151494498,v1:172.21.15.118:6835/3151494498] up:boot 2024-09-12T03:29:32.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:32 smithi006 ceph-mon[30991]: mds.? [v2:172.21.15.118:6836/1206250161,v1:172.21.15.118:6837/1206250161] up:boot 2024-09-12T03:29:32.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:32 smithi006 ceph-mon[30991]: mds.? [v2:172.21.15.118:6838/1554635848,v1:172.21.15.118:6839/1554635848] up:boot 2024-09-12T03:29:32.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:32 smithi006 ceph-mon[30991]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T03:29:32.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:32 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:29:32.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:32 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:29:32.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:32 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:29:32.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:32 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:29:32.726 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:32 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:29:32.726 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:32 smithi006 ceph-mon[30991]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:29:32.726 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:32 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/4091112821' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:29:32.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:32 smithi161 ceph-mon[35485]: mds.? [v2:172.21.15.161:6835/1168250036,v1:172.21.15.161:6838/1168250036] up:boot 2024-09-12T03:29:32.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:32 smithi161 ceph-mon[35485]: mds.? [v2:172.21.15.161:6834/1657492466,v1:172.21.15.161:6837/1657492466] up:boot 2024-09-12T03:29:32.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:32 smithi161 ceph-mon[35485]: mds.? [v2:172.21.15.161:6836/2540670629,v1:172.21.15.161:6839/2540670629] up:boot 2024-09-12T03:29:32.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:32 smithi161 ceph-mon[35485]: mds.? [v2:172.21.15.118:6834/3151494498,v1:172.21.15.118:6835/3151494498] up:boot 2024-09-12T03:29:32.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:32 smithi161 ceph-mon[35485]: mds.? [v2:172.21.15.118:6836/1206250161,v1:172.21.15.118:6837/1206250161] up:boot 2024-09-12T03:29:32.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:32 smithi161 ceph-mon[35485]: mds.? [v2:172.21.15.118:6838/1554635848,v1:172.21.15.118:6839/1554635848] up:boot 2024-09-12T03:29:32.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:32 smithi161 ceph-mon[35485]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T03:29:32.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:32 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:29:32.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:32 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:29:32.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:32 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:29:32.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:32 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:29:32.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:32 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:29:32.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:32 smithi161 ceph-mon[35485]: from='mgr.14150 172.21.15.6:0/2287665544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:29:32.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:32 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/4091112821' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:29:33.102 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T03:29:33.366 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:33.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:33 smithi006 ceph-mon[30991]: pgmap v589: 97 pgs: 97 active+clean; 159 MiB data, 100 GiB used, 973 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-12T03:29:33.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:33 smithi006 ceph-mon[30991]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:29:33.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:33 smithi006 ceph-mon[30991]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:29:33.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:33 smithi006 ceph-mon[30991]: Cluster is now healthy 2024-09-12T03:29:33.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:33 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/4091112821' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:29:33.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:33 smithi006 ceph-mon[30991]: fsmap 12 up:standby 2024-09-12T03:29:33.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:33 smithi118 ceph-mon[35048]: pgmap v589: 97 pgs: 97 active+clean; 159 MiB data, 100 GiB used, 973 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-12T03:29:33.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:33 smithi118 ceph-mon[35048]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:29:33.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:33 smithi118 ceph-mon[35048]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:29:33.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:33 smithi118 ceph-mon[35048]: Cluster is now healthy 2024-09-12T03:29:33.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:33 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/4091112821' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:29:33.713 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:33 smithi118 ceph-mon[35048]: fsmap 12 up:standby 2024-09-12T03:29:33.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:33 smithi161 ceph-mon[35485]: pgmap v589: 97 pgs: 97 active+clean; 159 MiB data, 100 GiB used, 973 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-12T03:29:33.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:33 smithi161 ceph-mon[35485]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:29:33.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:33 smithi161 ceph-mon[35485]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:29:33.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:33 smithi161 ceph-mon[35485]: Cluster is now healthy 2024-09-12T03:29:33.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:33 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/4091112821' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:29:33.864 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:33 smithi161 ceph-mon[35485]: fsmap 12 up:standby 2024-09-12T03:29:33.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.892+0000 7f4c96660640 1 -- 172.21.15.6:0/113442012 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c901027a0 msgr2=0x7f4c901067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:33.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.892+0000 7f4c96660640 1 --2- 172.21.15.6:0/113442012 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c901027a0 0x7f4c901067f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009900 tx=0x7f4c8402a3f0 comp rx=0 tx=0).stop 2024-09-12T03:29:33.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.892+0000 7f4c96660640 1 -- 172.21.15.6:0/113442012 shutdown_connections 2024-09-12T03:29:33.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.892+0000 7f4c96660640 1 --2- 172.21.15.6:0/113442012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c90106f20 0x7f4c9010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:33.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.892+0000 7f4c96660640 1 --2- 172.21.15.6:0/113442012 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c901027a0 0x7f4c901067f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:33.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.892+0000 7f4c96660640 1 --2- 172.21.15.6:0/113442012 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c90101df0 0x7f4c901021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:33.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.892+0000 7f4c96660640 1 -- 172.21.15.6:0/113442012 >> 172.21.15.6:0/113442012 conn(0x7f4c900fd8b0 msgr2=0x7f4c900ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:33.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.892+0000 7f4c96660640 1 -- 172.21.15.6:0/113442012 shutdown_connections 2024-09-12T03:29:33.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.892+0000 7f4c96660640 1 -- 172.21.15.6:0/113442012 wait complete. 2024-09-12T03:29:33.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.892+0000 7f4c96660640 1 Processor -- start 2024-09-12T03:29:33.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.892+0000 7f4c96660640 1 -- start start 2024-09-12T03:29:33.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.893+0000 7f4c96660640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c90101df0 0x7f4c90198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:33.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.893+0000 7f4c96660640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c901027a0 0x7f4c90198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:33.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.893+0000 7f4c96660640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c90106f20 0x7f4c9019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:33.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.893+0000 7f4c96660640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4c90111f80 con 0x7f4c90106f20 2024-09-12T03:29:33.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.893+0000 7f4c96660640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f4c90111e00 con 0x7f4c90101df0 2024-09-12T03:29:33.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.893+0000 7f4c96660640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4c90112100 con 0x7f4c901027a0 2024-09-12T03:29:33.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.893+0000 7f4c94bfd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c90106f20 0x7f4c9019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:33.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.893+0000 7f4c94bfd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c90106f20 0x7f4c9019ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50336/0 (socket says 172.21.15.6:50336) 2024-09-12T03:29:33.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.893+0000 7f4c94bfd640 1 -- 172.21.15.6:0/3112215588 learned_addr learned my addr 172.21.15.6:0/3112215588 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:29:33.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.893+0000 7f4c8ffff640 1 --2- 172.21.15.6:0/3112215588 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c90101df0 0x7f4c90198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:33.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.893+0000 7f4c8f7fe640 1 --2- 172.21.15.6:0/3112215588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c901027a0 0x7f4c90198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:33.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.893+0000 7f4c94bfd640 1 -- 172.21.15.6:0/3112215588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c901027a0 msgr2=0x7f4c90198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:33.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.893+0000 7f4c94bfd640 1 --2- 172.21.15.6:0/3112215588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c901027a0 0x7f4c90198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:33.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.893+0000 7f4c94bfd640 1 -- 172.21.15.6:0/3112215588 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c90101df0 msgr2=0x7f4c90198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:33.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.893+0000 7f4c94bfd640 1 --2- 172.21.15.6:0/3112215588 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c90101df0 0x7f4c90198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:33.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.893+0000 7f4c94bfd640 1 -- 172.21.15.6:0/3112215588 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9019d3d0 con 0x7f4c90106f20 2024-09-12T03:29:33.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.893+0000 7f4c8ffff640 1 --2- 172.21.15.6:0/3112215588 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c90101df0 0x7f4c90198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:33.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.893+0000 7f4c94bfd640 1 --2- 172.21.15.6:0/3112215588 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c90106f20 0x7f4c9019ccf0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800eb10 tx=0x7f4c8800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:33.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.894+0000 7f4c8d7fa640 1 -- 172.21.15.6:0/3112215588 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c88004b20 con 0x7f4c90106f20 2024-09-12T03:29:33.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.894+0000 7f4c8d7fa640 1 -- 172.21.15.6:0/3112215588 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c88004cc0 con 0x7f4c90106f20 2024-09-12T03:29:33.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.894+0000 7f4c96660640 1 -- 172.21.15.6:0/3112215588 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c9019d6c0 con 0x7f4c90106f20 2024-09-12T03:29:33.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.894+0000 7f4c8d7fa640 1 -- 172.21.15.6:0/3112215588 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c88010760 con 0x7f4c90106f20 2024-09-12T03:29:33.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.894+0000 7f4c96660640 1 -- 172.21.15.6:0/3112215588 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c901a4fa0 con 0x7f4c90106f20 2024-09-12T03:29:33.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.895+0000 7f4c96660640 1 -- 172.21.15.6:0/3112215588 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c901021d0 con 0x7f4c90106f20 2024-09-12T03:29:33.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.896+0000 7f4c8d7fa640 1 -- 172.21.15.6:0/3112215588 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4c88005020 con 0x7f4c90106f20 2024-09-12T03:29:33.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.897+0000 7f4c8d7fa640 1 --2- 172.21.15.6:0/3112215588 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4c780a2c00 0x7f4c780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:33.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.897+0000 7f4c8d7fa640 1 -- 172.21.15.6:0/3112215588 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8440+0+0 (secure 0 0 0) 0x7f4c880cf5a0 con 0x7f4c90106f20 2024-09-12T03:29:33.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.897+0000 7f4c8ffff640 1 --2- 172.21.15.6:0/3112215588 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4c780a2c00 0x7f4c780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:33.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.897+0000 7f4c8ffff640 1 --2- 172.21.15.6:0/3112215588 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4c780a2c00 0x7f4c780a50c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009fc0 tx=0x7f4c80005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:33.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:33.901+0000 7f4c8d7fa640 1 -- 172.21.15.6:0/3112215588 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4c8809c430 con 0x7f4c90106f20 2024-09-12T03:29:34.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:34.083+0000 7f4c96660640 1 -- 172.21.15.6:0/3112215588 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4c9010a1c0 con 0x7f4c90106f20 2024-09-12T03:29:34.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:34.378+0000 7f4c8d7fa640 1 -- 172.21.15.6:0/3112215588 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v84) ==== 181+0+0 (secure 0 0 0) 0x7f4c8809bd30 con 0x7f4c90106f20 2024-09-12T03:29:34.378 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_metadata' removed 2024-09-12T03:29:34.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:34.381+0000 7f4c96660640 1 -- 172.21.15.6:0/3112215588 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4c780a2c00 msgr2=0x7f4c780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:34.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:34.381+0000 7f4c96660640 1 --2- 172.21.15.6:0/3112215588 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4c780a2c00 0x7f4c780a50c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009fc0 tx=0x7f4c80005f90 comp rx=0 tx=0).stop 2024-09-12T03:29:34.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:34.381+0000 7f4c96660640 1 -- 172.21.15.6:0/3112215588 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c90106f20 msgr2=0x7f4c9019ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:34.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:34.381+0000 7f4c96660640 1 --2- 172.21.15.6:0/3112215588 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c90106f20 0x7f4c9019ccf0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800eb10 tx=0x7f4c8800c2d0 comp rx=0 tx=0).stop 2024-09-12T03:29:34.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:34.382+0000 7f4c96660640 1 -- 172.21.15.6:0/3112215588 shutdown_connections 2024-09-12T03:29:34.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:34.382+0000 7f4c96660640 1 --2- 172.21.15.6:0/3112215588 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f4c780a2c00 0x7f4c780a50c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:34.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:34.382+0000 7f4c96660640 1 --2- 172.21.15.6:0/3112215588 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c90106f20 0x7f4c9019ccf0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:34.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:34.382+0000 7f4c96660640 1 --2- 172.21.15.6:0/3112215588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c901027a0 0x7f4c90198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:34.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:34.382+0000 7f4c96660640 1 --2- 172.21.15.6:0/3112215588 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c90101df0 0x7f4c90198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:34.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:34.382+0000 7f4c96660640 1 -- 172.21.15.6:0/3112215588 >> 172.21.15.6:0/3112215588 conn(0x7f4c900fd8b0 msgr2=0x7f4c900febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:34.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:34.382+0000 7f4c96660640 1 -- 172.21.15.6:0/3112215588 shutdown_connections 2024-09-12T03:29:34.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:34.382+0000 7f4c96660640 1 -- 172.21.15.6:0/3112215588 wait complete. 2024-09-12T03:29:34.707 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:34 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3112215588' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:29:34.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:34 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3112215588' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:29:34.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:34 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3112215588' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:29:35.047 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 89d99938-70b4-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T03:29:35.310 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:35.637 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:35 smithi006 ceph-mon[30991]: pgmap v590: 97 pgs: 97 active+clean; 159 MiB data, 100 GiB used, 973 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 716 B/s wr, 6 op/s 2024-09-12T03:29:35.637 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:35 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/3112215588' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:29:35.637 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:35 smithi006 ceph-mon[30991]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T03:29:35.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:35 smithi118 ceph-mon[35048]: pgmap v590: 97 pgs: 97 active+clean; 159 MiB data, 100 GiB used, 973 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 716 B/s wr, 6 op/s 2024-09-12T03:29:35.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:35 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/3112215588' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:29:35.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:35 smithi118 ceph-mon[35048]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T03:29:35.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.831+0000 7f879c1ed640 1 -- 172.21.15.6:0/3927953031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87941036e0 msgr2=0x7f8794107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:35.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.831+0000 7f879c1ed640 1 --2- 172.21.15.6:0/3927953031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87941036e0 0x7f8794107b90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8784009930 tx=0x7f878402a420 comp rx=0 tx=0).stop 2024-09-12T03:29:35.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.831+0000 7f879c1ed640 1 -- 172.21.15.6:0/3927953031 shutdown_connections 2024-09-12T03:29:35.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.831+0000 7f879c1ed640 1 --2- 172.21.15.6:0/3927953031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87941082a0 0x7f879410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:35.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.831+0000 7f879c1ed640 1 --2- 172.21.15.6:0/3927953031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87941036e0 0x7f8794107b90 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:35.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.831+0000 7f879c1ed640 1 --2- 172.21.15.6:0/3927953031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8794102d30 0x7f8794103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:35.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.831+0000 7f879c1ed640 1 -- 172.21.15.6:0/3927953031 >> 172.21.15.6:0/3927953031 conn(0x7f87940fc7d0 msgr2=0x7f87940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:35.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.831+0000 7f879c1ed640 1 -- 172.21.15.6:0/3927953031 shutdown_connections 2024-09-12T03:29:35.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.831+0000 7f879c1ed640 1 -- 172.21.15.6:0/3927953031 wait complete. 2024-09-12T03:29:35.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.832+0000 7f879c1ed640 1 Processor -- start 2024-09-12T03:29:35.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.832+0000 7f879c1ed640 1 -- start start 2024-09-12T03:29:35.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.832+0000 7f879c1ed640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8794102d30 0x7f87941a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:35.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.832+0000 7f879c1ed640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87941036e0 0x7f87941a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:35.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.832+0000 7f879c1ed640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87941082a0 0x7f879419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:35.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.832+0000 7f879c1ed640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8794112060 con 0x7f87941036e0 2024-09-12T03:29:35.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.832+0000 7f879c1ed640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 -- 0x7f8794111ee0 con 0x7f87941082a0 2024-09-12T03:29:35.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.832+0000 7f879c1ed640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f87941121e0 con 0x7f8794102d30 2024-09-12T03:29:35.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.832+0000 7f8799788640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87941036e0 0x7f87941a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:35.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.832+0000 7f8799f89640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8794102d30 0x7f87941a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:35.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.832+0000 7f8799f89640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8794102d30 0x7f87941a0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.6:40320/0 (socket says 172.21.15.6:40320) 2024-09-12T03:29:35.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.832+0000 7f8799788640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87941036e0 0x7f87941a0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50362/0 (socket says 172.21.15.6:50362) 2024-09-12T03:29:35.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.832+0000 7f8799f89640 1 -- 172.21.15.6:0/47417318 learned_addr learned my addr 172.21.15.6:0/47417318 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-09-12T03:29:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.832+0000 7f879a78a640 1 --2- 172.21.15.6:0/47417318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87941082a0 0x7f879419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.833+0000 7f8799f89640 1 -- 172.21.15.6:0/47417318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87941082a0 msgr2=0x7f879419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.833+0000 7f8799f89640 1 --2- 172.21.15.6:0/47417318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87941082a0 0x7f879419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.833+0000 7f8799f89640 1 -- 172.21.15.6:0/47417318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87941036e0 msgr2=0x7f87941a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.833+0000 7f8799f89640 1 --2- 172.21.15.6:0/47417318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87941036e0 0x7f87941a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.833+0000 7f8799f89640 1 -- 172.21.15.6:0/47417318 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879419aea0 con 0x7f8794102d30 2024-09-12T03:29:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.833+0000 7f8799788640 1 --2- 172.21.15.6:0/47417318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87941036e0 0x7f87941a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.833+0000 7f879a78a640 1 --2- 172.21.15.6:0/47417318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87941082a0 0x7f879419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.833+0000 7f8799f89640 1 --2- 172.21.15.6:0/47417318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8794102d30 0x7f87941a0560 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f879000d950 tx=0x7f879000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.833+0000 7f878affd640 1 -- 172.21.15.6:0/47417318 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8790017030 con 0x7f8794102d30 2024-09-12T03:29:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.833+0000 7f879c1ed640 1 -- 172.21.15.6:0/47417318 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f879419b190 con 0x7f8794102d30 2024-09-12T03:29:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.833+0000 7f878affd640 1 -- 172.21.15.6:0/47417318 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8790004020 con 0x7f8794102d30 2024-09-12T03:29:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.833+0000 7f878affd640 1 -- 172.21.15.6:0/47417318 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87900047a0 con 0x7f8794102d30 2024-09-12T03:29:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.833+0000 7f879c1ed640 1 -- 172.21.15.6:0/47417318 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87941a74e0 con 0x7f8794102d30 2024-09-12T03:29:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.834+0000 7f879c1ed640 1 -- 172.21.15.6:0/47417318 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f879406a850 con 0x7f8794102d30 2024-09-12T03:29:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.835+0000 7f878affd640 1 -- 172.21.15.6:0/47417318 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f87900041d0 con 0x7f8794102d30 2024-09-12T03:29:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.836+0000 7f878affd640 1 --2- 172.21.15.6:0/47417318 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f87800a2c00 0x7f87800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.836+0000 7f8799788640 1 --2- 172.21.15.6:0/47417318 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f87800a2c00 0x7f87800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.836+0000 7f878affd640 1 -- 172.21.15.6:0/47417318 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7915+0+0 (secure 0 0 0) 0x7f87900ceb60 con 0x7f8794102d30 2024-09-12T03:29:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.837+0000 7f8799788640 1 --2- 172.21.15.6:0/47417318 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f87800a2c00 0x7f87800a50c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8784005940 tx=0x7f87840058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:35.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:35.839+0000 7f878affd640 1 -- 172.21.15.6:0/47417318 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f879009bc30 con 0x7f8794102d30 2024-09-12T03:29:35.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:35 smithi161 ceph-mon[35485]: pgmap v590: 97 pgs: 97 active+clean; 159 MiB data, 100 GiB used, 973 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 716 B/s wr, 6 op/s 2024-09-12T03:29:35.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:35 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/3112215588' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:29:35.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:35 smithi161 ceph-mon[35485]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T03:29:36.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:36.016+0000 7f879c1ed640 1 -- 172.21.15.6:0/47417318 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f879419bb60 con 0x7f8794102d30 2024-09-12T03:29:36.386 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:36 smithi161 ceph-mon[35485]: from='client.? 172.21.15.6:0/47417318' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:29:36.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:36.398+0000 7f878affd640 1 -- 172.21.15.6:0/47417318 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v85) ==== 169+0+0 (secure 0 0 0) 0x7f879009b530 con 0x7f8794102d30 2024-09-12T03:29:36.399 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data' removed 2024-09-12T03:29:36.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:36.401+0000 7f879c1ed640 1 -- 172.21.15.6:0/47417318 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f87800a2c00 msgr2=0x7f87800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:36.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:36.401+0000 7f879c1ed640 1 --2- 172.21.15.6:0/47417318 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f87800a2c00 0x7f87800a50c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8784005940 tx=0x7f87840058d0 comp rx=0 tx=0).stop 2024-09-12T03:29:36.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:36.401+0000 7f879c1ed640 1 -- 172.21.15.6:0/47417318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8794102d30 msgr2=0x7f87941a0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:36.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:36.401+0000 7f879c1ed640 1 --2- 172.21.15.6:0/47417318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8794102d30 0x7f87941a0560 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f879000d950 tx=0x7f879000de10 comp rx=0 tx=0).stop 2024-09-12T03:29:36.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:36.402+0000 7f879c1ed640 1 -- 172.21.15.6:0/47417318 shutdown_connections 2024-09-12T03:29:36.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:36.402+0000 7f879c1ed640 1 --2- 172.21.15.6:0/47417318 >> [v2:172.21.15.6:6800/5477767,v1:172.21.15.6:6801/5477767] conn(0x7f87800a2c00 0x7f87800a50c0 secure :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8784005940 tx=0x7f87840058d0 comp rx=0 tx=0).stop 2024-09-12T03:29:36.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:36.402+0000 7f879c1ed640 1 --2- 172.21.15.6:0/47417318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87941082a0 0x7f879419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:36.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:36.402+0000 7f879c1ed640 1 --2- 172.21.15.6:0/47417318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87941036e0 0x7f87941a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:36.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:36.402+0000 7f879c1ed640 1 --2- 172.21.15.6:0/47417318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8794102d30 0x7f87941a0560 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f879000d950 tx=0x7f879000de10 comp rx=0 tx=0).stop 2024-09-12T03:29:36.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:36.402+0000 7f879c1ed640 1 -- 172.21.15.6:0/47417318 >> 172.21.15.6:0/47417318 conn(0x7f87940fc7d0 msgr2=0x7f87941008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:36.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:36.403+0000 7f879c1ed640 1 -- 172.21.15.6:0/47417318 shutdown_connections 2024-09-12T03:29:36.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-09-12T03:29:36.403+0000 7f879c1ed640 1 -- 172.21.15.6:0/47417318 wait complete. 2024-09-12T03:29:36.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:36 smithi118 ceph-mon[35048]: from='client.? 172.21.15.6:0/47417318' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:29:36.712 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:36 smithi118 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:29:36.771 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:36 smithi006 ceph-mon[30991]: from='client.? 172.21.15.6:0/47417318' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:29:36.772 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:36 smithi006 ceph-mon[30991]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:29:36.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:36 smithi161 ceph-mon[35485]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:29:37.099 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:29:37.127 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:29:37.162 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:29:37.190 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-12T03:29:37.191 DEBUG:teuthology.orchestra.run.smithi006:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:29:37.207 DEBUG:teuthology.orchestra.run.smithi118:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:29:37.223 DEBUG:teuthology.orchestra.run.smithi161:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:29:37.253 INFO:tasks.cephadm:Stopping all daemons... 2024-09-12T03:29:37.254 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-12T03:29:37.254 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@mon.a 2024-09-12T03:29:37.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:37 smithi006 systemd[1]: Stopping Ceph mon.a for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:29:37.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:37 smithi118 ceph-mon[35048]: pgmap v592: 65 pgs: 65 active+clean; 577 KiB data, 100 GiB used, 973 GiB / 1.0 TiB avail 2024-09-12T03:29:37.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:37 smithi118 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:29:37.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:37 smithi118 ceph-mon[35048]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T03:29:37.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:37 smithi161 ceph-mon[35485]: pgmap v592: 65 pgs: 65 active+clean; 577 KiB data, 100 GiB used, 973 GiB / 1.0 TiB avail 2024-09-12T03:29:37.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:37 smithi161 ceph-mon[35485]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:29:37.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:37 smithi161 ceph-mon[35485]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T03:29:37.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:37 smithi006 ceph-mon[30991]: pgmap v592: 65 pgs: 65 active+clean; 577 KiB data, 100 GiB used, 973 GiB / 1.0 TiB avail 2024-09-12T03:29:37.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:37 smithi006 ceph-mon[30991]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:29:37.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:37 smithi006 ceph-mon[30991]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T03:29:37.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:37 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-a[30987]: 2024-09-12T03:29:37.591+0000 7f70461c1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:29:37.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:37 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-a[30987]: 2024-09-12T03:29:37.591+0000 7f70461c1640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-12T03:29:37.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:37 smithi006 podman[101538]: 2024-09-12 03:29:37.642129893 +0000 UTC m=+0.168018890 container died 29ec88f97057c5e91ed7f19e183beb5e3552940502247b6e1d99ce0a0195e70c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-12T03:29:38.277 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:38 smithi006 podman[101538]: 2024-09-12 03:29:38.051606909 +0000 UTC m=+0.577495911 container cleanup 29ec88f97057c5e91ed7f19e183beb5e3552940502247b6e1d99ce0a0195e70c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-12T03:29:38.277 INFO:journalctl@ceph.mon.a.smithi006.stdout:Sep 12 03:29:38 smithi006 bash[101538]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-a 2024-09-12T03:29:38.434 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mon.a.service' 2024-09-12T03:29:38.479 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:29:38.479 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-12T03:29:38.479 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-12T03:29:38.480 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@mon.b 2024-09-12T03:29:38.773 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:38 smithi118 systemd[1]: Stopping Ceph mon.b for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:29:39.198 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:38 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-b[35044]: 2024-09-12T03:29:38.772+0000 7f6dabb6c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:29:39.199 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:38 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-b[35044]: 2024-09-12T03:29:38.772+0000 7f6dabb6c640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-12T03:29:39.199 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:38 smithi118 podman[68957]: 2024-09-12 03:29:38.936027886 +0000 UTC m=+0.267542420 container died 7cca8700c6316e48b1e8da563821c7dd6c30e8809bbc422a8c8531913504aed3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-09-12T03:29:39.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:39 smithi118 podman[68957]: 2024-09-12 03:29:39.297709654 +0000 UTC m=+0.629224187 container cleanup 7cca8700c6316e48b1e8da563821c7dd6c30e8809bbc422a8c8531913504aed3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:29:39.462 INFO:journalctl@ceph.mon.b.smithi118.stdout:Sep 12 03:29:39 smithi118 bash[68957]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-b 2024-09-12T03:29:39.729 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mon.b.service' 2024-09-12T03:29:39.777 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:29:39.778 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-12T03:29:39.778 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-12T03:29:39.778 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@mon.c 2024-09-12T03:29:40.060 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:39 smithi161 systemd[1]: Stopping Ceph mon.c for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:29:40.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:40 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-c[35481]: 2024-09-12T03:29:40.058+0000 7f3ecc08a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:29:40.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:40 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-c[35481]: 2024-09-12T03:29:40.058+0000 7f3ecc08a640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-12T03:29:40.363 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:40 smithi161 podman[70121]: 2024-09-12 03:29:40.202575718 +0000 UTC m=+0.236089055 container died 892bf1db843aa52b55ab06aa4ebd35b4b33aee932e2216aa054c52bcd1fcae17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-c, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:29:40.760 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:40 smithi161 podman[70121]: 2024-09-12 03:29:40.5431517 +0000 UTC m=+0.576665036 container cleanup 892bf1db843aa52b55ab06aa4ebd35b4b33aee932e2216aa054c52bcd1fcae17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-c, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:29:40.760 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:40 smithi161 bash[70121]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-c 2024-09-12T03:29:41.025 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 12 03:29:40 smithi161 podman[70134]: 2024-09-12 03:29:40.761169561 +0000 UTC m=+0.556968519 container remove 892bf1db843aa52b55ab06aa4ebd35b4b33aee932e2216aa054c52bcd1fcae17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:29:41.029 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mon.c.service' 2024-09-12T03:29:41.074 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:29:41.074 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-12T03:29:41.074 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-12T03:29:41.074 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@mgr.x 2024-09-12T03:29:41.362 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:29:41 smithi006 systemd[1]: Stopping Ceph mgr.x for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:29:41.715 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:29:41 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:29:41.388+0000 7f8b35498640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:29:41.716 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:29:41 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x[31220]: 2024-09-12T03:29:41.388+0000 7f8b35498640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-12T03:29:41.716 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:29:41 smithi006 podman[101658]: 2024-09-12 03:29:41.420170496 +0000 UTC m=+0.160779454 container died 7792be15ececa21c77a936256cce69d498d0d2e60a52c2af05b04f6a8ccc6a7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-12T03:29:42.074 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:29:41 smithi006 podman[101658]: 2024-09-12 03:29:41.822786679 +0000 UTC m=+0.563395628 container cleanup 7792be15ececa21c77a936256cce69d498d0d2e60a52c2af05b04f6a8ccc6a7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:29:42.074 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:29:41 smithi006 bash[101658]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x 2024-09-12T03:29:42.306 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-12T03:29:42.328 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:29:42 smithi006 podman[101670]: 2024-09-12 03:29:42.074330199 +0000 UTC m=+0.650517700 container remove 7792be15ececa21c77a936256cce69d498d0d2e60a52c2af05b04f6a8ccc6a7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-12T03:29:42.328 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:29:42 smithi006 systemd[1]: ceph-89d99938-70b4-11ef-bcea-c7b262605968@mgr.x.service: Deactivated successfully. 2024-09-12T03:29:42.328 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:29:42 smithi006 systemd[1]: Stopped Ceph mgr.x for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:29:42.328 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Sep 12 03:29:42 smithi006 systemd[1]: ceph-89d99938-70b4-11ef-bcea-c7b262605968@mgr.x.service: Consumed 45.850s CPU time. 2024-09-12T03:29:42.341 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:29:42.342 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-12T03:29:42.342 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-12T03:29:42.342 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@mgr.y 2024-09-12T03:29:42.675 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:29:42 smithi118 systemd[1]: Stopping Ceph mgr.y for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:29:42.944 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:29:42 smithi118 podman[69091]: 2024-09-12 03:29:42.675929315 +0000 UTC m=+0.136336297 container died 19c6b21ba90fdbd48a95743c71bdd4c34e1f6e4dc2ba41bb45d6ca4ff72d2477 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:29:43.212 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:29:43 smithi118 podman[69091]: 2024-09-12 03:29:43.044968463 +0000 UTC m=+0.505375445 container cleanup 19c6b21ba90fdbd48a95743c71bdd4c34e1f6e4dc2ba41bb45d6ca4ff72d2477 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-09-12T03:29:43.212 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 12 03:29:43 smithi118 bash[69091]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-y 2024-09-12T03:29:43.499 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-12T03:29:43.535 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:29:43.536 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-12T03:29:43.536 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-12T03:29:43.536 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@mgr.z 2024-09-12T03:29:43.864 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:29:43 smithi161 systemd[1]: Stopping Ceph mgr.z for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:29:44.155 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:29:43 smithi161 podman[70245]: 2024-09-12 03:29:43.869401472 +0000 UTC m=+0.144824535 container died 27802b55bbb014b4d350737013d3cc92106d2f6924f73babadc91be61924bde2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:29:44.407 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:29:44 smithi161 podman[70245]: 2024-09-12 03:29:44.156079624 +0000 UTC m=+0.431502680 container cleanup 27802b55bbb014b4d350737013d3cc92106d2f6924f73babadc91be61924bde2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-12T03:29:44.407 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 12 03:29:44 smithi161 bash[70245]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-mgr-z 2024-09-12T03:29:44.600 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-12T03:29:44.637 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:29:44.637 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-12T03:29:44.638 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-12T03:29:44.638 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.0 2024-09-12T03:29:44.927 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:29:44 smithi006 systemd[1]: Stopping Ceph osd.0 for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:29:45.362 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:29:44 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-0[45348]: 2024-09-12T03:29:44.927+0000 7faa1a362640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:29:45.362 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:29:44 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-0[45348]: 2024-09-12T03:29:44.927+0000 7faa1a362640 -1 osd.0 85 *** Got signal Terminated *** 2024-09-12T03:29:45.362 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:29:44 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-0[45348]: 2024-09-12T03:29:44.927+0000 7faa1a362640 -1 osd.0 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:30:15.311 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:30:14 smithi006 bash[101777]: time="2024-09-12T03:30:14Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-0 in 30 seconds, resorting to SIGKILL" 2024-09-12T03:30:15.312 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:30:15 smithi006 podman[101777]: 2024-09-12 03:30:15.028180575 +0000 UTC m=+30.202514970 container died a72f5b2efba9d539d90281cd5dc8e6d8535e4c8c418d3b1a487b60166a19960b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:30:15.862 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:30:15 smithi006 podman[101777]: 2024-09-12 03:30:15.593451988 +0000 UTC m=+30.767786380 container cleanup a72f5b2efba9d539d90281cd5dc8e6d8535e4c8c418d3b1a487b60166a19960b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-12T03:30:15.862 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:30:15 smithi006 bash[101777]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-0 2024-09-12T03:30:16.219 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:30:15 smithi006 podman[101808]: 2024-09-12 03:30:15.894945448 +0000 UTC m=+0.862837102 container remove a72f5b2efba9d539d90281cd5dc8e6d8535e4c8c418d3b1a487b60166a19960b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-12T03:30:16.219 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:30:15 smithi006 systemd[1]: ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.0.service: Main process exited, code=exited, status=137/n/a 2024-09-12T03:30:16.472 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:30:16 smithi006 podman[101893]: 2024-09-12 03:30:16.122554765 +0000 UTC m=+0.016194554 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:30:16.473 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:30:16 smithi006 podman[101893]: 2024-09-12 03:30:16.229000061 +0000 UTC m=+0.122639833 container create ad923f3ea7c2dbcab0fc60bf7794d0a3b9e8cf8874ee18d2bcdf97d012298de6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:30:16.473 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:30:16 smithi006 podman[101893]: 2024-09-12 03:30:16.473054749 +0000 UTC m=+0.366694522 container init ad923f3ea7c2dbcab0fc60bf7794d0a3b9e8cf8874ee18d2bcdf97d012298de6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-12T03:30:16.862 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:30:16 smithi006 podman[101893]: 2024-09-12 03:30:16.476641184 +0000 UTC m=+0.370280958 container start ad923f3ea7c2dbcab0fc60bf7794d0a3b9e8cf8874ee18d2bcdf97d012298de6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731) 2024-09-12T03:30:16.862 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:30:16 smithi006 podman[101893]: 2024-09-12 03:30:16.564758631 +0000 UTC m=+0.458398423 container attach ad923f3ea7c2dbcab0fc60bf7794d0a3b9e8cf8874ee18d2bcdf97d012298de6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-12T03:30:16.862 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:30:16 smithi006 podman[101893]: 2024-09-12 03:30:16.673681815 +0000 UTC m=+0.567321587 container died ad923f3ea7c2dbcab0fc60bf7794d0a3b9e8cf8874ee18d2bcdf97d012298de6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-09-12T03:30:17.260 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.0.service' 2024-09-12T03:30:17.261 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:30:17 smithi006 podman[101916]: 2024-09-12 03:30:17.234968708 +0000 UTC m=+0.555183511 container remove ad923f3ea7c2dbcab0fc60bf7794d0a3b9e8cf8874ee18d2bcdf97d012298de6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-12T03:30:17.261 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:30:17 smithi006 systemd[1]: ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.0.service: Failed with result 'exit-code'. 2024-09-12T03:30:17.261 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:30:17 smithi006 systemd[1]: Stopped Ceph osd.0 for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:30:17.261 INFO:journalctl@ceph.osd.0.smithi006.stdout:Sep 12 03:30:17 smithi006 systemd[1]: ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.0.service: Consumed 8min 29.149s CPU time. 2024-09-12T03:30:17.302 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:30:17.302 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-12T03:30:17.302 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-12T03:30:17.302 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.1 2024-09-12T03:30:17.640 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:30:17 smithi118 systemd[1]: Stopping Ceph osd.1 for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:30:17.962 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:30:17 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-1[39554]: 2024-09-12T03:30:17.639+0000 7fc29c9fc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:30:17.962 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:30:17 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-1[39554]: 2024-09-12T03:30:17.639+0000 7fc29c9fc640 -1 osd.1 85 *** Got signal Terminated *** 2024-09-12T03:30:17.962 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:30:17 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-1[39554]: 2024-09-12T03:30:17.639+0000 7fc29c9fc640 -1 osd.1 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:30:23.398 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:30:23 smithi118 podman[69229]: 2024-09-12 03:30:23.092483007 +0000 UTC m=+5.579852237 container died cdbe673d9f8ea76c0a4308e7d988ceec5de9a9a5154949e3fd08c9f95b6d481a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-1, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-12T03:30:23.701 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:30:23 smithi118 podman[69229]: 2024-09-12 03:30:23.398505221 +0000 UTC m=+5.885874439 container cleanup cdbe673d9f8ea76c0a4308e7d988ceec5de9a9a5154949e3fd08c9f95b6d481a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-1, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:30:23.701 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:30:23 smithi118 bash[69229]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-1 2024-09-12T03:30:23.701 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:30:23 smithi118 podman[69245]: 2024-09-12 03:30:23.633246141 +0000 UTC m=+0.536463252 container remove cdbe673d9f8ea76c0a4308e7d988ceec5de9a9a5154949e3fd08c9f95b6d481a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-1, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-12T03:30:23.955 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:30:23 smithi118 podman[69334]: 2024-09-12 03:30:23.95094698 +0000 UTC m=+0.107887327 container create eb28231dbd83eff1060aee51bd018c88414de0478095cbb23c972ca37a9a7c58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3) 2024-09-12T03:30:23.955 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:30:23 smithi118 podman[69334]: 2024-09-12 03:30:23.858753116 +0000 UTC m=+0.015693478 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:30:24.212 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:30:24 smithi118 podman[69334]: 2024-09-12 03:30:24.187221406 +0000 UTC m=+0.344161753 container init eb28231dbd83eff1060aee51bd018c88414de0478095cbb23c972ca37a9a7c58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:30:24.212 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:30:24 smithi118 podman[69334]: 2024-09-12 03:30:24.190467475 +0000 UTC m=+0.347407823 container start eb28231dbd83eff1060aee51bd018c88414de0478095cbb23c972ca37a9a7c58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-12T03:30:24.681 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:30:24 smithi118 podman[69334]: 2024-09-12 03:30:24.303941799 +0000 UTC m=+0.460882155 container attach eb28231dbd83eff1060aee51bd018c88414de0478095cbb23c972ca37a9a7c58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-12T03:30:24.682 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 12 03:30:24 smithi118 podman[69334]: 2024-09-12 03:30:24.384240385 +0000 UTC m=+0.541180753 container died eb28231dbd83eff1060aee51bd018c88414de0478095cbb23c972ca37a9a7c58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-12T03:30:24.993 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.1.service' 2024-09-12T03:30:25.033 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:30:25.033 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-12T03:30:25.034 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-12T03:30:25.034 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.2 2024-09-12T03:30:25.332 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:30:25 smithi161 systemd[1]: Stopping Ceph osd.2 for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:30:25.602 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:30:25 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-2[40470]: 2024-09-12T03:30:25.331+0000 7f00151c2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:30:25.602 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:30:25 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-2[40470]: 2024-09-12T03:30:25.331+0000 7f00151c2640 -1 osd.2 85 *** Got signal Terminated *** 2024-09-12T03:30:25.602 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:30:25 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-2[40470]: 2024-09-12T03:30:25.331+0000 7f00151c2640 -1 osd.2 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:30:55.614 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:30:55 smithi161 bash[70383]: time="2024-09-12T03:30:55Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-2 in 30 seconds, resorting to SIGKILL" 2024-09-12T03:30:55.614 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:30:55 smithi161 podman[70383]: 2024-09-12 03:30:55.438690257 +0000 UTC m=+30.209465299 container died 8f9438d60b4df744969c12c595970592456a9ae5f73cd1bf9f9e491dbd606d7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-12T03:30:56.081 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:30:55 smithi161 podman[70383]: 2024-09-12 03:30:55.788823899 +0000 UTC m=+30.559598942 container cleanup 8f9438d60b4df744969c12c595970592456a9ae5f73cd1bf9f9e491dbd606d7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:30:56.081 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:30:55 smithi161 bash[70383]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-2 2024-09-12T03:30:56.363 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:30:56 smithi161 podman[70413]: 2024-09-12 03:30:56.081531244 +0000 UTC m=+0.644783404 container remove 8f9438d60b4df744969c12c595970592456a9ae5f73cd1bf9f9e491dbd606d7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-12T03:30:56.363 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:30:56 smithi161 systemd[1]: ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.2.service: Main process exited, code=exited, status=137/n/a 2024-09-12T03:30:56.750 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:30:56 smithi161 podman[70501]: 2024-09-12 03:30:56.301893169 +0000 UTC m=+0.016100092 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:30:56.751 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:30:56 smithi161 podman[70501]: 2024-09-12 03:30:56.415780388 +0000 UTC m=+0.129987314 container create c02199a086fcf331e420bee6a6175a1185a7570fc714a0ffabf14bc6f7bb406d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-12T03:30:56.751 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:30:56 smithi161 podman[70501]: 2024-09-12 03:30:56.634543167 +0000 UTC m=+0.348750095 container init c02199a086fcf331e420bee6a6175a1185a7570fc714a0ffabf14bc6f7bb406d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-12T03:30:56.751 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:30:56 smithi161 podman[70501]: 2024-09-12 03:30:56.637873466 +0000 UTC m=+0.352080391 container start c02199a086fcf331e420bee6a6175a1185a7570fc714a0ffabf14bc6f7bb406d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-12T03:30:56.751 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:30:56 smithi161 podman[70501]: 2024-09-12 03:30:56.751321352 +0000 UTC m=+0.465528293 container attach c02199a086fcf331e420bee6a6175a1185a7570fc714a0ffabf14bc6f7bb406d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-12T03:30:57.113 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 12 03:30:56 smithi161 podman[70501]: 2024-09-12 03:30:56.838749039 +0000 UTC m=+0.552955963 container died c02199a086fcf331e420bee6a6175a1185a7570fc714a0ffabf14bc6f7bb406d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-12T03:30:57.458 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.2.service' 2024-09-12T03:30:57.498 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:30:57.498 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-12T03:30:57.499 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-12T03:30:57.499 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.3 2024-09-12T03:30:57.802 INFO:journalctl@ceph.osd.3.smithi006.stdout:Sep 12 03:30:57 smithi006 systemd[1]: Stopping Ceph osd.3 for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:30:58.111 INFO:journalctl@ceph.osd.3.smithi006.stdout:Sep 12 03:30:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-3[51847]: 2024-09-12T03:30:57.801+0000 7f824d0fa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:30:58.112 INFO:journalctl@ceph.osd.3.smithi006.stdout:Sep 12 03:30:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-3[51847]: 2024-09-12T03:30:57.801+0000 7f824d0fa640 -1 osd.3 85 *** Got signal Terminated *** 2024-09-12T03:30:58.112 INFO:journalctl@ceph.osd.3.smithi006.stdout:Sep 12 03:30:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-3[51847]: 2024-09-12T03:30:57.801+0000 7f824d0fa640 -1 osd.3 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:31:16.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:15 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:15.961+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:17.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:16 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:16.939+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:17.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:17 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:17.534+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:17.976 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:17 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:17.655+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:18.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:17 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:17.976+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:18.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:18 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:18.342+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:18.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:18 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:18.581+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:18.983 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:18 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:18.640+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:19.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:18 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:18.982+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:19.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:19 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:19.327+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:19.613 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:19 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:19.521+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:19.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:19 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:19.593+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:19.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:19 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:19.587+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:20.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:19 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:19.976+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:20.613 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:20 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:20.510+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:20.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:20 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:20.289+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:20.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:20 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:20.582+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:20.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:20 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:20.550+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:21.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:20 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:20.950+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:21.560 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:21 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:21.308+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:21.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:21.598+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:21.863 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:21 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:21.560+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:21.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:21 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:21.531+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:22.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:21.971+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:22.613 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:22 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:22.520+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:22.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:22 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:22.290+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:22.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:22 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:22.569+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:22.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:22 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:22.552+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:23.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:22 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:22.980+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:23.613 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:23 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:23.541+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:23.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:23 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:23.321+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:23.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:23 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:23.600+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:23.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:23 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:23.600+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:23.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:23 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:23.586+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:23.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:23 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:23.586+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:24.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:23 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:23.995+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:24.462 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:24 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4[46088]: 2024-09-12T03:31:24.043+0000 7fde3f9de640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.600280+0000 front 2024-09-12T03:30:22.600298+0000 (oldest deadline 2024-09-12T03:31:23.100382+0000) 2024-09-12T03:31:24.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:24 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:24.356+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:24.613 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:24 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:24.551+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:24.613 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:24 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:24.551+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.977593+0000 front 2024-09-12T03:30:18.977760+0000 (oldest deadline 2024-09-12T03:31:24.277584+0000) 2024-09-12T03:31:24.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:24 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:24.597+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:24.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:24 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:24.597+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:24.995 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:24 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:24.641+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:24.995 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:24 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:24.641+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:25.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:24 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:24.995+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:25.363 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:25 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:25.049+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:25.462 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:25 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4[46088]: 2024-09-12T03:31:25.050+0000 7fde3f9de640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.600280+0000 front 2024-09-12T03:30:22.600298+0000 (oldest deadline 2024-09-12T03:31:23.100382+0000) 2024-09-12T03:31:25.863 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:25 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:25.554+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:25.863 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:25 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:25.554+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.977593+0000 front 2024-09-12T03:30:18.977760+0000 (oldest deadline 2024-09-12T03:31:24.277584+0000) 2024-09-12T03:31:25.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:25 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:25.375+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:25.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:25 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:25.375+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:25.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:25 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:25.637+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:25.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:25 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:25.637+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:26.009 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:25 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:25.684+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:26.009 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:25 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:25.684+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:26.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:26 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:26.009+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:26.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:26 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:26.009+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:26.363 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:26 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:26.062+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:26.462 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:26 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4[46088]: 2024-09-12T03:31:26.062+0000 7fde3f9de640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.600280+0000 front 2024-09-12T03:30:22.600298+0000 (oldest deadline 2024-09-12T03:31:23.100382+0000) 2024-09-12T03:31:26.462 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:25 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:25.987+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:26.863 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:26 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:26.540+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:26.863 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:26 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:26.540+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.977593+0000 front 2024-09-12T03:30:18.977760+0000 (oldest deadline 2024-09-12T03:31:24.277584+0000) 2024-09-12T03:31:26.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:26 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:26.395+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:26.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:26 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:26.395+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:26.961 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:26 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:26.700+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:26.961 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:26 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:26.700+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:26.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:26 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:26.608+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:26.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:26 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:26.608+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:27.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:26 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:26.961+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:27.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:26 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:26.961+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:27.363 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:27 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:27.094+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:27.462 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:27 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4[46088]: 2024-09-12T03:31:27.056+0000 7fde3f9de640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.600280+0000 front 2024-09-12T03:30:22.600298+0000 (oldest deadline 2024-09-12T03:31:23.100382+0000) 2024-09-12T03:31:27.462 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:27 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:27.014+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:27.863 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:27 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:27.539+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:27.863 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:27 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:27.539+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.977593+0000 front 2024-09-12T03:30:18.977760+0000 (oldest deadline 2024-09-12T03:31:24.277584+0000) 2024-09-12T03:31:27.863 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:27 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:27.409+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:27.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:27 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:27.409+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:27.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:27 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:27.575+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:27.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:27 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:27.575+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:28.111 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:27 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:27.917+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:28.112 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:27 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:27.917+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:28.112 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:27 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:27.722+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:28.112 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:27 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:27.722+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:28.112 INFO:journalctl@ceph.osd.3.smithi006.stdout:Sep 12 03:31:27 smithi006 bash[102006]: time="2024-09-12T03:31:27Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-3 in 30 seconds, resorting to SIGKILL" 2024-09-12T03:31:28.112 INFO:journalctl@ceph.osd.3.smithi006.stdout:Sep 12 03:31:27 smithi006 podman[102006]: 2024-09-12 03:31:27.926565997 +0000 UTC m=+30.228452768 container died 4e170146bd51606ea730d271e126889436ecbade9104f6e0009d7e667027adda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-3, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-12T03:31:28.363 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:28 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:28.108+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:28.462 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:28 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4[46088]: 2024-09-12T03:31:28.104+0000 7fde3f9de640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.600280+0000 front 2024-09-12T03:30:22.600298+0000 (oldest deadline 2024-09-12T03:31:23.100382+0000) 2024-09-12T03:31:28.463 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:27 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:27.969+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:28.785 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:28 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:28.674+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:28.785 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:28 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:28.674+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:28.785 INFO:journalctl@ceph.osd.3.smithi006.stdout:Sep 12 03:31:28 smithi006 podman[102006]: 2024-09-12 03:31:28.525679463 +0000 UTC m=+30.827566224 container cleanup 4e170146bd51606ea730d271e126889436ecbade9104f6e0009d7e667027adda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-3, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:31:28.785 INFO:journalctl@ceph.osd.3.smithi006.stdout:Sep 12 03:31:28 smithi006 bash[102006]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-3 2024-09-12T03:31:28.863 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:28 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:28.572+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:28.863 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:28 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:28.572+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.977593+0000 front 2024-09-12T03:30:18.977760+0000 (oldest deadline 2024-09-12T03:31:24.277584+0000) 2024-09-12T03:31:28.863 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:28 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:28.373+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:28.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:28 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:28.373+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:28.927 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:28 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:28.600+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:28.927 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:28 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:28.600+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:29.111 INFO:journalctl@ceph.osd.3.smithi006.stdout:Sep 12 03:31:28 smithi006 podman[102035]: 2024-09-12 03:31:28.785433881 +0000 UTC m=+0.854143532 container remove 4e170146bd51606ea730d271e126889436ecbade9104f6e0009d7e667027adda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:31:29.112 INFO:journalctl@ceph.osd.3.smithi006.stdout:Sep 12 03:31:28 smithi006 systemd[1]: ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.3.service: Main process exited, code=exited, status=137/n/a 2024-09-12T03:31:29.112 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:28 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:28.941+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:29.112 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:28 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:28.941+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:29.212 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:29 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4[46088]: 2024-09-12T03:31:29.124+0000 7fde3f9de640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.600280+0000 front 2024-09-12T03:30:22.600298+0000 (oldest deadline 2024-09-12T03:31:23.100382+0000) 2024-09-12T03:31:29.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:28 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:28.927+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:29.414 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:29 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:29.153+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:29.429 INFO:journalctl@ceph.osd.3.smithi006.stdout:Sep 12 03:31:29 smithi006 podman[102124]: 2024-09-12 03:31:29.066083006 +0000 UTC m=+0.016680881 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:31:29.429 INFO:journalctl@ceph.osd.3.smithi006.stdout:Sep 12 03:31:29 smithi006 podman[102124]: 2024-09-12 03:31:29.169568367 +0000 UTC m=+0.120166239 container create 853058c70f41c1669d7605badbc2acd45437d1d253a29d7c8788c55db4fac24e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-12T03:31:29.429 INFO:journalctl@ceph.osd.3.smithi006.stdout:Sep 12 03:31:29 smithi006 podman[102124]: 2024-09-12 03:31:29.430377084 +0000 UTC m=+0.380974957 container init 853058c70f41c1669d7605badbc2acd45437d1d253a29d7c8788c55db4fac24e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-12T03:31:29.716 INFO:journalctl@ceph.osd.3.smithi006.stdout:Sep 12 03:31:29 smithi006 podman[102124]: 2024-09-12 03:31:29.433906833 +0000 UTC m=+0.384504714 container start 853058c70f41c1669d7605badbc2acd45437d1d253a29d7c8788c55db4fac24e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:31:29.716 INFO:journalctl@ceph.osd.3.smithi006.stdout:Sep 12 03:31:29 smithi006 podman[102124]: 2024-09-12 03:31:29.513661171 +0000 UTC m=+0.464259053 container attach 853058c70f41c1669d7605badbc2acd45437d1d253a29d7c8788c55db4fac24e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-12T03:31:29.716 INFO:journalctl@ceph.osd.3.smithi006.stdout:Sep 12 03:31:29 smithi006 podman[102124]: 2024-09-12 03:31:29.639030532 +0000 UTC m=+0.589628404 container died 853058c70f41c1669d7605badbc2acd45437d1d253a29d7c8788c55db4fac24e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-12T03:31:29.863 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:29 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:29.611+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:29.863 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:29 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:29.611+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.977593+0000 front 2024-09-12T03:30:18.977760+0000 (oldest deadline 2024-09-12T03:31:24.277584+0000) 2024-09-12T03:31:29.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:29 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:29.413+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:29.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:29 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:29.413+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:29.962 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:29 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:29.886+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:29.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:29 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:29.648+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:29.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:29 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:29.648+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:30.091 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:29 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:29.715+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:30.092 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:29 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:29.715+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:30.092 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:29 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:29.910+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:30.092 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:29 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:29.910+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:30.387 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.3.service' 2024-09-12T03:31:30.427 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:31:30.428 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-12T03:31:30.428 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-12T03:31:30.428 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.4 2024-09-12T03:31:30.457 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:30 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4[46088]: 2024-09-12T03:31:30.144+0000 7fde3f9de640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.600280+0000 front 2024-09-12T03:30:22.600298+0000 (oldest deadline 2024-09-12T03:31:23.100382+0000) 2024-09-12T03:31:30.571 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:30 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:30.168+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:30.572 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:30 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:30.384+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:30.572 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:30 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:30.384+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:30.713 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:30 smithi118 systemd[1]: Stopping Ceph osd.4 for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:31:30.713 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:30 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:30.605+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:30.714 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:30 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:30.605+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:30.863 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:30 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:30.571+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:30.863 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:30 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:30.571+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.977593+0000 front 2024-09-12T03:30:18.977760+0000 (oldest deadline 2024-09-12T03:31:24.277584+0000) 2024-09-12T03:31:31.106 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:30 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:30.873+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:31.106 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:30 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4[46088]: 2024-09-12T03:31:30.766+0000 7fde473e1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:31:31.106 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:30 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4[46088]: 2024-09-12T03:31:30.766+0000 7fde473e1640 -1 osd.4 85 *** Got signal Terminated *** 2024-09-12T03:31:31.106 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:30 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4[46088]: 2024-09-12T03:31:30.766+0000 7fde473e1640 -1 osd.4 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:31:31.106 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:31 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4[46088]: 2024-09-12T03:31:31.106+0000 7fde3f9de640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.600280+0000 front 2024-09-12T03:30:22.600298+0000 (oldest deadline 2024-09-12T03:31:23.100382+0000) 2024-09-12T03:31:31.111 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:30 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:30.899+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:31.112 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:30 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:30.899+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:31.112 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:30 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:30.693+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:31.112 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:30 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:30.693+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:31.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:31 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:31.139+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:31.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:31 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:31.336+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:31.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:31 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:31.336+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:31.937 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:31 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:31.666+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:31.937 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:31 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:31.666+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:31.962 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:31 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:31.859+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:31.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:31 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:31.622+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:31.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:31 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:31.622+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:32.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:31 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:31.614+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:32.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:31 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:31.614+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.977593+0000 front 2024-09-12T03:30:18.977760+0000 (oldest deadline 2024-09-12T03:31:24.277584+0000) 2024-09-12T03:31:32.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:31 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:31.936+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:32.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:31 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:31.936+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:32.462 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:32 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4[46088]: 2024-09-12T03:31:32.137+0000 7fde3f9de640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.600280+0000 front 2024-09-12T03:30:22.600298+0000 (oldest deadline 2024-09-12T03:31:23.100382+0000) 2024-09-12T03:31:32.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:32 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:32.141+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:32.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:32 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:32.342+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:32.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:32 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:32.342+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:32.962 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:32 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:32.836+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:32.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:32 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:32.617+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:32.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:32 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:32.617+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:32.982 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:32 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:32.682+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:32.982 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:32 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:32.682+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:33.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:32 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:32.624+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:33.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:32 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:32.624+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.977593+0000 front 2024-09-12T03:30:18.977760+0000 (oldest deadline 2024-09-12T03:31:24.277584+0000) 2024-09-12T03:31:33.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:32 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:32.982+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:33.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:32 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:32.982+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:33.462 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:33 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4[46088]: 2024-09-12T03:31:33.125+0000 7fde3f9de640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.600280+0000 front 2024-09-12T03:30:22.600298+0000 (oldest deadline 2024-09-12T03:31:23.100382+0000) 2024-09-12T03:31:33.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:33 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:33.166+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:33.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:33 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:33.301+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:33.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:33 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:33.301+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:33.884 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:33 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:33.598+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:33.884 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:33 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:33.598+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:34.003 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:33 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:33.657+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:34.003 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:33 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:33.657+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:34.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:33 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:33.633+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:34.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:33 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:33.633+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.977593+0000 front 2024-09-12T03:30:18.977760+0000 (oldest deadline 2024-09-12T03:31:24.277584+0000) 2024-09-12T03:31:34.157 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:33 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:33.884+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:34.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:34 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:34.002+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:34.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:34 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:34.002+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:34.462 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:34 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4[46088]: 2024-09-12T03:31:34.158+0000 7fde3f9de640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.600280+0000 front 2024-09-12T03:30:22.600298+0000 (oldest deadline 2024-09-12T03:31:23.100382+0000) 2024-09-12T03:31:34.609 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:34 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:34.180+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:34.609 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:34 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:34.308+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:34.609 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:34 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:34.308+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:34.863 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:34 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:34.608+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:34.863 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:34 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:34.608+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.977593+0000 front 2024-09-12T03:30:18.977760+0000 (oldest deadline 2024-09-12T03:31:24.277584+0000) 2024-09-12T03:31:34.962 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:34 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:34.851+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:34.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:34 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:34.640+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:34.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:34 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:34.640+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:34.973 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:34 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:34.696+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:34.973 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:34 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:34.696+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:35.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:34 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:34.973+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:35.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:34 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:34.973+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:35.462 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:35 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4[46088]: 2024-09-12T03:31:35.171+0000 7fde3f9de640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.600280+0000 front 2024-09-12T03:30:22.600298+0000 (oldest deadline 2024-09-12T03:31:23.100382+0000) 2024-09-12T03:31:35.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:35 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:35.204+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:35.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:35 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:35.298+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:35.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:35 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:35.298+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:35.962 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:35 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:35.873+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:35.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:35 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:35.685+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:35.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:35 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:35.685+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:35.965 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:35 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:35.675+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:35.965 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:35 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:35.676+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:36.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:35 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:35.614+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:36.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:35 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:35.614+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.977593+0000 front 2024-09-12T03:30:18.977760+0000 (oldest deadline 2024-09-12T03:31:24.277584+0000) 2024-09-12T03:31:36.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:35 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:35.965+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:36.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:35 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:35.965+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:36.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:36 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:36.238+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:36.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:36 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:36.262+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:36.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:36 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:36.262+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:36.649 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:36 smithi118 podman[69462]: 2024-09-12 03:31:36.370761983 +0000 UTC m=+5.734269697 container died 25dddd345d378dcd4acb9c3a1625b5ca5ddc928ea49962f83d04ee97e5db7102 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-12T03:31:36.962 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:36 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:36.872+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:36.963 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:36 smithi118 podman[69462]: 2024-09-12 03:31:36.75831232 +0000 UTC m=+6.121820033 container cleanup 25dddd345d378dcd4acb9c3a1625b5ca5ddc928ea49962f83d04ee97e5db7102 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-12T03:31:36.963 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:36 smithi118 bash[69462]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4 2024-09-12T03:31:36.963 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:36 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:36.665+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:36.963 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:36 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:36.665+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:37.002 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:36 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:36.679+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:37.002 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:36 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:36.679+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:37.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:36 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:36.643+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:37.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:36 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:36.643+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.977593+0000 front 2024-09-12T03:30:18.977760+0000 (oldest deadline 2024-09-12T03:31:24.277584+0000) 2024-09-12T03:31:37.360 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:37 smithi118 podman[69481]: 2024-09-12 03:31:37.026672489 +0000 UTC m=+0.654141380 container remove 25dddd345d378dcd4acb9c3a1625b5ca5ddc928ea49962f83d04ee97e5db7102 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:31:37.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:37 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:37.002+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:37.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:37 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:37.002+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:37.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:37 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:37.219+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:37.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:37 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:37.291+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:37.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:37 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:37.291+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:37.671 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:37 smithi118 podman[69570]: 2024-09-12 03:31:37.360799938 +0000 UTC m=+0.102997694 container create bf9cef5c7d02b74358f014e782ab87917d588d1af67e075206d69aa22a15ca49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True) 2024-09-12T03:31:37.671 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:37 smithi118 podman[69570]: 2024-09-12 03:31:37.273793368 +0000 UTC m=+0.015991123 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:31:37.671 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:37 smithi118 podman[69570]: 2024-09-12 03:31:37.588687283 +0000 UTC m=+0.330885038 container init bf9cef5c7d02b74358f014e782ab87917d588d1af67e075206d69aa22a15ca49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-12T03:31:37.671 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:37 smithi118 podman[69570]: 2024-09-12 03:31:37.592126061 +0000 UTC m=+0.334323821 container start bf9cef5c7d02b74358f014e782ab87917d588d1af67e075206d69aa22a15ca49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-12T03:31:37.671 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:37 smithi118 podman[69570]: 2024-09-12 03:31:37.671756239 +0000 UTC m=+0.413953993 container attach bf9cef5c7d02b74358f014e782ab87917d588d1af67e075206d69aa22a15ca49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:31:37.962 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:37 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:37.844+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:37.962 INFO:journalctl@ceph.osd.4.smithi118.stdout:Sep 12 03:31:37 smithi118 podman[69570]: 2024-09-12 03:31:37.776818053 +0000 UTC m=+0.519015808 container died bf9cef5c7d02b74358f014e782ab87917d588d1af67e075206d69aa22a15ca49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-12T03:31:37.963 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:37 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:37.697+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:37.963 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:37 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:37.697+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:37.990 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:37 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:37.649+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:37.990 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:37 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:37.649+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:38.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:37 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:37.690+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:38.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:37 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:37.690+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.977593+0000 front 2024-09-12T03:30:18.977760+0000 (oldest deadline 2024-09-12T03:31:24.277584+0000) 2024-09-12T03:31:38.338 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.4.service' 2024-09-12T03:31:38.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:37 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:37.990+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:38.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:37 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:37.990+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:38.419 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:31:38.419 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-12T03:31:38.419 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-12T03:31:38.420 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.5 2024-09-12T03:31:38.613 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:38 smithi161 systemd[1]: Stopping Ceph osd.5 for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:31:38.614 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:38 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:38.244+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:38.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:38 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:38.311+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:38.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:38 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:38.311+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:38.709 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:38 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:38.687+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:38.709 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:38 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:38.687+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:38.962 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:38 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:38.827+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:39.008 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:38 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:38.631+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:39.008 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:38 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:38.631+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:39.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:38 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:38.672+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:39.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:38 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:38.672+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.977593+0000 front 2024-09-12T03:30:18.977760+0000 (oldest deadline 2024-09-12T03:31:24.277584+0000) 2024-09-12T03:31:39.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:38 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:38.713+0000 7fee0a8b6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:31:39.114 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:38 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:38.713+0000 7fee0a8b6640 -1 osd.5 85 *** Got signal Terminated *** 2024-09-12T03:31:39.114 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:38 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:38.713+0000 7fee0a8b6640 -1 osd.5 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:31:39.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:39 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:39.007+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:39.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:39 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:39.007+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:39.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:39 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:39.225+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:39.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:39 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:39.268+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:39.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:39 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:39.268+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:39.962 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:39 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:39.807+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:39.963 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:39 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:39.647+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:39.963 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:39 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:39.647+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:39.985 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:39 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:39.635+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:39.986 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:39 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:39.636+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:40.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:39 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:39.710+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:40.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:39 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:39.710+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.977593+0000 front 2024-09-12T03:30:18.977760+0000 (oldest deadline 2024-09-12T03:31:24.277584+0000) 2024-09-12T03:31:40.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:39 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:39.985+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:40.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:39 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:39.985+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:40.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:40 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:40.212+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:40.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:40 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:40.249+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:40.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:40 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:40.249+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:40.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:40 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:40.611+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:40.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:40 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:40.611+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:40.962 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:40 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:40.835+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:40.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:40 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:40.624+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:40.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:40 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:40.624+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:41.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:40 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:40.751+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:41.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:40 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:40.751+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.977593+0000 front 2024-09-12T03:30:18.977760+0000 (oldest deadline 2024-09-12T03:31:24.277584+0000) 2024-09-12T03:31:41.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:40 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:40.968+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:41.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:40 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:40.968+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:41.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:41 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:41.237+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:41.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:41 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:41.279+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:41.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:41 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:41.279+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:41.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:41 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:41.566+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:41.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:41 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:41.566+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:41.962 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:41 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:41.841+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:41.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:41 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:41.656+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:41.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:41 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:41.656+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:42.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:41 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:41.740+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:42.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:41 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:41.740+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.977593+0000 front 2024-09-12T03:30:18.977760+0000 (oldest deadline 2024-09-12T03:31:24.277584+0000) 2024-09-12T03:31:42.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:41 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:41.932+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:42.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:41 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:41.932+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:42.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:42 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:42.230+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:42.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:42 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:42.281+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:42.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:42 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:42.281+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:42.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:42 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:42.570+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:42.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:42 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:42.570+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:42.886 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:42 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:42.631+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:42.887 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:42 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:42.631+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:43.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:42 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:42.740+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.373445+0000 front 2024-09-12T03:29:49.373245+0000 (oldest deadline 2024-09-12T03:31:18.977161+0000) 2024-09-12T03:31:43.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:31:42 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5[47452]: 2024-09-12T03:31:42.740+0000 7fee06ebb640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.977593+0000 front 2024-09-12T03:30:18.977760+0000 (oldest deadline 2024-09-12T03:31:24.277584+0000) 2024-09-12T03:31:43.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:42 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:42.887+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:43.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:42 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:42.970+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:43.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:42 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:42.970+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:43.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:43 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:43.202+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:43.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:43 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:43.322+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:43.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:43 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:43.322+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:43.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:43 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:43.588+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:43.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:43 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:43.588+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:43.962 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:43 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:43.923+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:43.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:43 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:43.677+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:43.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:43 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:43.677+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:44.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:43 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:43.976+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:44.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:43 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:43.976+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:44.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:44 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:44.219+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:44.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:44 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:44.335+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:44.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:44 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:44.335+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:44.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:44 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:44.545+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:44.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:44 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:44.545+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:45.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:44 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:44.888+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:45.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:44 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:44.715+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:45.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:44 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:44.715+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:45.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:44 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:44.954+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:45.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:44 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:44.954+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:45.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:45 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:45.267+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:45.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:45 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:45.346+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:45.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:45 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:45.346+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:45.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:45 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:45.532+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:45.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:45 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:45.532+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:46.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:45 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:45.852+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:46.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:45 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:45.717+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:46.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:45 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:45.717+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:46.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:45 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:45.952+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:46.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:45 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:45.952+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:46.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:46 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:46.289+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:46.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:46 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:46.381+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:46.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:46 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:46.381+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:46.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:46 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:46.535+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:46.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:46 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:46.535+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:47.113 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:46 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:46.824+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:47.114 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:46 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:46.751+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:47.114 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:46 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:46.751+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:47.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:46 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:46.965+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:47.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:46 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:46.965+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:47.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:47 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:47.270+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:47.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:47 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:47.366+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:47.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:47 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:47.366+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:47.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:47 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:47.525+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:47.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:47 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:47.525+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:48.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:47 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:47.816+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:48.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:47 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:47.789+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:48.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:47 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:47.789+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:48.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:47 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:47.927+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:48.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:47 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:47.927+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:48.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:48 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:48.289+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:48.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:48 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:48.390+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:48.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:48 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:48.390+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:48.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:48 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:48.542+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:48.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:48 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:48.542+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:49.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:48 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:48.832+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:49.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:48 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:48.792+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:49.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:48 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:48.792+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:49.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:48 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:48.964+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:49.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:48 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:48.964+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:49.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:49 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:49.249+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:49.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:49 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:49.420+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:49.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:49 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:49.420+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:49.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:49 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:49.495+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:49.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:49 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:49.495+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:50.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:49 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:49.812+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:50.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:49 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:49.799+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:50.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:49 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:49.799+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:50.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:49 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:49.990+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:50.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:49 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:49.990+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:50.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:50 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:50.249+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:50.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:50 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:50.444+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:50.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:50 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:50.444+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:50.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:50 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:50.474+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:50.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:50 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:50.474+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:51.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:50 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:50.829+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:51.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:50 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:50.755+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:51.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:50 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:50.755+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:51.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:51 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:51.002+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:51.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:51 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:51.002+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:51.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:51 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:51.283+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:51.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:51 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:51.476+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:51.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:51 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:51.476+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:51.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:51 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:51.480+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:51.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:51 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:51.480+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:52.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:51 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:51.791+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:52.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:51 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:51.795+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:52.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:51 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:51.795+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:52.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:52 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:52.050+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:52.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:52 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:52.050+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:52.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:52 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:52.236+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:52.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:52 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:52.449+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:52.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:52 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:52.449+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:52.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:52 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:52.497+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:52.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:52 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:52.497+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:53.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:52 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:52.773+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:53.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:52 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:52.784+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:53.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:52 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:52.784+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:53.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:53 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:53.086+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:53.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:53 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:53.086+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:53.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:53 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:53.216+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:53.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:53 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:53.422+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:53.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:53 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:53.422+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:53.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:53 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:53.523+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:53.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:53 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:53.523+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:54.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:53 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:53.764+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:54.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:53 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:53.768+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:54.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:53 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:53.768+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:54.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:54 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:54.040+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:54.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:54 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:54.040+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:54.614 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:54 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:54.219+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:54.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:54 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:54.418+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:54.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:54 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:54.418+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:54.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:54 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:54.509+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:54.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:54 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:54.509+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:55.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:54 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:54.739+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:55.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:54 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:54.729+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:55.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:54 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:54.729+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:55.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:55 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:55.062+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:55.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:55 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:55.062+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:55.514 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:55 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:55.182+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:55.515 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:55 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:55.367+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:55.515 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:55 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:55.367+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:55.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:55 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:55.463+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:55.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:55 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:55.463+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:56.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:55 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:55.775+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:56.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:55 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:55.771+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:56.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:55 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:55.771+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:56.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:56 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:56.033+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:56.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:56 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:56.033+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:56.529 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:56 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:56.190+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:56.529 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:56 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:56.374+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:56.529 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:56 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:56.374+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:56.529 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:56 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:56.374+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:31:56.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:56 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:56.462+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:56.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:56 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:56.462+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:56.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:56 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:56.462+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:31:57.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:56 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:56.729+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:57.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:56 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:56.729+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:57.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:56 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:56.808+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:57.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:56 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:56.808+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:31:57.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:57.047+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:57.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:57.047+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:57.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:57.047+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.520873+0000 front 2024-09-12T03:30:27.520901+0000 (oldest deadline 2024-09-12T03:31:56.723782+0000) 2024-09-12T03:31:57.614 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:57 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:57.232+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:57.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:57 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:57.375+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:57.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:57 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:57.375+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:57.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:57 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:57.375+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:31:57.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:57.507+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:57.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:57.507+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:57.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:57 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:57.507+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:31:58.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:57 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:57.805+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:58.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:57 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:57.805+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:31:58.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:57 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:57.762+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:58.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:57 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:57.762+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:58.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:58 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:58.047+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:58.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:58 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:58.047+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:58.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:58 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:58.047+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.520873+0000 front 2024-09-12T03:30:27.520901+0000 (oldest deadline 2024-09-12T03:31:56.723782+0000) 2024-09-12T03:31:58.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:58 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:58.247+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:58.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:58 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:58.416+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:58.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:58 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:58.416+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:58.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:58 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:58.416+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:31:58.630 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:58 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:58.484+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:58.630 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:58 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:58.484+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:58.630 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:58 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:58.484+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:31:59.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:58 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:58.832+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:59.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:58 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:58.832+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:31:59.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:58 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:58.730+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:59.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:58 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:58.730+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:59.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:59.052+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:31:59.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:59.052+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:31:59.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:31:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:31:59.052+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.520873+0000 front 2024-09-12T03:30:27.520901+0000 (oldest deadline 2024-09-12T03:31:56.723782+0000) 2024-09-12T03:31:59.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:59 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:59.231+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:31:59.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:31:59 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:31:59.231+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:31:59.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:59 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:59.410+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:31:59.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:59 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:59.410+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:31:59.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:31:59 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:31:59.410+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:31:59.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:59.531+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:31:59.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:59.531+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:31:59.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:31:59 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:31:59.531+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:31:59.962 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:59 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:59.797+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:31:59.962 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:31:59 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:31:59.797+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:31:59.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:59 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:59.698+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:31:59.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:59 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:59.698+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:31:59.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:31:59 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:31:59.698+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:00.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:00 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:00.093+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:32:00.361 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:00 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:00.093+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:32:00.362 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:00 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:00.093+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.520873+0000 front 2024-09-12T03:30:27.520901+0000 (oldest deadline 2024-09-12T03:31:56.723782+0000) 2024-09-12T03:32:00.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:00 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:00.230+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:00.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:00 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:00.230+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:00.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:00 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:00.393+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:00.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:00 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:00.393+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:00.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:00 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:00.393+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:00.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:00 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:00.522+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:00.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:00 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:00.522+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:00.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:00 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:00.522+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:01.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:00 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:00.815+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:01.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:00 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:00.815+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:01.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:00 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:00.730+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:01.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:00 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:00.730+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:01.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:00 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:00.730+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:01.506 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:01 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:01.137+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:32:01.506 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:01 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:01.137+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:32:01.507 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:01 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:01.137+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.520873+0000 front 2024-09-12T03:30:27.520901+0000 (oldest deadline 2024-09-12T03:31:56.723782+0000) 2024-09-12T03:32:01.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:01 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:01.277+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:01.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:01 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:01.277+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:01.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:01 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:01.427+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:01.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:01 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:01.427+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:01.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:01 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:01.427+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:01.861 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:01 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:01.506+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:01.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:01 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:01.506+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:01.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:01 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:01.506+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:02.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:01 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:01.860+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:02.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:01 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:01.860+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:02.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:01 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:01.778+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:02.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:01 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:01.778+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:02.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:01 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:01.778+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:02.465 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:02 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:02.162+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:32:02.465 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:02 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:02.162+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:32:02.465 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:02 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:02.162+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.520873+0000 front 2024-09-12T03:30:27.520901+0000 (oldest deadline 2024-09-12T03:31:56.723782+0000) 2024-09-12T03:32:02.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:02 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:02.248+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:02.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:02 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:02.248+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:02.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:02 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:02.381+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:02.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:02 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:02.381+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:02.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:02 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:02.381+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:02.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:02 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:02.463+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:02.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:02 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:02.463+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:02.862 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:02 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:02.463+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:03.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:02 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:02.869+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:03.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:02 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:02.869+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:03.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:02 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:02.790+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:03.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:02 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:02.790+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:03.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:02 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:02.790+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:03.611 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:03 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:03.204+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:32:03.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:03 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:03.204+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:32:03.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:03 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:03.204+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.520873+0000 front 2024-09-12T03:30:27.520901+0000 (oldest deadline 2024-09-12T03:31:56.723782+0000) 2024-09-12T03:32:03.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:03 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:03.424+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:03.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:03 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:03.424+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:03.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:03 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:03.424+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:03.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:03 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:03.273+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:03.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:03 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:03.273+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:03.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:03 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:03.429+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:03.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:03 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:03.429+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:03.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:03 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:03.429+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:04.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:03 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:03.879+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:04.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:03 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:03.879+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:04.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:03 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:03.826+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:04.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:03 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:03.826+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:04.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:03 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:03.826+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:04.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:04 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:04.391+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:04.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:04 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:04.391+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:04.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:04 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:04.391+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:04.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:04 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:04.193+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:32:04.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:04 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:04.193+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:32:04.613 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:04 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:04.193+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.520873+0000 front 2024-09-12T03:30:27.520901+0000 (oldest deadline 2024-09-12T03:31:56.723782+0000) 2024-09-12T03:32:04.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:04 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:04.290+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:04.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:04 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:04.290+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:04.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:04 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:04.455+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:04.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:04 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:04.455+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:04.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:04 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:04.455+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:05.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:04 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:04.906+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:05.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:04 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:04.906+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:05.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:04 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:04.833+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:05.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:04 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:04.833+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:05.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:04 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:04.833+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:05.611 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:05 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:05.406+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:05.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:05 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:05.406+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:05.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:05 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:05.406+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:05.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:05 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:05.189+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:32:05.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:05 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:05.189+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:32:05.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:05 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:05.189+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.520873+0000 front 2024-09-12T03:30:27.520901+0000 (oldest deadline 2024-09-12T03:31:56.723782+0000) 2024-09-12T03:32:05.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:05 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:05.329+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:05.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:05 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:05.329+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:05.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:05 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:05.474+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:05.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:05 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:05.474+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:05.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:05 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:05.474+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:06.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:05 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:05.886+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:06.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:05 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:05.886+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:06.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:05 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:05.832+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:06.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:05 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:05.832+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:06.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:05 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:05.832+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:06.611 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:06 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:06.363+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:06.611 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:06 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:06.363+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:06.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:06 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:06.363+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:06.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:06 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:06.196+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:32:06.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:06 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:06.196+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:32:06.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:06 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:06.196+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.520873+0000 front 2024-09-12T03:30:27.520901+0000 (oldest deadline 2024-09-12T03:31:56.723782+0000) 2024-09-12T03:32:06.863 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:06 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:06.377+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:06.863 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:06 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:06.377+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:06.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:06 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:06.445+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:06.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:06 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:06.445+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:06.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:06 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:06.445+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:07.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:06 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:06.858+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:07.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:06 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:06.858+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:07.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:06 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:06.841+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:07.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:06 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:06.842+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:07.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:06 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:06.842+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:07.611 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:07 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:07.245+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:32:07.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:07 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:07.245+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:32:07.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:07 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:07.245+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.520873+0000 front 2024-09-12T03:30:27.520901+0000 (oldest deadline 2024-09-12T03:31:56.723782+0000) 2024-09-12T03:32:07.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:07 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:07.318+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:07.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:07 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:07.318+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:07.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:07 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:07.318+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:07.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:07 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:07.370+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:07.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:07 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:07.370+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:07.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:07 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:07.471+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:07.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:07 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:07.471+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:07.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:07 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:07.471+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:08.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:07 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:07.859+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:08.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:07 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:07.859+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:08.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:07 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:07.795+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:08.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:07 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:07.795+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:08.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:07 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:07.795+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:08.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:08 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:08.247+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:32:08.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:08 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:08.247+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:32:08.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:08 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:08.247+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.520873+0000 front 2024-09-12T03:30:27.520901+0000 (oldest deadline 2024-09-12T03:31:56.723782+0000) 2024-09-12T03:32:08.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:08 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:08.287+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:08.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:08 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:08.287+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:08.613 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:08 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:08.287+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:08.722 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:08 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:08.376+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:08.722 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:08 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:08.376+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:08.722 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:08 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:08.438+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:08.722 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:08 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:08.438+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:08.722 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:08 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:08.438+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:09.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:32:08 smithi161 bash[70614]: time="2024-09-12T03:32:08Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5 in 30 seconds, resorting to SIGKILL" 2024-09-12T03:32:09.113 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:32:08 smithi161 podman[70614]: 2024-09-12 03:32:08.827783871 +0000 UTC m=+30.219155141 container died faec383de15871793d435568a2928c26dfb195f60598882eaf15c764f6081c65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-12T03:32:09.213 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:08 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:08.904+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:09.213 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:08 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:08.904+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:09.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:08 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:08.786+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:09.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:08 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:08.786+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:09.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:08 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:08.786+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:09.614 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:09 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:09.264+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:32:09.614 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:09 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:09.264+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:32:09.614 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:09 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:09.264+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.520873+0000 front 2024-09-12T03:30:27.520901+0000 (oldest deadline 2024-09-12T03:31:56.723782+0000) 2024-09-12T03:32:09.615 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:09 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:09.287+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:09.615 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:09 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:09.287+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:09.615 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:09 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:09.287+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:09.770 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:32:09 smithi161 podman[70614]: 2024-09-12 03:32:09.486764163 +0000 UTC m=+30.878135441 container cleanup faec383de15871793d435568a2928c26dfb195f60598882eaf15c764f6081c65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T03:32:09.771 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:32:09 smithi161 bash[70614]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5 2024-09-12T03:32:09.771 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:09 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:09.416+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:09.771 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:09 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:09.416+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:09.771 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:09 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:09.400+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:09.771 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:09 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:09.400+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:09.771 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:09 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:09.400+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:10.097 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:32:09 smithi161 podman[70642]: 2024-09-12 03:32:09.77102737 +0000 UTC m=+0.937189978 container remove faec383de15871793d435568a2928c26dfb195f60598882eaf15c764f6081c65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:32:10.098 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:32:09 smithi161 systemd[1]: ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.5.service: Main process exited, code=exited, status=137/n/a 2024-09-12T03:32:10.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:09 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:09.864+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:10.213 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:09 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:09.864+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:10.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:09 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:09.744+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:10.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:09 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:09.744+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:10.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:09 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:09.744+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:10.363 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:32:10 smithi161 podman[70732]: 2024-09-12 03:32:10.001131466 +0000 UTC m=+0.016354066 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:32:10.363 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:32:10 smithi161 podman[70732]: 2024-09-12 03:32:10.105033795 +0000 UTC m=+0.120256396 container create 2b957ed727023481dd083b1be15e4525fea46b206dbb0acc83d7d33feeea98ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T03:32:10.364 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:32:10 smithi161 podman[70732]: 2024-09-12 03:32:10.31559841 +0000 UTC m=+0.330821031 container init 2b957ed727023481dd083b1be15e4525fea46b206dbb0acc83d7d33feeea98ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-12T03:32:10.364 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:32:10 smithi161 podman[70732]: 2024-09-12 03:32:10.318808927 +0000 UTC m=+0.334031530 container start 2b957ed727023481dd083b1be15e4525fea46b206dbb0acc83d7d33feeea98ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-12T03:32:10.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:10 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:10.271+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:32:10.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:10 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:10.271+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:32:10.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:10 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:10.271+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.520873+0000 front 2024-09-12T03:30:27.520901+0000 (oldest deadline 2024-09-12T03:31:56.723782+0000) 2024-09-12T03:32:10.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:10 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:10.243+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:10.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:10 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:10.243+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:10.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:10 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:10.243+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:10.793 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:10 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:10.388+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:10.793 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:10 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:10.388+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:10.793 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:32:10 smithi161 podman[70732]: 2024-09-12 03:32:10.39907697 +0000 UTC m=+0.414299570 container attach 2b957ed727023481dd083b1be15e4525fea46b206dbb0acc83d7d33feeea98ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:32:10.793 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 12 03:32:10 smithi161 podman[70732]: 2024-09-12 03:32:10.509497052 +0000 UTC m=+0.524719657 container died 2b957ed727023481dd083b1be15e4525fea46b206dbb0acc83d7d33feeea98ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-12T03:32:10.794 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:10 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:10.378+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:10.794 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:10 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:10.378+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:10.794 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:10 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:10.378+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:10.962 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:10 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:10.873+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:10.962 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:10 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:10.873+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:10.963 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:10 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:10.709+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:10.963 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:10 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:10.709+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:10.963 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:10 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:10.709+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:11.144 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.5.service' 2024-09-12T03:32:11.184 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:32:11.184 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-12T03:32:11.185 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-12T03:32:11.185 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.6 2024-09-12T03:32:11.409 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:11 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:11.365+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:11.409 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:11 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:11.365+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:11.409 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:11 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:11.365+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:11.501 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:11 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:11.243+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:11.502 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:11 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:11.243+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:11.502 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:11 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:11.243+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:11.502 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:11 smithi006 systemd[1]: Stopping Ceph osd.6 for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:32:11.502 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:11 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:11.284+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:32:11.502 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:11 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:11.284+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:32:11.502 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:11 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:11.284+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.520873+0000 front 2024-09-12T03:30:27.520901+0000 (oldest deadline 2024-09-12T03:31:56.723782+0000) 2024-09-12T03:32:11.502 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:11 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:11.501+0000 7ffb3c108640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:32:11.502 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:11 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:11.501+0000 7ffb3c108640 -1 osd.6 85 *** Got signal Terminated *** 2024-09-12T03:32:11.502 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:11 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:11.501+0000 7ffb3c108640 -1 osd.6 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:32:11.865 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:11 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:11.409+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:11.865 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:11 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:11.409+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:12.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:11 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:11.879+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:12.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:11 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:11.879+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:12.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:11 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:11.729+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:12.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:11 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:11.729+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:12.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:11 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:11.729+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:12.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:12 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:12.233+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:12.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:12 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:12.233+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:12.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:12 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:12.233+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:12.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:12 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:12.287+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:32:12.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:12 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:12.287+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:32:12.613 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:12 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:12.287+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.520873+0000 front 2024-09-12T03:30:27.520901+0000 (oldest deadline 2024-09-12T03:31:56.723782+0000) 2024-09-12T03:32:12.614 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:12 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:12.455+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:12.614 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:12 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:12.455+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:12.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:12 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:12.330+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:12.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:12 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:12.330+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:12.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:12 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:12.330+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:13.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:12 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:12.891+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:13.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:12 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:12.891+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:13.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:12 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:12.741+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:13.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:12 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:12.741+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:13.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:12 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:12.741+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:13.611 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:13 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:13.250+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:13.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:13 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:13.250+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:13.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:13 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:13.250+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:13.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:13 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:13.255+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:32:13.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:13 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:13.255+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:32:13.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:13 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:13.255+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.520873+0000 front 2024-09-12T03:30:27.520901+0000 (oldest deadline 2024-09-12T03:31:56.723782+0000) 2024-09-12T03:32:13.863 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:13 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:13.486+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:13.863 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:13 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:13.486+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:13.863 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:13 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:13.365+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:13.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:13 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:13.365+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:13.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:13 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:13.365+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:14.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:13 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:13.915+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:14.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:13 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:13.915+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:14.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:13 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:13.712+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:14.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:13 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:13.712+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:14.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:13 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:13.712+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:14.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:14 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:14.292+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:32:14.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:14 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:14.292+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:32:14.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:14 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:14.292+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.520873+0000 front 2024-09-12T03:30:27.520901+0000 (oldest deadline 2024-09-12T03:31:56.723782+0000) 2024-09-12T03:32:14.613 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:14 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:14.300+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:14.613 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:14 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:14.300+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:14.613 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:14 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:14.300+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:14.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:14 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:14.482+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:14.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:14 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:14.482+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:14.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:14 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:14.331+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:14.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:14 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:14.331+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:14.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:14 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:14.331+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:15.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:14 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:14.934+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:15.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:14 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:14.934+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:15.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:14 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:14.761+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:15.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:14 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:14.761+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:15.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:14 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:14.761+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:15.611 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:15 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:15.301+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:32:15.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:15 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:15.301+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:32:15.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:15 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:15.301+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.520873+0000 front 2024-09-12T03:30:27.520901+0000 (oldest deadline 2024-09-12T03:31:56.723782+0000) 2024-09-12T03:32:15.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:15 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:15.308+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:15.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:15 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:15.308+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:15.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:15 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:15.308+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:15.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:15 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:15.457+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:15.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:15 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:15.457+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:15.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:15 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:15.343+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:15.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:15 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:15.343+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:15.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:15 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:15.343+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:16.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:15 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:15.942+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:16.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:15 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:15.942+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:16.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:15 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:15.769+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:16.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:15 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:15.769+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:16.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:15 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:15.769+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:16.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:16 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:16.319+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:49.115509+0000 front 2024-09-12T03:29:49.115549+0000 (oldest deadline 2024-09-12T03:31:15.318918+0000) 2024-09-12T03:32:16.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:16 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:16.319+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:21.720179+0000 front 2024-09-12T03:30:21.720393+0000 (oldest deadline 2024-09-12T03:31:25.220136+0000) 2024-09-12T03:32:16.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:16 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6[58386]: 2024-09-12T03:32:16.319+0000 7ffb34705640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.520873+0000 front 2024-09-12T03:30:27.520901+0000 (oldest deadline 2024-09-12T03:31:56.723782+0000) 2024-09-12T03:32:16.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:16 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:16.337+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:16.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:16 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:16.337+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:16.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:16 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:16.337+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:16.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:16 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:16.467+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:16.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:16 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:16.467+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:16.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:16 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:16.335+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:16.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:16 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:16.335+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:16.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:16 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:16.335+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:17.113 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:16 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:16.952+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:17.113 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:16 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:16.952+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:17.113 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:16 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:16.747+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:17.114 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:16 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:16.747+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:17.114 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:16 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:16.747+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:17.236 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:16 smithi006 podman[102240]: 2024-09-12 03:32:16.956127885 +0000 UTC m=+5.557059146 container died 9a117b4b6b039a364157c12f2d13dbb9b74737ad8803b5fa42a0d66748f0eb17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-12T03:32:17.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:17 smithi006 podman[102240]: 2024-09-12 03:32:17.410140581 +0000 UTC m=+6.011071845 container cleanup 9a117b4b6b039a364157c12f2d13dbb9b74737ad8803b5fa42a0d66748f0eb17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:32:17.612 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:17 smithi006 bash[102240]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6 2024-09-12T03:32:17.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:17 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:17.323+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:17.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:17 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:17.323+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:17.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:17 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:17.323+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:17.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:17 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:17.434+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:17.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:17 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:17.434+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:17.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:17 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:17.344+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:17.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:17 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:17.344+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:17.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:17 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:17.344+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:18.035 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:17 smithi006 podman[102257]: 2024-09-12 03:32:17.703591067 +0000 UTC m=+0.738840687 container remove 9a117b4b6b039a364157c12f2d13dbb9b74737ad8803b5fa42a0d66748f0eb17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731) 2024-09-12T03:32:18.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:17 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:17.955+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:18.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:17 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:17.955+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:18.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:17 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:17.737+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:18.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:17 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:17.737+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:18.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:17 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:17.737+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:18.290 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:18 smithi006 podman[102342]: 2024-09-12 03:32:17.938207876 +0000 UTC m=+0.016427653 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:32:18.290 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:18 smithi006 podman[102342]: 2024-09-12 03:32:18.162704507 +0000 UTC m=+0.240924326 container create ccd006a57872810719305cd992b8b94e5a0ccb293752055065af3b6093bc2321 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:32:18.291 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:18 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:18.275+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:18.291 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:18 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:18.275+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:18.291 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:18 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:18.275+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:18.605 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:18 smithi006 podman[102342]: 2024-09-12 03:32:18.406621612 +0000 UTC m=+0.484841389 container init ccd006a57872810719305cd992b8b94e5a0ccb293752055065af3b6093bc2321 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-12T03:32:18.605 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:18 smithi006 podman[102342]: 2024-09-12 03:32:18.410055324 +0000 UTC m=+0.488275098 container start ccd006a57872810719305cd992b8b94e5a0ccb293752055065af3b6093bc2321 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-12T03:32:18.606 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:18 smithi006 podman[102342]: 2024-09-12 03:32:18.48997054 +0000 UTC m=+0.568190312 container attach ccd006a57872810719305cd992b8b94e5a0ccb293752055065af3b6093bc2321 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:32:18.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:18 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:18.418+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:18.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:18 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:18.418+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:18.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:18 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:18.299+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:18.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:18 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:18.299+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:18.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:18 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:18.299+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:18.864 INFO:journalctl@ceph.osd.6.smithi006.stdout:Sep 12 03:32:18 smithi006 podman[102342]: 2024-09-12 03:32:18.605673014 +0000 UTC m=+0.683892783 container died ccd006a57872810719305cd992b8b94e5a0ccb293752055065af3b6093bc2321 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-12T03:32:19.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:18 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:18.914+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:19.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:18 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:18.914+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:19.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:18 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:18.764+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:19.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:18 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:18.764+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:19.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:18 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:18.764+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:19.248 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.6.service' 2024-09-12T03:32:19.291 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:32:19.291 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-12T03:32:19.291 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-12T03:32:19.293 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.7 2024-09-12T03:32:19.512 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:19 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:19.277+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:19.513 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:19 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:19.277+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:19.513 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:19 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:19.277+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:19.593 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:19 smithi118 systemd[1]: Stopping Ceph osd.7 for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:32:19.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:19 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:19.452+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:19.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:19 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:19.452+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:19.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:19 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:19.334+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:19.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:19 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:19.334+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:19.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:19 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:19.334+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:19.866 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:19 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:19.765+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:19.866 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:19 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:19.765+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:19.866 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:19 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:19.765+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:19.866 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:19 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:19.593+0000 7f241e985640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:32:19.866 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:19 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:19.593+0000 7f241e985640 -1 osd.7 85 *** Got signal Terminated *** 2024-09-12T03:32:19.867 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:19 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:19.593+0000 7f241e985640 -1 osd.7 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:32:19.867 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:19 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:19.866+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:19.867 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:19 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:19.866+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:20.611 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:20 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:20.278+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:20.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:20 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:20.278+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:20.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:20 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:20.278+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:20.863 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:20 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:20.474+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:20.863 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:20 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:20.474+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:20.863 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:20 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:20.384+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:20.863 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:20 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:20.384+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:20.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:20 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:20.384+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:21.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:20 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:20.851+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:21.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:20 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:20.851+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:21.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:20 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:20.799+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:21.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:20 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:20.799+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:21.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:20 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:20.799+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:21.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:21.320+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:21.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:21.320+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:21.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:21 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:21.320+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:21.863 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:21 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:21.473+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:21.863 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:21 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:21.473+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:21.863 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:21 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:21.363+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:21.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:21 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:21.363+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:21.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:21 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:21.363+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:22.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:21 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:21.812+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:22.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:21 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:21.812+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:22.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:21 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:21.772+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:22.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:21 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:21.772+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:22.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:21 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:21.772+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:22.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:22 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:22.347+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:22.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:22 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:22.347+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:22.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:22 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:22.347+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:22.863 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:22 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:22.500+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:22.863 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:22 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:22.500+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:22.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:22 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:22.401+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:22.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:22 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:22.401+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:22.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:22 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:22.401+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:23.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:22 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:22.842+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:23.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:22 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:22.842+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:23.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:22 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:22.729+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:23.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:22 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:22.729+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:23.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:22 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:22.729+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:23.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:23 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:23.354+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:23.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:23 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:23.354+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:23.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:23 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:23.354+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:23.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:23 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:23.501+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:23.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:23 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:23.501+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:23.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:23 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:23.358+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:23.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:23 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:23.358+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:23.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:23 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:23.358+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:23.964 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:23 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:23.798+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:22.455906+0000 front 2024-09-12T03:30:22.455865+0000 (oldest deadline 2024-09-12T03:31:25.356088+0000) 2024-09-12T03:32:23.964 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:23 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7[53108]: 2024-09-12T03:32:23.798+0000 7f241af8a640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:25.356578+0000 front 2024-09-12T03:30:25.356507+0000 (oldest deadline 2024-09-12T03:31:56.160921+0000) 2024-09-12T03:32:23.965 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:23 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:23.705+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:23.965 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:23 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:23.705+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:23.965 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:23 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:23.705+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:24.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:24 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:24.353+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:24.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:24 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:24.353+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:24.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:24 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:24.353+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:24.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:24 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:24.494+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:24.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:24 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:24.494+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:24.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:24 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:24.342+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:24.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:24 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:24.342+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:24.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:24 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:24.342+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:24.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:24 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:24.671+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:24.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:24 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:24.671+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:24.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:24 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:24.671+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:25.235 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:24 smithi118 podman[69685]: 2024-09-12 03:32:24.967331098 +0000 UTC m=+5.475149161 container died 62fe00bb357df15aaa1998d314bec6f7dbb79b782e0c213293ed0b9ddd21ed9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-12T03:32:25.585 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:25 smithi118 podman[69685]: 2024-09-12 03:32:25.335263033 +0000 UTC m=+5.843081095 container cleanup 62fe00bb357df15aaa1998d314bec6f7dbb79b782e0c213293ed0b9ddd21ed9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T03:32:25.586 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:25 smithi118 bash[69685]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7 2024-09-12T03:32:25.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:25 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:25.354+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:25.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:25 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:25.354+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:25.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:25 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:25.354+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:25.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:25 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:25.458+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:25.614 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:25 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:25.458+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:25.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:25 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:25.301+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:25.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:25 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:25.301+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:25.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:25 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:25.301+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:25.945 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:25 smithi118 podman[69701]: 2024-09-12 03:32:25.586575793 +0000 UTC m=+0.616880417 container remove 62fe00bb357df15aaa1998d314bec6f7dbb79b782e0c213293ed0b9ddd21ed9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:32:25.945 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:25 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:25.679+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:25.945 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:25 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:25.679+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:25.945 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:25 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:25.679+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:26.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:25 smithi118 podman[69787]: 2024-09-12 03:32:25.848295565 +0000 UTC m=+0.015884892 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:32:26.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:25 smithi118 podman[69787]: 2024-09-12 03:32:25.94575682 +0000 UTC m=+0.113346181 container create 6d8aeee252622c42f82fdba9f0e464a2ca7f7d11d952cc166ddd74788a213c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-12T03:32:26.212 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:26 smithi118 podman[69787]: 2024-09-12 03:32:26.173346187 +0000 UTC m=+0.340935505 container init 6d8aeee252622c42f82fdba9f0e464a2ca7f7d11d952cc166ddd74788a213c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-12T03:32:26.213 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:26 smithi118 podman[69787]: 2024-09-12 03:32:26.176680668 +0000 UTC m=+0.344269986 container start 6d8aeee252622c42f82fdba9f0e464a2ca7f7d11d952cc166ddd74788a213c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-12T03:32:26.611 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:26 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:26.310+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:26.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:26 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:26.310+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:26.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:26 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:26.310+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:26.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:26 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:26.433+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:26.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:26 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:26.433+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:26.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:26 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:26.286+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:26.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:26 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:26.286+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:26.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:26 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:26.286+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:26.659 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:26 smithi118 podman[69787]: 2024-09-12 03:32:26.273461629 +0000 UTC m=+0.441050951 container attach 6d8aeee252622c42f82fdba9f0e464a2ca7f7d11d952cc166ddd74788a213c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0) 2024-09-12T03:32:26.659 INFO:journalctl@ceph.osd.7.smithi118.stdout:Sep 12 03:32:26 smithi118 podman[69787]: 2024-09-12 03:32:26.363791737 +0000 UTC m=+0.531381056 container died 6d8aeee252622c42f82fdba9f0e464a2ca7f7d11d952cc166ddd74788a213c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731) 2024-09-12T03:32:26.943 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:26 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:26.714+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:26.944 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:26 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:26.714+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:26.944 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:26 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:26.714+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:26.972 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.7.service' 2024-09-12T03:32:27.012 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:32:27.013 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-12T03:32:27.013 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-12T03:32:27.013 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.8 2024-09-12T03:32:27.330 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:27 smithi161 systemd[1]: Stopping Ceph osd.8 for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:32:27.330 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:27 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:27.255+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:27.330 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:27 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:27.255+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:27.330 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:27 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:27.255+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:27.611 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:27 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:27.326+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:27.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:27 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:27.326+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:27.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:27 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:27.326+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:27.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:27 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:27.329+0000 7fd906361640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:32:27.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:27 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:27.329+0000 7fd906361640 -1 osd.8 85 *** Got signal Terminated *** 2024-09-12T03:32:27.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:27 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:27.329+0000 7fd906361640 -1 osd.8 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:32:27.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:27 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:27.444+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:27.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:27 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:27.444+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:28.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:27 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:27.719+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:28.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:27 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:27.719+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:28.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:27 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:27.719+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:28.611 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:28 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:28.319+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:28.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:28 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:28.319+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:28.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:28 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:28.319+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:28.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:28 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:28.493+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:28.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:28 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:28.493+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:28.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:28 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:28.290+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:28.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:28 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:28.290+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:28.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:28 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:28.290+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:29.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:28 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:28.764+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:29.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:28 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:28.764+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:29.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:28 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:28.764+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:29.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:28 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:28.764+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.939646+0000 front 2024-09-12T03:31:01.939732+0000 (oldest deadline 2024-09-12T03:32:28.544033+0000) 2024-09-12T03:32:29.611 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:29 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:29.338+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:29.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:29 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:29.338+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:29.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:29 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:29.338+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:29.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:29 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:29.512+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:29.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:29 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:29.512+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:29.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:29 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:29.288+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:29.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:29 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:29.288+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:29.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:29 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:29.288+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:30.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:29 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:29.761+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:30.244 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:29 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:29.761+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:30.244 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:29 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:29.761+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:30.244 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:29 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:29.761+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.939646+0000 front 2024-09-12T03:31:01.939732+0000 (oldest deadline 2024-09-12T03:32:28.544033+0000) 2024-09-12T03:32:30.589 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:30 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:30.317+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:30.589 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:30 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:30.317+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:30.589 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:30 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:30.317+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:30.589 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:30 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:30.317+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.548478+0000 front 2024-09-12T03:31:01.548498+0000 (oldest deadline 2024-09-12T03:32:30.250437+0000) 2024-09-12T03:32:30.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:30 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:30.496+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:30.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:30 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:30.496+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:30.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:30 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:30.331+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:30.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:30 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:30.331+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:30.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:30 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:30.331+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:31.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:30 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:30.715+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:31.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:30 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:30.715+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:31.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:30 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:30.715+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:31.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:30 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:30.715+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.939646+0000 front 2024-09-12T03:31:01.939732+0000 (oldest deadline 2024-09-12T03:32:28.544033+0000) 2024-09-12T03:32:31.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:31 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:31.306+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:31.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:31 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:31.306+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:31.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:31 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:31.306+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:31.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:31 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:31.306+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.548478+0000 front 2024-09-12T03:31:01.548498+0000 (oldest deadline 2024-09-12T03:32:30.250437+0000) 2024-09-12T03:32:31.863 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:31 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:31.522+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:17.942162+0000 front 2024-09-12T03:30:17.942173+0000 (oldest deadline 2024-09-12T03:31:24.942665+0000) 2024-09-12T03:32:31.863 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:31 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:31.522+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.243749+0000 front 2024-09-12T03:30:27.243769+0000 (oldest deadline 2024-09-12T03:31:58.747210+0000) 2024-09-12T03:32:31.863 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:31 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8[54282]: 2024-09-12T03:32:31.522+0000 7fd8fe95e640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:30:58.747793+0000 front 2024-09-12T03:30:58.747731+0000 (oldest deadline 2024-09-12T03:32:30.852247+0000) 2024-09-12T03:32:31.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:31 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:31.371+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:31.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:31 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:31.371+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:31.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:31 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:31.371+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:31.864 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:31 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:31.371+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:31.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:31 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:31.701+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:31.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:31 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:31.701+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:31.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:31 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:31.701+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:31.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:31 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:31.701+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.939646+0000 front 2024-09-12T03:31:01.939732+0000 (oldest deadline 2024-09-12T03:32:28.544033+0000) 2024-09-12T03:32:32.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:32 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:32.278+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:32.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:32 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:32.278+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:32.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:32 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:32.278+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:32.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:32 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:32.278+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.548478+0000 front 2024-09-12T03:31:01.548498+0000 (oldest deadline 2024-09-12T03:32:30.250437+0000) 2024-09-12T03:32:32.811 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:32 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:32.371+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:32.811 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:32 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:32.371+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:32.811 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:32 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:32.371+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:32.811 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:32 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:32.371+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:32.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:32 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:32.711+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:32.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:32 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:32.711+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:32.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:32 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:32.711+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:32.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:32 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:32.711+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.939646+0000 front 2024-09-12T03:31:01.939732+0000 (oldest deadline 2024-09-12T03:32:28.544033+0000) 2024-09-12T03:32:33.079 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:32 smithi161 podman[70847]: 2024-09-12 03:32:32.811585717 +0000 UTC m=+5.615626394 container died 4b3a0dd4c5cb9eef7cea639b290e3932c8d6c994806bb396a35ffade6562c076 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:32:33.358 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:33 smithi161 podman[70847]: 2024-09-12 03:32:33.188141578 +0000 UTC m=+5.992182263 container cleanup 4b3a0dd4c5cb9eef7cea639b290e3932c8d6c994806bb396a35ffade6562c076 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:32:33.358 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:33 smithi161 bash[70847]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8 2024-09-12T03:32:33.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:33 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:33.299+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:33.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:33 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:33.299+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:33.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:33 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:33.299+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:33.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:33 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:33.299+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.548478+0000 front 2024-09-12T03:31:01.548498+0000 (oldest deadline 2024-09-12T03:32:30.250437+0000) 2024-09-12T03:32:33.613 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:33 smithi161 podman[70862]: 2024-09-12 03:32:33.439423631 +0000 UTC m=+0.628307822 container remove 4b3a0dd4c5cb9eef7cea639b290e3932c8d6c994806bb396a35ffade6562c076 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-12T03:32:33.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:33 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:33.357+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:33.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:33 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:33.357+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:33.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:33 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:33.357+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:33.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:33 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:33.357+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:34.066 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:33 smithi161 podman[70948]: 2024-09-12 03:32:33.657739638 +0000 UTC m=+0.015991997 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:32:34.066 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:33 smithi161 podman[70948]: 2024-09-12 03:32:33.773442075 +0000 UTC m=+0.131694428 container create 5eae7a220b69ddf39893c09bfce63714147029ca628c441f80e8111c49dea4fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:32:34.066 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:33 smithi161 podman[70948]: 2024-09-12 03:32:33.983842888 +0000 UTC m=+0.342095242 container init 5eae7a220b69ddf39893c09bfce63714147029ca628c441f80e8111c49dea4fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8-deactivate, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-12T03:32:34.067 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:33 smithi161 podman[70948]: 2024-09-12 03:32:33.987009128 +0000 UTC m=+0.345261482 container start 5eae7a220b69ddf39893c09bfce63714147029ca628c441f80e8111c49dea4fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:32:34.067 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:34 smithi161 podman[70948]: 2024-09-12 03:32:34.067086735 +0000 UTC m=+0.425339100 container attach 5eae7a220b69ddf39893c09bfce63714147029ca628c441f80e8111c49dea4fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-12T03:32:34.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:33 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:33.757+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:34.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:33 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:33.757+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:34.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:33 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:33.757+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:34.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:33 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:33.757+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.939646+0000 front 2024-09-12T03:31:01.939732+0000 (oldest deadline 2024-09-12T03:32:28.544033+0000) 2024-09-12T03:32:34.323 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 12 03:32:34 smithi161 podman[70948]: 2024-09-12 03:32:34.176016752 +0000 UTC m=+0.534269121 container died 5eae7a220b69ddf39893c09bfce63714147029ca628c441f80e8111c49dea4fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-12T03:32:34.611 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:34 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:34.292+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:34.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:34 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:34.292+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:34.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:34 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:34.292+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:34.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:34 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:34.292+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.548478+0000 front 2024-09-12T03:31:01.548498+0000 (oldest deadline 2024-09-12T03:32:30.250437+0000) 2024-09-12T03:32:34.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:34 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:34.323+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:34.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:34 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:34.323+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:34.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:34 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:34.323+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:34.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:34 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:34.323+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:34.773 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.8.service' 2024-09-12T03:32:34.815 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:32:34.816 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-12T03:32:34.816 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-12T03:32:34.816 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.9 2024-09-12T03:32:35.111 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:34 smithi006 systemd[1]: Stopping Ceph osd.9 for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:32:35.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:34 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:34.723+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:35.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:34 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:34.723+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:35.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:34 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:34.723+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:35.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:34 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:34.723+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.939646+0000 front 2024-09-12T03:31:01.939732+0000 (oldest deadline 2024-09-12T03:32:28.544033+0000) 2024-09-12T03:32:35.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:35 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:35.129+0000 7fcec3f2a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:32:35.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:35 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:35.129+0000 7fcec3f2a640 -1 osd.9 85 *** Got signal Terminated *** 2024-09-12T03:32:35.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:35 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:35.129+0000 7fcec3f2a640 -1 osd.9 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:32:35.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:35 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:35.275+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:35.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:35 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:35.275+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:35.613 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:35 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:35.275+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:35.613 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:35 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:35.275+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.548478+0000 front 2024-09-12T03:31:01.548498+0000 (oldest deadline 2024-09-12T03:32:30.250437+0000) 2024-09-12T03:32:35.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:35 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:35.358+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:35.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:35 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:35.358+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:35.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:35 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:35.358+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:35.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:35 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:35.358+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:36.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:35 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:35.728+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:36.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:35 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:35.728+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:36.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:35 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:35.728+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:36.214 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:35 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:35.728+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.939646+0000 front 2024-09-12T03:31:01.939732+0000 (oldest deadline 2024-09-12T03:32:28.544033+0000) 2024-09-12T03:32:36.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:36 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:36.266+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:36.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:36 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:36.266+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:36.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:36 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:36.266+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:36.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:36 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:36.266+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.548478+0000 front 2024-09-12T03:31:01.548498+0000 (oldest deadline 2024-09-12T03:32:30.250437+0000) 2024-09-12T03:32:36.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:36 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:36.335+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:36.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:36 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:36.335+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:36.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:36 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:36.335+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:36.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:36 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:36.335+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:36.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:36 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:36.678+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:36.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:36 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:36.678+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:36.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:36 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:36.678+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:36.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:36 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:36.678+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.939646+0000 front 2024-09-12T03:31:01.939732+0000 (oldest deadline 2024-09-12T03:32:28.544033+0000) 2024-09-12T03:32:36.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:36 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:36.678+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:33.745613+0000 front 2024-09-12T03:31:33.745553+0000 (oldest deadline 2024-09-12T03:32:36.645708+0000) 2024-09-12T03:32:37.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:37 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:37.274+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:37.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:37 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:37.274+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:37.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:37 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:37.274+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:37.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:37 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:37.274+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.548478+0000 front 2024-09-12T03:31:01.548498+0000 (oldest deadline 2024-09-12T03:32:30.250437+0000) 2024-09-12T03:32:37.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:37 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:37.363+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:37.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:37 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:37.363+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:37.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:37 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:37.363+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:37.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:37 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:37.363+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:37.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:37 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:37.698+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:37.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:37 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:37.698+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:37.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:37 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:37.698+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:37.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:37 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:37.698+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.939646+0000 front 2024-09-12T03:31:01.939732+0000 (oldest deadline 2024-09-12T03:32:28.544033+0000) 2024-09-12T03:32:37.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:37 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:37.698+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:33.745613+0000 front 2024-09-12T03:31:33.745553+0000 (oldest deadline 2024-09-12T03:32:36.645708+0000) 2024-09-12T03:32:38.611 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:38 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:38.252+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:38.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:38 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:38.252+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:38.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:38 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:38.252+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:38.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:38 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:38.252+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.548478+0000 front 2024-09-12T03:31:01.548498+0000 (oldest deadline 2024-09-12T03:32:30.250437+0000) 2024-09-12T03:32:38.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:38 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:38.317+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:38.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:38 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:38.317+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:38.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:38 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:38.317+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:38.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:38 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:38.317+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:38.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:38 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:38.317+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:35.546219+0000 front 2024-09-12T03:31:35.546205+0000 (oldest deadline 2024-09-12T03:32:37.846027+0000) 2024-09-12T03:32:38.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:38 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:38.704+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:38.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:38 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:38.704+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:38.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:38 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:38.704+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:38.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:38 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:38.704+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.939646+0000 front 2024-09-12T03:31:01.939732+0000 (oldest deadline 2024-09-12T03:32:28.544033+0000) 2024-09-12T03:32:38.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:38 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:38.704+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:33.745613+0000 front 2024-09-12T03:31:33.745553+0000 (oldest deadline 2024-09-12T03:32:36.645708+0000) 2024-09-12T03:32:39.611 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:39 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:39.208+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.338091+0000 front 2024-09-12T03:29:45.337990+0000 (oldest deadline 2024-09-12T03:31:16.842111+0000) 2024-09-12T03:32:39.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:39 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:39.208+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.542979+0000 front 2024-09-12T03:30:18.542928+0000 (oldest deadline 2024-09-12T03:31:23.242951+0000) 2024-09-12T03:32:39.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:39 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:39.208+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.044331+0000 front 2024-09-12T03:30:29.044313+0000 (oldest deadline 2024-09-12T03:31:55.846941+0000) 2024-09-12T03:32:39.612 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:39 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9[65258]: 2024-09-12T03:32:39.208+0000 7fcec052f640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.548478+0000 front 2024-09-12T03:31:01.548498+0000 (oldest deadline 2024-09-12T03:32:30.250437+0000) 2024-09-12T03:32:39.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:39 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:39.317+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:39.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:39 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:39.318+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:39.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:39 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:39.318+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:39.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:39 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:39.318+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:39.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:39 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:39.318+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:35.546219+0000 front 2024-09-12T03:31:35.546205+0000 (oldest deadline 2024-09-12T03:32:37.846027+0000) 2024-09-12T03:32:40.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:39 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:39.739+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:40.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:39 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:39.739+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:40.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:39 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:39.739+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:40.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:39 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:39.739+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.939646+0000 front 2024-09-12T03:31:01.939732+0000 (oldest deadline 2024-09-12T03:32:28.544033+0000) 2024-09-12T03:32:40.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:39 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:39.739+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:33.745613+0000 front 2024-09-12T03:31:33.745553+0000 (oldest deadline 2024-09-12T03:32:36.645708+0000) 2024-09-12T03:32:40.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:40 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:40.359+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:40.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:40 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:40.359+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:40.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:40 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:40.359+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:40.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:40 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:40.359+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:40.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:40 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:40.359+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:35.546219+0000 front 2024-09-12T03:31:35.546205+0000 (oldest deadline 2024-09-12T03:32:37.846027+0000) 2024-09-12T03:32:41.055 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:40 smithi006 podman[102456]: 2024-09-12 03:32:40.664642302 +0000 UTC m=+5.643351640 container died 0b3e2ef4936efe9cb7323009338b7a3528c64c8e6ddef262cce3e358e3e0abb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-12T03:32:41.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:40 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:40.750+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:41.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:40 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:40.750+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:41.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:40 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:40.750+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:41.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:40 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:40.750+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.939646+0000 front 2024-09-12T03:31:01.939732+0000 (oldest deadline 2024-09-12T03:32:28.544033+0000) 2024-09-12T03:32:41.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:40 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:40.750+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:33.745613+0000 front 2024-09-12T03:31:33.745553+0000 (oldest deadline 2024-09-12T03:32:36.645708+0000) 2024-09-12T03:32:41.362 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:41 smithi006 podman[102456]: 2024-09-12 03:32:41.246028801 +0000 UTC m=+6.224738121 container cleanup 0b3e2ef4936efe9cb7323009338b7a3528c64c8e6ddef262cce3e358e3e0abb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3) 2024-09-12T03:32:41.362 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:41 smithi006 bash[102456]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9 2024-09-12T03:32:41.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:41 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:41.344+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:41.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:41 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:41.344+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:41.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:41 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:41.344+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:41.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:41 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:41.344+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:41.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:41 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:41.344+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:35.546219+0000 front 2024-09-12T03:31:35.546205+0000 (oldest deadline 2024-09-12T03:32:37.846027+0000) 2024-09-12T03:32:41.816 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:41 smithi006 podman[102474]: 2024-09-12 03:32:41.480689214 +0000 UTC m=+0.819283911 container remove 0b3e2ef4936efe9cb7323009338b7a3528c64c8e6ddef262cce3e358e3e0abb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-12T03:32:42.100 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:41 smithi006 podman[102561]: 2024-09-12 03:32:41.719811556 +0000 UTC m=+0.016310327 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:32:42.100 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:41 smithi006 podman[102561]: 2024-09-12 03:32:41.822986631 +0000 UTC m=+0.119485403 container create 35b4e4d954460a7a40b91cef795b74fb083a7049fb8104218a7d3c8c77a0c76b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T03:32:42.100 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:42 smithi006 podman[102561]: 2024-09-12 03:32:42.033678528 +0000 UTC m=+0.330177302 container init 35b4e4d954460a7a40b91cef795b74fb083a7049fb8104218a7d3c8c77a0c76b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-12T03:32:42.100 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:42 smithi006 podman[102561]: 2024-09-12 03:32:42.037246106 +0000 UTC m=+0.333744886 container start 35b4e4d954460a7a40b91cef795b74fb083a7049fb8104218a7d3c8c77a0c76b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731) 2024-09-12T03:32:42.100 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:42 smithi006 podman[102561]: 2024-09-12 03:32:42.100234657 +0000 UTC m=+0.396733437 container attach 35b4e4d954460a7a40b91cef795b74fb083a7049fb8104218a7d3c8c77a0c76b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-12T03:32:42.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:41 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:41.727+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:42.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:41 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:41.727+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:42.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:41 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:41.727+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:42.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:41 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:41.727+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.939646+0000 front 2024-09-12T03:31:01.939732+0000 (oldest deadline 2024-09-12T03:32:28.544033+0000) 2024-09-12T03:32:42.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:41 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:41.727+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:33.745613+0000 front 2024-09-12T03:31:33.745553+0000 (oldest deadline 2024-09-12T03:32:36.645708+0000) 2024-09-12T03:32:42.362 INFO:journalctl@ceph.osd.9.smithi006.stdout:Sep 12 03:32:42 smithi006 podman[102561]: 2024-09-12 03:32:42.231547419 +0000 UTC m=+0.528046193 container died 35b4e4d954460a7a40b91cef795b74fb083a7049fb8104218a7d3c8c77a0c76b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-12T03:32:42.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:42 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:42.323+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:42.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:42 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:42.323+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:42.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:42 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:42.323+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:42.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:42 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:42.323+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:42.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:42 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:42.323+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:35.546219+0000 front 2024-09-12T03:31:35.546205+0000 (oldest deadline 2024-09-12T03:32:37.846027+0000) 2024-09-12T03:32:42.820 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.9.service' 2024-09-12T03:32:42.862 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:32:42.862 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-12T03:32:42.862 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-12T03:32:42.862 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.10 2024-09-12T03:32:43.171 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:42 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:42.755+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:43.172 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:42 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:42.755+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:43.172 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:42 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:42.755+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:43.172 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:42 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:42.755+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.939646+0000 front 2024-09-12T03:31:01.939732+0000 (oldest deadline 2024-09-12T03:32:28.544033+0000) 2024-09-12T03:32:43.172 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:42 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:42.755+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:33.745613+0000 front 2024-09-12T03:31:33.745553+0000 (oldest deadline 2024-09-12T03:32:36.645708+0000) 2024-09-12T03:32:43.172 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:42 smithi118 systemd[1]: Stopping Ceph osd.10 for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:32:43.172 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:43 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:43.170+0000 7fd3dc13a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:32:43.172 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:43 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:43.170+0000 7fd3dc13a640 -1 osd.10 85 *** Got signal Terminated *** 2024-09-12T03:32:43.172 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:43 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:43.170+0000 7fd3dc13a640 -1 osd.10 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:32:43.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:43 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:43.285+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:43.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:43 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:43.285+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:43.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:43 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:43.285+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:43.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:43 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:43.285+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:43.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:43 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:43.285+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:35.546219+0000 front 2024-09-12T03:31:35.546205+0000 (oldest deadline 2024-09-12T03:32:37.846027+0000) 2024-09-12T03:32:44.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:43 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:43.768+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:44.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:43 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:43.768+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:44.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:43 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:43.768+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:44.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:43 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:43.768+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.939646+0000 front 2024-09-12T03:31:01.939732+0000 (oldest deadline 2024-09-12T03:32:28.544033+0000) 2024-09-12T03:32:44.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:43 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:43.768+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:33.745613+0000 front 2024-09-12T03:31:33.745553+0000 (oldest deadline 2024-09-12T03:32:36.645708+0000) 2024-09-12T03:32:44.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:44 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:44.250+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:44.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:44 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:44.250+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:44.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:44 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:44.250+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:44.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:44 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:44.250+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:44.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:44 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:44.250+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:35.546219+0000 front 2024-09-12T03:31:35.546205+0000 (oldest deadline 2024-09-12T03:32:37.846027+0000) 2024-09-12T03:32:45.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:44 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:44.741+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:45.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:44 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:44.741+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:45.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:44 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:44.741+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:45.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:44 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:44.741+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.939646+0000 front 2024-09-12T03:31:01.939732+0000 (oldest deadline 2024-09-12T03:32:28.544033+0000) 2024-09-12T03:32:45.213 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:44 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:44.741+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:33.745613+0000 front 2024-09-12T03:31:33.745553+0000 (oldest deadline 2024-09-12T03:32:36.645708+0000) 2024-09-12T03:32:45.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:45 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:45.236+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:45.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:45 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:45.236+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:45.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:45 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:45.236+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:45.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:45 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:45.236+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:45.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:45 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:45.236+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:35.546219+0000 front 2024-09-12T03:31:35.546205+0000 (oldest deadline 2024-09-12T03:32:37.846027+0000) 2024-09-12T03:32:46.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:45 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:45.725+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:46.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:45 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:45.725+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:46.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:45 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:45.725+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:46.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:45 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:45.725+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.939646+0000 front 2024-09-12T03:31:01.939732+0000 (oldest deadline 2024-09-12T03:32:28.544033+0000) 2024-09-12T03:32:46.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:45 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:45.725+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:33.745613+0000 front 2024-09-12T03:31:33.745553+0000 (oldest deadline 2024-09-12T03:32:36.645708+0000) 2024-09-12T03:32:46.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:46 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:46.251+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:46.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:46 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:46.251+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:46.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:46 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:46.251+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:46.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:46 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:46.251+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:46.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:46 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:46.251+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:35.546219+0000 front 2024-09-12T03:31:35.546205+0000 (oldest deadline 2024-09-12T03:32:37.846027+0000) 2024-09-12T03:32:47.113 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:46 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:46.763+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:47.114 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:46 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:46.763+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:47.114 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:46 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:46.763+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:47.114 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:46 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:46.763+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.939646+0000 front 2024-09-12T03:31:01.939732+0000 (oldest deadline 2024-09-12T03:32:28.544033+0000) 2024-09-12T03:32:47.114 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:46 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:46.763+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:33.745613+0000 front 2024-09-12T03:31:33.745553+0000 (oldest deadline 2024-09-12T03:32:36.645708+0000) 2024-09-12T03:32:47.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:47 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:47.225+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:47.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:47 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:47.225+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:47.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:47 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:47.225+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:47.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:47 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:47.225+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:47.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:47 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:47.225+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:35.546219+0000 front 2024-09-12T03:31:35.546205+0000 (oldest deadline 2024-09-12T03:32:37.846027+0000) 2024-09-12T03:32:48.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:47 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:47.720+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:44.324633+0000 front 2024-09-12T03:29:44.324450+0000 (oldest deadline 2024-09-12T03:31:17.330893+0000) 2024-09-12T03:32:48.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:47 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:47.720+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:18.431638+0000 front 2024-09-12T03:30:18.431659+0000 (oldest deadline 2024-09-12T03:31:23.131877+0000) 2024-09-12T03:32:48.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:47 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:47.720+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:29.032806+0000 front 2024-09-12T03:30:29.032858+0000 (oldest deadline 2024-09-12T03:31:59.038692+0000) 2024-09-12T03:32:48.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:47 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:47.720+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.939646+0000 front 2024-09-12T03:31:01.939732+0000 (oldest deadline 2024-09-12T03:32:28.544033+0000) 2024-09-12T03:32:48.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:47 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-12T03:32:47.720+0000 7fd3d4737640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:33.745613+0000 front 2024-09-12T03:31:33.745553+0000 (oldest deadline 2024-09-12T03:32:36.645708+0000) 2024-09-12T03:32:48.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:48 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:48.182+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:48.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:48 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:48.182+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:48.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:48 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:48.182+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:48.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:48 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:48.182+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:48.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:48 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:48.182+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:35.546219+0000 front 2024-09-12T03:31:35.546205+0000 (oldest deadline 2024-09-12T03:32:37.846027+0000) 2024-09-12T03:32:48.962 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:48 smithi118 podman[69903]: 2024-09-12 03:32:48.687252227 +0000 UTC m=+5.635779837 container died 6a4aeb014b960f7272e3864a63b4331e40f14f837d22d9413767a39a4640d953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:32:49.322 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:49 smithi118 podman[69903]: 2024-09-12 03:32:49.071883294 +0000 UTC m=+6.020410895 container cleanup 6a4aeb014b960f7272e3864a63b4331e40f14f837d22d9413767a39a4640d953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-12T03:32:49.322 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:49 smithi118 bash[69903]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10 2024-09-12T03:32:49.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:49 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:49.174+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:49.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:49 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:49.174+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:49.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:49 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:49.174+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:49.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:49 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:49.174+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:49.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:49 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:49.174+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:35.546219+0000 front 2024-09-12T03:31:35.546205+0000 (oldest deadline 2024-09-12T03:32:37.846027+0000) 2024-09-12T03:32:49.631 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:49 smithi118 podman[69934]: 2024-09-12 03:32:49.323235983 +0000 UTC m=+0.627893537 container remove 6a4aeb014b960f7272e3864a63b4331e40f14f837d22d9413767a39a4640d953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-12T03:32:49.934 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:49 smithi118 podman[70021]: 2024-09-12 03:32:49.632352628 +0000 UTC m=+0.109780515 container create de6e6abe08c9b55dffd918ab6bb128a5cdbd3eaa02ea8be4cd880812bf1e626c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-12T03:32:49.934 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:49 smithi118 podman[70021]: 2024-09-12 03:32:49.538430196 +0000 UTC m=+0.015858098 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:32:49.934 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:49 smithi118 podman[70021]: 2024-09-12 03:32:49.851387659 +0000 UTC m=+0.328815546 container init de6e6abe08c9b55dffd918ab6bb128a5cdbd3eaa02ea8be4cd880812bf1e626c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-12T03:32:49.935 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:49 smithi118 podman[70021]: 2024-09-12 03:32:49.854741938 +0000 UTC m=+0.332169835 container start de6e6abe08c9b55dffd918ab6bb128a5cdbd3eaa02ea8be4cd880812bf1e626c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-12T03:32:49.935 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:49 smithi118 podman[70021]: 2024-09-12 03:32:49.934713403 +0000 UTC m=+0.412141307 container attach de6e6abe08c9b55dffd918ab6bb128a5cdbd3eaa02ea8be4cd880812bf1e626c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-12T03:32:50.212 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:50 smithi118 podman[70021]: 2024-09-12 03:32:50.042918328 +0000 UTC m=+0.520346215 container died de6e6abe08c9b55dffd918ab6bb128a5cdbd3eaa02ea8be4cd880812bf1e626c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:32:50.617 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:50 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:50.176+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:50.617 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:50 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:50.176+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:50.617 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:50 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:50.176+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:50.618 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:50 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:50.176+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:50.618 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:50 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:50.176+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:35.546219+0000 front 2024-09-12T03:31:35.546205+0000 (oldest deadline 2024-09-12T03:32:37.846027+0000) 2024-09-12T03:32:50.641 INFO:journalctl@ceph.osd.10.smithi118.stdout:Sep 12 03:32:50 smithi118 podman[70045]: 2024-09-12 03:32:50.613347231 +0000 UTC m=+0.564853543 container remove de6e6abe08c9b55dffd918ab6bb128a5cdbd3eaa02ea8be4cd880812bf1e626c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:32:50.646 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.10.service' 2024-09-12T03:32:50.687 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:32:50.687 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-12T03:32:50.687 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-12T03:32:50.688 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.11 2024-09-12T03:32:50.981 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:50 smithi161 systemd[1]: Stopping Ceph osd.11 for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:32:51.363 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:50 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:50.981+0000 7f57e016b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:32:51.363 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:50 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:50.981+0000 7f57e016b640 -1 osd.11 85 *** Got signal Terminated *** 2024-09-12T03:32:51.363 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:50 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:50.981+0000 7f57e016b640 -1 osd.11 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:32:51.364 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:51 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:51.160+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:51.364 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:51 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:51.160+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:51.364 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:51 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:51.160+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:51.364 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:51 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:51.160+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:51.364 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:51 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:51.160+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:35.546219+0000 front 2024-09-12T03:31:35.546205+0000 (oldest deadline 2024-09-12T03:32:37.846027+0000) 2024-09-12T03:32:52.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:52 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:52.176+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:52.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:52 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:52.176+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:52.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:52 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:52.176+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:52.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:52 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:52.176+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:52.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:52 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:52.176+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:35.546219+0000 front 2024-09-12T03:31:35.546205+0000 (oldest deadline 2024-09-12T03:32:37.846027+0000) 2024-09-12T03:32:53.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:53 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:53.169+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:53.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:53 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:53.169+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:53.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:53 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:53.169+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:53.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:53 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:53.169+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:53.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:53 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:53.169+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:35.546219+0000 front 2024-09-12T03:31:35.546205+0000 (oldest deadline 2024-09-12T03:32:37.846027+0000) 2024-09-12T03:32:54.613 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:54 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:54.155+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:54.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:54 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:54.155+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:54.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:54 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:54.155+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:54.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:54 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:54.155+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:54.614 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:54 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:54.155+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:35.546219+0000 front 2024-09-12T03:31:35.546205+0000 (oldest deadline 2024-09-12T03:32:37.846027+0000) 2024-09-12T03:32:55.515 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:55 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:55.128+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-09-12T03:29:45.929806+0000 front 2024-09-12T03:29:45.929716+0000 (oldest deadline 2024-09-12T03:31:17.834159+0000) 2024-09-12T03:32:55.515 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:55 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:55.128+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6804 osd.1 since back 2024-09-12T03:30:19.535294+0000 front 2024-09-12T03:30:19.535365+0000 (oldest deadline 2024-09-12T03:31:24.835135+0000) 2024-09-12T03:32:55.515 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:55 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:55.128+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-12T03:30:27.735858+0000 front 2024-09-12T03:30:27.735864+0000 (oldest deadline 2024-09-12T03:31:55.539599+0000) 2024-09-12T03:32:55.515 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:55 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:55.128+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-09-12T03:31:01.141470+0000 front 2024-09-12T03:31:01.141546+0000 (oldest deadline 2024-09-12T03:32:30.944873+0000) 2024-09-12T03:32:55.515 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:55 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11[60903]: 2024-09-12T03:32:55.128+0000 7f57dc770640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.118:6812 osd.4 since back 2024-09-12T03:31:35.546219+0000 front 2024-09-12T03:31:35.546205+0000 (oldest deadline 2024-09-12T03:32:37.846027+0000) 2024-09-12T03:32:56.805 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:56 smithi161 podman[71052]: 2024-09-12 03:32:56.579294886 +0000 UTC m=+5.690173568 container died db84530bb03580c4672c0622287c413731061d82541375a17f196973cc7a7085 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-12T03:32:57.113 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:56 smithi161 podman[71052]: 2024-09-12 03:32:56.938662332 +0000 UTC m=+6.049541002 container cleanup db84530bb03580c4672c0622287c413731061d82541375a17f196973cc7a7085 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:32:57.113 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:56 smithi161 bash[71052]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11 2024-09-12T03:32:57.566 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:57 smithi161 podman[71067]: 2024-09-12 03:32:57.214990715 +0000 UTC m=+0.636318002 container remove db84530bb03580c4672c0622287c413731061d82541375a17f196973cc7a7085 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-12T03:32:57.817 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:57 smithi161 podman[71166]: 2024-09-12 03:32:57.46982743 +0000 UTC m=+0.016174925 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:32:57.817 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:57 smithi161 podman[71166]: 2024-09-12 03:32:57.582192147 +0000 UTC m=+0.128539628 container create adeab35d39a596a03503a7a028d2718bee3ee848eb1f4cd367846b69d7dd346a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:32:57.817 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:57 smithi161 podman[71166]: 2024-09-12 03:32:57.776162151 +0000 UTC m=+0.322509632 container init adeab35d39a596a03503a7a028d2718bee3ee848eb1f4cd367846b69d7dd346a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11-deactivate, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True) 2024-09-12T03:32:57.817 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:57 smithi161 podman[71166]: 2024-09-12 03:32:57.779519946 +0000 UTC m=+0.325867430 container start adeab35d39a596a03503a7a028d2718bee3ee848eb1f4cd367846b69d7dd346a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-12T03:32:58.113 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:57 smithi161 podman[71166]: 2024-09-12 03:32:57.893629772 +0000 UTC m=+0.439977251 container attach adeab35d39a596a03503a7a028d2718bee3ee848eb1f4cd367846b69d7dd346a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:32:58.113 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:57 smithi161 podman[71166]: 2024-09-12 03:32:57.974313814 +0000 UTC m=+0.520661291 container died adeab35d39a596a03503a7a028d2718bee3ee848eb1f4cd367846b69d7dd346a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-12T03:32:58.468 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.11.service' 2024-09-12T03:32:58.491 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:58 smithi161 podman[71190]: 2024-09-12 03:32:58.454574732 +0000 UTC m=+0.474621997 container remove adeab35d39a596a03503a7a028d2718bee3ee848eb1f4cd367846b69d7dd346a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-12T03:32:58.491 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:58 smithi161 systemd[1]: ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.11.service: Deactivated successfully. 2024-09-12T03:32:58.492 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:58 smithi161 systemd[1]: Stopped Ceph osd.11 for 89d99938-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:32:58.492 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 12 03:32:58 smithi161 systemd[1]: ceph-89d99938-70b4-11ef-bcea-c7b262605968@osd.11.service: Consumed 8min 627ms CPU time. 2024-09-12T03:32:58.506 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:32:58.507 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-12T03:32:58.507 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-12T03:32:58.507 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.a 2024-09-12T03:32:58.861 INFO:journalctl@ceph.mds.a.smithi006.stdout:Sep 12 03:32:58 smithi006 systemd[1]: Stopping Ceph mds.a for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:32:59.361 INFO:journalctl@ceph.mds.a.smithi006.stdout:Sep 12 03:32:58 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-a[72474]: 2024-09-12T03:32:58.930+0000 7fb29f214640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:32:59.362 INFO:journalctl@ceph.mds.a.smithi006.stdout:Sep 12 03:32:58 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-a[72474]: 2024-09-12T03:32:58.930+0000 7fb29f214640 -1 mds.a *** got signal Terminated *** 2024-09-12T03:33:02.862 INFO:journalctl@ceph.mds.a.smithi006.stdout:Sep 12 03:33:02 smithi006 podman[102664]: 2024-09-12 03:33:02.606426807 +0000 UTC m=+3.896952479 container died c12ad20163c7b7b84e06925a96668a96042004c8fa4b77d308c9fae1aea03b99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:33:03.331 INFO:journalctl@ceph.mds.a.smithi006.stdout:Sep 12 03:33:03 smithi006 podman[102664]: 2024-09-12 03:33:03.088746753 +0000 UTC m=+4.379272421 container cleanup c12ad20163c7b7b84e06925a96668a96042004c8fa4b77d308c9fae1aea03b99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-12T03:33:03.331 INFO:journalctl@ceph.mds.a.smithi006.stdout:Sep 12 03:33:03 smithi006 bash[102664]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-a 2024-09-12T03:33:03.480 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.a.service' 2024-09-12T03:33:03.518 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:33:03.518 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-12T03:33:03.519 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-12T03:33:03.519 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.d 2024-09-12T03:33:03.851 INFO:journalctl@ceph.mds.d.smithi006.stdout:Sep 12 03:33:03 smithi006 systemd[1]: Stopping Ceph mds.d for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:33:04.112 INFO:journalctl@ceph.mds.d.smithi006.stdout:Sep 12 03:33:03 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-d[71290]: 2024-09-12T03:33:03.851+0000 7efd70459640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:33:04.112 INFO:journalctl@ceph.mds.d.smithi006.stdout:Sep 12 03:33:03 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-d[71290]: 2024-09-12T03:33:03.851+0000 7efd70459640 -1 mds.d *** got signal Terminated *** 2024-09-12T03:33:08.093 INFO:journalctl@ceph.mds.d.smithi006.stdout:Sep 12 03:33:07 smithi006 podman[102800]: 2024-09-12 03:33:07.807720616 +0000 UTC m=+4.062579727 container died 1076a04284222cad7fac2c536fcf2ead2bffec19d60caab482edc907ee3c9304 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-d, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-12T03:33:08.362 INFO:journalctl@ceph.mds.d.smithi006.stdout:Sep 12 03:33:08 smithi006 podman[102800]: 2024-09-12 03:33:08.192498339 +0000 UTC m=+4.447357460 container cleanup 1076a04284222cad7fac2c536fcf2ead2bffec19d60caab482edc907ee3c9304 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T03:33:08.362 INFO:journalctl@ceph.mds.d.smithi006.stdout:Sep 12 03:33:08 smithi006 bash[102800]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-d 2024-09-12T03:33:08.573 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.d.service' 2024-09-12T03:33:08.608 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:33:08.608 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-12T03:33:08.609 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-12T03:33:08.609 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.g 2024-09-12T03:33:09.112 INFO:journalctl@ceph.mds.g.smithi006.stdout:Sep 12 03:33:08 smithi006 systemd[1]: Stopping Ceph mds.g for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:33:09.112 INFO:journalctl@ceph.mds.g.smithi006.stdout:Sep 12 03:33:08 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-g[72064]: 2024-09-12T03:33:08.929+0000 7fdedacb0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:33:09.112 INFO:journalctl@ceph.mds.g.smithi006.stdout:Sep 12 03:33:08 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-g[72064]: 2024-09-12T03:33:08.929+0000 7fdedacb0640 -1 mds.g *** got signal Terminated *** 2024-09-12T03:33:12.014 INFO:journalctl@ceph.mds.g.smithi006.stdout:Sep 12 03:33:11 smithi006 podman[102922]: 2024-09-12 03:33:11.642689879 +0000 UTC m=+2.812431377 container died d6b829cebcbd88c7a9e1c83470276f567d7e4c75dd425c60f6d6e91346507042 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-09-12T03:33:12.356 INFO:journalctl@ceph.mds.g.smithi006.stdout:Sep 12 03:33:12 smithi006 podman[102922]: 2024-09-12 03:33:12.113339211 +0000 UTC m=+3.283080708 container cleanup d6b829cebcbd88c7a9e1c83470276f567d7e4c75dd425c60f6d6e91346507042 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-12T03:33:12.356 INFO:journalctl@ceph.mds.g.smithi006.stdout:Sep 12 03:33:12 smithi006 bash[102922]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-g 2024-09-12T03:33:12.505 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.g.service' 2024-09-12T03:33:12.539 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:33:12.539 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-12T03:33:12.539 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-12T03:33:12.539 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.j 2024-09-12T03:33:13.112 INFO:journalctl@ceph.mds.j.smithi006.stdout:Sep 12 03:33:12 smithi006 systemd[1]: Stopping Ceph mds.j for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:33:13.112 INFO:journalctl@ceph.mds.j.smithi006.stdout:Sep 12 03:33:12 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-j[70879]: 2024-09-12T03:33:12.858+0000 7f33b7609640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:33:13.112 INFO:journalctl@ceph.mds.j.smithi006.stdout:Sep 12 03:33:12 smithi006 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-j[70879]: 2024-09-12T03:33:12.858+0000 7f33b7609640 -1 mds.j *** got signal Terminated *** 2024-09-12T03:33:16.611 INFO:journalctl@ceph.mds.j.smithi006.stdout:Sep 12 03:33:16 smithi006 podman[103044]: 2024-09-12 03:33:16.266924156 +0000 UTC m=+3.501139389 container died cda4ab06d3e1be23aaf127ad205f5708e36dbf88a3aff661a17554db262192b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-12T03:33:16.975 INFO:journalctl@ceph.mds.j.smithi006.stdout:Sep 12 03:33:16 smithi006 podman[103044]: 2024-09-12 03:33:16.716933497 +0000 UTC m=+3.951148729 container cleanup cda4ab06d3e1be23aaf127ad205f5708e36dbf88a3aff661a17554db262192b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-j, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-12T03:33:16.975 INFO:journalctl@ceph.mds.j.smithi006.stdout:Sep 12 03:33:16 smithi006 bash[103044]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-j 2024-09-12T03:33:17.152 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.j.service' 2024-09-12T03:33:17.186 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:33:17.186 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-12T03:33:17.186 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-12T03:33:17.186 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.b 2024-09-12T03:33:17.481 INFO:journalctl@ceph.mds.b.smithi118.stdout:Sep 12 03:33:17 smithi118 systemd[1]: Stopping Ceph mds.b for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:33:17.735 INFO:journalctl@ceph.mds.b.smithi118.stdout:Sep 12 03:33:17 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-b[65505]: 2024-09-12T03:33:17.480+0000 7f3738b2d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:33:17.736 INFO:journalctl@ceph.mds.b.smithi118.stdout:Sep 12 03:33:17 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-b[65505]: 2024-09-12T03:33:17.480+0000 7f3738b2d640 -1 mds.b *** got signal Terminated *** 2024-09-12T03:33:21.799 INFO:journalctl@ceph.mds.b.smithi118.stdout:Sep 12 03:33:21 smithi118 podman[70125]: 2024-09-12 03:33:21.486249524 +0000 UTC m=+4.108927733 container died e937849e5ddb6f857a968dc3070fe4c29cbdc6c40ada7db7c1d62e89c823f53b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-12T03:33:22.123 INFO:journalctl@ceph.mds.b.smithi118.stdout:Sep 12 03:33:21 smithi118 podman[70125]: 2024-09-12 03:33:21.88064707 +0000 UTC m=+4.503325268 container cleanup e937849e5ddb6f857a968dc3070fe4c29cbdc6c40ada7db7c1d62e89c823f53b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-b, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-12T03:33:22.123 INFO:journalctl@ceph.mds.b.smithi118.stdout:Sep 12 03:33:21 smithi118 bash[70125]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-b 2024-09-12T03:33:22.258 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.b.service' 2024-09-12T03:33:22.295 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:33:22.295 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-12T03:33:22.295 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-12T03:33:22.296 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.e 2024-09-12T03:33:22.701 INFO:journalctl@ceph.mds.e.smithi118.stdout:Sep 12 03:33:22 smithi118 systemd[1]: Stopping Ceph mds.e for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:33:22.962 INFO:journalctl@ceph.mds.e.smithi118.stdout:Sep 12 03:33:22 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-e[64900]: 2024-09-12T03:33:22.700+0000 7f71557ad640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:33:22.962 INFO:journalctl@ceph.mds.e.smithi118.stdout:Sep 12 03:33:22 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-e[64900]: 2024-09-12T03:33:22.700+0000 7f71557ad640 -1 mds.e *** got signal Terminated *** 2024-09-12T03:33:26.712 INFO:journalctl@ceph.mds.e.smithi118.stdout:Sep 12 03:33:26 smithi118 podman[70259]: 2024-09-12 03:33:26.431106146 +0000 UTC m=+3.917253021 container died 3075a89eb6ea44a7e415849ca256c16068f0e9fb6387a5465c1a103169a43fc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-12T03:33:27.052 INFO:journalctl@ceph.mds.e.smithi118.stdout:Sep 12 03:33:26 smithi118 podman[70259]: 2024-09-12 03:33:26.801439604 +0000 UTC m=+4.287586477 container cleanup 3075a89eb6ea44a7e415849ca256c16068f0e9fb6387a5465c1a103169a43fc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-12T03:33:27.052 INFO:journalctl@ceph.mds.e.smithi118.stdout:Sep 12 03:33:26 smithi118 bash[70259]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-e 2024-09-12T03:33:27.181 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.e.service' 2024-09-12T03:33:27.214 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:33:27.215 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-12T03:33:27.215 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-12T03:33:27.215 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.h 2024-09-12T03:33:27.464 INFO:journalctl@ceph.mds.h.smithi118.stdout:Sep 12 03:33:27 smithi118 systemd[1]: Stopping Ceph mds.h for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:33:27.962 INFO:journalctl@ceph.mds.h.smithi118.stdout:Sep 12 03:33:27 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-h[65203]: 2024-09-12T03:33:27.562+0000 7fbbb1328640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:33:27.962 INFO:journalctl@ceph.mds.h.smithi118.stdout:Sep 12 03:33:27 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-h[65203]: 2024-09-12T03:33:27.562+0000 7fbbb1328640 -1 mds.h *** got signal Terminated *** 2024-09-12T03:33:30.712 INFO:journalctl@ceph.mds.h.smithi118.stdout:Sep 12 03:33:30 smithi118 podman[70380]: 2024-09-12 03:33:30.440966676 +0000 UTC m=+3.005111602 container died 8070a91c59c69526155be6cb0a1840fed9f6b0c27275e388ad1ca6fb9e195842 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2) 2024-09-12T03:33:31.064 INFO:journalctl@ceph.mds.h.smithi118.stdout:Sep 12 03:33:30 smithi118 podman[70380]: 2024-09-12 03:33:30.805558385 +0000 UTC m=+3.369703313 container cleanup 8070a91c59c69526155be6cb0a1840fed9f6b0c27275e388ad1ca6fb9e195842 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-h, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:33:31.064 INFO:journalctl@ceph.mds.h.smithi118.stdout:Sep 12 03:33:30 smithi118 bash[70380]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-h 2024-09-12T03:33:31.224 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.h.service' 2024-09-12T03:33:31.258 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:33:31.258 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-12T03:33:31.258 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-12T03:33:31.259 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.k 2024-09-12T03:33:31.584 INFO:journalctl@ceph.mds.k.smithi118.stdout:Sep 12 03:33:31 smithi118 systemd[1]: Stopping Ceph mds.k for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:33:31.964 INFO:journalctl@ceph.mds.k.smithi118.stdout:Sep 12 03:33:31 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-k[64585]: 2024-09-12T03:33:31.584+0000 7f063120a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:33:31.964 INFO:journalctl@ceph.mds.k.smithi118.stdout:Sep 12 03:33:31 smithi118 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-k[64585]: 2024-09-12T03:33:31.584+0000 7f063120a640 -1 mds.k *** got signal Terminated *** 2024-09-12T03:33:33.452 INFO:journalctl@ceph.mds.k.smithi118.stdout:Sep 12 03:33:33 smithi118 podman[70500]: 2024-09-12 03:33:33.181410599 +0000 UTC m=+1.706515693 container died 89deea317135b27996571d99c6023a62135d0ed66377239baf21718f956782bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T03:33:33.712 INFO:journalctl@ceph.mds.k.smithi118.stdout:Sep 12 03:33:33 smithi118 podman[70500]: 2024-09-12 03:33:33.551575035 +0000 UTC m=+2.076680128 container cleanup 89deea317135b27996571d99c6023a62135d0ed66377239baf21718f956782bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-k, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:33:33.712 INFO:journalctl@ceph.mds.k.smithi118.stdout:Sep 12 03:33:33 smithi118 bash[70500]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-k 2024-09-12T03:33:33.967 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.k.service' 2024-09-12T03:33:33.998 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:33:33.998 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-12T03:33:33.998 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-12T03:33:33.998 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.c 2024-09-12T03:33:34.320 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 12 03:33:34 smithi161 systemd[1]: Stopping Ceph mds.c for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:33:34.613 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 12 03:33:34 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-c[66065]: 2024-09-12T03:33:34.320+0000 7fea6c02b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:33:34.613 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 12 03:33:34 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-c[66065]: 2024-09-12T03:33:34.320+0000 7fea6c02b640 -1 mds.c *** got signal Terminated *** 2024-09-12T03:33:38.612 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 12 03:33:38 smithi161 podman[71267]: 2024-09-12 03:33:38.326270197 +0000 UTC m=+4.135582019 container died 9abf9587f1ed710013ce0d4893f616920182d479a5655abe604af13a88120f57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2) 2024-09-12T03:33:38.863 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 12 03:33:38 smithi161 podman[71267]: 2024-09-12 03:33:38.703357977 +0000 UTC m=+4.512669789 container cleanup 9abf9587f1ed710013ce0d4893f616920182d479a5655abe604af13a88120f57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-12T03:33:38.863 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 12 03:33:38 smithi161 bash[71267]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-c 2024-09-12T03:33:39.140 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.c.service' 2024-09-12T03:33:39.173 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:33:39.174 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-12T03:33:39.174 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-12T03:33:39.174 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.f 2024-09-12T03:33:39.490 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 12 03:33:39 smithi161 systemd[1]: Stopping Ceph mds.f for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:33:39.863 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 12 03:33:39 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-f[65762]: 2024-09-12T03:33:39.490+0000 7f0189f2e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:33:39.863 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 12 03:33:39 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-f[65762]: 2024-09-12T03:33:39.490+0000 7f0189f2e640 -1 mds.f *** got signal Terminated *** 2024-09-12T03:33:42.747 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 12 03:33:42 smithi161 podman[71386]: 2024-09-12 03:33:42.422363983 +0000 UTC m=+3.026253801 container died 7aa5e78212500ef71291ba37a9a8e24c7dc229174b1ca4ae9d58a156b1e2201b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:33:42.748 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 12 03:33:42 smithi161 podman[71386]: 2024-09-12 03:33:42.748415749 +0000 UTC m=+3.352305567 container cleanup 7aa5e78212500ef71291ba37a9a8e24c7dc229174b1ca4ae9d58a156b1e2201b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-f, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T03:33:43.113 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 12 03:33:42 smithi161 bash[71386]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-f 2024-09-12T03:33:43.113 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 12 03:33:42 smithi161 podman[71399]: 2024-09-12 03:33:42.93301372 +0000 UTC m=+0.502538546 container remove 7aa5e78212500ef71291ba37a9a8e24c7dc229174b1ca4ae9d58a156b1e2201b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-12T03:33:43.185 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.f.service' 2024-09-12T03:33:43.221 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:33:43.222 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-12T03:33:43.222 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-12T03:33:43.222 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.i 2024-09-12T03:33:43.551 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 12 03:33:43 smithi161 systemd[1]: Stopping Ceph mds.i for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:33:43.863 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 12 03:33:43 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-i[66368]: 2024-09-12T03:33:43.551+0000 7f1a7d8c1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:33:43.863 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 12 03:33:43 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-i[66368]: 2024-09-12T03:33:43.551+0000 7f1a7d8c1640 -1 mds.i *** got signal Terminated *** 2024-09-12T03:33:46.701 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 12 03:33:46 smithi161 podman[71508]: 2024-09-12 03:33:46.420669931 +0000 UTC m=+2.972508157 container died 12bd8a72daec7123574e5cf07bbf51a9f89794f2d61fa69b0271e2c1918a6749 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-i, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:33:47.056 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 12 03:33:46 smithi161 podman[71508]: 2024-09-12 03:33:46.793517449 +0000 UTC m=+3.345355662 container cleanup 12bd8a72daec7123574e5cf07bbf51a9f89794f2d61fa69b0271e2c1918a6749 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-12T03:33:47.056 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 12 03:33:46 smithi161 bash[71508]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-i 2024-09-12T03:33:47.212 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.i.service' 2024-09-12T03:33:47.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:33:47.244 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-12T03:33:47.244 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-12T03:33:47.244 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.l 2024-09-12T03:33:47.563 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 12 03:33:47 smithi161 systemd[1]: Stopping Ceph mds.l for 89d99938-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:33:47.863 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 12 03:33:47 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-l[65462]: 2024-09-12T03:33:47.563+0000 7f2a12fca640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:33:47.863 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 12 03:33:47 smithi161 ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-l[65462]: 2024-09-12T03:33:47.563+0000 7f2a12fca640 -1 mds.l *** got signal Terminated *** 2024-09-12T03:33:50.897 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 12 03:33:50 smithi161 podman[71629]: 2024-09-12 03:33:50.616584467 +0000 UTC m=+3.156065960 container died 456b8c4a58a3ac1ba9a389d56d24eeb6b977c18a53082232d712696b35a7edd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3) 2024-09-12T03:33:51.230 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 12 03:33:50 smithi161 podman[71629]: 2024-09-12 03:33:50.988745093 +0000 UTC m=+3.528226587 container cleanup 456b8c4a58a3ac1ba9a389d56d24eeb6b977c18a53082232d712696b35a7edd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-l, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-12T03:33:51.231 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 12 03:33:50 smithi161 bash[71629]: ceph-89d99938-70b4-11ef-bcea-c7b262605968-mds-l 2024-09-12T03:33:51.368 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89d99938-70b4-11ef-bcea-c7b262605968@mds.l.service' 2024-09-12T03:33:51.400 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:33:51.400 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-12T03:33:51.400 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 89d99938-70b4-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:33:51.612 INFO:teuthology.orchestra.run.smithi006.stdout:Deleting cluster with fsid: 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:33:54.205 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 89d99938-70b4-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:33:54.405 INFO:teuthology.orchestra.run.smithi118.stdout:Deleting cluster with fsid: 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:33:56.881 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 89d99938-70b4-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:33:57.081 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:33:59.445 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:33:59.479 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:33:59.512 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:33:59.539 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-12T03:33:59.541 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901668/remote/smithi006/crash 2024-09-12T03:33:59.542 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:33:59.588 INFO:teuthology.orchestra.run.smithi006.stderr:tar: /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:33:59.588 INFO:teuthology.orchestra.run.smithi006.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:33:59.591 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901668/remote/smithi118/crash 2024-09-12T03:33:59.592 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:33:59.635 INFO:teuthology.orchestra.run.smithi118.stderr:tar: /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:33:59.636 INFO:teuthology.orchestra.run.smithi118.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:33:59.638 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901668/remote/smithi161/crash 2024-09-12T03:33:59.639 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:33:59.677 INFO:teuthology.orchestra.run.smithi161.stderr:tar: /var/lib/ceph/89d99938-70b4-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:33:59.677 INFO:teuthology.orchestra.run.smithi161.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:33:59.679 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-12T03:33:59.679 DEBUG:teuthology.orchestra.run.smithi006:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-12T03:33:59.720 INFO:tasks.cephadm:Compressing logs... 2024-09-12T03:33:59.720 DEBUG:teuthology.orchestra.run.smithi006:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:33:59.765 DEBUG:teuthology.orchestra.run.smithi118:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:33:59.771 DEBUG:teuthology.orchestra.run.smithi161:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:33:59.792 INFO:teuthology.orchestra.run.smithi006.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-12T03:33:59.793 INFO:teuthology.orchestra.run.smithi006.stderr:: No such file or directory 2024-09-12T03:33:59.793 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-12T03:33:59.793 INFO:teuthology.orchestra.run.smithi006.stderr:gzip/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.0.log: -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:33:59.793 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:33:59.793 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-12T03:33:59.795 INFO:teuthology.orchestra.run.smithi161.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:33:59.796 INFO:teuthology.orchestra.run.smithi161.stderr:: No such file or directory 2024-09-12T03:33:59.796 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:33:59.796 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-12T03:33:59.798 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-12T03:33:59.798 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:33:59.798 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-12T03:33:59.800 INFO:teuthology.orchestra.run.smithi118.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-12T03:33:59.800 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-12T03:33:59.800 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-12T03:33:59.800 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-12T03:33:59.800 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.10.log: /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:33:59.800 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-12T03:33:59.800 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-12T03:33:59.800 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-12T03:33:59.800 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:33:59.801 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:33:59.804 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:33:59.804 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-12T03:33:59.804 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-12T03:33:59.805 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.cephadm.log: /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mon.a.log: 82.7% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:33:59.805 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.g.log: 90.3% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:33:59.805 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-12T03:33:59.805 INFO:teuthology.orchestra.run.smithi006.stderr: 94.8% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-12T03:33:59.805 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:33:59.805 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-12T03:33:59.805 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-12T03:33:59.805 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.log: /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.d.log: 95.0% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-12T03:33:59.805 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-12T03:33:59.805 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mgr.x.log: 86.3% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:33:59.805 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-12T03:33:59.805 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.j.log: 95.0% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-12T03:33:59.808 INFO:teuthology.orchestra.run.smithi006.stderr: 90.9% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:33:59.808 INFO:teuthology.orchestra.run.smithi006.stderr: 95.0% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-12T03:33:59.808 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:33:59.810 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-12T03:33:59.810 INFO:teuthology.orchestra.run.smithi161.stderr: 82.6% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:33:59.810 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-12T03:33:59.810 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.2.log: --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:33:59.810 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-12T03:33:59.810 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.log: 90.4% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:33:59.811 INFO:teuthology.orchestra.run.smithi161.stderr: 86.3% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:33:59.811 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-12T03:33:59.811 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-12T03:33:59.811 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.8.log: 95.0% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-12T03:33:59.811 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-12T03:33:59.811 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mgr.z.log: 90.9% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:33:59.815 INFO:teuthology.orchestra.run.smithi161.stderr: 92.7% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-12T03:33:59.815 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-12T03:33:59.815 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:33:59.816 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.audit.log: 90.4% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:33:59.816 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-12T03:33:59.816 INFO:teuthology.orchestra.run.smithi118.stderr: 94.8% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-12T03:33:59.816 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:33:59.816 INFO:teuthology.orchestra.run.smithi118.stderr: 82.7%/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mgr.y.log: -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:33:59.816 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-12T03:33:59.817 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.log: 86.3% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:33:59.817 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-12T03:33:59.817 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.h.log: 92.6% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-12T03:33:59.817 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-12T03:33:59.817 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mon.b.log: 90.9% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:33:59.817 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:33:59.821 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:33:59.825 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:33:59.828 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:33:59.871 INFO:teuthology.orchestra.run.smithi161.stderr: 93.9% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-12T03:33:59.874 INFO:teuthology.orchestra.run.smithi118.stderr: 94.1% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-12T03:34:00.039 INFO:teuthology.orchestra.run.smithi161.stderr: 94.2% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-12T03:34:00.054 INFO:teuthology.orchestra.run.smithi118.stderr: 94.1% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-12T03:34:00.058 INFO:teuthology.orchestra.run.smithi006.stderr: 90.2% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-12T03:34:00.186 INFO:teuthology.orchestra.run.smithi161.stderr: 91.7% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-12T03:34:00.200 INFO:teuthology.orchestra.run.smithi118.stderr: 91.7% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-12T03:34:00.860 INFO:teuthology.orchestra.run.smithi006.stderr: 90.3% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-12T03:34:03.932 INFO:teuthology.orchestra.run.smithi161.stderr: 96.7% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-12T03:34:11.192 INFO:teuthology.orchestra.run.smithi006.stderr: 93.2% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-12T03:34:13.143 INFO:teuthology.orchestra.run.smithi118.stderr: 93.4% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-12T03:34:13.419 INFO:teuthology.orchestra.run.smithi161.stderr: 93.2% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-12T03:34:14.969 INFO:teuthology.orchestra.run.smithi161.stderr: 93.2% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-12T03:34:16.057 INFO:teuthology.orchestra.run.smithi118.stderr: 93.2% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-12T03:34:17.062 INFO:teuthology.orchestra.run.smithi006.stderr: 93.2% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-12T03:34:18.085 INFO:teuthology.orchestra.run.smithi161.stderr: 93.1% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-12T03:34:19.763 INFO:teuthology.orchestra.run.smithi118.stderr: 93.0% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-12T03:34:20.524 INFO:teuthology.orchestra.run.smithi161.stderr: 93.1% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-12T03:34:20.524 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-09-12T03:34:20.525 INFO:teuthology.orchestra.run.smithi161.stderr:real 0m20.730s 2024-09-12T03:34:20.525 INFO:teuthology.orchestra.run.smithi161.stderr:user 1m9.991s 2024-09-12T03:34:20.525 INFO:teuthology.orchestra.run.smithi161.stderr:sys 0m2.517s 2024-09-12T03:34:21.786 INFO:teuthology.orchestra.run.smithi006.stderr: 93.0% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-12T03:34:24.196 INFO:teuthology.orchestra.run.smithi006.stderr: 93.1% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-12T03:34:24.201 INFO:teuthology.orchestra.run.smithi006.stderr: 2024-09-12T03:34:24.201 INFO:teuthology.orchestra.run.smithi006.stderr:real 0m24.418s 2024-09-12T03:34:24.201 INFO:teuthology.orchestra.run.smithi006.stderr:user 1m13.841s 2024-09-12T03:34:24.201 INFO:teuthology.orchestra.run.smithi006.stderr:sys 0m2.460s 2024-09-12T03:34:24.963 INFO:teuthology.orchestra.run.smithi118.stderr: 93.2% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-12T03:34:30.756 INFO:teuthology.orchestra.run.smithi118.stderr: 94.7% -- replaced with /var/log/ceph/89d99938-70b4-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-12T03:34:30.758 INFO:teuthology.orchestra.run.smithi118.stderr: 2024-09-12T03:34:30.758 INFO:teuthology.orchestra.run.smithi118.stderr:real 0m30.974s 2024-09-12T03:34:30.758 INFO:teuthology.orchestra.run.smithi118.stderr:user 1m42.768s 2024-09-12T03:34:30.758 INFO:teuthology.orchestra.run.smithi118.stderr:sys 0m3.418s 2024-09-12T03:34:30.759 INFO:tasks.cephadm:Archiving logs... 2024-09-12T03:34:30.759 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901668/remote/smithi006/log 2024-09-12T03:34:30.760 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:34:41.583 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901668/remote/smithi118/log 2024-09-12T03:34:41.584 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:34:55.530 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901668/remote/smithi161/log 2024-09-12T03:34:55.531 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:35:05.560 INFO:tasks.cephadm:Removing cluster... 2024-09-12T03:35:05.560 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 89d99938-70b4-11ef-bcea-c7b262605968 --force 2024-09-12T03:35:05.773 INFO:teuthology.orchestra.run.smithi006.stdout:Deleting cluster with fsid: 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:35:06.145 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 89d99938-70b4-11ef-bcea-c7b262605968 --force 2024-09-12T03:35:06.344 INFO:teuthology.orchestra.run.smithi118.stdout:Deleting cluster with fsid: 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:35:06.745 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 89d99938-70b4-11ef-bcea-c7b262605968 --force 2024-09-12T03:35:06.949 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 89d99938-70b4-11ef-bcea-c7b262605968 2024-09-12T03:35:07.305 INFO:tasks.cephadm:Removing cephadm ... 2024-09-12T03:35:07.306 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T03:35:07.326 DEBUG:teuthology.orchestra.run.smithi118:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T03:35:07.343 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T03:35:07.359 INFO:tasks.cephadm:Teardown complete 2024-09-12T03:35:07.359 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T03:35:07.370 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T03:35:07.371 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:35:07.373 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:35:07.386 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:35:07.526 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T03:35:07.537 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T03:35:07.537 DEBUG:teuthology.orchestra.run.smithi006:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:35:07.540 DEBUG:teuthology.orchestra.run.smithi118:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:35:07.543 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:35:07.556 INFO:teuthology.orchestra.run.smithi006.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:35:07.556 INFO:teuthology.orchestra.run.smithi118.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:35:07.557 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:35:07.560 INFO:teuthology.orchestra.run.smithi118.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:35:07.560 INFO:teuthology.orchestra.run.smithi118.stdout:=============================================================================== 2024-09-12T03:35:07.561 INFO:teuthology.orchestra.run.smithi118.stdout:^* hv01.front.sepia.ceph.com 3 6 377 38 -2934us[-2881us] +/- 54ms 2024-09-12T03:35:07.561 INFO:teuthology.orchestra.run.smithi118.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 40 +1821us[+1874us] +/- 69ms 2024-09-12T03:35:07.561 INFO:teuthology.orchestra.run.smithi118.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +2372us[+2425us] +/- 80ms 2024-09-12T03:35:07.561 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:35:07.562 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:35:07.562 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-09-12T03:35:07.562 INFO:teuthology.orchestra.run.smithi161.stdout:^* hv01.front.sepia.ceph.com 3 6 377 41 -2913us[-2915us] +/- 54ms 2024-09-12T03:35:07.562 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 40 +1819us[+1819us] +/- 69ms 2024-09-12T03:35:07.562 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +2389us[+2389us] +/- 80ms 2024-09-12T03:35:07.562 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:35:07.563 INFO:teuthology.orchestra.run.smithi006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:35:07.563 INFO:teuthology.orchestra.run.smithi006.stdout:=============================================================================== 2024-09-12T03:35:07.563 INFO:teuthology.orchestra.run.smithi006.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 -2916us[-2902us] +/- 54ms 2024-09-12T03:35:07.563 INFO:teuthology.orchestra.run.smithi006.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 41 +1810us[+1824us] +/- 69ms 2024-09-12T03:35:07.563 INFO:teuthology.orchestra.run.smithi006.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 +2392us[+2392us] +/- 80ms 2024-09-12T03:35:07.564 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:35:07.564 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T03:35:07.588 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T03:35:07.589 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T03:35:07.599 DEBUG:teuthology.orchestra.run.smithi006:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:35:07.691 DEBUG:teuthology.orchestra.run.smithi118:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:35:07.760 DEBUG:teuthology.orchestra.run.smithi161:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:35:07.829 DEBUG:teuthology.orchestra.run.smithi006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:35:07.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:35:07.862 DEBUG:teuthology.orchestra.run.smithi118:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:35:07.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:35:07.891 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:35:07.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:35:07.919 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T03:35:07.937 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T03:35:07.948 INFO:teuthology.task.internal:Duration was 1848.537943 seconds 2024-09-12T03:35:07.949 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T03:35:07.959 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T03:35:07.960 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:35:07.963 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:35:07.965 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:35:08.002 INFO:teuthology.orchestra.run.smithi118.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:35:08.004 INFO:teuthology.orchestra.run.smithi006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:35:08.004 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:35:08.504 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T03:35:08.505 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi006.front.sepia.ceph.com 2024-09-12T03:35:08.505 DEBUG:teuthology.orchestra.run.smithi006:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:35:08.529 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi118.front.sepia.ceph.com 2024-09-12T03:35:08.529 DEBUG:teuthology.orchestra.run.smithi118:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:35:08.566 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2024-09-12T03:35:08.566 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:35:08.596 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T03:35:08.597 DEBUG:teuthology.orchestra.run.smithi006:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:35:08.599 DEBUG:teuthology.orchestra.run.smithi118:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:35:08.608 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:35:08.666 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T03:35:08.667 DEBUG:teuthology.orchestra.run.smithi006:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:35:08.673 DEBUG:teuthology.orchestra.run.smithi118:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:35:08.676 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:35:09.557 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T03:35:09.568 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T03:35:09.568 DEBUG:teuthology.orchestra.run.smithi006:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:35:09.594 DEBUG:teuthology.orchestra.run.smithi118:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:35:09.622 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:35:09.651 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T03:35:09.662 DEBUG:teuthology.orchestra.run.smithi006:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:35:09.665 DEBUG:teuthology.orchestra.run.smithi118:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:35:09.667 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:35:09.686 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern = core 2024-09-12T03:35:09.692 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern = core 2024-09-12T03:35:09.718 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2024-09-12T03:35:09.746 DEBUG:teuthology.orchestra.run.smithi006:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:35:09.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:35:09.762 DEBUG:teuthology.orchestra.run.smithi118:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:35:09.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:35:09.793 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:35:09.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:35:09.809 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T03:35:09.821 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T03:35:09.822 DEBUG:teuthology.misc:Transferring archived files from smithi006:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901668/remote/smithi006 2024-09-12T03:35:09.822 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:35:09.879 DEBUG:teuthology.misc:Transferring archived files from smithi118:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901668/remote/smithi118 2024-09-12T03:35:09.879 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:35:09.916 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901668/remote/smithi161 2024-09-12T03:35:09.916 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:35:09.955 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T03:35:09.955 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:35:09.958 DEBUG:teuthology.orchestra.run.smithi118:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:35:09.960 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:35:10.005 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T03:35:10.019 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T03:35:10.019 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T03:35:10.048 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T03:35:10.049 DEBUG:teuthology.orchestra.run.smithi006:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:35:10.051 DEBUG:teuthology.orchestra.run.smithi118:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:35:10.053 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:35:10.064 INFO:teuthology.orchestra.run.smithi006.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:35 /home/ubuntu/cephtest 2024-09-12T03:35:10.066 INFO:teuthology.orchestra.run.smithi118.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:35 /home/ubuntu/cephtest 2024-09-12T03:35:10.069 INFO:teuthology.orchestra.run.smithi161.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:35 /home/ubuntu/cephtest 2024-09-12T03:35:10.070 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T03:35:10.082 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T03:35:10.108 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} duration: 1848.5379431247711 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-12T03:35:10.108 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T03:35:10.212 INFO:teuthology.run:pass